Incident 29: Image Classification of Battle Tanks

Description: A potentially apocryphal story in which an image classifier was produced to differentiate types of battle tanks, but the resulting model keyed in on environmental attributes rather than tank attributes

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscoverView HistoryView History

Entities

View all entities
Alleged: United States Government developed and deployed an AI system, which harmed United States Government.

Incident Stats

Incident ID
29
Report Count
3
Incident Date
2011-09-20
Editors
Sean McGregor
Applied Taxonomies
CSETv1, GMF, MIT

CSETv1 Taxonomy Classifications

Taxonomy Details

Incident Number

29

Estimated Date

No

Lives Lost

0

Injuries

0

Estimated Harm Quantities

No

There is a potentially identifiable specific entity that experienced the harm

No

MIT Taxonomy Classifications

Machine-Classified
Taxonomy Details

Risk Subdomain

7.3. Lack of capability or robustness

Risk Domain

  1. AI system safety, failures, and limitations

Entity

AI

Timing

Pre-deployment

Intent

Unintentional

gwern.net · 2011

Drawing on Google/Google Books/Google Scholar/Libgen/LessWrong/Hacker News/Twitter, I have compiled a large number of variants of the story from various sources; below, in reverse chronological order by decade.

A similar thing happened here…

Tales from the Trenches: AI Disaster Stories (GDC talk)
lobste.rs · 2016

His team was working on running simulations of long-distance manned spaceflight. In particular, the goal of their simulations was to determine an algorithm that would optimally allocate food, water, and electricity to 3 crew members. The de…

AI Incident Database Incidents Converted to Issues
github.com · 2022

The following former incidents have been converted to "issues" following an update to the incident definition and ingestion criteria.

21: Tougher Turing Test Exposes Chatbots’ Stupidity

Description: The 2016 Winograd Schema Challenge highli…

Variants

A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.

Similar Incidents

By textual similarity

Did our AI mess up? Flag the unrelated incidents

Northpointe Risk Models

Northpointe Risk Models

· 15 reports
AI Beauty Judge Did Not Like Dark Skin

AI Beauty Judge Did Not Like Dark Skin

· 10 reports
All Image Captions Produced are Violent

All Image Captions Produced are Violent

· 28 reports
Previous IncidentNext Incident