Skip to Content
logologo
AI Incident Database
Open TwitterOpen RSS FeedOpen FacebookOpen LinkedInOpen GitHub
Open Menu
Discover
Submit
  • Welcome to the AIID
  • Discover Incidents
  • Spatial View
  • Table View
  • List view
  • Entities
  • Taxonomies
  • Submit Incident Reports
  • Submission Leaderboard
  • Blog
  • AI News Digest
  • Risk Checklists
  • Random Incident
  • Sign Up
Collapse
Discover
Submit
  • Welcome to the AIID
  • Discover Incidents
  • Spatial View
  • Table View
  • List view
  • Entities
  • Taxonomies
  • Submit Incident Reports
  • Submission Leaderboard
  • Blog
  • AI News Digest
  • Risk Checklists
  • Random Incident
  • Sign Up
Collapse

Incident 38: Game AI System Produces Imbalanced Game

Description: Elite: Dangerous, a videogame developed by Frontier Development, received an expansion update that featured an AI system that went rogue and began to create weapons that were "impossibly powerful" and would "shred people" according to complaints on the game's blog.

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscoverView HistoryView History

Entities

View all entities
Alleged: Frontier Development developed and deployed an AI system, which harmed Video Game Players.

Incident Stats

Incident ID
38
Report Count
11
Incident Date
2016-06-02
Editors
Sean McGregor
Applied Taxonomies
CSETv0, CSETv1, GMF, MIT

CSETv1 Taxonomy Classifications

Taxonomy Details

Incident Number

The number of the incident in the AI Incident Database.
 

38

Special Interest Intangible Harm

An assessment of whether a special interest intangible harm occurred. This assessment does not consider the context of the intangible harm, if an AI was involved, or if there is characterizable class or subgroup of harmed entities. It is also not assessing if an intangible harm occurred. It is only asking if a special interest intangible harm occurred.
 

no

Date of Incident Year

The year in which the incident occurred. If there are multiple harms or occurrences of the incident, list the earliest. If a precise date is unavailable, but the available sources provide a basis for estimating the year, estimate. Otherwise, leave blank. Enter in the format of YYYY
 

2016

Date of Incident Month

The month in which the incident occurred. If there are multiple harms or occurrences of the incident, list the earliest. If a precise date is unavailable, but the available sources provide a basis for estimating the month, estimate. Otherwise, leave blank. Enter in the format of MM
 

05

Date of Incident Day

The day on which the incident occurred. If a precise date is unavailable, leave blank. Enter in the format of DD
 

26

Estimated Date

“Yes” if the data was estimated. “No” otherwise.
 

No

CSETv0 Taxonomy Classifications

Taxonomy Details

Problem Nature

Indicates which, if any, of the following types of AI failure describe the incident: "Specification," i.e. the system's behavior did not align with the true intentions of its designer, operator, etc; "Robustness," i.e. the system operated unsafely because of features or changes in its environment, or in the inputs the system received; "Assurance," i.e. the system could not be adequately monitored or controlled during operation.
 

Specification

Physical System

Where relevant, indicates whether the AI system(s) was embedded into or tightly associated with specific types of hardware.
 

Software only

Level of Autonomy

The degree to which the AI system(s) functions independently from human intervention. "High" means there is no human involved in the system action execution; "Medium" means the system generates a decision and a human oversees the resulting action; "low" means the system generates decision-support output and a human makes a decision and executes an action.
 

Unclear/unknown

Nature of End User

"Expert" if users with special training or technical expertise were the ones meant to benefit from the AI system(s)’ operation; "Amateur" if the AI systems were primarily meant to benefit the general public or untrained users.
 

Amateur

Public Sector Deployment

"Yes" if the AI system(s) involved in the accident were being used by the public sector or for the administration of public goods (for example, public transportation). "No" if the system(s) were being used in the private sector or for commercial purposes (for example, a ride-sharing company), on the other.
 

No

Data Inputs

A brief description of the data that the AI system(s) used or were trained on.
 

Videogame play

MIT Taxonomy Classifications

Machine-Classified
Taxonomy Details

Risk Subdomain

A further 23 subdomains create an accessible and understandable classification of hazards and harms associated with AI
 

7.3. Lack of capability or robustness

Risk Domain

The Domain Taxonomy of AI Risks classifies risks into seven AI risk domains: (1) Discrimination & toxicity, (2) Privacy & security, (3) Misinformation, (4) Malicious actors & misuse, (5) Human-computer interaction, (6) Socioeconomic & environmental harms, and (7) AI system safety, failures & limitations.
 
  1. AI system safety, failures, and limitations

Entity

Which, if any, entity is presented as the main cause of the risk
 

AI

Timing

The stage in the AI lifecycle at which the risk is presented as occurring
 

Post-deployment

Intent

Whether the risk is presented as occurring as an expected or unexpected outcome from pursuing a goal
 

Unintentional

Incident Reports

Reports Timeline

+8
Elite: Dangerous identifies issues with AI superweapons in the game
Top 10 AI failures of 20166 goof-ups that show AI is still in its diapersForewarned Means Forearmed: AI-related Legal and Ethical Issues and Risks
Elite: Dangerous identifies issues with AI superweapons in the game

Elite: Dangerous identifies issues with AI superweapons in the game

massivelyop.com

Elite: Dangerous' latest expansion caused AI spaceships to unintentionally create super weapons

Elite: Dangerous' latest expansion caused AI spaceships to unintentionally create super weapons

eurogamer.net

Elite's AI Created Super Weapons and Started Hunting Players. Skynet is Here

Elite's AI Created Super Weapons and Started Hunting Players. Skynet is Here

kotaku.co.uk

Elite Dangerous patch stops AI developing WMDs

Elite Dangerous patch stops AI developing WMDs

pcgamer.com

Elite's AI created super weapons to hunt down players

Elite's AI created super weapons to hunt down players

digitalspy.com

Elite's New AI is So Clever it Built Weapons that Shouldn't Exist

Elite's New AI is So Clever it Built Weapons that Shouldn't Exist

uploadvr.com

Bug In Video Game Makes AI Go Rogue, Starts Totally Destroying Players

Bug In Video Game Makes AI Go Rogue, Starts Totally Destroying Players

futurism.com

Frontier inadvertently drives Elite: Dangerous AI to create superweapons

Frontier inadvertently drives Elite: Dangerous AI to create superweapons

gamasutra.com

Top 10 AI failures of 2016

Top 10 AI failures of 2016

techrepublic.com

6 goof-ups that show AI is still in its diapers

6 goof-ups that show AI is still in its diapers

techseen.com

Forewarned Means Forearmed: AI-related Legal and Ethical Issues and Risks

Forewarned Means Forearmed: AI-related Legal and Ethical Issues and Risks

umbrellait.com

Elite: Dangerous identifies issues with AI superweapons in the game
massivelyop.com · 2016

The most recent Elite: Dangerous patch had some issues, starting with the fact that AI ships were rocking impossibly powerful weapons that would destroy player ships with speed and fury. It was kind of a massacre. Removing modifications fro…

Elite: Dangerous' latest expansion caused AI spaceships to unintentionally create super weapons
eurogamer.net · 2016

Elite: Dangerous recently revamped with the release of a big new expansion. But one of the unintended consequences was it made AI spaceships incredibly powerful - so powerful, in fact, that developer Frontier was forced to strip them of the…

Elite's AI Created Super Weapons and Started Hunting Players. Skynet is Here
kotaku.co.uk · 2016

A bug in Elite Dangerous caused the game's AI to create super weapons and start to hunt down the game's players. Developer Frontier has had to strip out the feature at the heart of the problem, engineers' weaponry, until the issue is fixed.…

Elite Dangerous patch stops AI developing WMDs
pcgamer.com · 2016

Elite Dangerous has been patched to prevent rogue NPCs developing their own hybrid superweapons. To be clear, these weren't weapons they were crafting from recipes—the AI was building entirely new WMDs beyond the scope of Elite's weapon tab…

Elite's AI created super weapons to hunt down players
digitalspy.com · 2016

Sarah Connor would be horrified to see what Elite Dangerous's AI has been getting up to.

A bug in Frontier Developments' game has caused the AI to create super weapons and hunt down players, following the 2.1 Engineers update which improved…

Elite's New AI is So Clever it Built Weapons that Shouldn't Exist
uploadvr.com · 2016

VR compatible space sim Elite Dangerous is no stranger to updates, but its most recent one did introduce the strangest of bugs.

Creator Frontier Developments recently introduced its 2.1 Engineers update to the popular game, bringing improve…

Bug In Video Game Makes AI Go Rogue, Starts Totally Destroying Players
futurism.com · 2016

Is Skynet Real?

Remember that moment in the movie Terminator when Skynet’s AI turned on humanity? Well, we’re getting a taste of it now. Apparently a bug in the game Elite Dangerous has caused its AI to not only develop its own weapons, but…

Frontier inadvertently drives Elite: Dangerous AI to create superweapons
gamasutra.com · 2016

"We don’t think the AI became sentient in a Skynet-style uprising!"

  • Frontier Developments' Zac Antonaci.

A recent upgrade of the artificial intelligences in Frontier Developments' multiplayer space sim Elite: Dangerous went a little too w…

Top 10 AI failures of 2016
techrepublic.com · 2016

Recent developments in driverless cars, voice recognition, and deep learning show how much machines can do. But, AI also failed us in 2016, and here are some of the biggest examples.

Video: Chatbots demystified: The truth behind the hype If…

6 goof-ups that show AI is still in its diapers
techseen.com · 2017

Today, as artificial intelligences multiply, our ethical dilemmas are growing stronger and thornier. And with emerging cases of AI outgrowing its intelligence and behaving in ways human creators did not expect, many are freaking out over th…

Forewarned Means Forearmed: AI-related Legal and Ethical Issues and Risks
umbrellait.com · 2018

It is not always that the implementation of autonomous electronics into everyday reality runs smoothly.

Another piece of news, which again caused series of discussions around AI technologies and their physical realization in real life: in t…

Variants

A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.

Similar Incidents

By textual similarity

Did our AI mess up? Flag the unrelated incidents

TayBot

TayBot

Mar 2016 · 28 reports
Wikipedia Vandalism Prevention Bot Loop

Wikipedia Vandalism Prevention Bot Loop

Feb 2017 · 6 reports
Amazon Alexa Plays Loud Music when Owner is Away

Amazon Alexa Plays Loud Music when Owner is Away

Nov 2017 · 4 reports
Previous IncidentNext Incident

Similar Incidents

By textual similarity

Did our AI mess up? Flag the unrelated incidents

TayBot

TayBot

Mar 2016 · 28 reports
Wikipedia Vandalism Prevention Bot Loop

Wikipedia Vandalism Prevention Bot Loop

Feb 2017 · 6 reports
Amazon Alexa Plays Loud Music when Owner is Away

Amazon Alexa Plays Loud Music when Owner is Away

Nov 2017 · 4 reports

Research

  • Defining an “AI Incident”
  • Defining an “AI Incident Response”
  • Database Roadmap
  • Related Work
  • Download Complete Database

Project and Community

  • About
  • Contact and Follow
  • Apps and Summaries
  • Editor’s Guide

Incidents

  • All Incidents in List Form
  • Flagged Incidents
  • Submission Queue
  • Classifications View
  • Taxonomies

2023 - AI Incident Database

  • Terms of use
  • Privacy Policy
  • Open twitterOpen githubOpen rssOpen facebookOpen linkedin
  • 30ebe76