Skip to Content
logologo
AI Incident Database
Open TwitterOpen RSS FeedOpen FacebookOpen LinkedInOpen GitHub
Open Menu
Discover
Submit
  • Welcome to the AIID
  • Discover Incidents
  • Spatial View
  • Table View
  • List view
  • Entities
  • Taxonomies
  • Submit Incident Reports
  • Submission Leaderboard
  • Blog
  • AI News Digest
  • Risk Checklists
  • Random Incident
  • Sign Up
Collapse
Discover
Submit
  • Welcome to the AIID
  • Discover Incidents
  • Spatial View
  • Table View
  • List view
  • Entities
  • Taxonomies
  • Submit Incident Reports
  • Submission Leaderboard
  • Blog
  • AI News Digest
  • Risk Checklists
  • Random Incident
  • Sign Up
Collapse

Incident 41: All Image Captions Produced are Violent

Description: MIT Media Lab researchers create AI-powered "psychopath" named Norman by training a model on "dark corners" of Reddit.

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscoverView HistoryView History

Entities

View all entities
Alleged: MIT Media Lab developed and deployed an AI system, which harmed unknown.

Incident Stats

Incident ID
41
Report Count
28
Incident Date
2018-04-02
Editors
Sean McGregor
Applied Taxonomies
CSETv0, CSETv1, GMF, MIT

CSETv1 Taxonomy Classifications

Taxonomy Details

Incident Number

The number of the incident in the AI Incident Database.
 

41

CSETv0 Taxonomy Classifications

Taxonomy Details

Problem Nature

Indicates which, if any, of the following types of AI failure describe the incident: "Specification," i.e. the system's behavior did not align with the true intentions of its designer, operator, etc; "Robustness," i.e. the system operated unsafely because of features or changes in its environment, or in the inputs the system received; "Assurance," i.e. the system could not be adequately monitored or controlled during operation.
 

Unknown/unclear

Physical System

Where relevant, indicates whether the AI system(s) was embedded into or tightly associated with specific types of hardware.
 

Software only

Level of Autonomy

The degree to which the AI system(s) functions independently from human intervention. "High" means there is no human involved in the system action execution; "Medium" means the system generates a decision and a human oversees the resulting action; "low" means the system generates decision-support output and a human makes a decision and executes an action.
 

High

Nature of End User

"Expert" if users with special training or technical expertise were the ones meant to benefit from the AI system(s)’ operation; "Amateur" if the AI systems were primarily meant to benefit the general public or untrained users.
 

Expert

Public Sector Deployment

"Yes" if the AI system(s) involved in the accident were being used by the public sector or for the administration of public goods (for example, public transportation). "No" if the system(s) were being used in the private sector or for commercial purposes (for example, a ride-sharing company), on the other.
 

No

Data Inputs

A brief description of the data that the AI system(s) used or were trained on.
 

Violent content from Reddit for the Norman algorithm, MSCOCO dataset for the control algorithm.

MIT Taxonomy Classifications

Machine-Classified
Taxonomy Details

Risk Subdomain

A further 23 subdomains create an accessible and understandable classification of hazards and harms associated with AI
 

7.3. Lack of capability or robustness

Risk Domain

The Domain Taxonomy of AI Risks classifies risks into seven AI risk domains: (1) Discrimination & toxicity, (2) Privacy & security, (3) Misinformation, (4) Malicious actors & misuse, (5) Human-computer interaction, (6) Socioeconomic & environmental harms, and (7) AI system safety, failures & limitations.
 
  1. AI system safety, failures, and limitations

Entity

Which, if any, entity is presented as the main cause of the risk
 

Human

Timing

The stage in the AI lifecycle at which the risk is presented as occurring
 

Pre-deployment

Intent

Whether the risk is presented as occurring as an expected or unexpected outcome from pursuing a goal
 

Intentional

Incident Reports

Reports Timeline

+1
Meet 'Norman,' the Darkest, Most Disturbed AI the World Has Ever Seen
Are you scared yet? Meet Norman, the psychopathic AI+18
MIT Scientists Create Norman, The World's First "Psychopathic" AI
+4
Scientists have created a murder-obsessed 'psychopath' AI called Norman -- and it learned everything it knows from Reddit
"Norman" Is MIT's New Psychopathic AIThis Is Why You Should Know About Norman the AI Psychopath
Meet 'Norman,' the Darkest, Most Disturbed AI the World Has Ever Seen

Meet 'Norman,' the Darkest, Most Disturbed AI the World Has Ever Seen

livescience.com

Are you scared yet? Meet Norman, the psychopathic AI

Are you scared yet? Meet Norman, the psychopathic AI

bbc.co.uk

MIT Scientists Create Norman, The World's First "Psychopathic" AI

MIT Scientists Create Norman, The World's First "Psychopathic" AI

iflscience.com

MIT Creates An AI Psychopath Because Someone Had To Eventually

MIT Creates An AI Psychopath Because Someone Had To Eventually

geek.com

Norman the psychopathic AI offers a warning on biased data

Norman the psychopathic AI offers a warning on biased data

internetofbusiness.com

MIT Creates AI-Powered Psychopath Called 'Norman'

MIT Creates AI-Powered Psychopath Called 'Norman'

extremetech.com

Norman Bates-a ‘psychopath’ AI created by MIT-was trained by studying reddit posts of “gruesome deaths.”

Norman Bates-a ‘psychopath’ AI created by MIT-was trained by studying reddit posts of “gruesome deaths.”

reddit.com

MIT Creates A Psychopathic AI Named Norman, Because Why Not

MIT Creates A Psychopathic AI Named Norman, Because Why Not

analyticsindiamag.com

MIT Trains Psychopath Robot 'Norman' Using Only Gruesome Reddit Images

MIT Trains Psychopath Robot 'Norman' Using Only Gruesome Reddit Images

newsweek.com

Meet Norman, the world's first 'psychopathic' AI

Meet Norman, the world's first 'psychopathic' AI

zdnet.com

MIT Scientists Create 'Psychopath' AI Named Norman

MIT Scientists Create 'Psychopath' AI Named Norman

fortune.com

MIT researchers use Reddit to create the first "psychopath AI"

MIT researchers use Reddit to create the first "psychopath AI"

cnet.com

MIT scientists created an AI-powered 'psychopath' named Norman

MIT scientists created an AI-powered 'psychopath' named Norman

money.cnn.com

MIT fed an AI data from Reddit, and now it only thinks about murder

MIT fed an AI data from Reddit, and now it only thinks about murder

theverge.com

'Psychopath AI' Offers A Cautionary Tale for Technologists

'Psychopath AI' Offers A Cautionary Tale for Technologists

blogs.discovermagazine.com

MIT's Psycho Norman AI Proves Reddit Can Make You Insane

MIT's Psycho Norman AI Proves Reddit Can Make You Insane

hothardware.com

MIT scientists created a “psychopath” AI by feeding it violent content from Reddit

MIT scientists created a “psychopath” AI by feeding it violent content from Reddit

news.avclub.com

MIT Scientists Unveil First Psychopath AI, ‘Norman’

MIT Scientists Unveil First Psychopath AI, ‘Norman’

rollingstone.com

MIT Trained an A.I. With Reddit, and it Became a Psychopath

MIT Trained an A.I. With Reddit, and it Became a Psychopath

digitaltrends.com

We Asked MIT Researchers Why They Made a ‘Psychotic AI' That Only Sees Death

We Asked MIT Researchers Why They Made a ‘Psychotic AI' That Only Sees Death

motherboard.vice.com

Meet Norman, the 'psychopath AI' that's here to teach us a lesson

Meet Norman, the 'psychopath AI' that's here to teach us a lesson

independent.co.uk

Scientists have created a murder-obsessed 'psychopath' AI called Norman -- and it learned everything it knows from Reddit

Scientists have created a murder-obsessed 'psychopath' AI called Norman -- and it learned everything it knows from Reddit

businessinsider.com.au

Scientists Made an Psychopathic AI Based on Reddit

Scientists Made an Psychopathic AI Based on Reddit

popularmechanics.com

Meet Norman, the world’s first psychopath AI

Meet Norman, the world’s first psychopath AI

livemint.com

'Norman,' when artificial intelligence goes psycho

'Norman,' when artificial intelligence goes psycho

phys.org

Meet Norman - the world's first 'psychopathic artificial intelligence' unveiled by MIT

Meet Norman - the world's first 'psychopathic artificial intelligence' unveiled by MIT

news.sky.com

"Norman" Is MIT's New Psychopathic AI

"Norman" Is MIT's New Psychopathic AI

curiosity.com

This Is Why You Should Know About Norman the AI Psychopath

This Is Why You Should Know About Norman the AI Psychopath

inc.com

Meet 'Norman,' the Darkest, Most Disturbed AI the World Has Ever Seen
livescience.com · 2018

A neural network named "Norman" is disturbingly different from other types of artificial intelligence (AI).

Housed at MIT Media Lab, a research laboratory that investigates AI and machine learning, Norman's computer brain was allegedly warp…

Are you scared yet? Meet Norman, the psychopathic AI
bbc.co.uk · 2018

Image copyright MIT Image caption Norman was named after Alfred Hitchcock's Norman Bates from his classic horror film Psycho

Norman is an algorithm trained to understand pictures but, like its namesake Hitchcock's Norman Bates, it does not …

MIT Scientists Create Norman, The World's First "Psychopathic" AI
iflscience.com · 2018

A team of scientists at the Massachusetts Institute of Technology (MIT) have built a psychopathic AI using image captions pulled from Reddit. Oh, and they’ve named it Norman after Alfred Hitchcock’s Norman Bates. This is how our very own Te…

MIT Creates An AI Psychopath Because Someone Had To Eventually
geek.com · 2018

In one of the big musical numbers from The Life Of Brian, Eric Idle reminds us to “always look on the bright side of life.” Norman, a new artificial intelligence project from MIT, doesn’t know how to do that.

That’s because Norman is a psyc…

Norman the psychopathic AI offers a warning on biased data
internetofbusiness.com · 2018

Many people are concerned about the potential rise of malignant AI, with UK newspapers, in particular, worried about the ‘Terminator’ scenario of machines that are hostile to humanity.

Researchers at MIT have decided to explore this concept…

MIT Creates AI-Powered Psychopath Called 'Norman'
extremetech.com · 2018

Artificial intelligence researchers have thus far attempted to make well-rounded algorithms that can be helpful to humanity. However, a team from MIT has undertaken a project to do the exact opposite. Researchers from the MIT Media Lab have…

Norman Bates-a ‘psychopath’ AI created by MIT-was trained by studying reddit posts of “gruesome deaths.”
reddit.com · 2018

I appreciate what MIT has done here. However, the work described in the article is useless, misleading, or both. All the training methods I've encountered in the literature do not maintain any kind of measure of the mutual information betwe…

MIT Creates A Psychopathic AI Named Norman, Because Why Not
analyticsindiamag.com · 2018

The Massachusets Institute of Technology’s website for Norman, world’s first psychopathic AI is oddly cheerful and optimistic. A creepy combination of Norman Bates (from the 1960 Alfred Hitchcock Movie Psycho) and a robot stares at you and …

MIT Trains Psychopath Robot 'Norman' Using Only Gruesome Reddit Images
newsweek.com · 2018

Scientists at the Massachusetts Institute of Technology (MIT) trained an artificial intelligence algorithm dubbed "Norman" to become a psychopath by only exposing it to macabre Reddit images of gruesome deaths and violence, according to a n…

Meet Norman, the world's first 'psychopathic' AI
zdnet.com · 2018

Norman: MIT's artificial intelligence with psychopathic traits The experiment is based on the 1921 Rorschach test, which identifies psychopathic traits based on users' perception of inkblots.

Researchers at the Massachusetts Institute of Te…

MIT Scientists Create 'Psychopath' AI Named Norman
fortune.com · 2018

MIT scientists’ newest artificial intelligence algorithm endeavor birthed a “psychopath” by the name of Norman.

Scientists, Pinar Yanardag, Manuel Cebrian, and Iyad Rahwan, exposed AI Norman, named after Anthony Perkins’ character in Alfred…

MIT researchers use Reddit to create the first "psychopath AI"
cnet.com · 2018

MIT Media Lab

Meet Norman.

He's not your everyday AI. His algorithms won't help filter through your Facebook feed or recommend you new songs to listen to on Spotify.

Nope -- Norman is a "psychopath AI", created by researchers at the MIT Med…

MIT scientists created an AI-powered 'psychopath' named Norman
money.cnn.com · 2018

Norman always sees the worst in things.

That's because Norman is a "psychopath" powered by artificial intelligence and developed by the MIT Media Lab.

Norman is an algorithm meant to show how the data behind AI matters deeply.

MIT researche…

MIT fed an AI data from Reddit, and now it only thinks about murder
theverge.com · 2018

For some, the phrase “artificial intelligence” conjures nightmare visions — something out of the ’04 Will Smith flick I, Robot, perhaps, or the ending of Ex Machina — like a boot smashing through the glass of a computer screen to stamp on a…

'Psychopath AI' Offers A Cautionary Tale for Technologists
blogs.discovermagazine.com · 2018

Researchers at MIT have created a psychopath. They call him Norman. He's a computer. Actually, that's not really right. Though the team calls Norman a psychopath (and the chilling lead graphic on their homepage certainly backs that up), wha…

MIT's Psycho Norman AI Proves Reddit Can Make You Insane
hothardware.com · 2018

MIT has created a new AI that we sincerely hope never escapes into the wild because this IS how you get Skynet. This AI is called Norman and it is a psychopath. How did Norman turn into a psycho? All the image data MIT fed Norman came from …

MIT scientists created a “psychopath” AI by feeding it violent content from Reddit
news.avclub.com · 2018

Photo: Matt Cardy (Getty Images)

We’ve all seen evil machines in The Terminator or The Matrix, but how does a machine become evil? Is it like Project Satan from Futurama, where scientists combined parts from various evil cars to create the …

MIT Scientists Unveil First Psychopath AI, ‘Norman’
rollingstone.com · 2018

Scientists at the Massachusetts Institute of Technology unveiled the first artificial intelligence algorithm trained to be a psychopath. The AI was fittingly dubbed “Norman” after Norman Bates, the notorious killer in Alfred Hitchcock’s Psy…

MIT Trained an A.I. With Reddit, and it Became a Psychopath
digitaltrends.com · 2018

Share

Science fiction has given us many iconic malevolent A.I. characters. However, these are often figures like Terminator’s T-800 or Alien’s Ash who commit emotionless murder to pursue an end goal. Those which exhibit more unhinged parano…

We Asked MIT Researchers Why They Made a ‘Psychotic AI' That Only Sees Death
motherboard.vice.com · 2018

MIT researchers created an artificial intelligence that they call “psychopathic” in order to show the biases that are inherent in AI research. When asked to look at Rorschach test blots, “Norman” always sees death. Here’s what Norman saw, c…

Meet Norman, the 'psychopath AI' that's here to teach us a lesson
independent.co.uk · 2018

The development of artificial intelligence, Stephen Hawking once warned, will be “either the best or the worst thing ever to happen to humanity”. A new AI algorithm exposed to the most macabre corners of the internet demonstrates how we cou…

Scientists have created a murder-obsessed 'psychopath' AI called Norman -- and it learned everything it knows from Reddit
businessinsider.com.au · 2018

Researchers at MIT have programmed an AI using exclusively violent and gruesome content from Reddit.

They called it “Norman.”

As a result, Norman only sees death in everything.

This isn’t the first time an AI has been turned dark by the int…

Scientists Made an Psychopathic AI Based on Reddit
popularmechanics.com · 2018

Scientists at MIT have created an AI psychopath trained on images from a particularly disturbing thread on Reddit. Norman is designed to illustrate that the data used for machine learning can significantly impact its outcome. “Norman suffer…

Meet Norman, the world’s first psychopath AI
livemint.com · 2018

Artificial Intelligence (AI) is solving many problems for humans. But, as Google CEO Sundar Pichai said in the company’s manifesto for AI, such a powerful technology “raises equally powerful questions about its use". Google (Alphabet Inc.) …

'Norman,' when artificial intelligence goes psycho
phys.org · 2018

Researchers at MIT have created 'Norman', the first psychopathic artificial intelligence to explain how algorithms are made, and to make people aware of AI's potential dangers

No, it's not a new horror film. It's Norman: also known as the f…

Meet Norman - the world's first 'psychopathic artificial intelligence' unveiled by MIT
news.sky.com · 2018

Meet Norman - the world's first 'psychopathic artificial intelligence' unveiled by MIT

Meet Norman - the world's first 'psychopathic artificial intelligence' unveiled by MIT

It shares his name with the knife-wielding killer in Alfred Hitchc…

"Norman" Is MIT's New Psychopathic AI
curiosity.com · 2018

This experiment with Norman shows that it doesn't matter how rigorous the actual algorithm is if it's overexposed to harmful content. Algorithms like pymetrics actually do have a beneficial effect, but it's vital that it's looked over by a …

This Is Why You Should Know About Norman the AI Psychopath
inc.com · 2018

You can learn a lot from a psychopath, and you don't even have to binge-watch Netflix's "Mindhunter" to do it. You just need to study Norman, the artificial intelligence psychopath aptly named for the pivotal figure in Alfred Hitchcock's "P…

Variants

A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.

Similar Incidents

By textual similarity

Did our AI mess up? Flag the unrelated incidents

Deepfake Obama Introduction of Deepfakes

Deepfake Obama Introduction of Deepfakes

Jul 2017 · 29 reports
TayBot

TayBot

Mar 2016 · 28 reports
AI Beauty Judge Did Not Like Dark Skin

AI Beauty Judge Did Not Like Dark Skin

Sep 2016 · 10 reports
Previous IncidentNext Incident

Similar Incidents

By textual similarity

Did our AI mess up? Flag the unrelated incidents

Deepfake Obama Introduction of Deepfakes

Deepfake Obama Introduction of Deepfakes

Jul 2017 · 29 reports
TayBot

TayBot

Mar 2016 · 28 reports
AI Beauty Judge Did Not Like Dark Skin

AI Beauty Judge Did Not Like Dark Skin

Sep 2016 · 10 reports

Research

  • Defining an “AI Incident”
  • Defining an “AI Incident Response”
  • Database Roadmap
  • Related Work
  • Download Complete Database

Project and Community

  • About
  • Contact and Follow
  • Apps and Summaries
  • Editor’s Guide

Incidents

  • All Incidents in List Form
  • Flagged Incidents
  • Submission Queue
  • Classifications View
  • Taxonomies

2023 - AI Incident Database

  • Terms of use
  • Privacy Policy
  • Open twitterOpen githubOpen rssOpen facebookOpen linkedin
  • 9d70fba