Skip to Content
logologo
AI Incident Database
Open TwitterOpen RSS FeedOpen FacebookOpen LinkedInOpen GitHub
Open Menu
Discover
Submit
  • Welcome to the AIID
  • Discover Incidents
  • Spatial View
  • Table View
  • List view
  • Entities
  • Taxonomies
  • Submit Incident Reports
  • Submission Leaderboard
  • Blog
  • AI News Digest
  • Risk Checklists
  • Random Incident
  • Sign Up
Collapse
Discover
Submit
  • Welcome to the AIID
  • Discover Incidents
  • Spatial View
  • Table View
  • List view
  • Entities
  • Taxonomies
  • Submit Incident Reports
  • Submission Leaderboard
  • Blog
  • AI News Digest
  • Risk Checklists
  • Random Incident
  • Sign Up
Collapse
Entities

Instagram users

Incidents Harmed By

Incident 26710 Report
Clearview AI Algorithm Built on Photos Scraped from Social Media Profiles without Consent

2017-06-15

Face-matching algorithm by Clearview AI was built using scraped images from social media sites such as Instagram and Facebook without user consent, violating social media site policies, and allegedly privacy regulations.

More

Incident 3246 Report
GAN Faces Deployed by The BL's Fake Account Network to Push Pro-Trump Content on Meta Platforms

2019-11-12

A large network of pages, groups, and fake accounts having GAN-generated face photos associated with The BL, a US-based media outlet, reportedly bypassed Facebook moderation systems to push "pro-Trump" narratives on its platform and Instagram.

More

Incident 8855 Report
Meta AI Characters Allegedly Exhibited Racism, Fabricated Identities, and Exploited User Trust

2025-01-03

Meta deployed AI-generated profiles on its platforms, including Instagram and Facebook, as part of an experiment. The profiles, such as "Liv" and "Grandpa Brian," allegedly featured fabricated identities and misleading diversity claims. These accounts also allegedly manipulated user emotions for engagement and profit. Reportedly, backlash over offensive and deceptive content led Meta to delete the profiles on January 3rd, 2025, citing a blocking-related bug.

More

Incident 1904 Report
ByteDance Allegedly Trained "For You" Algorithm Using Content Scraped without Consent from Other Social Platforms

2017-01-15

ByteDance allegedly scraped short-form videos, usernames, profile pictures, and descriptions of accounts on Instagram, Snapchat, and other sources, and uploaded them without consent on Flipagram, TikTok’s predecessor, in order to improve its “For You” algorithm's performance on American users.

More

Incidents involved as Deployer

Incident 7975 Report
Teenager Makes Deepfake Pornography of 50 Girls at Bacchus Marsh Grammar School in Australia

2024-06-07

At Bacchus Marsh Grammar, a school in Victoria state in Australia, an adolescent male made deepfake pornography of 50 girls, ages 9 to 12. He then allegedly uploaded the pictures to Instagram, and others subsequently shared them on Snapchat.

More

Related Entities
Other entities that are related to the same incident. For example, if the developer of an incident is this entity but the deployer is another entity, they are marked as related entities.
 

Entity

ByteDance

Incidents involved as both Developer and Deployer
  • Incident 190
    4 Reports

    ByteDance Allegedly Trained "For You" Algorithm Using Content Scraped without Consent from Other Social Platforms

More
Entity

Snapchat users

Incidents Harmed By
  • Incident 190
    4 Reports

    ByteDance Allegedly Trained "For You" Algorithm Using Content Scraped without Consent from Other Social Platforms

Incidents involved as Deployer
  • Incident 797
    5 Reports

    Teenager Makes Deepfake Pornography of 50 Girls at Bacchus Marsh Grammar School in Australia

More
Entity

American social media users

Incidents Harmed By
  • Incident 190
    4 Reports

    ByteDance Allegedly Trained "For You" Algorithm Using Content Scraped without Consent from Other Social Platforms

More
Entity

Clearview AI

Incidents involved as both Developer and Deployer
  • Incident 267
    10 Reports

    Clearview AI Algorithm Built on Photos Scraped from Social Media Profiles without Consent

More
Entity

social media users

Incidents Harmed By
  • Incident 267
    10 Reports

    Clearview AI Algorithm Built on Photos Scraped from Social Media Profiles without Consent

More
Entity

Facebook users

Incidents Harmed By
  • Incident 267
    10 Reports

    Clearview AI Algorithm Built on Photos Scraped from Social Media Profiles without Consent

  • Incident 324
    6 Reports

    GAN Faces Deployed by The BL's Fake Account Network to Push Pro-Trump Content on Meta Platforms

More
Entity

Instagram

Incidents involved as both Developer and Deployer
  • Incident 331
    2 Reports

    Bug in Instagram’s “Related Hashtags” Algorithm Allegedly Caused Disproportionate Treatment of Political Hashtags

  • Incident 343
    2 Reports

    Facebook, Instagram, and Twitter Failed to Proactively Remove Targeted Racist Remarks via Automated Systems

Incidents involved as Deployer
  • Incident 469
    3 Reports

    Automated Adult Content Detection Tools Showed Bias against Women Bodies

  • Incident 723
    2 Reports

    Instagram Algorithms Reportedly Directed Children's Merchandise Ad Campaign to Adult Men and Sex Offenders

Incidents implicated systems
  • Incident 885
    5 Reports

    Meta AI Characters Allegedly Exhibited Racism, Fabricated Identities, and Exploited User Trust

  • Incident 999
    2 Reports

    Attackers Reportedly Deployed Simulated AI Support Chatbot to Trick Instagram Business Users into Adding Malicious 2FA Login

More
Entity

The BL

Incidents involved as Deployer
  • Incident 324
    6 Reports

    GAN Faces Deployed by The BL's Fake Account Network to Push Pro-Trump Content on Meta Platforms

More
Entity

unknown

Incidents involved as Developer
  • Incident 324
    6 Reports

    GAN Faces Deployed by The BL's Fake Account Network to Push Pro-Trump Content on Meta Platforms

More
Entity

Facebook

Incidents involved as both Developer and Deployer
  • Incident 343
    2 Reports

    Facebook, Instagram, and Twitter Failed to Proactively Remove Targeted Racist Remarks via Automated Systems

  • Incident 359
    1 Report

    Facebook, Instagram, and Twitter Cited Errors in Automated Systems as Cause for Blocking pro-Palestinian Content on Israeli-Palestinian Conflict

Incidents involved as Deployer
  • Incident 469
    3 Reports

    Automated Adult Content Detection Tools Showed Bias against Women Bodies

  • Incident 758
    1 Report

    Teen's Overdose Reportedly Linked to Meta's AI Systems Failing to Block Ads for Illegal Drugs

Incidents implicated systems
  • Incident 885
    5 Reports

    Meta AI Characters Allegedly Exhibited Racism, Fabricated Identities, and Exploited User Trust

More
Entity

Twitter

Incidents involved as both Developer and Deployer
  • Incident 343
    2 Reports

    Facebook, Instagram, and Twitter Failed to Proactively Remove Targeted Racist Remarks via Automated Systems

  • Incident 359
    1 Report

    Facebook, Instagram, and Twitter Cited Errors in Automated Systems as Cause for Blocking pro-Palestinian Content on Israeli-Palestinian Conflict

More
Entity

Marcus Rashford

Incidents Harmed By
  • Incident 343
    2 Reports

    Facebook, Instagram, and Twitter Failed to Proactively Remove Targeted Racist Remarks via Automated Systems

More
Entity

Jadon Sancho

Incidents Harmed By
  • Incident 343
    2 Reports

    Facebook, Instagram, and Twitter Failed to Proactively Remove Targeted Racist Remarks via Automated Systems

More
Entity

Bukayo Saka

Incidents Harmed By
  • Incident 343
    2 Reports

    Facebook, Instagram, and Twitter Failed to Proactively Remove Targeted Racist Remarks via Automated Systems

More
Entity

Twitter Users

Incidents Harmed By
  • Incident 343
    2 Reports

    Facebook, Instagram, and Twitter Failed to Proactively Remove Targeted Racist Remarks via Automated Systems

  • Incident 359
    1 Report

    Facebook, Instagram, and Twitter Cited Errors in Automated Systems as Cause for Blocking pro-Palestinian Content on Israeli-Palestinian Conflict

More
Entity

Meta

Incidents involved as both Developer and Deployer
  • Incident 885
    5 Reports

    Meta AI Characters Allegedly Exhibited Racism, Fabricated Identities, and Exploited User Trust

  • Incident 723
    2 Reports

    Instagram Algorithms Reportedly Directed Children's Merchandise Ad Campaign to Adult Men and Sex Offenders

Incidents involved as Deployer
  • Incident 469
    3 Reports

    Automated Adult Content Detection Tools Showed Bias against Women Bodies

More
Entity

LinkedIn

Incidents involved as Deployer
  • Incident 469
    3 Reports

    Automated Adult Content Detection Tools Showed Bias against Women Bodies

More
Entity

Microsoft

Incidents involved as Developer
  • Incident 469
    3 Reports

    Automated Adult Content Detection Tools Showed Bias against Women Bodies

More
Entity

Google

Incidents involved as Developer
  • Incident 469
    3 Reports

    Automated Adult Content Detection Tools Showed Bias against Women Bodies

More
Entity

Amazon

Incidents involved as Developer
  • Incident 469
    3 Reports

    Automated Adult Content Detection Tools Showed Bias against Women Bodies

More
Entity

LinkedIn users

Incidents Harmed By
  • Incident 469
    3 Reports

    Automated Adult Content Detection Tools Showed Bias against Women Bodies

More
Entity

Palestinian social media users

Incidents Harmed By
  • Incident 359
    1 Report

    Facebook, Instagram, and Twitter Cited Errors in Automated Systems as Cause for Blocking pro-Palestinian Content on Israeli-Palestinian Conflict

More
Entity

Facebook employees having families affected by the conflict

Incidents Harmed By
  • Incident 359
    1 Report

    Facebook, Instagram, and Twitter Cited Errors in Automated Systems as Cause for Blocking pro-Palestinian Content on Israeli-Palestinian Conflict

More
Entity

Instagram sellers

Incidents Harmed By
  • Incident 723
    2 Reports

    Instagram Algorithms Reportedly Directed Children's Merchandise Ad Campaign to Adult Men and Sex Offenders

More
Entity

Children

Incidents Harmed By
  • Incident 723
    2 Reports

    Instagram Algorithms Reportedly Directed Children's Merchandise Ad Campaign to Adult Men and Sex Offenders

  • Incident 788
    1 Report

    Instagram's Algorithm Reportedly Recommended Sexual Content to Teenagers' Accounts

More
Entity

Meta Platforms

Incidents involved as both Developer and Deployer
  • Incident 758
    1 Report

    Teen's Overdose Reportedly Linked to Meta's AI Systems Failing to Block Ads for Illegal Drugs

  • Incident 788
    1 Report

    Instagram's Algorithm Reportedly Recommended Sexual Content to Teenagers' Accounts

More
Entity

Elijah Ott

Incidents Harmed By
  • Incident 758
    1 Report

    Teen's Overdose Reportedly Linked to Meta's AI Systems Failing to Block Ads for Illegal Drugs

More
Entity

minors

Incidents Harmed By
  • Incident 788
    1 Report

    Instagram's Algorithm Reportedly Recommended Sexual Content to Teenagers' Accounts

  • Incident 957
    1 Report

    Alleged Instagram Algorithm Malfunction Floods Users’ Reels Feeds with Violent and Graphic Content

More
Entity

General public

Incidents Harmed By
  • Incident 788
    1 Report

    Instagram's Algorithm Reportedly Recommended Sexual Content to Teenagers' Accounts

More
Entity

Unnamed adolescent male

Incidents involved as Deployer
  • Incident 797
    5 Reports

    Teenager Makes Deepfake Pornography of 50 Girls at Bacchus Marsh Grammar School in Australia

More
Entity

Unknown deepfake technology developer

Incidents involved as Developer
  • Incident 797
    5 Reports

    Teenager Makes Deepfake Pornography of 50 Girls at Bacchus Marsh Grammar School in Australia

More
Entity

Bacchus Marsh Grammar students

Incidents Harmed By
  • Incident 797
    5 Reports

    Teenager Makes Deepfake Pornography of 50 Girls at Bacchus Marsh Grammar School in Australia

More
Entity

Bacchus Marsh Grammar girls

Incidents Harmed By
  • Incident 797
    5 Reports

    Teenager Makes Deepfake Pornography of 50 Girls at Bacchus Marsh Grammar School in Australia

More
Entity

Meta users

Incidents Harmed By
  • Incident 885
    5 Reports

    Meta AI Characters Allegedly Exhibited Racism, Fabricated Identities, and Exploited User Trust

  • Incident 999
    2 Reports

    Attackers Reportedly Deployed Simulated AI Support Chatbot to Trick Instagram Business Users into Adding Malicious 2FA Login

More
Entity

Meta AI Characters

Incidents implicated systems
  • Incident 885
    5 Reports

    Meta AI Characters Allegedly Exhibited Racism, Fabricated Identities, and Exploited User Trust

More
Entity

Instagram recommendation algorithm

Incidents implicated systems
  • Incident 957
    1 Report

    Alleged Instagram Algorithm Malfunction Floods Users’ Reels Feeds with Violent and Graphic Content

More
Entity

Instagram Reels

Incidents implicated systems
  • Incident 957
    1 Report

    Alleged Instagram Algorithm Malfunction Floods Users’ Reels Feeds with Violent and Graphic Content

More
Entity

Scammers impersonating Meta support

Incidents involved as Deployer
  • Incident 999
    2 Reports

    Attackers Reportedly Deployed Simulated AI Support Chatbot to Trick Instagram Business Users into Adding Malicious 2FA Login

More
Entity

scammers

Incidents involved as Deployer
  • Incident 999
    2 Reports

    Attackers Reportedly Deployed Simulated AI Support Chatbot to Trick Instagram Business Users into Adding Malicious 2FA Login

More
Entity

Phishers

Incidents involved as Deployer
  • Incident 999
    2 Reports

    Attackers Reportedly Deployed Simulated AI Support Chatbot to Trick Instagram Business Users into Adding Malicious 2FA Login

More
Entity

Fraudsters

Incidents involved as Deployer
  • Incident 999
    2 Reports

    Attackers Reportedly Deployed Simulated AI Support Chatbot to Trick Instagram Business Users into Adding Malicious 2FA Login

More
Entity

Unknown generative AI developers

Incidents involved as Developer
  • Incident 999
    2 Reports

    Attackers Reportedly Deployed Simulated AI Support Chatbot to Trick Instagram Business Users into Adding Malicious 2FA Login

More
Entity

Instagram Business users

Incidents Harmed By
  • Incident 999
    2 Reports

    Attackers Reportedly Deployed Simulated AI Support Chatbot to Trick Instagram Business Users into Adding Malicious 2FA Login

More
Entity

Unknown chatbot interface

Incidents implicated systems
  • Incident 999
    2 Reports

    Attackers Reportedly Deployed Simulated AI Support Chatbot to Trick Instagram Business Users into Adding Malicious 2FA Login

More

Research

  • Defining an “AI Incident”
  • Defining an “AI Incident Response”
  • Database Roadmap
  • Related Work
  • Download Complete Database

Project and Community

  • About
  • Contact and Follow
  • Apps and Summaries
  • Editor’s Guide

Incidents

  • All Incidents in List Form
  • Flagged Incidents
  • Submission Queue
  • Classifications View
  • Taxonomies

2023 - AI Incident Database

  • Terms of use
  • Privacy Policy
  • Open twitterOpen githubOpen rssOpen facebookOpen linkedin
  • 9d70fba