Incidents involved as both Developer and Deployer
Incident 1412 Reports
California Police Turned on Music to Allegedly Trigger Instagram’s DCMA to Avoid Being Live-Streamed
2021-02-05
A police officer in Beverly Hills played copyrighted music on his phone when realizing that his interactions were being recorded on a livestream, allegedly hoping the Instagram's automated copyright detection system to end or mute the stream.
MoreIncident 3432 Reports
Facebook, Instagram, and Twitter Failed to Proactively Remove Targeted Racist Remarks via Automated Systems
2021-07-11
Facebook's, Instagram's, and Twitter's automated content moderation failed to proactively remove racist remarks and posts directing at Black football players after finals loss, allegedly largely relying on user reports of harassment.
MoreIncident 3942 Reports
Social Media's Automated Word-Flagging without Context Shifted Content Creators' Language Use
2017-03-15
TikTok's, YouTube's, Instagram's, and Twitch's use of algorithms to flag certain words devoid of context changed content creators' use of everyday language or discussion about certain topics in fear of their content getting flagged or auto-demonetized by mistake.
MoreIncident 4472 Reports
Footballer's "X-Rated" Comment Created by Instagram's Mistranslation
2022-12-19
Instagram's English translation of a footballer's comment on his wife's post in Spanish made the message seem "racy" and "X-rated," which some fans found amusing.
MoreIncidents involved as Deployer
Incident 4693 Reports
Automated Adult Content Detection Tools Showed Bias against Women Bodies
2006-02-25
Automated content moderation tools to detect sexual explicitness or "raciness" reportedly exhibited bias against women bodies, resulting in suppression of reach despite not breaking platform policies.
MoreIncident 7232 Reports
Instagram Algorithms Reportedly Directed Children's Merchandise Ad Campaign to Adult Men and Sex Offenders
2024-05-13
An Instagram ad campaign for children's merchandise was intended to reach adult women but was instead predominantly shown to adult men, including convicted sex offenders, due to Instagram's algorithmic targeting. This failure is reported to have led to direct solicitations for sex with a 5-year-old model in the ads.
MoreIncident 5761 Report
Alleged Misuse of PicSo AI for Generating Inappropriate Content Emphasizing "Girls"
2023-10-24
PicSo AI, which appears to be getting advertised by Meta over Instagram, is allegedly being used for generating inappropriate content with an emphasis on "girls." This raises concerns about the misuse of generative AI technologies for creating offensive and potentially sexually explicit material that could be used for nefarious and criminal purposes.
MoreIncident 7581 Report
Teen's Overdose Reportedly Linked to Meta's AI Systems Failing to Block Ads for Illegal Drugs
2023-09-11
Meta's AI moderation systems reportedly failed to block ads for illegal drugs on Facebook and Instagram, allowing users to access dangerous substances. The system's failure is linked to the overdose death of Elijah Ott, a 15-year-old boy who sought drugs through Instagram.
MoreIncidents implicated systems
Incident 8852 Reports
Meta AI Characters Allegedly Exhibited Racism, Fabricated Identities, and Exploited User Trust
2025-01-03
Meta deployed AI-generated profiles on its platforms, including Instagram and Facebook, as part of an experiment. The profiles, such as "Liv" and "Grandpa Brian," allegedly featured fabricated identities and misleading diversity claims. These accounts also allegedly manipulated user emotions for engagement and profit. Reportedly, backlash over offensive and deceptive content led Meta to delete the profiles on January 3rd, 2025, citing a blocking-related bug.
MoreRelated Entities
Other entities that are related to the same incident. For example, if the developer of an incident is this entity but the deployer is another entity, they are marked as related entities.
Related Entities
Incidents involved as both Developer and Deployer
- Incident 3432 Reports
Facebook, Instagram, and Twitter Failed to Proactively Remove Targeted Racist Remarks via Automated Systems
- Incident 3591 Report
Facebook, Instagram, and Twitter Cited Errors in Automated Systems as Cause for Blocking pro-Palestinian Content on Israeli-Palestinian Conflict
Incidents involved as Deployer
- Incident 4693 Reports
Automated Adult Content Detection Tools Showed Bias against Women Bodies
- Incident 7581 Report
Teen's Overdose Reportedly Linked to Meta's AI Systems Failing to Block Ads for Illegal Drugs
Incidents implicated systems
Meta
Incidents involved as both Developer and Deployer
- Incident 7232 Reports
Instagram Algorithms Reportedly Directed Children's Merchandise Ad Campaign to Adult Men and Sex Offenders
- Incident 8852 Reports
Meta AI Characters Allegedly Exhibited Racism, Fabricated Identities, and Exploited User Trust