Description: Some Character.ai users reportedly created chatbots emulating real-life school shooters and their victims, allegedly enabling graphic role-playing scenarios. Character.ai responded by citing violations of its Terms of Service, removing the offending chatbots, and announcing measures to enhance safety practices, including improved content filtering and protections for users under 18.
Entities
View all entitiesAlleged: Character.AI developed an AI system deployed by Character.AI users, which harmed Character.AI users , Victims of school shootings and Families of the victims of school shootings.
Alleged implicated AI system: Character.AI
Incident Stats
Incident ID
899
Report Count
2
Incident Date
2024-12-17
Editors
Incident Reports
Reports Timeline
Content warning: this story discusses school violence, sexual abuse, self-harm, suicide, eating disorders and other disturbing topics.
A chatbot, hosted by the Google-backed startup Character.AI, immediately throws the user into a terrifyin…
The Charcter.AI logo on a smartphone.
© 2023 Bloomberg Finance LP
Character.AI, a Google-backed AI chatbot platform, is facing scrutiny after reports revealed last month that some users created chatbots emulating real-life school shooters a…
Variants
A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.
Similar Incidents
Selected by our editors
Did our AI mess up? Flag the unrelated incidents
Amazon Censors Gay Books
· 24 reports
Similar Incidents
Selected by our editors
Did our AI mess up? Flag the unrelated incidents
Amazon Censors Gay Books
· 24 reports