Skip to Content
logologo
AI Incident Database
Open TwitterOpen RSS FeedOpen FacebookOpen LinkedInOpen GitHub
Open Menu
Discover
Submit
  • Welcome to the AIID
  • Discover Incidents
  • Spatial View
  • Table View
  • List view
  • Entities
  • Taxonomies
  • Submit Incident Reports
  • Submission Leaderboard
  • Blog
  • AI News Digest
  • Risk Checklists
  • Random Incident
  • Sign Up
Collapse
Discover
Submit
  • Welcome to the AIID
  • Discover Incidents
  • Spatial View
  • Table View
  • List view
  • Entities
  • Taxonomies
  • Submit Incident Reports
  • Submission Leaderboard
  • Blog
  • AI News Digest
  • Risk Checklists
  • Random Incident
  • Sign Up
Collapse

Incident 60: FaceApp Racial Filters

Description: FaceApp is criticized for offering racist filters.

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscoverView HistoryView History

Entities

View all entities
Alleged: FaceApp developed and deployed an AI system, which harmed Minority Groups.

Incident Stats

Incident ID
60
Report Count
23
Incident Date
2017-04-25
Editors
Sean McGregor
Applied Taxonomies
CSETv0, CSETv1, GMF, MIT

CSETv1 Taxonomy Classifications

Taxonomy Details

Incident Number

The number of the incident in the AI Incident Database.
 

60

AI Tangible Harm Level Notes

Notes about the AI tangible harm level assessment
 

Annotator 1:

3.3 - Although there was no tangible harm, the adverse outcomes of the incident were directly related to the AI.

Notes (special interest intangible harm)

Input any notes that may help explain your answers.
 

Annotator 1:

4.4 - FaceApp added a filter that allowed users to look like different races/ethnicities. It also added a "hot" filter that automatically lightened skin color.

Special Interest Intangible Harm

An assessment of whether a special interest intangible harm occurred. This assessment does not consider the context of the intangible harm, if an AI was involved, or if there is characterizable class or subgroup of harmed entities. It is also not assessing if an intangible harm occurred. It is only asking if a special interest intangible harm occurred.
 

yes

CSETv0 Taxonomy Classifications

Taxonomy Details

Problem Nature

Indicates which, if any, of the following types of AI failure describe the incident: "Specification," i.e. the system's behavior did not align with the true intentions of its designer, operator, etc; "Robustness," i.e. the system operated unsafely because of features or changes in its environment, or in the inputs the system received; "Assurance," i.e. the system could not be adequately monitored or controlled during operation.
 

Unknown/unclear

Physical System

Where relevant, indicates whether the AI system(s) was embedded into or tightly associated with specific types of hardware.
 

Software only

Level of Autonomy

The degree to which the AI system(s) functions independently from human intervention. "High" means there is no human involved in the system action execution; "Medium" means the system generates a decision and a human oversees the resulting action; "low" means the system generates decision-support output and a human makes a decision and executes an action.
 

Low

Nature of End User

"Expert" if users with special training or technical expertise were the ones meant to benefit from the AI system(s)’ operation; "Amateur" if the AI systems were primarily meant to benefit the general public or untrained users.
 

Amateur

Public Sector Deployment

"Yes" if the AI system(s) involved in the accident were being used by the public sector or for the administration of public goods (for example, public transportation). "No" if the system(s) were being used in the private sector or for commercial purposes (for example, a ride-sharing company), on the other.
 

No

Data Inputs

A brief description of the data that the AI system(s) used or were trained on.
 

Photos of faces

MIT Taxonomy Classifications

Machine-Classified
Taxonomy Details

Risk Subdomain

A further 23 subdomains create an accessible and understandable classification of hazards and harms associated with AI
 

1.1. Unfair discrimination and misrepresentation

Risk Domain

The Domain Taxonomy of AI Risks classifies risks into seven AI risk domains: (1) Discrimination & toxicity, (2) Privacy & security, (3) Misinformation, (4) Malicious actors & misuse, (5) Human-computer interaction, (6) Socioeconomic & environmental harms, and (7) AI system safety, failures & limitations.
 
  1. Discrimination and Toxicity

Entity

Which, if any, entity is presented as the main cause of the risk
 

AI

Timing

The stage in the AI lifecycle at which the risk is presented as occurring
 

Post-deployment

Intent

Whether the risk is presented as occurring as an expected or unexpected outcome from pursuing a goal
 

Unintentional

Incident Reports

Reports Timeline

+8
FaceApp CEO Apologizes For Whitening Skin With 'Hot' Filter
How Does FaceApp Work? Here's How You Can Get In On The Viral Face Mashing TrendHow popular smartphones make your skin look 'whiter' in selfies+12
Photo-editing app FaceApp updates with Asian, Black, Caucasian and Indian filters
FaceApp CEO Apologizes For Whitening Skin With 'Hot' Filter

FaceApp CEO Apologizes For Whitening Skin With 'Hot' Filter

dailydot.com

FaceApp sorry for 'racist' filter that lightens skin to make users 'hot'

FaceApp sorry for 'racist' filter that lightens skin to make users 'hot'

bbc.co.uk

FaceApp Blames AI for Whitening up Black People

FaceApp Blames AI for Whitening up Black People

gizmodo.co.uk

FaceApp deletes new black, white and Asian filters after racism storm

FaceApp deletes new black, white and Asian filters after racism storm

telegraph.co.uk

That Viral 'FaceApp' Is Making Everyone Look Whiter

That Viral 'FaceApp' Is Making Everyone Look Whiter

motherboard.vice.com

FaceApp apologises for 'racist' filter that lightens users' skintone

FaceApp apologises for 'racist' filter that lightens users' skintone

theguardian.com

FaceApp Removes 'Spark' Filter Following Whitewashing Accusations

FaceApp Removes 'Spark' Filter Following Whitewashing Accusations

colorlines.com

FaceApp: Experts have privacy concerns about popular face transformation app

FaceApp: Experts have privacy concerns about popular face transformation app

abc.net.au

How Does FaceApp Work? Here's How You Can Get In On The Viral Face Mashing Trend

How Does FaceApp Work? Here's How You Can Get In On The Viral Face Mashing Trend

bustle.com

How popular smartphones make your skin look 'whiter' in selfies

How popular smartphones make your skin look 'whiter' in selfies

businessinsider.com.au

Photo-editing app FaceApp updates with Asian, Black, Caucasian and Indian filters

Photo-editing app FaceApp updates with Asian, Black, Caucasian and Indian filters

mic.com

Popular face-aging app now offers ‘Black,’ ‘Indian,’ and ‘Asian’ filters

Popular face-aging app now offers ‘Black,’ ‘Indian,’ and ‘Asian’ filters

theverge.com

FaceApp Removes Black, Asian, and Indian Filters After Backlash They Should Have Expected

FaceApp Removes Black, Asian, and Indian Filters After Backlash They Should Have Expected

complex.com

FaceApp removes 'Ethnicity Filters' after racism storm

FaceApp removes 'Ethnicity Filters' after racism storm

dailymail.co.uk

FaceApp briefly let users change their skin color. Bad idea.

FaceApp briefly let users change their skin color. Bad idea.

slate.com

FaceApp forced to pull 'racist' filters that allow 'digital blackface'

FaceApp forced to pull 'racist' filters that allow 'digital blackface'

theguardian.com

FaceApp Launches Digital Blackface Options Because The World Is Filled With Idiots

FaceApp Launches Digital Blackface Options Because The World Is Filled With Idiots

gizmodo.com.au

FaceApp's Indian and Black photo filters are quite obviously racist

FaceApp's Indian and Black photo filters are quite obviously racist

dailyo.in

FaceApp releases change-your-race filters

FaceApp releases change-your-race filters

stuff.co.nz

'Racist' FaceApp photo filters encouraged users to black up

'Racist' FaceApp photo filters encouraged users to black up

independent.co.uk

FaceApp Introduced Blackface And Yellowface Selfie Filters, Then Removed Them A Few Hours Later

FaceApp Introduced Blackface And Yellowface Selfie Filters, Then Removed Them A Few Hours Later

buzzfeednews.com

FaceApp Is Being Called "Racist" Again for Featuring Problematic Filters

FaceApp Is Being Called "Racist" Again for Featuring Problematic Filters

allure.com

FaceApp Ethnicity Filters Make You Look Like a Different Race

FaceApp Ethnicity Filters Make You Look Like a Different Race

cosmopolitan.com

FaceApp CEO Apologizes For Whitening Skin With 'Hot' Filter
dailydot.com · 2017

Looks like FaceApp, the new face-morphing craze, already needs a makeover.

Launched in January and hailing from Russia, FaceApp is catching interest for its artificial intelligence technology and intriguing filters that can make you look ol…

FaceApp sorry for 'racist' filter that lightens skin to make users 'hot'
bbc.co.uk · 2017

The makers of a face-morphing app have apologised after users accused them of creating a "racist" filter.

FaceApp can change portraits to make people look older, change gender or become "more attractive".

But when users with darker skin ton…

FaceApp Blames AI for Whitening up Black People
gizmodo.co.uk · 2017

People who actually want to see their faces reflected in their phone screens have been having fun with FaceApp recently, with Facebook and Twitter currently overloaded with images of users sharing what they look like young, old and "hot" as…

FaceApp deletes new black, white and Asian filters after racism storm
telegraph.co.uk · 2017

A viral app that adds filters to users' selfies to change their appearance has backtracked on its latest update after was accused of racism over its new range of ethnic filters.

FaceApp, which uses facial recognition to change users' expres…

That Viral 'FaceApp' Is Making Everyone Look Whiter
motherboard.vice.com · 2017

Twitter/Terrance AB Johnson

It happened—a tech company did a bad thing again. This time, it's the super popular face-morphing app called, wait for it…FaceApp.

For some reason unknown to me, FaceApp went gangbusters this week after launching…

FaceApp apologises for 'racist' filter that lightens users' skintone
theguardian.com · 2017

The creator of an app which changes your selfies using artificial intelligence has apologised because its “hot” filter automatically lightened people’s skin.

FaceApp is touted as an app which uses “neural networks” to change facial characte…

FaceApp Removes 'Spark' Filter Following Whitewashing Accusations
colorlines.com · 2017

A selfie app that renders users’ faces in varying styles removed one of its filters following criticism that it Whitewashed the faces of people of color.

Mic reports that after receiving complaints, FaceApp developers first changed the func…

FaceApp: Experts have privacy concerns about popular face transformation app
abc.net.au · 2017

FaceApp: Experts have privacy concerns about popular face transformation app

Updated

FaceApp has gone viral. Why? Because it lets you transform your face in hilarious ways.

But privacy advocates warn you could be giving up much more informa…

How Does FaceApp Work? Here's How You Can Get In On The Viral Face Mashing Trend
bustle.com · 2017

Social media is awash right now in selfies with weirdly realistic facial transformations thanks to a viral app, appropriately called “FaceApp.” FaceApp lets users add filters to faces to make them smile, age, and change gender, with a surpr…

How popular smartphones make your skin look 'whiter' in selfies
businessinsider.com.au · 2017

Shona Ghosh/Business Insider Beauty filters on popular smartphones turn you paler in selfies

If you own a phone which runs Android, the chances are it has a camera setting you don’t know about.

Popular smartphones like the Samsung Galaxy S8…

Photo-editing app FaceApp updates with Asian, Black, Caucasian and Indian filters
mic.com · 2017

On Wednesday morning, the photo-editing app FaceApp released new photo filters that change the ethnic appearance of your face.

The app first became popular earlier in 2017 due to its ability to transform people into elderly versions of them…

Popular face-aging app now offers ‘Black,’ ‘Indian,’ and ‘Asian’ filters
theverge.com · 2017

FaceApp’s CEO says: “The new controversial filters will be removed in the next few hours.”

You might remember FaceApp — a selfie-editing app that transforms users’ pictures by making them look older, younger, or giving them an artificial sm…

FaceApp Removes Black, Asian, and Indian Filters After Backlash They Should Have Expected
complex.com · 2017

Remember FaceApp, the smartphone app that, among other things, allows you to add a filter to your face to make you look younger or older? The one that somehow fascinates every single person you know and encourages them to post photos on the…

FaceApp removes 'Ethnicity Filters' after racism storm
dailymail.co.uk · 2017

A viral app that added Asian, Black, Caucasian and Indian filters to people's selfies has removed them after being accused of racism.

The update which launched yesterday was met with backlash - with many people criticising it for propagatin…

FaceApp briefly let users change their skin color. Bad idea.
slate.com · 2017

FaceApp’s racial filters lived only briefly, but they should never have happened in the first place.

FaceApp

For a brief moment on Wednesday, FaceApp—the app that went viral in April for taking a photo of someone’s face and making them look…

FaceApp forced to pull 'racist' filters that allow 'digital blackface'
theguardian.com · 2017

AI-powered program allowed users to edit selfies to fit into ‘Caucasian, Asian, Indian or Black’ categories causing outrage and immediate U-turn

Popular AI-powered selfie program FaceApp was forced to pull new filters that allowed users to …

FaceApp Launches Digital Blackface Options Because The World Is Filled With Idiots
gizmodo.com.au · 2017

This really isn't that hard, people. Just last year, Snapchat apologised for adding an offensive Bob Marley selfie filter to its stable. Now, FaceApp — the silly photo editor you probably downloaded in March and totally forgot about — has o…

FaceApp's Indian and Black photo filters are quite obviously racist
dailyo.in · 2017

Popular photo filter application, FaceApp, which allows users to take selfies and then add fun filters to them to change their appearance by adding a range of photo filters, faced major backlash after its developers on August 9 rolled out a…

FaceApp releases change-your-race filters
stuff.co.nz · 2017

FaceApp boss Yaroslav Goncharov has defended the new filters. "The ethnicity change filters have been designed to be equal in all aspects."

​FaceApp, an app that uses neural networks to transform your selfies in some rather eerie ways, has …

'Racist' FaceApp photo filters encouraged users to black up
independent.co.uk · 2017

FaceApp has removed a number of racially themed photo filters after being accused of racism.

The app, which uses artificial intelligence to edit pictures, this week launched a number of “ethnicity change filters”.

They claimed to show users…

FaceApp Introduced Blackface And Yellowface Selfie Filters, Then Removed Them A Few Hours Later
buzzfeednews.com · 2017

"New filters: Asian, Black, Caucasian, Indian." WHAT. Update: Faceapp has removed the four new filters.

Remember FaceApp? Back in April, it was a suddenly popular Internet Thing(™) that would make your face look like you were a baby, an old…

FaceApp Is Being Called "Racist" Again for Featuring Problematic Filters
allure.com · 2017

Remember that viral face-filtering app, FaceApp, that was accused of being racist after featuring a filter called "Hot" that actually lightened people's skin](https://www.allure.com/story/faceapp-accused-of-racism-with-skin-whitening-featur…

FaceApp Ethnicity Filters Make You Look Like a Different Race
cosmopolitan.com · 2017

FaceApp

In recent months, FaceApp has been a fun app you’ve probably seen on your social media feeds, letting you and your friends morph faces to look older, younger, or smiling when you frown. You can even see what you’d look like if you p…

Variants

A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.

Similar Incidents

By textual similarity

Did our AI mess up? Flag the unrelated incidents

Images of Black People Labeled as Gorillas

Images of Black People Labeled as Gorillas

Jun 2015 · 24 reports
Biased Google Image Results

Biased Google Image Results

Mar 2016 · 18 reports
TayBot

TayBot

Mar 2016 · 28 reports
Previous IncidentNext Incident

Similar Incidents

By textual similarity

Did our AI mess up? Flag the unrelated incidents

Images of Black People Labeled as Gorillas

Images of Black People Labeled as Gorillas

Jun 2015 · 24 reports
Biased Google Image Results

Biased Google Image Results

Mar 2016 · 18 reports
TayBot

TayBot

Mar 2016 · 28 reports

Research

  • Defining an “AI Incident”
  • Defining an “AI Incident Response”
  • Database Roadmap
  • Related Work
  • Download Complete Database

Project and Community

  • About
  • Contact and Follow
  • Apps and Summaries
  • Editor’s Guide

Incidents

  • All Incidents in List Form
  • Flagged Incidents
  • Submission Queue
  • Classifications View
  • Taxonomies

2023 - AI Incident Database

  • Terms of use
  • Privacy Policy
  • Open twitterOpen githubOpen rssOpen facebookOpen linkedin
  • 30ebe76