Skip to Content
logologo
AI Incident Database
Open TwitterOpen RSS FeedOpen FacebookOpen LinkedInOpen GitHub
Open Menu
Discover
Submit
  • Welcome to the AIID
  • Discover Incidents
  • Spatial View
  • Table View
  • List view
  • Entities
  • Taxonomies
  • Submit Incident Reports
  • Submission Leaderboard
  • Blog
  • AI News Digest
  • Risk Checklists
  • Random Incident
  • Sign Up
Collapse
Discover
Submit
  • Welcome to the AIID
  • Discover Incidents
  • Spatial View
  • Table View
  • List view
  • Entities
  • Taxonomies
  • Submit Incident Reports
  • Submission Leaderboard
  • Blog
  • AI News Digest
  • Risk Checklists
  • Random Incident
  • Sign Up
Collapse

Incident 189: Opaque Fraud Detection Algorithm by the UK’s Department of Work and Pensions Allegedly Discriminated against People with Disabilities

Description: People with disabilities were allegedly disproportionately targeted by a benefit fraud detection algorithm which the UK’s Department of Work and Pensions was urged to disclose.

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscoverView HistoryView History

Entities

View all entities
Alleged: UiPath developed an AI system deployed by UK Department of Work and Pensions, which harmed people with disabilities.

Incident Stats

Incident ID
189
Report Count
6
Incident Date
2019-10-15
Editors
Sean McGregor, Khoa Lam
Applied Taxonomies
GMF, MIT

MIT Taxonomy Classifications

Machine-Classified
Taxonomy Details

Risk Subdomain

A further 23 subdomains create an accessible and understandable classification of hazards and harms associated with AI
 

1.1. Unfair discrimination and misrepresentation

Risk Domain

The Domain Taxonomy of AI Risks classifies risks into seven AI risk domains: (1) Discrimination & toxicity, (2) Privacy & security, (3) Misinformation, (4) Malicious actors & misuse, (5) Human-computer interaction, (6) Socioeconomic & environmental harms, and (7) AI system safety, failures & limitations.
 
  1. Discrimination and Toxicity

Entity

Which, if any, entity is presented as the main cause of the risk
 

Human

Timing

The stage in the AI lifecycle at which the risk is presented as occurring
 

Post-deployment

Intent

Whether the risk is presented as occurring as an expected or unexpected outcome from pursuing a goal
 

Intentional

Incident Reports

Reports Timeline

Incident OccurrenceHow the Government Spies on Welfare Claimants+2
DWP urged to reveal algorithm that ‘targets’ disabled for benefit fraud
DWP faces legal action to reveal 'algorithms' that flag claims of benefit fraudLegal letter asks DWP for information on ‘discriminatory’ secret algorithm
How the Government Spies on Welfare Claimants

How the Government Spies on Welfare Claimants

vice.com

DWP urged to reveal algorithm that ‘targets’ disabled for benefit fraud

DWP urged to reveal algorithm that ‘targets’ disabled for benefit fraud

amp-theguardian-com.cdn.ampproject.org

DWP bosses quizzed by MPs over secret benefit fraud algorithm

DWP bosses quizzed by MPs over secret benefit fraud algorithm

disabilitynewsservice.com

Disabled people ‘forced on gruelling and invasive hamster wheel’ of benefits appeals

Disabled people ‘forced on gruelling and invasive hamster wheel’ of benefits appeals

theoldhamtimes.co.uk

DWP faces legal action to reveal 'algorithms' that flag claims of benefit fraud

DWP faces legal action to reveal 'algorithms' that flag claims of benefit fraud

mirror.co.uk

Legal letter asks DWP for information on ‘discriminatory’ secret algorithm

Legal letter asks DWP for information on ‘discriminatory’ secret algorithm

disabilitynewsservice.com

How the Government Spies on Welfare Claimants
vice.com · 2021

The UK government uses invasive data gathering techniques to try and prove that people claiming benefits are actually living lavish lifestyles.

Social media monitoring, covert physical surveillance, and data gathering from airlines, PayPal …

DWP urged to reveal algorithm that ‘targets’ disabled for benefit fraud
amp-theguardian-com.cdn.ampproject.org · 2021

Manchester group launches action after people with disabilities report high number of stressful checks for potential scams

Disabled people are being subjected to stressful checks and months of frustrating bureaucracy after being identified …

DWP bosses quizzed by MPs over secret benefit fraud algorithm
disabilitynewsservice.com · 2021

The Department for Work and Pensions (DWP) has been questioned by MPs over its use of a secret set of instructions that are used by its computers to identify potential benefit fraudsters, and which appear to disproportionately target disabl…

Disabled people ‘forced on gruelling and invasive hamster wheel’ of benefits appeals
theoldhamtimes.co.uk · 2021

The government is ‘forcing disabled people into gruelling and invasive benefit fraud investigations', a disability group has claimed.

A Manchester disabled people’s group has issued the government’s Department for Work and Pensions with a l…

DWP faces legal action to reveal 'algorithms' that flag claims of benefit fraud
mirror.co.uk · 2022

Campaigners say disabled people are being unfairly targeted and living in 'fear of the brown envelope' thanks to mysterious computer processes that the government has never fully explained.

The Department for Work and Pensions is facing fre…

Legal letter asks DWP for information on ‘discriminatory’ secret algorithm
disabilitynewsservice.com · 2022

Disabled campaigners have sent a legal letter to the Department for Work and Pensions (DWP) to question how its use of a secret algorithm could be discriminating against disabled people in the way it selects claimants to face benefit fraud …

Variants

A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.
Previous IncidentNext Incident

Research

  • Defining an “AI Incident”
  • Defining an “AI Incident Response”
  • Database Roadmap
  • Related Work
  • Download Complete Database

Project and Community

  • About
  • Contact and Follow
  • Apps and Summaries
  • Editor’s Guide

Incidents

  • All Incidents in List Form
  • Flagged Incidents
  • Submission Queue
  • Classifications View
  • Taxonomies

2023 - AI Incident Database

  • Terms of use
  • Privacy Policy
  • Open twitterOpen githubOpen rssOpen facebookOpen linkedin
  • 30ebe76