Skip to Content
logologo
AI Incident Database
Open TwitterOpen RSS FeedOpen FacebookOpen LinkedInOpen GitHub
Open Menu
Discover
Submit
  • Welcome to the AIID
  • Discover Incidents
  • Spatial View
  • Table View
  • List view
  • Entities
  • Taxonomies
  • Submit Incident Reports
  • Submission Leaderboard
  • Blog
  • AI News Digest
  • Risk Checklists
  • Random Incident
  • Sign Up
Collapse
Discover
Submit
  • Welcome to the AIID
  • Discover Incidents
  • Spatial View
  • Table View
  • List view
  • Entities
  • Taxonomies
  • Submit Incident Reports
  • Submission Leaderboard
  • Blog
  • AI News Digest
  • Risk Checklists
  • Random Incident
  • Sign Up
Collapse

Incident 172: NarxCare’s Risk Score Model Allegedly Lacked Validation and Trained on Data with High Risk of Bias

Description: NarxCare's overdose risk algorithm, lacking peer-reviewed validation, uses sensitive data like doctor visits, prescriptions, and possibly genetic information, leading to significant biases against women and Black patients. Factors like sexual abuse and criminal records exacerbate stigmas and disparities, often resulting in unjust denial of necessary pain medication. The newly approved AvertD genetic test shares similar issues, further complicating and potentially harming medical treatment decisions.

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscoverView HistoryView History

Entities

View all entities
Alleged: Appriss developed an AI system deployed by Appriss , NarxCare and AvertD, which harmed American physicians , American pharmacists , American patients of minority groups and American patients.

Incident Stats

Incident ID
172
Report Count
3
Incident Date
2020-07-01
Editors
Sean McGregor, Khoa Lam
Applied Taxonomies
GMF, MIT

MIT Taxonomy Classifications

Machine-Classified
Taxonomy Details

Risk Subdomain

A further 23 subdomains create an accessible and understandable classification of hazards and harms associated with AI
 

7.3. Lack of capability or robustness

Risk Domain

The Domain Taxonomy of AI Risks classifies risks into seven AI risk domains: (1) Discrimination & toxicity, (2) Privacy & security, (3) Misinformation, (4) Malicious actors & misuse, (5) Human-computer interaction, (6) Socioeconomic & environmental harms, and (7) AI system safety, failures & limitations.
 
  1. AI system safety, failures, and limitations

Entity

Which, if any, entity is presented as the main cause of the risk
 

AI

Timing

The stage in the AI lifecycle at which the risk is presented as occurring
 

Post-deployment

Intent

Whether the risk is presented as occurring as an expected or unexpected outcome from pursuing a goal
 

Unintentional

Incident Reports

Reports Timeline

Incident OccurrenceThe Pain Was Unbearable. So Why Did Doctors Turn Her Away?Artificial Intelligence May Influence Whether You Can Get Pain MedicationSay Hello to Your Addiction Risk Score — Courtesy of the Tech Industry
The Pain Was Unbearable. So Why Did Doctors Turn Her Away?

The Pain Was Unbearable. So Why Did Doctors Turn Her Away?

wired.com

Artificial Intelligence May Influence Whether You Can Get Pain Medication

Artificial Intelligence May Influence Whether You Can Get Pain Medication

kffhealthnews.org

Say Hello to Your Addiction Risk Score — Courtesy of the Tech Industry

Say Hello to Your Addiction Risk Score — Courtesy of the Tech Industry

nytimes.com

The Pain Was Unbearable. So Why Did Doctors Turn Her Away?
wired.com · 2021

ONE EVENING IN July of 2020, a woman named Kathryn went to the hospital in excruciating pain.

A 32-year-old psychology grad student in Michigan, Kathryn lived with endometriosis, an agonizing condition that causes uterine-like cells to abno…

Artificial Intelligence May Influence Whether You Can Get Pain Medication
kffhealthnews.org · 2023

Elizabeth Amirault had never heard of a Narx Score. But she said she learned last year the tool had been used to track her medication use.

During an August 2022 visit to a hospital in Fort Wayne, Indiana, Amirault told a nurse practitioner …

Say Hello to Your Addiction Risk Score — Courtesy of the Tech Industry
nytimes.com · 2024

Before Dr. Bobby Mukkamala --- an ear, nose, and throat specialist in Michigan --- prescribed postsurgical opioids recently, he checked state records of his patient's existing controlled substance prescriptions, as legally required. A score…

Variants

A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.

Similar Incidents

By textual similarity

Did our AI mess up? Flag the unrelated incidents

Arkansas's Opaque Algorithm to Allocate Health Care Excessively Cut Down Hours for Beneficiaries

Arkansas's Opaque Algorithm to Allocate Health Care Excessively Cut Down Hours for Beneficiaries

Jan 2016 · 2 reports
Uber AV Killed Pedestrian in Arizona

Uber AV Killed Pedestrian in Arizona

Mar 2018 · 25 reports
AI Tools Failed to Sufficiently Predict COVID Patients, Some Potentially Harmful

AI Tools Failed to Sufficiently Predict COVID Patients, Some Potentially Harmful

Jul 2021 · 1 report
Previous IncidentNext Incident

Similar Incidents

By textual similarity

Did our AI mess up? Flag the unrelated incidents

Arkansas's Opaque Algorithm to Allocate Health Care Excessively Cut Down Hours for Beneficiaries

Arkansas's Opaque Algorithm to Allocate Health Care Excessively Cut Down Hours for Beneficiaries

Jan 2016 · 2 reports
Uber AV Killed Pedestrian in Arizona

Uber AV Killed Pedestrian in Arizona

Mar 2018 · 25 reports
AI Tools Failed to Sufficiently Predict COVID Patients, Some Potentially Harmful

AI Tools Failed to Sufficiently Predict COVID Patients, Some Potentially Harmful

Jul 2021 · 1 report

Research

  • Defining an “AI Incident”
  • Defining an “AI Incident Response”
  • Database Roadmap
  • Related Work
  • Download Complete Database

Project and Community

  • About
  • Contact and Follow
  • Apps and Summaries
  • Editor’s Guide

Incidents

  • All Incidents in List Form
  • Flagged Incidents
  • Submission Queue
  • Classifications View
  • Taxonomies

2023 - AI Incident Database

  • Terms of use
  • Privacy Policy
  • Open twitterOpen githubOpen rssOpen facebookOpen linkedin
  • 30ebe76