Incident 290: False Negatives for Water Quality-Associated Beach Closures

Description: Toronto’s use of AI predictive modeling (AIPM) which had replaced existing methodology as the only determiner of beach water quality raised concerns about its accuracy, after allegedly conflicting results were found by a local water advocacy group using traditional means.

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscoverView HistoryView History

Incident Stats

Incident ID
290
Report Count
3
Incident Date
2022-06-03
Editors
Khoa Lam
Applied Taxonomies
CSETv1, GMF, MIT

CSETv1 Taxonomy Classifications

Taxonomy Details

Incident Number

290

MIT Taxonomy Classifications

Machine-Classified
Taxonomy Details

Risk Subdomain

7.3. Lack of capability or robustness

Risk Domain

  1. AI system safety, failures, and limitations

Entity

AI

Timing

Post-deployment

Intent

Unintentional

Safe for swimming? Toronto’s new tool for measuring water quality at its beaches is misleading, say advocates
thestar.com · 2022

A safe water advocacy group is concerned for the health of Toronto beachgoers after the city’s new water quality monitoring system appears to have repeatedly allowed contaminated beaches to remain open.

This summer, the city quietly adopted…

Toronto Tapped Artificial Intelligence to Warn Swimmers. The Experiment Failed
theinformation.com · 2022

Earlier this year, Toronto's public health department quietly flipped the switch on an experiment targeting the city's most pollution-prone beaches.

Instead of relying on day-old laboratory tests to ensure that people don't swim in unsafe w…

The bait and switch behind AI risk prediction tools
aisnakeoil.substack.com · 2022

Toronto recently used an AI tool to predict when a public beach will be safe. It went horribly awry. 

The developer claimed the tool achieved over 90% accuracy in predicting when beaches would be safe to swim in. But the tool did much worse…

Variants

A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.
Previous IncidentNext Incident