AI gone astray: How subtle shifts in patient data send popular algorithms reeling, undermining patient safety

A novel investigation by STAT and the Massachusetts Institute of Technology found that subtle shifts in data fed into popular health care algorithms — used to warn caregivers of impending medical crises — can cause their accuracy to plummet over time, raising the prospect AI could do more harm than good in many hospitals.

In a monthslong experiment, STAT and MIT traced the performance of algorithms past their early days of peak performance into the grinding years that follow, when the hype has faded and they must prove their reliability to caregivers. Instead of transforming care, the algorithms withered in the face of fast-moving clinical conditions — unable to keep up with the pace of change.

Their frailty exposes gaping holes in the governance of products whose quiet deterioration in hospitals around the country threatens to mislead doctors and undermine patient safety. The initial signs of dysfunction are often faint, making it difficult to root out faulty information before it bleeds into decision-making.

Get unlimited access to award-winning journalism and exclusive events.

Subscribe

Log In