News
Study: Most FDA-cleared AI medical devices were never tested on patient outcomes
A report published August 21 in Healio found that the majority of the more than 1,000 AI/ML medical devices cleared by the FDA were evaluated on technical performance — sensitivity, specificity, AUC — rather than on whether patients who received AI-assisted care had better outcomes than those who did not. The finding is consistent with earlier analyses but gains urgency as cleared devices reach broader deployment.
- FDA
- clinical-validation
- evidence
- AI-devices
- patient-outcomes
- 510k
This is not a new finding — earlier analyses reaching the same conclusion appeared in JAMA and The BMJ — but the timing matters. The FDA’s device list crossed 1,000 AI/ML authorizations earlier this year, and that milestone triggered a fresh wave of coverage framing the number as unambiguous progress. This study is the corrective: a large number of cleared devices is not the same as a large number of devices with proven clinical benefit.
The distinction matters practically for procurement and governance. A device cleared through 510(k) on a substantial-equivalence predicate does not have to demonstrate that patients do better; it has to demonstrate that the device performs comparably to a predicate. For many imaging AI tools, the predicate is itself another imaging AI tool, meaning the evidence chain can be multiple generations removed from any randomized clinical outcome data.
Health system AI governance committees that treat FDA clearance as the end of the evidence review are doing inadequate due diligence. The appropriate follow-up questions are: was the device cleared on outcomes data or technical-performance data? Is there any post-market evidence of clinical-outcome benefit? Is the health system generating internal data on outcomes it could share?
The FDA is aware of this gap. The agency’s evolving guidance on real-world performance monitoring for AI/ML-based Software as a Medical Device (SaMD) reflects an attempt to move the industry toward continuous post-market evidence generation. But guidance is not yet a mandate, and most deployed AI devices are not generating structured outcome data that would feed back into evidence.
Primary source: Read the full original on Healio ↗