Study on FDA-approved AI medical devices

Artificial intelligence (AI) has practically limitless applications in healthcare, ranging from auto-drafting patient messages in MyChart to optimizing organ transplantation and improving tumor removal accuracy. 

Despite their potential benefit to doctors and patients alike, these tools have been met with skepticism because of patient privacy concerns, the possibility of bias, and device accuracy. 

“Although AI device manufacturers boast of the credibility of their technology with FDA authorization, clearance does not mean that the devices have been properly evaluated for clinical effectiveness using real patient data,” said Chouffani El Fassi.

Of the 521 device authorizations, 144 were labelled as “retrospectively validated,” 148 were “prospectively validated,” and 22 were validated using randomized controlled trials. 

Most notably, 226 of 521 FDA-approved medical devices, or approximately 43%, lacked published clinical validation data. 

A few of the devices used “phantom images” or computer-generated images that were not from a real patient, which did not technically meet the requirements for clinical validation.  

Furthermore, the researchers found that the latest draft guidance, published by the FDA in September 2023, does not clearly distinguish between different types of clinical validation studies in its recommendations to manufacturers.

“With these findings, we hope to encourage the FDA and industry to boost the credibility of device authorization by conducting clinical validation studies on these technologies and making the results of such studies publicly available,” said Chouffani El Fassi.