Clinician-AI combination is best in diagnosing diabetic retinopathy

Assisted by a deep-learning algorithm, physicians were able to more accurately diagnose diabetic retinopathy—a potentially blinding eye disease—than either a clinician or algorithm alone.

That’s the finding of a new study from the Google AI research group, which will be published in the April edition of Ophthalmology, the journal of the American Academy of Ophthalmology.

In the study, 10 ophthalmologists—five general ophthalmologists, four retina specialists and one retina fellow—read images for diabetic retinopathy severity based on a severity scale in each of three conditions—unassisted, grades only and grades plus heatmap. In total, the study’s participants utilized 1,796 retinal fundus images from 1,612 diabetic patients.

“We found a trend toward higher accuracy and confidence, but also higher grading times, with model assistance,” state the study’s authors. “We demonstrated that assistance can help even well-trained specialists, increasing sensitivity without decreasing specificity.”

Also See: FDA clears AI-based system to detect diabetic retinopathy

In fact, the study showed that general ophthalmologists who did not receive AI assistance were significantly less accurate than the algorithm, while retina specialists were not significantly more accurate than the algorithm. Further, with assistance, general ophthalmologists matched but do not exceed the accuracy of the model, while retina specialists started to exceed the model’s performance.

“What we found is that AI can do more than simply automate eye screening, it can assist physicians in more accurately diagnosing diabetic retinopathy,” says Rory Sayres, the study’s lead researcher and quantitative user experience researcher at Google. “AI and physicians working together can be more accurate than either alone.”

“There’s an analogy in driving,” adds Sayres. “There are self-driving vehicles, and there are tools to help drivers, like Android Auto. The first is automation, the second is augmentation. The findings of our study indicate that there may be space for augmentation in classifying medical images like retinal fundus images. When the combination of clinician and assistant outperforms either alone, this provides an argument for up-leveling clinicians with intelligent tools.”

While the study’s authors acknowledged that their results are preliminary, they contend that their research “has implications for how model assistance may impact the diagnosis of diabetic retinopathy and other eye conditions in clinical settings,” adding that “the use of an assisted tool such as this could increase the accuracy of the read at all levels of severity, including in treated populations.”

The study builds on previous work by Google AI on the development and validation of a deep learning algorithm for the detection of diabetic retinopathy in retinal fundus photographs, as well as a paper on improving their model by moving toward a more granular five-point grading scale (vs. the previous two-class system) and incorporating adjudication by a panel of retinal specialists.