Google’s AI can outperform experts in diagnosing eye disease

Google’s DeepMind AI arm has created a new deep learning framework that diagnoses eye diseases and triages treatment options.

Research suggests that the approach can do just as well as—and, in some cases, better—than eye specialists, and could help patients receive treatment faster and thus avoid the loss of sight.

The AI approach uses studies from ocular coherence tomography (OCT), a three-dimensional volumetric medical imaging technology that measures the reflection of near infrared light. There’s a widespread availability of OCT images in ophthalmology.

However, there aren’t enough experts to interpret the scans in a timely manner. As a result, diagnosis and referral are frequently delayed, increasing the risk that a bleed or other problem isn’t caught quickly. This is particularly problematic because of the increase in sight-threatening diseases, such as diabetes and age-related macular degeneration, for which OCT is the gold standard in initial assessment.

Researchers from DeepMind teamed up with Moorfields Eye Hospital NHS Foundation Trust in England to create a new AI technique to review OCT images and make triage recommendations—urgent, semi urgent, routine and observation only. These referral recommendations correspond to the ones the Moorfield Hospital currently uses.

The team also wanted to avoid the “black box” problem inherent with many AI systems—not knowing why the software made a particular recommendation.

They did so by combining two different neural networks. The first one analyzes the OCT scan to map the tissue and features of disease it sees, such as lesions or hemorrhage. The second network analyzes the map to provide the diagnosis and a referral recommendation. This framework more closely matches the clinical decision-making process, separating judgments about the scan from the referral decision.

To train the network, the researchers assembled 14,884 OCT scan volumes from 7,621 patients who had been referred to the Hospital with symptoms suggestive of macular pathology. They tested the system on a separate dataset. They then compared the network’s performance to that of eight expert clinicians reviewing the same dataset.

The framework’s referral decisions on more than 50 sight threatening eye diseases were 94 percent accurate.

The software matched the two best retina specialists and performed significantly higher than the other two retina specialists and all four optometrists when the clinicians used only OCT scans to diagnose the patients. The experts fared better when they also had access to scans of the patients’ fundus (the back of the eye), but the AI was still as good as the five best experts and continued to outperform the other three.

The AI also made no clinically serious wrong decisions, such as referring a patient who needed an urgent referral to observation only.

Notably, the technology is device-independent and can be applied to different types of eye scanners.

“This might seem inconsequential, but it means that the technology could be applied across the world with relative ease, massively increasing the number of patients who could potentially benefit. This also ensures the system can still be used in hospitals and other clinical settings even as OCR scanners are upgraded or replaced over time,” says Mustafa Suleyman, DeepMind co-founder and head of applied AI in a related blog post.

The next step is to test the framework in a clinical trial, according to the study authors. The study appears in Nature Medicine.

Resolving the black box problem was also “critically important,” says Suleyman.

“[E]yecare professionals are always going to play a key role in deciding the type of care and treatment a patient receives. Enabling them to scrutinise the technology’s recommendations is key to making the system usable in practice,” he says.