Dual site external validation of artificial intelligence-enabled treatment monitoring for neovascular age-related macular degeneration in England

Background
Monitoring neovascular age-related macular degeneration (nAMD) is a significant contributor to ophthalmology demands in the NHS, with clinical capacity struggling to meet the demand. This task depends upon interpreting retinal optical coherence tomography (OCT) imaging, where artificial intelligence (AI) could rebalance clinical demand and capacity. However, evidence of safety and effectiveness in nAMD monitoring is lacking.

Methods
Using a published non-inferiority design protocol, 521 pairs of ipsilateral retinal OCTs from consecutive visits for nAMD treatment were collected from two NHS ophthalmology services. Real-world binary assessments of nAMD disease activity or stability were compared to an independent ophthalmic reading centre reference standard. An AI system capable of retinal OCT segmentation analysed the OCTs, applying thresholds for intraretinal and subretinal fluid to generate binary assessments. The relative negative predictive value (rNPV) of AI versus real-world assessments was calculated.

Results
Real-world assessments of nAMD activity showed a NPV of 81.6% (57.3–81.6%) and a positive predictive value (PPV) of 41.5% (17.8–62.3%). Optimised thresholds for intraretinal fluid increase (>1,000,000 µm³) and subretinal fluid increase (>2,000,000 µm³) for the AI system assessments produced an NPV of 95.3% (85.5–97.9%) and PPV of 57.8% (29.4–76.0%). The rNPV of 1.17 (1.11–1.23) met predefined criteria for clinical and statistical superiority and accompanied an rPPV of 1.39 (1.10–1.76).

Conclusions
This study suggests that the same thresholds for interpreting OCT-based AI analysis could reduce undertreatment and overtreatment in nAMD monitoring at different centres. Interventional research is needed to test the potential of supportive or autonomous AI assessments of nAMD disease activity to improve the quality and efficiency of services.