FusionFM Systematically Evaluates and Fuses Foundation Models for Ophthalmic Image Analysis

Recent advances in artificial intelligence offer exciting possibilities for improving the diagnosis of eye diseases and even detecting systemic health problems from retinal scans. Ke Zou, Somfai, and colleagues from institutions including the Spross Research Institute and the National University of Singapore, now present a systematic evaluation of several cutting-edge ‘foundational models’ designed for ophthalmic image analysis. This research addresses a critical question, which model performs best, and can combining these models improve diagnostic accuracy? The team introduces FusionFM, a comprehensive framework for benchmarking four state-of-the-art models, and demonstrates that combining them through a ‘gating’ approach offers modest improvements in predicting conditions like glaucoma and hypertension, paving the way for more reliable and broadly applicable diagnostic tools in eye care.

Retinal Disease Diagnosis with Foundation Model Fusion

This document summarizes research evaluating and comparing foundation models (FMs) in ophthalmology, with a focus on improving detection of both eye diseases and systemic conditions. The study systematically investigates single and fused ophthalmic FMs to optimize performance in detecting retinal diseases and conditions affecting the wider body, based on retinal imaging. Researchers explored methods for combining the strengths of multiple FMs, including approaches that mimic expert systems and intelligently route information. Models were tested on diverse datasets, assessing their ability to detect diseases like glaucoma and diabetic retinopathy, and to predict systemic conditions.

Key findings reveal that DINORET and RETZERO consistently outperformed other models in both ocular and systemic disease prediction, with RETZERO demonstrating better generalization to external datasets, highlighting the importance of diverse training data. A Gating-based fusion strategy proved most beneficial for tasks like glaucoma detection, age-related macular degeneration detection, and predicting hypertension, although predicting systemic diseases, particularly hypertension from retinal images, remains a difficult problem. The research demonstrates the potential of foundation models in ophthalmology, and advocates for broader clinical deployment of these models to maximize their real-world impact.

FusionFM Evaluates Ophthalmology Foundation Model Performance

The research team developed FusionFM, a comprehensive evaluation suite designed to rigorously assess and compare the performance of foundation models, large, pre-trained artificial intelligence systems, in ophthalmology. Recognizing the increasing potential of these models, the researchers aimed to determine which performs best, whether performance varies across different tasks, and if combining multiple models could yield even better results. This approach addresses a critical gap in understanding how to effectively leverage these powerful tools for diagnosing and predicting eye and systemic diseases. FusionFM distinguishes itself through a carefully curated collection of datasets, combining publicly available resources with newly constructed private datasets, ensuring a robust and diverse evaluation encompassing various populations and imaging devices.

The suite focuses on detecting common eye diseases, glaucoma, diabetic retinopathy, and age-related macular degeneration, and predicting systemic conditions like diabetes and hypertension, leveraging the wealth of information contained within retinal images. A key innovation lies in the “frozen backbone” strategy employed throughout the evaluation, where the core, pre-trained components of each foundation model remain unchanged during testing, with only a small, task-specific “classification head” being trained. This ensures a fair comparison of the underlying representational quality of each model. Furthermore, the team explored two distinct fusion strategies to combine the strengths of multiple foundation models, representing novel approaches to integrating information from different sources. The comprehensive nature of FusionFM and its innovative methodology provide a valuable framework for advancing ophthalmic AI and improving patient care.

Foundation Model Benchmarking for Retinal Imaging

Recent advances in artificial intelligence have yielded foundation models, versatile AI systems trained on vast datasets, with promising applications in medical image analysis. Researchers have begun applying these models to ophthalmology, aiming to improve diagnosis and prediction of both eye diseases and systemic conditions detectable through retinal imaging. A new benchmarking suite, FusionFM, systematically evaluates several leading ophthalmic foundation models, providing crucial insights into their capabilities and limitations. The study assessed four state-of-the-art models, RETFound, VisionFM, RetiZero, and DINORET, across a range of tasks, including detection of glaucoma, diabetic retinopathy, and age-related macular degeneration, as well as prediction of systemic diseases like diabetes and hypertension.

Performance was evaluated using standardized datasets collected from multiple countries, ensuring a robust and comparative analysis. Results demonstrate that DINORET and RetiZero consistently achieve superior performance across both ophthalmic and systemic disease tasks, suggesting they represent the current leading edge in this field. Notably, RetiZero exhibited stronger generalization capabilities when tested on datasets not used during its initial training, indicating a greater ability to adapt to new clinical settings. Researchers also investigated whether combining these models could further enhance performance.

A fusion strategy termed “Gating-based” yielded modest improvements in predicting glaucoma, age-related macular degeneration, and hypertension, suggesting that combining models can offer incremental gains. Despite these advances, accurately predicting systemic diseases, particularly hypertension in independent patient groups, remains a significant challenge, highlighting the need for further research focused on improving the ability of these models to detect subtle indicators of systemic health within retinal images. FusionFM provides a valuable, evidence-based evaluation of ophthalmic foundation models, demonstrating the benefits of model fusion and identifying key areas for future development.

Foundational Models Excel in Ophthalmic Disease Prediction

This study presents a systematic evaluation of foundational models (FMs) applied to ophthalmic image analysis, and explores the benefits of combining these models. Researchers benchmarked four state-of-the-art FMs, RETFound, VisionFM, RetiZero, and DINORET, across tasks including the detection of glaucoma, diabetic retinopathy, and age-related macular degeneration, as well as the prediction of systemic diseases like diabetes and hypertension. Results indicate that DINORET and RetiZero consistently achieve superior performance in both ophthalmic and systemic disease tasks, with RetiZero demonstrating stronger generalisation when tested on external datasets. The team also investigated fusion strategies, finding that a Gating-based approach offers modest improvements in predicting glaucoma and hypertension. While model fusion shows promise, predicting systemic diseases, particularly hypertension in new patient groups, remains a significant challenge.