AI Model Creates 3D Ultrasound Images Without Bulky Sensors

Pusan National University researchers have developed MoGLo-Net, a deep learning model eliminating the need for external sensors in 3D medical imaging, potentially reducing equipment costs by up to 30 per cent. Published on 13 June 2025 in IEEE Transactions on Medical Imaging, the technology reconstructs three-dimensional images from two-dimensional ultrasound and photoacoustic scans by tracking transducer motion using tissue speckle data. This advancement, demonstrated with both proprietary and public datasets, promises to broaden access to 3D imaging in resource-constrained clinical settings and improve procedural accuracy.

Advancing 3D Medical Imaging

Conventional handheld photoacoustic and ultrasound imaging (PAUS) techniques, while offering procedural flexibility, are limited by their two-dimensional visualisation of target regions. Achieving comprehensive three-dimensional reconstruction typically requires either expensive, bulky equipment or freehand methods that rely on the accurate tracking of the imaging transducer. A recently developed deep learning model, MoGLo-Net, addresses these limitations by enabling three-dimensional reconstruction from two-dimensional PAUS images without the need for external tracking sensors.

The architecture of MoGLo-Net comprises an encoder, based on the ResNet deep learning framework, and a Long-Short Term Memory (LSTM)-based motion estimator. The encoder extracts correlations between sequential images using tissue speckle patterns – naturally occurring variations in ultrasound reflectivity – to capture both in-plane and out-of-plane motion of the transducer. A novel self-attention mechanism within the encoder prioritises local image features based on broader contextual information. The resulting features are then processed by the LSTM estimator, which utilises temporal memory to estimate transducer movement over time accurately. Customised loss functions incorporated into the model further enhance reconstruction accuracy.

Evaluation using both proprietary and publicly available datasets demonstrated MoGLo-Net’s superior performance compared to existing state-of-the-art models. Notably, the research team achieved, for the first time, the reconstruction of three-dimensional images combining both ultrasound and photoacoustic data, specifically visualising blood vessels. This advancement in 3D medical imaging promises to improve the safety and efficacy of diagnostic and interventional procedures.

The elimination of external sensor requirements is particularly significant, potentially broadening access to ultrasound technology in clinical settings lacking specialist resources. This democratisation of 3D medical imaging could have a substantial impact on healthcare provision, particularly in resource-constrained environments.

Overcoming Limitations of Current Techniques

The reliance on external tracking systems in conventional freehand 3D reconstruction represents a substantial practical and economic barrier. These systems often suffer from inaccuracies stemming from calibration errors and patient movement, necessitating frequent recalibration and potentially compromising image quality. Furthermore, the cost of these sensors, combined with the associated infrastructure requirements, limits their accessibility, particularly in smaller clinics or field-based medical facilities. MoGLo-Net circumvents these issues by deriving motion estimates directly from the ultrasound data itself, effectively transforming the inherent information within the images into a means of spatial reconstruction.

The model’s innovative use of tissue speckle – the granular pattern observed in ultrasound images resulting from the scattering of sound waves – is central to its functionality. By analysing subtle changes in this speckle pattern between consecutive image frames, the ResNet encoder can accurately determine the transducer’s displacement in three-dimensional space. This approach, analogous to visual odometry in robotics, allows for continuous tracking without reliance on external references. The incorporation of a self-attention mechanism within the encoder further refines this process by weighting the importance of different image features, ensuring that local motion estimates are consistent with the broader anatomical context.

The successful integration of ultrasound and photoacoustic imaging within the MoGLo-Net framework represents a significant step towards multi-modal 3D medical imaging. Photoacoustic imaging, which leverages the photoelectric effect to generate ultrasound waves from absorbed laser light, provides complementary information regarding tissue oxygenation and vascularity. Combining this data with conventional ultrasound allows for a more comprehensive visualisation of anatomical structures and physiological processes, potentially aiding in the diagnosis of a wider range of pathologies. The demonstration of three-dimensional blood vessel reconstruction, a challenging task due to their small size and complex geometry, underscores the model’s potential in vascular imaging and related applications.

MoGLo-Net Architecture and Functionality

The ResNet-driven encoder within MoGLo-Net employs correlation operations to extract relationships between consecutive images, effectively capturing transducer motion in all three planes. This technique leverages the naturally occurring variations in ultrasound reflectivity – known as tissue speckle – to identify subtle changes indicative of transducer displacement. The incorporation of a novel self-attention mechanism within the encoder further refines this process by highlighting local features based on global image information, thereby ensuring consistency and accuracy in motion estimation.

The LSTM-based motion estimator receives features extracted by the ResNet encoder and utilises long-term memory to estimate transducer motion over time. This is crucial for maintaining accuracy during scans where the transducer may move rapidly or in complex patterns. By considering the entire sequence of images, the LSTM estimator can effectively filter noise and compensate for temporary inaccuracies in the motion estimate. Customised loss functions incorporated into the model are designed to further optimise the accuracy of the reconstruction, penalising errors in both spatial position and image quality.

The successful integration of ultrasound and photoacoustic imaging within the MoGLo-Net framework represents a significant step towards multi-modal 3D medical imaging. Photoacoustic imaging, which leverages the photoelectric effect to generate ultrasound waves from absorbed laser light, provides complementary information regarding tissue oxygenation and vascularity. Combining this data with conventional ultrasound allows for a more comprehensive visualisation of anatomical structures and physiological processes, potentially aiding in the diagnosis of a wider range of pathologies. The demonstration of three-dimensional blood vessel reconstruction, a challenging task due to their small size and complex geometry, underscores the model’s potential in vascular imaging and related applications.

Performance and Validation of the Model

Evaluation employed both proprietary and publicly available datasets, consistently demonstrating MoGLo-Net’s superior performance relative to existing state-of-the-art models across all assessed metrics. This rigorous testing confirms the model’s capacity to generate more realistic three-dimensional ultrasound images, indicative of improved spatial accuracy and image fidelity.

The model’s architecture, specifically the ResNet-driven encoder, utilises correlation operations to discern relationships between sequential images, thereby capturing transducer motion across all three spatial dimensions. This technique leverages inherent variations in ultrasound reflectivity – tissue speckle – to identify subtle changes indicative of transducer displacement. The integrated self-attention mechanism refines this process by weighting the importance of different image features, ensuring consistency and accuracy in motion estimation.

The LSTM-based motion estimator receives features extracted by the ResNet encoder and employs long-term memory to estimate transducer motion over time. This is crucial for maintaining accuracy during scans involving rapid or complex transducer movements. By considering the entire image sequence, the LSTM estimator effectively filters noise and compensates for temporary inaccuracies in motion estimation. Customised loss functions incorporated into the model further optimise reconstruction accuracy, penalising errors in both spatial positioning and image quality.

The successful integration of ultrasound and photoacoustic imaging within the MoGLo-Net framework represents a significant step towards multi-modal 3D medical imaging. Photoacoustic imaging, which leverages the photoelectric effect to generate ultrasound waves from absorbed laser light, provides complementary information regarding tissue oxygenation and vascularity. Combining this data with conventional ultrasound allows for a more comprehensive visualisation of anatomical structures and physiological processes, potentially aiding in the diagnosis of a wider range of pathologies. The demonstration of three-dimensional blood vessel reconstruction, a challenging task due to their small size and complex geometry, underscores the model’s potential in vascular imaging and related applications.

Clinical Potential and Research Team

The research underpinning MoGLo-Net is led by Associate Professor MinWoo Kim of Pusan National University, Korea, whose expertise lies in the development of artificial intelligence techniques for medical instrumentation and data analysis. Professor Kim’s prior experience includes postdoctoral training at the University of Washington’s Matt O’Donnells Lab and a PhD in Electrical and Computer Engineering from the University of Illinois at Urbana-Champaign (UIUC), completed in 2018. His research group, the AI Medical Imaging Lab, actively pursues advancements in AI-powered ultrasound and photoacoustic imaging, as evidenced by their publicly available resources and publications (https://pnu-amilab.github.io/index.html; ORCID id: https://orcid.org/0000-0001-7547-2596).

The potential clinical impact of MoGLo-Net extends beyond improved diagnostic imaging. The model’s capacity to generate accurate three-dimensional reconstructions without reliance on external sensors could significantly enhance the efficacy of image-guided interventions, such as biopsies and minimally invasive surgery. By providing surgeons and interventional radiologists with a more comprehensive understanding of anatomical structures, the technology promises to improve procedural accuracy and reduce the risk of complications. Furthermore, the model’s accessibility – stemming from the elimination of expensive hardware requirements – could democratise access to advanced imaging capabilities, particularly in resource-constrained healthcare settings.

The research team’s successful combination of ultrasound and photoacoustic data to reconstruct three-dimensional images of blood vessels represents a noteworthy advancement in multi-modal imaging. Photoacoustic imaging provides functional information regarding tissue oxygenation and vascularity, complementing the anatomical detail provided by ultrasound. This synergistic approach could prove particularly valuable in applications such as cancer detection, cardiovascular disease assessment, and monitoring of tissue viability following therapeutic interventions. The ability to visualise blood vessels in three dimensions, without the need for invasive angiography, offers a less risky and more accessible alternative for evaluating vascular health.