Vision-Based Neuronavigation Achieves Real-Time TMS Tracking with Multiple Cameras

Researchers are tackling the challenges of precise, non-invasive brain stimulation with a new vision-based neuronavigation system. Xuyi Hu, Ke Ma, and Siwei Liu, from the Department of Engineering at the University of Cambridge, alongside Per Ola Kristensson and Stephan Goetz, have developed a framework utilising multi-camera optical tracking and augmented reality. This innovation moves beyond costly and often inaccurate traditional tracking methods, offering real-time tracking of both patient and transcranial magnetic stimulation (TMS) equipment. By integrating consumer-grade cameras with a dynamic 3D brain model and AR projection, the system visualises neural targets directly onto the patient’s head, promising improved spatial precision, accuracy, and a more intuitive experience for clinicians guiding TMS procedures.

This breakthrough addresses limitations found in traditional neuronavigation, which often relies on expensive, cumbersome, and error-prone tracking technologies. The research team achieved real-time tracking of both the patient’s head and TMS instrumentation using a self-coordinating system integrating multiple consumer-grade cameras and visible tags. This innovative approach feeds data into a dynamic 3D brain model constructed in Unity, allowing clinicians to intuitively visualise neural targets and the current position of the stimulation coil.

The core of this work lies in the creation of a digital twin that accurately reflects the patient’s anatomy and the TMS coil’s location, updating in real-time to represent the estimated stimulation point. To seamlessly integrate this digital information with the physical world, the researchers incorporated an Augmented reality (AR) module. This module projects the 3D brain model directly onto the patient’s head using AR headsets or mobile devices, enabling clinicians to interactively view and adjust the placement of the stimulation transducer. This intuitive AR guidance replaces the need for abstract numbers and 6D crosshairs displayed on external screens, promising a more natural and efficient workflow.
Experiments demonstrate that the proposed technique provides both improved spatial precision and accuracy in TMS delivery. The system’s ability to track movement and update the brain model in real-time is crucial for maintaining precise targeting during stimulation. A case study involving ten participants with medical backgrounds confirmed the system’s high usability, suggesting it can be readily adopted into clinical practice. This advancement is particularly important given that accurate targeting is essential for the success of TMS, a non-invasive technique used to modulate neural circuits for therapeutic and research purposes.

The research establishes a new paradigm for neuronavigation, moving away from complex and costly systems towards accessible and user-friendly solutions. By leveraging readily available consumer technology, the team has created a system that can potentially broaden access to precise TMS treatment and research. This work opens avenues for improved cortical mapping, pre-surgical planning, and the treatment of neurological and psychiatric disorders such as depression, obsessive-compulsive disorder, and chronic pain, ultimately enhancing the efficacy and reproducibility of non-invasive brain stimulation.

Low-cost vision tracking for TMS neuronavigation is becoming

Scientists developed a vision-based neuronavigation system to address limitations inherent in traditional transcranial magnetic stimulation (TMS) guidance methods. The research team engineered a low-cost system utilising multiple consumer-grade cameras and visible tags to achieve real-time tracking of both the patient’s head and TMS instrumentation with an accuracy of within 5mm. This innovative approach circumvents the need for expensive and often error-prone tracking systems typically employed in neuronavigation, reducing the total system cost to approximately £60. The study pioneered the integration of a self-coordinating tracking system with a dynamic 3D brain model constructed in Unity, enabling real-time updates reflecting the current stimulation coil position and estimated stimulation point.

Researchers harnessed computer vision-based tracking to precisely localise the TMS coil and patient head in space, providing a practical alternative to optical and electromagnetic tracking systems. This method achieves improved spatial precision and accuracy compared to conventional techniques, which often suffer from limitations such as obstructed lines of sight or susceptibility to metallic interference. To bridge the gap between digital visualisation and the clinical environment, the team incorporated an augmented reality (AR) module. This module projects the 3D brain model directly onto the patient’s head using AR headsets or mobile devices, allowing clinicians to interactively view and adjust coil placement without relying on abstract numerical data or 6D crosshairs on an external screen.

The brain model was modified to support both side-by-side display and spatially registered in-situ overlay, enhancing intuitive understanding of neural targets. A case study involving ten participants with medical backgrounds demonstrated high usability of the system, validating its potential for clinical implementation. The work introduces two novel evaluation methods to comprehensively demonstrate the system’s practicality and competitiveness against existing high-cost neuronavigation solutions, achieving an accuracy-cost ratio significantly lower than commercial systems, exceeding £20000 per mm−1. This research offers a lightweight, operator-guided solution that eliminates the need for complex robotic infrastructure, making precise TMS guidance more accessible and affordable for resource-limited clinics and mobile applications.

AR-guided TMS with consumer-grade vision tracking offers a

Scientists have developed a vision-based neuronavigation system offering a cost-effective alternative to traditional methods used in transcranial magnetic stimulation (TMS). The system employs consumer-grade cameras and visible tags to track both the patient and TMS instrumentation in real time, feeding data into a dynamic 3D brain model displayed within a digital twin. This allows for precise coil placement and visualisation of stimulation targets, improving spatial accuracy and potentially enhancing stimulation efficacy. The research integrates an augmented reality (AR) module, projecting the brain model onto the patient’s head, enabling clinicians to intuitively view and adjust transducer placement rather than relying on abstract numerical guidance.

A case study involving ten participants with medical backgrounds demonstrated high usability of the system. The authors acknowledge limitations regarding the scope of usability testing, suggesting further studies with a broader range of end-users are needed to fully assess the system’s effectiveness and impact on clinical workflows. Future research will focus on in-depth clinical trials to validate the findings and explore the system’s potential in various therapeutic applications. This work establishes the viability of optical tags for neuronavigation, particularly in settings with limited resources, and provides a valuable tool for the AR-based medical and neuronavigation research community. By balancing accuracy with affordability and ease of use, the system offers a simpler setup compared to existing approaches, potentially streamlining clinical procedures and improving patient outcomes. The findings suggest that this technology could facilitate more efficient and accessible TMS treatments.