Neuromorphic Hardware Enables Low-Power, Event-Based Pupil Tracking with Microsecond Resolution

Eye tracking underpins a growing range of applications, but creating systems that are both accurate and exceptionally energy efficient remains a significant hurdle for truly wearable technology. Federico Paredes-Valles, Yoshitaka Miyatani from Sony Semiconductor Solutions Corporation, and Kirk Y. W. Scheper, alongside their colleagues, now present the first fully integrated, battery-powered pupil-tracking system designed for wearable platforms. This achievement combines event-based vision sensors, which capture changes with remarkable speed, with a custom-designed, low-power processing solution. The team’s innovative system employs a novel spiking neural network that efficiently decodes eye movements, and validation on a new dataset demonstrates robust binocular tracking at 100Hz with an average power consumption of less than 5mW per eye, paving the way for practical, always-on eye tracking in next-generation wearable devices.

Speck2F Neuromorphic System: Configuration and Control

This document serves as a comprehensive guide for configuring, programming, and operating a neuromorphic system built around the SynSense Speck2f chip, intended for researchers and engineers deeply involved in neuromorphic computing hardware and software. It details the low-level Serial Peripheral Interface (SPI) communication protocols, memory organization, and configuration procedures necessary to fully utilize the chip’s capabilities, explaining how configuration data generated by SynSense’s Samna API is compressed and reformatted for efficient transmission and storage. A key feature is a method for sequentially accessing spike counts from all 16 readout neurons, crucial for monitoring the activity of the neural network. The document meticulously outlines the SPI communication protocol, detailing command formatting and data transfer procedures, and reveals the chip’s memory organization and how kernel weights are stored. Understanding SPI, memory mapping, and register configuration are crucial for successful operation, as data is stored in little-endian format. In conclusion, this document is a highly technical and detailed guide for working with the SynSense Speck2f chip, providing the necessary information for researchers and engineers to develop, integrate, and optimize neuromorphic systems based on this chip.

Binocular Pupil Tracking with Neuromorphic Vision

Scientists engineered a fully integrated, battery-powered system for wearable pupil-center tracking, combining event-based sensing and neuromorphic processing on commercially available Speck2f system-on-chips with a low-power microcontroller. This system pioneered a novel uncertainty-quantifying spiking neural network with gated temporal decoding, specifically optimized for the strict memory and bandwidth constraints of the Speck2f, enabling real-time inference with minimal power consumption. The core of the system utilizes the SynSense Speck2f, a digital system-on-chip integrating a 128×128 event-based vision sensor with nine asynchronous convolutional spiking neural network cores, facilitating efficient, low-power, event-driven sensing and processing. Researchers deployed the system on dual Speck2f SoCs, achieving robust binocular pupil tracking at 100Hz with an average power consumption below 5mW per eye. To validate the system, researchers created a new multi-user dataset and constructed a wearable prototype, demonstrating a significant advancement over prior work by achieving fully integrated, real-time, low-power event-based eye tracking on neuromorphic hardware.

Low-Power Wearable Binocular Eye Tracking Demonstrated

Scientists have developed a fully integrated, wearable system for tracking pupil center with exceptionally low power consumption, achieving robust binocular eye tracking at 100Hz while consuming less than 5mW per eye. This breakthrough combines event-based vision sensing with on-chip processing, demonstrating a practical solution for always-on eye tracking in wearable technology. Experiments using a custom-built prototype and a new multi-user dataset reveal high tracking accuracy and energy efficiency, closely matching performance achieved with more powerful GPU-based systems. Models incorporating uncertainty estimation demonstrated up to 24% error reduction, highlighting the importance of quantifying prediction confidence.

Detailed analysis of on-chip performance shows that gated decoding configurations outperform direct decoding, reducing errors by nearly 50%, underscoring the value of temporal reasoning for accurate eye tracking. Measurements confirm that the system maintains an average power consumption of approximately 0. 13mW for the microcontroller, contributing to the overall low-power design. These findings pave the way for next-generation energy-efficient wearable systems capable of continuous, real-time eye tracking.

Neuromorphic Eye Tracking System Achieves Low Power

This work demonstrates a fully integrated, battery-powered system for continuous eye tracking, bridging the gap between neuromorphic hardware capabilities and practical deployment requirements. Researchers have successfully combined event-based vision sensing with on-chip processing, achieving robust pupil-center tracking at 100Hz with an average power consumption of less than 5mW per eye. This advancement relies on a novel spiking neural network, optimized for limited memory and bandwidth, and a gated decoding mechanism that enhances performance under challenging conditions. The resulting system demonstrates the potential of neuromorphic processors to deliver practical solutions for always-on wearable applications.