Hybrid Neural Networks Achieve Fast Obstacle Detection and Precise Localization

Researchers are addressing the challenge of reliable robotic navigation in complex environments with a vision-based obstacle detection and localization framework. Reza Ahmadvand, Sarah Safura Sharif, and Yaser Mike Banad, from the School of Electrical and Computer Engineering at the University of Oklahoma, present a system that integrates Hybrid Neural Networks (HNNs) with Spiking Neural Networks (SNNs) to improve a robot’s ability to perceive and respond to unforeseen obstacles. This innovative approach combines the precision of artificial neural networks in processing static spatial features with the speed and energy efficiency of SNNs for dynamic, event-based data, avoiding cumbersome domain conversions and enabling real-time performance. By separating low-frequency spatial analysis to ANNs and high-frequency, event-driven temporal analysis to SNNs in a dual-pathway design, the system enhances responsiveness to unmodeled obstacles while reducing computational overhead. A key component is the implementation of an SNN governed by leaky integrate-and-fire dynamics, where the membrane potential of each neuron evolves according to τduidt=−g(ui(t))+∑jwijsj(t)\tau \frac{du_i}{dt} = -g(u_i(t)) + \sum_j w_{ij} s_j(t)τdtdui​​=−g(ui​(t))+∑j​wij​sj​(t), and spikes are generated when the potential exceeds a threshold, Si(t)=H(ui−vth)S_i(t) = H(u_i – v_\text{th})Si​(t)=H(ui​−vth​). This enables high-precision encoding of frame-level object characteristics and robust temporal tracking.

To further improve reliability, the architecture employs a pre-developed SNN-based filter (SNN-EMSIF) for robust state estimation, which allows accurate localization and velocity estimation of obstacles while maintaining low-latency response. The framework is capable of operating without online learning and demonstrates resilience to errors, making it suitable for deployment in unpredictable, real-world environments. Simulation results confirm the system’s effectiveness, showing comparable accuracy to conventional methods but with significantly lower computational cost. By leveraging the complementary strengths of HNNs and SNNs, along with the dual-pathway design and robust estimation filter, this approach represents a substantial advance in autonomous robotic navigation and real-time obstacle avoidance.

ANN-SNN Hybrid Enables Dynamic Obstacle Navigation in complex

The research introduces a system capable of detecting and localizing unmodeled obstacles with improved efficiency and accuracy in dynamic environments. This innovative approach bypasses traditional domain conversion mechanisms by directly utilizing a pre-developed SNN-based filter that processes spike-encoded inputs for localization and state estimation. The team measured performance through simulation, demonstrating acceptable detection accuracy while maintaining computational efficiency comparable to SNN-only implementations, which operate at a significantly reduced resource cost. Data shows the system effectively validates detected anomalies using contextual information from the ANN pathway, ensuring robust tracking of obstacles to support anticipatory navigation strategies.

Specifically, the SNN-EMSIF filter, integrated into the framework, embeds system dynamics directly into the weight matrices of Leaky Integrate-and-Fire (LIF) neurons, eliminating the need for online learning and enhancing resilience to neuron silencing. Results demonstrate the framework’s ability to process sensor data through parallel pathways: high-resolution camera images flow through the ANN path, generating feature maps of the static environment, while a dynamic vision camera (DVC) feeds the SNN path, producing real-time spike-encoded representations. The system efficiently divides processing between static features processed by the ANN and dynamic changes handled by the SNN, enabling a focused computational approach. Measurements confirm that the high-frequency pre-attentive SNN processing monitors for temporal anomalies, triggering focused attention when potential obstacles are identified, and the SNN-based filter maintains state estimates for precise localization.

The breakthrough delivers a unified architecture that fuses the perception capabilities of a Hybrid Sensing Network (HSN) with the robustness of SNN-EMSIF filtering, achieving high accuracy, low latency, and minimal computational burden. Tests prove the system’s suitability for real-time robotics applications in dynamically changing environments with unmodeled moving intruders. The ANN path extracts static spatial features using a feedforward dynamic, calculated as a = f” $% w#a$% # ‘ (1), while the SNN component focuses on temporal dynamics and event-based information, creating a synergistic system for robust navigation.

Hybrid ANNs and SNNs for Robotic Navigation offer

The system utilises a pre-developed SNN-based filter, termed SNN-EMSIF, for robust state estimation, minimising computational burden. Simulation results confirm the feasibility of this approach, demonstrating promising accuracy in object localisation and velocity estimation while maintaining low-latency response and efficient computational performance. The framework’s ability to function without online learning and its resilience to partial observability and neuron failure suggest its suitability for edge robotics and autonomous systems operating in unpredictable environments. Researchers acknowledge limitations related to the scope of testing, primarily within simulated environments. Future work may investigate applying the framework’s outputs to decision-making and path planning within obstacle avoidance control systems.