Robbyant’s LingBot-Depth AI Cuts Depth Error by 70% for Robotics

Robbyant, an Ant Group subsidiary, has unveiled LingBot-Depth, a new AI model that dramatically improves depth sensing for robotics, cutting depth error by over 70% in challenging environments. The high-precision spatial perception model tackles a long-standing industry problem – unreliable data from reflective or transparent surfaces – by intelligently reconstructing missing 3D information. This breakthrough, achieved through Robbyant’s Masked Depth Modeling (MDM), was co-optimized with hardware from strategic partner Orbbec and validated using their Gemini 330 cameras. “Reliable 3D vision is critical to the advancement of embodied AI,” says Zhu Xing, Chief Executive Officer of Robbyant, adding that open-sourcing LingBot-Depth aims to “lower the barrier to advanced spatial perception and accelerate the adoption of embodied intelligence.”

LingBot-Depth Achieves 70% REL Reduction on Sparse Depth Completion

LingBot-Depth, a new spatial perception AI model from Robbyant, is demonstrably improving depth completion in challenging robotic environments. Benchmark evaluations on both NYUv2 and ETH3D datasets reveal significant performance gains; the model achieves over a 70% reduction in relative error (REL) when completing sparse depth data in indoor settings. Furthermore, root mean squared error (RMSE) on the complex Structure-from-Motion (SfM) task decreased by approximately 47%, indicating a substantial leap in 3D environmental understanding for robots. This advancement addresses a critical limitation in depth sensing systems – their struggle with reflective and transparent surfaces.

These materials commonly introduce errors in depth data, potentially causing operational failures or safety hazards for robots navigating real-world spaces. Robbyant tackled this issue with Masked Depth Modeling (MDM), enabling LingBot-Depth to infer and reconstruct missing depth information by analyzing RGB image features like texture and context. The model’s compatibility with existing hardware is noteworthy, requiring no alterations to current sensor designs. Orbbec’s Gemini 330 stereo 3D cameras, leveraging their MX6800 depth engine chip, played a vital role in both data collection and validation, with LingBot-Depth fine-tuned on high-quality RGB-depth pairs produced by the cameras.

Len Zhong, Head of Product Management of Orbbec, stated, “Robbyant’s work…complements Orbbec’s expertise…It’s a great example that demonstrates close coupling between a robot’s sensing hardware and its perception intelligence.” Robbyant also collected approximately 10 million raw samples and curated 2 million RGB-depth pairs for training, with plans to open-source this dataset.

Masked Depth Modeling Reconstructs Data for Challenging Optical Surfaces

Depth sensing systems routinely encounter difficulties with surfaces like glass and polished metal, leading to incomplete or noisy data that can compromise robotic operation and safety; however, a new approach called Masked Depth Modeling (MDM) is offering a solution. This allows robots to perceive a more complete and accurate 3D map of their surroundings, even in visually complex scenarios. LingBot-Depth demonstrably improves performance on demanding tasks, achieving a greater than 70% reduction in relative error in indoor scenes when benchmarked against models like PromptDA and PriorDA.

Crucially, the model also reduced root mean squared error on a sparse Structure-from-Motion task by approximately 47%. Robbyant also plans to open-source a dataset of approximately 2 million RGB-depth pairs used to train LingBot-Depth, fostering wider innovation in the field.

Orbbec’s Gemini 330 & MX6800 Chip Enable Robust Data Acquisition

The development of reliable 3D vision is being significantly propelled by a collaborative effort between Robbyant and Orbbec, focusing on data acquisition at the hardware level. This chip combines active and passive imaging, delivering consistent 3D data even in challenging lighting – from complete darkness to direct sunlight – and reducing system latency through on-device computation. LingBot-Depth was co-optimized and validated using Orbbec’s platforms, with the Gemini 330 providing key hardware resources and technical expertise. Leveraging raw depth data from the Gemini 330, the model intelligently reconstructs missing information, bolstering a robot’s ability to perceive accurately in optically complex environments.

The partnership highlights the importance of integrating high-quality chip-level data with advanced perception algorithms, a strategy that allows for substantial performance gains without altering existing sensor designs.