CSVO: Complementary-Pathway Spatial-Enhanced Visual Odometry for Extreme Environments with Brain-Inspired Vision Sensors

Yihan Lin, Zhaoxi Zhang, Yuguo Chen, Taoyi Wang, Rong Zhao

Abstract: Visual Odometry (VO) estimates the pose and motion trajectory of the camera based on visual input, serving as a fundamental technique for robotic positioning and navigation. However, existing VO methods face challenges in visual degradation in extreme environments, e.g., high dynamic range or fast-motion conditions. Although event-based sensing schemes offer partial solutions to this problem, they are limited by unstable features and noise. Recently, a novel brain-inspired vision sensor, Tianmouc, has been reported, incorporating two complementary pathways: a cognition-oriented pathway (COP) for precise color intensity and an action-oriented pathway (AOP) for fast spatiotemporal sensing, considered a promising visual input for VO tasks. Here, we develop Complementary Pathway Spatial Enhanced Visual Odometry (CSVO) to cope with extreme scenarios by fusing the COP and AOP information of Tianmouc. To leverage the dynamic range expansion brought about by dual-pathway fusion, as well as the low-latency spatial difference data in AOP to address high-speed motion, we design an asynchronous dual-pathway feature encoder considering synchronous multimodal fusion and asynchronous cross-modal feature matching. To train and evaluate CSVO, we transform two conventional VO datasets, TartanAir and Apollo, to Tianmouc modality through simulation and collect a real- world Tianmouc-VO dataset in challenging scenes. Our results demonstrate state-of-the-art performance over existing methods on these datasets. Our work sheds light on the generalizability of agents working in extreme scenarios. The codes and data sets are available at https://github.com/Tianmouc/CSVO.

External IDs:dblp:conf/iros/LinZCWZ25