Velociraptor: Leveraging Visual Foundation Models for Label-Free, Risk-Aware Off-Road Navigation

Samuel Triest, Matthew Sivaprakasam, Shubhra Aich, David Fan, Wenshan Wang, Sebastian Scherer

Keywords: Field Robotics, Self-Supervised Learning, Visual Foundation Models

TL;DR: We propose a fully self-supervised traversability learning method that leverages visual foundation model features and demonstrate it in high-speed off-road navigation.

Abstract: Traversability analysis in off-road regimes is a challenging task that requires understanding of multi-modal inputs such as camera and LiDAR. These measurements are often sparse, noisy, and difficult to interpret, particularly in the off-road setting. Existing systems are very engineering-intensive, often requiring hand-tuning of traversability rules and manual annotation of semantic labels. Furthermore, existing methods for analyzing traversability risk and uncertainty are computationally expensive or not well-calibrated. We propose Velociraptor, a traversability analysis system that performs [veloci]ty-informed, [r]isk-[a]ware [p]erception and [t]raversability for [o]ff-[r]oad driving without any human annotations. We achieve this via the use of visual foundation models (VFMs) and geometric mapping to produce a rich visual-geometric representation of the robot's local environment. We then leverage this representation to produce costmaps, speedmaps, and uncertainty maps using state-of-the-art fully self-supervised techniques. Our approach enables intelligent high-speed off-road navigation with zero human annotation, and with about forty minutes of expert data, outperforms several geometric and semantic traversability baselines, both in offline and real-world robot trials across multiple challenging off-road sites.

Video: https://drive.google.com/file/d/1ONdMZaoTQDQhYK4B9kfFbP0UGNh24N0R/view?usp=sharing

Student Paper: yes

Submission Number: 409