Zero-Shot Context Identification through Clustering and Foundation Modeling for Friction Estimation

Renukanandan Tumu, Ahmad Amine, Lee Milburn, Rajnish Gupta, Urara Kono, Rahul Mangharam

Keywords: Foundation models, Friction estimation, Field robotics, Zero-shot learning

TL;DR: This paper presents an approach to integrating foundation models with unsupervised clustering algorithms to identify terrain contexts, and fit friction parameters for each one, enabling vehicles to traverse unknown numbers of unseen terrains.

Abstract: Off-road autonomous navigation demands accurate estimation of terrain-dependent parameters, particularly tire-ground friction, which directly impacts control performance and safety. Traditional methods for friction estimation—whether proprioceptive, vision-based, or hybrid—struggle to adapt to abrupt terrain transitions and lack generalization to previously unseen environments. This paper introduces Physics-Constrained and Vision-Informed Friction Estimation (PC-VFE), a framework that combines semantic visual understanding through the use of foundation models with physics-based dynamics modeling to estimate friction in real time. PC-VFE first identifies terrain contexts using a vision-language model and unsupervised clustering, then estimates context-specific friction parameters via a constrained optimization process. Our approach requires no prior knowledge of terrain types, adapts in a zero-shot manner, and enables rapid re-identification of known surfaces.

Submission Number: 36