RACE: Real-Time Adaptive Camera-Intrinsics Estimation via Control Theory

ICLR 2026 Conference Submission21580 Authors

Keywords: Computer vision, control theory, online camera calibration

Abstract: Modern embodied AI systems, from mobile robots to AR devices, rely on accurate camera intrinsics to ensure reliable perception. Yet in real-world operation, intrinsics drift due to heating, zoom events, mechanical shocks, a single hard landing, or simply incorrect factory calibration, violating the fixed-parameter assumption that underpins most vision and learning pipelines. This induces a distribution shift in the visual input, which in turn degrades the performance of downstream models and tasks that rely on stable camera geometry. We introduce RACE (Real-time Adaptive Camera-intrinsic Estimation), a provably stable online learning algorithm that continually estimates camera intrinsics directly from continuous monocular image stream. RACE updates parameters through a Lyapunov-stable adaptive law, guaranteeing global asymptotic convergence of the reprojection error dynamics and recovery of the true intrinsics under persistent excitation. Unlike prior batch optimization, heuristic self-calibration or learning-based approaches, RACE requires no training data, bundle adjustment, or retraining, and provides the first theoretical bridge between adaptive control and online learning for camera models. Empirically, we evaluate RACE across public benchmarks (EuRoC, TUM, and TartanAir), showing that it matches or surpasses state-of-the-art learning-based calibration while adapting in real time with negligible computational overhead. Our results highlight RACE as a new class of theoretically grounded continual learners for camera intrinsics, enabling robust long-term perception in embodied agents.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 21580