Beyond Reactive Adaptation: Long-Horizon Memory for Autonomous Racing via State Space Models

Grzegorz Czechmanowski, Jan Węgrzynowski, Piotr Kicki, Krzysztof Walas

Keywords: Autonomous Racing, Reinforcement Learning, State Space Models, In-context Meta-Learning

TL;DR: Acting as an in-context meta-learner, our Mamba-based policy learns how to drive on tracks with unknown friction, leveraging its memory to achieve continuous lap-to-lap improvement.

Abstract: Autonomous racing pushes vehicles to their phys- ical limits, requiring control policies that can rapidly adapt to localized changes in track conditions, such as varying surface friction. Current Reinforcement Learning (RL) approaches rely either on ground-truth system identification, which is imprac- tical in the real world, or short-horizon reactive adaptations (e.g., Rapid Motor Adaptation (RMA)) that cannot remember spatial disturbances across multiple laps. In this extended abstract, we propose a novel RL architecture based on Mamba, a structured State Space Model (SSM), for autonomous racing. By fusing vehicle state with Fourier features of vehicle position on the racetrack, our Mamba-based policy builds a long-horizon episodic memory. This allows the policy not only to adapt to unknown friction online but also to map and memorize slippery zones for future laps. Evaluated in a simulated F1Tenth environment, our approach demonstrates continuous lap-to- lap improvement, approaching the performance of an ”oracle” policy trained on exact ground-truth friction, whereas standard Multi-Layer Perceptron (MLP) and Recurrent Neural Network (RNN) baselines plateau at inferior performance levels.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 2