Multimodal Model-Based Reinforcement Learning for Autonomous Racing

Elena Shrestha, Hanxi Wan, Chetan Reddy, Yulun Zhuang, Ram Vasudevan

Keywords: reinforcement learning, model-based reinforcement learning, world model, autonomous racing, multi-agent

Abstract: Model-based reinforcement learning (MBRL) techniques have recently yielded promising results for real-world autonomous racing using high-dimensional observations. MBRL agents solve long-horizon tasks by building a world model and planning actions by latent imagination. This approach involves explicitly learning a model of the system dynamics and using it to learn the optimal policy for continuous control over multiple timesteps. As a result, agents may converge to sub-optimal policies if the world model is inaccurate. This paper proposes Lucid Dreamer, a end-to-end multimodal MBRL agent that leverages egocentric LiDAR and RGB camera observations through self-supervised sensor fusion. The zero-shot performance of MBRL agents is empirically evaluated on a 1:10 scale rover in simulation for unseen racing conditions and in a real-world environment to demonstrate sim-to-real transfer. Although only trained against five static obstacles in simulation, Lucid Dreamer safely avoided collisions with a dynamic rule-based agent in a zero-shot manner. This paper illustrates that multimodal perception improves robustness of the world model without requiring additional training data.

Submission Number: 23