DriveGPT: Scaling Autoregressive Behavior Models for Driving

Xin Huang, Eric M Wolff, Paul Vernaza, Tung Phan-Minh, Hongge Chen, David S Hayden, Mark Edmonds, Brian Pierce, Xinxin Chen, Pratik Elias Jacob, Xiaobai Chen, Chingiz Tairbekov, Pratik Agarwal, Tianshi Gao, Yuning Chai, Siddhartha Srinivasa

TL;DR: Our paper studies the scaling properties in driving, across multiple orders of magnitude in data, model size, and compute.

Abstract: We present DriveGPT, a scalable behavior model for autonomous driving. We model driving as a sequential decision-making task, and learn a transformer model to predict future agent states as tokens in an autoregressive fashion. We scale up our model parameters and training data by multiple orders of magnitude, enabling us to explore the scaling properties in terms of dataset size, model parameters, and compute. We evaluate DriveGPT across different scales in a planning task, through both quantitative metrics and qualitative examples, including closed-loop driving in complex real-world scenarios. In a separate prediction task, DriveGPT outperforms state-of-the-art baselines and exhibits improved performance by pretraining on a large-scale dataset, further validating the benefits of data scaling.

Lay Summary:

* Our paper is the first published study of large-scale scaling laws in autonomous driving. The largest model was trained on 100M+ high-quality human demonstrations in dense urban driving with 1B+ parameters.
* Our scaling experiments validate the benefits of increasing both data and compute, revealing better model scalability as training data increases -- consistent with trends observed in language models.
* We quantitatively and qualitatively evaluate models across scales, including real-world deployment of our model as a real-time planner in complex urban scenarios.
* Our model achieves state-of-the-art performance on the Waymo Open Motion Dataset across key geometric metrics, highlighting its strong generalization in motion prediction.

Application-Driven Machine Learning: This submission is on Application-Driven Machine Learning.

Primary Area: Applications->Robotics

Keywords: Autonomous Driving, Foundation Models, Behavior Modeling

Submission Number: 12237