METEOR: A Dense, Heterogeneous, and Unstructured Traffic Dataset With Rare Behaviors

Rohan Chandra, Xijun Wang, Mridul Mahajan, Rahul Kala, Rishitha Palugulla, Chandrababu Naidu Nallagopu, Alok Jain, Dinesh Manocha

Abstract: We present a new traffic dataset, {\textsc{Meteor}\xspace}, which captures traffic patterns and multi-agent driving behaviors in unstructured scenarios. {\textsc{Meteor}\xspace}~consists of more than $1000$ one-minute videos, over $2$ million annotated frames with bounding boxes and GPS trajectories for $16$ unique agent categories, and more than $13$ million bounding boxes for traffic agents. {\textsc{Meteor}\xspace} is a dataset for rare and interesting, multi-agent driving behaviors that are grouped into traffic violations, atypical interactions, and diverse scenarios. Every video in {\textsc{Meteor}\xspace} is tagged using a diverse range of factors corresponding to weather, time of the day, road conditions, and traffic density. We use \rain to benchmark perception methods for object detection and multi-agent behavior prediction. Our key finding is that state-of-the-art models for object detection and behavior prediction, which otherwise succeed on existing datasets such as Waymo, fail on the {\textsc{Meteor}\xspace} dataset. {\textsc{Meteor}\xspace} is a step towards developing more sophisticated perception models for dense, heterogeneous, and unstructured scenarios.