SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptation

TAO SUN, Mattia Segu, Janis Postels, Yuxuan Wang, Luc Van Gool, Bernt Schiele, Federico Tombari, Fisher Yu

Abstract:

Adapting to a continuously evolving environment is a
safety-critical challenge inevitably faced by all autonomous
driving systems. Existing image and video driving datasets,
however, fall short of capturing the mutable nature of the
real world. In this paper, we introduce the largest multitask synthetic dataset for autonomous driving, SHIFT. It
presents discrete and continuous shifts in cloudiness, rain
and fog intensity, time of day, and vehicle and pedestrian
density. Featuring a comprehensive sensor suite and annotations for several mainstream perception tasks, SHIFT
allows investigating the degradation of a perception system performance at increasing levels of domain shift, fostering the development of continuous adaptation strategies
to mitigate this problem and assess model robustness and
generality. Our dataset and benchmark toolkit are publicly
available at www.vis.xyz/shift.