MSU-4S - The Michigan State University Four Seasons Dataset

Daniel Kent, Mohammed Alyaqoub, Xiaohu Lu, Sayed Hamed Khatounabadi, Kookjin Sung, Cole Scheller, Alexander Dalat, Xinwei Guo, Asma Bin Thabit, Roberto Muntaner Whitley, Hayder Radha

Abstract:

Public datasets, such as KITTI, nuScenes, and Waymo,
have played a key role in the research and development of
autonomous vehicles and advanced driver assistance systems. However, many of these datasets fail to incorporate a full range of driving conditions; some datasets only
contain clear-weather conditions, underrepresenting or entirely missing colder weather conditions such as snow or
autumn scenes with bright colorful foliage. In this paper, we
present the Michigan State University Four Seasons (MSU4S) Dataset, which contains real-world collections of autonomous vehicle data from varied types of driving scenarios. These scenarios were recorded throughout a full range
of seasons, and capture clear, rainy, snowy, and fall weather
conditions, at varying times of day. MSU-4S contains more
than 100,000 two- and three-dimensional frames for camera, lidar, and radar data, as well as Global Navigation
Satellite System (GNSS), wheel speed, and steering data,
all annotated with weather, time-of-day, and time-of-year.
Our data includes cluttered scenes that have large numbers
of vehicles and pedestrians; and it also captures industrial
scenes, busy traffic thoroughfare with traffic lights and numerous signs, and scenes with dense foliage. While providing a diverse set of scenes, our data incorporate an important feature: virtually every scene and its corresponding
lidar, camera, and radar frames were captured in four different seasons, enabling unparalleled object detection analysis and testing of the domain shift problem across weather
conditions. In that context, we present detailed analyses for
3D and 2D object detection showing a strong domain shift
effect among MSU-4S data segments collected across different conditions. MSU-4S will also enable advanced multimodal fusion research including different combinations of
camera-lidar-radar fusion, which continues to be of strong
interest for the computer vision, autonomous driving and
ADAS development communities. The MSU-4S dataset is
available online at https://egr.msu.edu/waves/msu4s.