MM-DCDR: A Benchmark of Device Configuration and Data Representation for mmWave-Based Human Sensing

Yating Gao, Ruixu Geng, Dongheng Zhang, Yang Hu, Hui Lin, Yan Chen

Abstract: Millimeter-wave radar-based human sensing has become increasingly popular due to its non-invasive and lightrobust characteristics. While existing studies focus on improving performance with specific setups, they fail to consider the impact of different device configurations and data representations, which determine hardware capabilities and contain distinct features, respectively. Moreover, exploring the impact of those two factors presents inherent trade-offs such as balancing the velocity resolution and angle resolution, as well as choosing between heatmap or point clouds. To address these issues, we introduce MM-DCDR, the first large-scale multi-radar dataset consisting of both a highvelocity resolution radar and a high-angle resolution radar. The dataset comprises 352K frames of time-aligned radar and camera frames, covering 8 activities performed by 11 subjects. Leveraging this dataset, we systematically evaluate the trade-offs between velocity and angle resolution in their impact on human sensing. We also conduct extensive experiments with various deep neural networks to investigate the performance gap between different data representations. We believe our large-scale dataset and comprehensive benchmark would provide valuable insights into exploring optimal device configurations and data representations for practical radar-based sensing systems. The dataset is publicly available at https://github.com/yating-gao/MM-DCDR.