Radar Spectra-language Model for Automotive Scene Parsing
Primary Area: representation learning for computer vision, audio, language, and other modalities
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: radar spectra, radar perception, radar object detection, free space segmentation, autonomous driving, radar classification
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: Radar sensors are low cost, long-range, and weather-resilient and provide direct velocity measurements. Therefore, they are widely used for driver assistance functions, and are expected to be crucial for the success of autonomous driving in the future. In many perception tasks only pre-processed radar point clouds are considered. In contrast, radar spectra are a dense and raw form of radar measurements and contain more information than radar point clouds. However, radar spectra are rather difficult to interpret. In this work, we aim to explore the semantic information extracted from spectra in the context of automotive driving, thereby moving towards to a better interpretability of radar spectra. To this end, we create a radar spectra-language model, allowing us to query radar spectra measurements for the presence of scene elements by using free text. We overcome the scarcity of radar spectra data by matching the embedding space of an existing vision-language model (VLM). Recognizing that off-the-shelf VLMs underperform on our target domain, we develop a fine-tuning approach tailored to automotive scenes. Finally, we explore the benefit of the learned representation for scene parsing, obtaining improvements in drivable space segmentation and object detection merely by injecting the spectra embedding into a baseline model.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 5702
Loading