Metric Learning Based Class Specific Experts for Open-Set Recognition of Traffic Participants in Urban Areas Using Infrastructure Sensors

Karthikeyan Chandra Sekaran, Lakshman Balasubramanian, Michael Botsch, Wolfgang Utschick

Abstract: Sensors installed in the infrastructure can make a significant contribution to the advancement of Advanced Driver Assistance Systems (ADAS) and connected mobility. Thermal cameras provide protection against the abuse of personalised data and perform robustly in challenging environmental conditions, making them an excellent choice for infrastructural perception. The goal of this work is to solve the crucial problem of Open-Set Recognition (OSR) for thermal camera-based perception systems installed in the infrastructure. In this paper, a novel modular architecture for OSR called Class Specific Experts (CSE) is proposed, in which, class specialization is achieved using individual feature spaces. The proposed methodology can be easily embedded in an object detection setting and provides as a main advantage, the possibility of online incremental learning without catastrophic forgetting. This work also introduces a open-source classification dataset called Infrastructure Thermal Dataset (ITD) containing image snippets captured by a thermal camera mounted in the infrastructure. The proposed approach outperforms the compared baselines for the task of OSR on many publicly available thermal and non-thermal datasets, as well as the new ITD dataset.