Audio-Visual Feature Fusion for Vehicles Classification in a Surveillance System

Tao Wang, Zhigang Zhu, Riad I. Hammoud

Abstract: In this paper we tackle the challenging problem of multimodal feature selection and fusion for vehicle categorization. Our proposed framework utilizes a boosting-based feature learning technique to learn the optimal combinations of feature modalities. New multimodal features are learned from the existing uni-modal features which are initially extracted from the data acquired by a novel audio-visual sensing system under different sensing conditions (long range, moving vehicles, and various environments). Experiments on a challenging dataset collected with our long-range sensing system demonstrated that the proposed technique is robust to noise and can find the best among multiple good feature modalities from training in terms of classification performance than the feature modality selection using a sequential based technique which tends to stay on a local maxima.