Input dropout for spatially aligned modalities

Sébastien de Blois, Mathieu Garon, Christian Gagné, Jean-Francois Lalonde

Abstract:

Computer vision datasets containing multiple modalities
such as color, depth, and thermal properties are now commonly
accessible and useful for solving a wide array of challenging
tasks. However, deploying multi-sensor heads is not possible
in many scenarios. As such many practical solutions tend to
be based on simpler sensors, mostly for cost, simplicity and
robustness considerations. In this work, we propose a training
methodology to take advantage of these additional modali-
ties available in datasets, even if they are not available at test
time. By assuming that the modalities have a strong spatial
correlation, we propose Input Dropout, a simple technique that
consists in stochastic hiding of one or many input modalities
at training time, while using only the canonical (e.g. RGB)
modalities at test time. We demonstrate that Input Dropout triv-
ially combines with existing deep convolutional architectures,
and improves their performance on a wide range of computer
vision tasks such as dehazing, 6-DOF object tracking, pedes-
trian detection and object classification.