Optical Flow in Dense Foggy Scenes using Semi-Supervised Learning

Wending Yan, Aashish Sharma, Robby T. Tan

Abstract:

In dense foggy scenes, existing optical flow methods are
erroneous. This is due to the degradation caused by dense
fog particles that break the optical flow basic assumptions
such as brightness and gradient constancy. To address
the problem, we introduce a semi-supervised deep learning technique that employs real fog images without optical flow ground-truths in the training process. Our network
integrates the domain transformation and optical flow networks in one framework. Initially, given a pair of synthetic
fog images, its corresponding clean images and optical flow
ground-truths, in one training batch we train our network in
a supervised manner. Subsequently, given a pair of real fog
images and a pair of clean images that are not corresponding to each other (unpaired), in the next training batch, we
train our network in an unsupervised manner. We then alternate the training of synthetic and real data iteratively. We
use real data without ground-truths, since to have groundtruths in such conditions is intractable, and also to avoid
the overfitting problem of synthetic data training, where the
knowledge learned on synthetic data cannot be generalized
to real data testing. Together with the network architecture
design, we propose a new training strategy that combines
supervised synthetic-data training and unsupervised realdata training. Experimental results show that our method
is effective and outperforms the state-of-the-art methods in
estimating optical flow in dense foggy scenes.