Unsupervised Spatiotemporal Data Inpainting

Yuan Yin, Arthur Pajot, Emmanuel de Bézenac, Patrick Gallinari

Abstract: We tackle the problem of inpainting occluded area in spatiotemporal sequences, such as cloud occluded satellite observations, in an unsupervised manner. We place ourselves in the setting where there is neither access to paired nor unpaired training data. We consider several cases in which the underlying information of the observed sequence in certain areas is lost through an observation operator. In this case, the only available information is provided by the observation of the sequence, the nature of the measurement process and its associated statistics. We propose an unsupervised-learning framework to retrieve the most probable sequence using a generative adversarial network. We demonstrate the capacity of our model to exhibit strong reconstruction capacity on several video datasets such as satellite sequences or natural videos.

Code: https://sites.google.com/view/unsup-video-inpaiting/

Keywords: Deep Learning, Adversarial, MAP, GAN, neural networks, video