PDE-Driven Spatiotemporal Disentanglement

Jérémie Donà, Jean-Yves Franceschi, sylvain lamprier, patrick gallinari

Keywords: disentanglement, spatiotemporal prediction, representation learning, dynamical systems, separation of variables

Abstract: A recent line of work in the machine learning community addresses the problem of predicting high-dimensional spatiotemporal phenomena by leveraging specific tools from the differential equations theory. Following this direction, we propose in this article a novel and general paradigm for this task based on a resolution method for partial differential equations: the separation of variables. This inspiration allows us to introduce a dynamical interpretation of spatiotemporal disentanglement. It induces a principled model based on learning disentangled spatial and temporal representations of a phenomenon to accurately predict future observations. We experimentally demonstrate the performance and broad applicability of our method against prior state-of-the-art models on physical and synthetic video datasets.

One-sentence Summary: We introduce a novel interpretation of spatiotemporal disentanglement, inducing a simple and performant disentangled prediction model.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Code: [ JeremDona/spatiotemporal_variable_separation](https://github.com/JeremDona/spatiotemporal_variable_separation)

Data: [Chairs](https://paperswithcode.com/dataset/chairs), [MNIST](https://paperswithcode.com/dataset/mnist), [Moving MNIST](https://paperswithcode.com/dataset/moving-mnist)

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:2008.01352/code)