Policy Disentangled Variational Autoencoder

Tae Yoon Lim, Kanghoon Lee, Woohyung Lim

Primary Area: generative models

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Video generative model; Disentangled representation learning; Generating video conditioned to the policy

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We present PDVAE which learns the representation of policy in discrete space without label information and the dynamics of environment conditioned to the policy. PDVAE can generate diverse scenarios aligned with the specified policy.

Abstract: Deep generative models for video primarily treat videos as visual representations of agents (e.g., people or objects) performing actions, often overlooking the underlying intentions driving those actions. In reinforcement learning, the policy determines actions based on the current context and is analogous to the underlying intention guiding those actions. Through the acquisition of policy representations, we can generate a video capturing how an agent would behave when following a specific policy in a given context. In this paper, we aim to learn the representation of the policy without supervision and the dynamics of the environment conditioned to the policy. We propose Policy Disentangled Variational Autoencoder (PDVAE) which can generate diverse videos aligned with the specified policy where the user can alter the policy during the generation. We demonstrate PDVAE with three video datasets: Moving MNIST, KTH action dataset, and VizDoom.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 2317