Action as vector to the goal state in latent space

Keno Harada, Masahiro Suzuki, Yutaka Matsuo

Abstract:

In reinforcement learning, action is treated as a point in the action space, with little emphasis on the design of
the action space. In contrast to the existing reinforcement learning frameworks, we consider action as the amount
of change in the latent space to reach the target state, referring to the human action process, and define this
as latent action. We propose a representation learning method using Predictive Variational Autoencoder which
enables that taking latent action to minimize the distance to the goal state in the latent space corresponds to the
optimal action in the actual input space. We verify by experiments that action selection by latent actions using
Predictive Variational Autoencoder can achieve more stable control compared to the method which uses Variational
Autoencoder for current observation and selects actions based on errors from the control goal in the input space.