We introduce a new unsupervised reinforcement learning method for discovering the set of intrinsic options available to an agent. This set is learned by maximizing the number of different states an agent can reliably reach, as measured by the mutual information between the set of options and option termination states. To this end, we instantiate two policy gradient based algorithms, one that creates an explicit embedding space of options and one that represents options implicitly. Both algorithms also yield a tractable and explicit empowerment measure, which is useful for empowerment maximizing agents. Furthermore, they scale well with function approximation and we demonstrate their applicability on a range of tasks.

Variational Intrinsic Control

"Too Long; Didn't Read": a short sentence describing your paper

Comma separated list of author email addresses, in the same order as above.

Comma separated list of author names, as they appear in the paper.

Comma separated list of email domains of people who would have a conflict of interest in reviewing this paper, (e.g., cs.umass.edu;google.com, etc.).

Either upload a PDF file or provide a direct link to your PDF on ArXiv (link must begin with http(s) and end with .pdf)

The users who will be allowed to read the above content.

How your identity will be displayed with the above content.

Variational Intrinsic Control

Karol Gregor, Danilo Jimenez Rezende, Daan Wierstra