Learning Hierarchical Abstractions of Complex Dynamics using Active Predictive Coding

Vishwas Sathish, Rajesh P. N. Rao

Abstract: Learning robot dynamics for real-world environments is difficult, if not impossible, using a single monolithic state transition network. A more practical approach, inspired by sensory-motor hierarchies in the mammalian brain, is to model complex state transition dynamics as a sequence of simpler dynamics, which in turn are modeled using even simpler dynamics, and so on. We demonstrate the efficacy of such an approach using Active Predictive Coding (APC), a structured model for learning hierarchical dynamics and hierarchical policies that is inspired by the neocortex. APC leverages higher-level abstract states as priors for generating lower-level transition functions using hypernetworks. Additionally, APC learns a transition function for the dynamics between abstract states, leading to a tractable approach to learning complex dynamics in terms of a hierarchy of dynamically generated transition functions. We apply our approach to compositional navigation problems and show its capability for rapid planning and transfer to novel scenarios. In both traditional grid-world-style navigation problems as well as in the more complex Habitat vision-based navigation domain, APC learns to abstract the transition dynamics of a robot within and between different rooms in an unsupervised manner. Our results suggest that APC offers a promising framework for learning complex robot dynamics using a compositional and hierarchical approach.