Composing Task Knowledge with Modular Successor Feature Approximators

Wilka Torrico Carvalho, Angelos Filos, Richard Lewis, Honglak Lee, Satinder Singh

Abstract:

Recently, the Successor Features and Generalized Policy Improvement (SF&GPI)
framework has been proposed as a method for learning, composing and transferring
predictive knowledge and behavior. SF&GPI works by having an agent learn
predictive representations (SFs) that can be combined for transfer to new tasks
with GPI. However, to be effective this approach requires state features that are
useful to predict, and these state-features are typically hand-designed. In this
work, we present a novel neural network architecture, “Modular Successor Feature
Approximators” (MSFA), where modules both discover what is useful to predict,
and learn their own predictive representations. We show that MSFA is able to
better generalize compared to baseline architectures for learning SFs and a modular
network that discovers factored state representations.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2301.12305/code)