Towards A Unified Agent with Foundation ModelsDownload PDF

Published: 03 Mar 2023, Last Modified: 19 Apr 2023RRL 2023 OralReaders: Everyone
Keywords: foundation models, language models, exploration, transfer, skills, imitation
TL;DR: We leverage foundation models to create a more unified RL framework that improves on core RL challenges such as exploration, experience reuse and transfer, skills scheduling and learning from observation.
Abstract: Language Models and Vision Language Models have recently demonstrated unprecedented capabilities in terms of understanding human intentions, reasoning, scene understanding, and planning-like behaviour, in text form, among many others. In this work, we investigate how to embed and leverage such abilities in Reinforcement Learning (RL) agents. We design a framework that uses language as the core reasoning tool, exploring how this enables an agent to tackle a series of fundamental RL challenges, such as efficient exploration, reusing experience data, scheduling skills, and learning from observations, which traditionally require separate, vertically designed algorithms. We test our method on a sparse-reward simulated robotic manipulation environment, where a robot needs to stack a set of objects. We demonstrate substantial performance improvements over baselines in exploration efficiency and ability to reuse data from offline datasets, and illustrate how to reuse learned skills to solve novel tasks or imitate videos of human experts.
Track: Technical Paper
Supplementary Material: zip
Confirmation: I have read and agree with the workshop's policy on behalf of myself and my co-authors.
2 Replies