Meta-Learning and Meta-Reinforcement Learning - Tracing the Path towards Deep Mind's Adaptive Agent

TMLR Paper6536 Authors

Abstract:

Humans are highly effective at utilizing prior knowledge to adapt to novel tasks, a capability that standard machine learning models struggle to replicate due to their reliance on task-specific training.
Meta-learning overcomes this limitation by allowing models to acquire transferable knowledge from various tasks, enabling rapid adaptation to new challenges with minimal data.
This survey provides a rigorous, task-based formalization of meta‑learning and meta-reinforcement learning and uses that paradigm to chronicle the landmark algorithms that paved the way for DeepMind’s Adaptive Agent, consolidating the essential concepts needed to understand the Adaptive Agent and other generalist approaches.

Submission Type: Long submission (more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=TG1QDSqTP1

Changes Since Last Submission:

1. Significantly shortened the timeline (Section 3) by mooving PEARL to the Appendix, merging overview tables into one and removing the timeline plots.
2. Re-Structured timeline section so that each subsection contains only one single landmark algorithm. Thereby, we moved the paragraphs about performance into a specific "performance analysis" paragraph for each algorithm.
3. Slightly shortened section 2 by reformulating the meta-RL paradigm with less redundant language.
4. Sharpened the introduction to better address the paper focus and corresponding paper audience.

Assigned Action Editor: ~Tim_Genewein1

Submission Number: 6536