The Role of Representation Transfer in Multitask Imitation Learning
Primary Area: transfer learning, meta learning, and lifelong learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Imitation Learning, Multitask Learning, Task Diversity, Representation Learning
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
TL;DR: We present a statistical sample complexity bound and empirical evaluation to demonstrate improved sample efficiency with representation transfer using multitask data
Abstract: Transferring representation for multitask imitation learning has the potential to provide improved sample efficiency on learning new tasks, when compared to learning from scratch. In this work, we provide a statistical guarantee indicating that we can indeed achieve improved sample efficiency on the target task when a representation is trained using sufficiently diverse source tasks. Our theoretical results can be readily extended to account for commonly used neural network architectures such as multilayer perceptrons and convolutional networks with realistic assumptions. Inspired by the theory, we propose a practical metric that estimates the notion of task diversity. We conduct empirical analyses that align with our theoretical findings on five simulated environments—in particular leveraging more data from source tasks can improve sample efficiency on learning in the new task. Our experiments further demonstrate that our proposed task diversity metric is positively correlated to the imitation performance.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 3065
Loading