$\pi$2vec: Policy Representation with Successor Features
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Policy representation, offline policy selection, robotics, evaluation
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
TL;DR: This paper introduces `pi2vec`, a method that represents black box policies as feature vectors by leveraging a pretrained foundation model and successor feature framework, enabling efficient policy evaluation in resource constrained environments.
Abstract: This paper introduces $\pi$2vec, a method for representing black box policies as comparable feature vectors.
Our method combines the strengths of foundation models that serve as generic and powerful state representations and successor features that can model the future occurrence of the states for a policy.
$\pi$2vec represents the behavior of policies by capturing the statistics of the features from a pretrained model with the help of successor feature framework.
We focus on the offline setting where policies and their representations are trained on a fixed dataset of trajectories.
Finally, we employ linear regression on $\pi$2vec vector representations to predict the performance of held out policies.
The synergy of these techniques results in a method for efficient policy evaluation in resource constrained environments.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Submission Number: 1825
Loading