Asymmetric Norms to Approximate the Minimum Action Distance

Lorenzo Steccanella, Anders Jonsson

Confirmation: I have read and confirm that at least one author will be attending the workshop in person if the submission is accepted

Keywords: Distance Learning, Representation Learning, Goal Conditioned Reinforcement Learning

TL;DR: We propose a constrained optimization problem to approximate the Minimum Action Distance in asymmetric environments.

Abstract:

This paper presents a state representation for reward-free Markov decision processes. The idea is to learn, in a self-supervised manner, an embedding space where distances between pairs of embedded states correspond to the minimum number of actions needed to transition between them. Unlike previous methods, our approach incorporates an asymmetric norm parametrization, enabling accurate approximations of minimum action distances in environments with inherent asymmetry. We show how this representation can be leveraged to learn goal-conditioned policies, providing a notion of similarity between states and goals and a useful heuristic distance to guide planning.

To validate our approach, we conduct empirical experiments on both symmetric and asymmetric environments. Our results show that our asymmetric norm parametrization performs comparably to symmetric norms in symmetric environments and surpasses symmetric norms in asymmetric environments.

Submission Number: 37