On Feasible Rewards in Multi-Agent Inverse Reinforcement Learning

Till Freihaut, Giorgia Ramponi

Keywords: Multi Agent Inverse Reinforcement Learning, Inverse Reinforcement Learning, Multi Agent Reinforcement Learning

Abstract:

Multi-agent inverse reinforcement learning (MAIRL) aims to recover agent reward
functions from expert demonstrations. We characterize the feasible reward set in
Markov games, identifying all reward functions that rationalize a given equilibrium.
However, equilibrium-based observations are often ambiguous: a single Nash
equilibrium can correspond to many reward structures, potentially changing the
game’s nature in multi-agent systems. We address this by introducing entropy-
regularized Markov games, which yield a unique equilibrium while preserving
strategic incentives. For this setting, we provide a sample complexity analysis
detailing how errors affect learned policy performance. Our work establishes
theoretical foundations and practical insights for MAIRL.

Confirmation: I understand that authors of each paper submitted to EWRL may be asked to review 2-3 other submissions to EWRL.

Serve As Reviewer: ~Till_Freihaut1

Track: Regular Track: unpublished work

Submission Number: 16