Robust Inverse Reinforcement Learning under State Adversarial Perturbations
Keywords: inverse reinforcement learning, state adversarial attacks, robustness
TL;DR: This paper introduces a novel Max-Margin Inverse Reinforcement Learning (IRL) approach for State-Adversarial Markov Decision Processes, emphasizing optimality under adversarial perturbations and advancing IRL strategies for resilient applications.
Abstract: State adversarial perturbations –such as sensor noise, environmental interference, or targeted attacks– are common in real-world systems, often leading to compromised state observations. Despite this, Inverse Reinforcement Learning (IRL) in the context of State-Adversarial Markov Decision Processes (SA-MDPs) has received limited attention, primarily because conventional notions of optimality do not apply. In this paper, we introduce a novel definition of optimality that ensures the existence of an optimal policy within SA-MDPs. Building on this foundation, we propose the State-Adversarial Max-Margin IRL (SAMM-IRL) algorithm, designed for robustness against state adversarial perturbations. Our theoretical analysis, supported by empirical validation, demonstrates that SAMM-IRL significantly enhances IRL performance in adversarial environments, providing a robust framework for real-world applications that demand resilience.
Primary Area: reinforcement learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 10604
Loading