Plug-in Sample Complexity For Constrained Reinforcement Learning

Ruosong Wang, Lin F. Yang, Zihan Zhang

Keywords: Reinforcement learning, plug-in algorithm, generative model

TL;DR: We demonstrate that strategically perturbing the reward function enhances the sample efficiency of linear programming in constrained reinforcement learning.

Abstract: We present a novel plug-in approach for constrained reinforcement learning that achieves the sample complexity of $\tilde{O}\left(\frac{SAH^4}{\epsilon^2\zeta^2}\right)$ using a generative model. Unlike previous specialized algorithms, our method is general: it requires only black-box access to an optimization oracle that solves the empirical CMDP. The core of our approach is a reward perturbation technique that guarantees the oracle's solution is valid for the original problem.

Primary Area: learning theory

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2026/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 18400