Policy Learning Using Weak Supervision

Jingkang Wang, Hongyi Guo, Zhaowei Zhu, Yang Liu

Keywords: Weak Supervision, Policy Learning, Correlated Agreement

Abstract: Most existing policy learning solutions require the learning agents to receive high-quality supervision signals, e.g., rewards in reinforcement learning (RL) or high-quality expert demonstrations in behavior cloning (BC). These quality supervisions are either infeasible or prohibitively expensive to obtain in practice. We aim for a unified framework that leverages the weak supervisions to perform policy learning efficiently. To handle this problem, we treat the ``weak supervisions'' as imperfect information coming from a \emph{peer agent}, and evaluate the learning agent's policy based on a ``correlated agreement'' with the peer agent's policy (instead of simple agreements). Our way of leveraging peer agent's information offers us a family of solutions that learn effectively from weak supervisions with theoretical guarantees. Extensive evaluations on tasks including RL with noisy reward, BC with weak demonstrations, and standard policy co-training (RL + BC) show that the proposed approach leads to substantial improvements, especially when the complexity or the noise of the learning environments grows.

One-sentence Summary: We introduce a novel framework to handle the common "weakly supervision" problem in policy learning based on a correlated agreement.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Community Implementations: [ 4 code implementations](https://www.catalyzex.com/paper/arxiv:2010.01748/code)

Reviewed Version (pdf): https://openreview.net/references/pdf?id=j9mFYYhc_M