Make You Better: Reinforcement Learning from Human Gain

Yiming Gao, Feiyu Liu, Liang Wang, Weixuan Wang, Wenjin Yang, Zhenjie Lian, Siqin Li, Xianliang Wang, Xianhan Zeng, Wenhui Chen, Jing Dai, QIANG FU, Yang Wei, Lanxiao Huang, Wei Liu

Keywords: deep reinforcement learning, human enhancement, human-agent collaboration, game playing

TL;DR: We propose the Reinforcement Learning from Human Gain (RLHG) method for effectively enhancing human goal-achievement abilities in collaborative tasks with known human goals.

Abstract: In human-agent collaboration tasks, it is essential to explore ways for developing assistive agents that can improve humans' performance in achieving their goals. In this paper, we propose the Reinforcement Learning from Human Gain (RLHG) approach, designed to effectively enhance human goal-achievement abilities in collaborative tasks with known human goals. Firstly, the RLHG method trains a value network to estimate primitive human performance in achieving goals. Subsequently, the RLHG method trains a gain network to estimate the positive gain of human performance in achieving goals when subjected to effective enhancement, in comparison to the primitive performance. The positive gains are used for guiding the agent to learn effective enhancement behaviors. Distinct from directly integrating human goal rewards into optimization objectives, the RLHG method largely mitigates the human-agent credit assignment issues encountered by agents in learning to enhance humans. We evaluate the RLHG agent in the widely popular Multi-player Online Battle Arena (MOBA) game, Honor of Kings, by conducting experiments in both simulated environments and real-world human-agent tests. Experimental results demonstrate that the RLHG agent effectively improves the goal-achievement performance of participants across varying levels.

Submission Number: 11548