Learning Human Objectives by Evaluating Hypothetical Behavior

Siddharth Reddy, Anca D. Dragan, Sergey Levine, Shane Legg, Jan Leike

Abstract: We seek to align agent behavior with a user’s objectives in a reinforcement learning setting with unknown dynamics, an unknown reward function, and unknown unsafe states. The user knows the rewards...