Reward tampering problems and solutions in reinforcement learning: a causal influence diagram perspective

Tom Everitt, Marcus Hutter, Ramana Kumar, Victoria Krakovna