Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals

Rohin Shah, Vikrant Varma, Ramana Kumar, Mary Phuong, Victoria Krakovna, Jonathan Uesato, Zac Kenton