Unit-level surprise in neural networks

Cian Eastwood, Ian Mason, Christopher K. I. Williams

Abstract: To adapt to changes in real-world data distributions, neural networks must update their parameters. We argue that unit-level surprise should be useful for: (i) determining which few parameters shou...