Learning to Give Checkable Answers with Prover-Verifier Games

Cem Anil, Guodong Zhang, Yuhuai Wu, Roger Baker Grosse

Keywords: AI Safety, verifiable learning, robustness, adversarial learning, proof systems

Abstract: Our ability to know when to trust the decisions made by machine learning systems has not kept up with the staggering improvements in their performance, limiting their applicability in high-stakes applications. We propose Prover-Verifier Games (PVGs), a game-theoretic framework to encourage neural networks to solve decision problems in a verifiable manner. The PVG consists of two learners with competing objectives: a trusted verifier network tries to choose the correct answer, and a more powerful but untrusted prover network attempts to persuade the verifier of a particular answer, regardless of its correctness. The goal is for a reliable justification protocol to emerge from this game. We analyze several variants of the basic framework, including both simultaneous and sequential games, and narrow the space down to a subset of games which provably have the desired equilibria. We then develop practical instantiations of the PVG for several algorithmic tasks, and show that in practice, the verifier is able to receive useful and reliable information from an untrusted prover. Importantly, the protocol still works even when the verifier is frozen and the prover's message is directly optimized to convince the verifier.

One-sentence Summary: We propose Prover Verifier Networks, a game-theoretic framework inspired by interactive proof systems to encourage neural networks to solve decision problems in a verifiable manner.