Quit While You’re Ahead: Sequential Training Can Harm Simulation-Based Inference

Kai Samaroo, Yunyi Shen, Robin J. Ryder, Tamara Broderick

Abstract: Simulation-based inference (SBI) is widely used to infer parameters of simulators whose likelihood functions are intractable. Recent work has suggested that sequential, multi-round training can improve upon a single round. While these conclusions are supported by evaluations on benchmark tasks with at most 20 parameters, simulators of practical interest can exhibit parameter dimensions exceeding 100. In the present work, we systematically study the performance of both single-round and sequential SBI methods as parameter dimension increases to 100. We show empirically that sequential methods often perform worse than single-round methods in dimensions higher than 20.

Keywords: simulation-based inference, likelihood-free inference, neural posterior estimation, neural likelihood estimation, neural ratio estimation

TLDR: We show empirically that sequential SBI methods often perform worse than single-round methods in parameter dimensions higher than 20.

Submission Number: 18