Boosted GFlowNets: Improving Exploration via Sequential Learning

Pedro Dall'Antonia, Tiago Silva, Daniel Augusto de Souza, César Lincoln Mattos, Diego Mesquita

TL;DR: We propose a method to sequentially train and combine multiple GFlowNets, where each new model optimizes a residual reward function that accounts for reward captured so far by the existing ensemble, thereby focusing on missed modes.

Abstract:

Generative Flow Networks (GFlowNets) are a class of powerful samplers for compositional objects that, by design, sample proportionally to any given non-negative reward. In practice, however, GFlowNets often fail to cover multiple high-reward regions, suffering from limited exploration of the reward landscape.

We propose Boosted GFlowNets, a method that sequentially trains and combines multiple GFlowNets, in which each new model optimizes a residual reward function that accounts for the reward captured so far by the existing ensemble, thereby focusing on missed modes.

Empirically, we demonstrate that our method enhances exploration and sample diversity on synthetic and real-world multimodal distributions. Furthermore, under mild assumptions, we establish a monotone non-degradation guarantee for the learned distribution: adding boosting rounds cannot worsen it and typically improves it, making the method broadly applicable.

Submission Number: 987