Evolution Guided Generative Flow Networks

Zarif Ikram, Ling Pan, Dianbo Liu

Track: Machine learning: computational method and/or computational results

Keywords: Machine learning, Biological sequence design, Probabilistic inference, GFlowNets, Evolutionary algorithms, Reward sparsity

TL;DR: A powerful method for the GFlowNets training using Evolutionary algorithms that enhances its robustness to long trajectory and sparse rewards

Abstract: Generative Flow Networks (GFlowNets) are a family of probabilistic generative models that learn to sample compositional objects proportional to their rewards. One big challenge of GFlowNets is training them effectively when dealing with long time horizons and sparse rewards. To address this, we propose Evolution guided generative flow networks (EGFN), a simple but powerful augmentation to the GFlowNets training using Evolutionary algorithms (EA). Our method can work on top of any GFlowNets training objective, by training a set of agent parameters using EA, storing the resulting trajectories in the prioritized replay buffer, and training the GFlowNets agent using the stored trajectories. We present a thorough investigation over a wide range of toy and real-world benchmark tasks showing the effectiveness of our method in handling long trajectories and sparse rewards.

Submission Number: 20