DGFN: Double Generative Flow Networks

Elaine Lau, Nikhil Murali Vemgal, Doina Precup, Emmanuel Bengio

Keywords: GFlowNets, Reinforcement learning, Generative modelling, Drug discovery, Q-learning

TL;DR: Introducing a new technique that can be applied in Generative Flow Networks (GFlowNets) to enhance exploration in sparse reward domains.

Abstract: Deep learning is emerging as an effective tool in drug discovery, with potential applications in both predictive and generative models. Generative Flow Networks (GFlowNets/GFNs) are a recently introduced method recognized for the ability to generate diverse candidates, in particular in small molecule generation tasks. In this work, we introduce double GFlowNets (DGFNs). Drawing inspiration from reinforcement learning and Double Deep Q-Learning, we introduce a target network used to sample trajectories, while updating the main network with these sampled trajectories. Empirical results confirm that DGFNs effectively enhance exploration in sparse reward domains and high-dimensional state spaces, both challenging aspects of de-novo design in drug discovery.

Submission Number: 29