Improved Exploration in GFlownets via Enhanced Epistemic Neural Networks

Sajan Muhammad, Salem Lahlou

Track: AI for Science

Keywords: GFlowNets, joint predictions, epistemic neural network, ENN-GFN-Enhanced, exploration

TL;DR: To the best of our knowledge, this is the first work to integrate Epistemic Neural Networks with GFlowNets to enable uncertainty-driven exploration.

Abstract:

Efficiently identifying the right trajectories for training remains an open problem
in GFlowNets. To address this, it is essential to prioritize exploration in regions of
the state space where the reward distribution has not been sufficiently learned. This
calls for uncertainty-driven exploration, in other words, the agent should be aware
of what it does not know. This attribute can be measured by joint predictions, which
are particularly important for combinatorial and sequential decision problems. In
this research, we integrate epistemic neural networks (ENN) with the conventional
architecture of GFlowNets to enable more efficient joint predictions and better
uncertainty quantification, thereby improving exploration and the identification of
optimal trajectories. Our proposed algorithm, ENN-GFN-Enhanced, is compared
to the baseline method in GFlownets and evaluated in grid environments and
structured sequence generation in various settings, demonstrating both its efficacy
and efficiency.

Serve As Reviewer: ~Salem_Lahlou3, ~Sajan_Muhammad1

Submission Number: 105