# $\alpha$-GFN: Generalized Mixing in GFlowNets for Better Exploration-Exploitation Trade-off
The official implementation for the paper $\alpha$-GFN: Generalized Mixing in GFlowNets for Better Exploration-Exploitation Trade-off.


## Experiments

### Set Generation
We implement our method based on the  [open-source code](https://github.com/ling-pan/FL-GFN/tree/main) of the paper [Better Training of GFlowNets with Local Credit and Incomplete Trajectories](https://openreview.net/pdf?id=beHp3L9KXc). Please refer to that for detailed documentation. We add the requirement for wandb installation.

Our implementation of $\alpha$-GFNs for this experiment is in the file fl-gfn/set/gflownet.py

Here's an example for experiments on small sets and $\alpha=0.5$:
```
cd fl-gfn/set/
```
- $\alpha$-DB
```
python gflownet.py --method db_gfn --size small --mode_threshold 0.25 --sampling_seed 0 --alpha 0.5 --use_alpha_scheduler 1 --alpha_warm_frac 0.9 --use_grad_clip 0 --exp_weight 0.05 --use_exp_weight_decay 1
```
- $\alpha$-TB
```
python gflownet.py --method tb_gfn --size small --mode_threshold 0.25 --sampling_seed 0 --alpha 0.5 --use_alpha_scheduler 1 --alpha_warm_frac 0.9 --use_grad_clip 0 --exp_weight 0.05 --use_exp_weight_decay 1

```
- $\alpha$-FL-DB
```
python gflownet.py --method db_gfn --fl 1 --size small --mode_threshold 0.25 --sampling_seed 0 --alpha 0.5 --use_alpha_scheduler 1 --alpha_warm_frac 0.9 --use_grad_clip 0 --exp_weight 0.05 --use_exp_weight_decay 1

```

### Bit Sequence Generation
We implement our method based on the  [open-source code](https://github.com/d-tiapkin/gflownet-rl/tree/main) of the paper [Generative Flow Networks as Entropy-Regularized RL](https://arxiv.org/abs/2310.12934). Please refer to that for detailed documentation. We add the requirement for wandb installation.

Our implementation of $\alpha$-GFNs for this experiment is in the file gflownet-rl/bitseq/run.py

Here's an example for experiments with the setting $k=8$ and $\alpha=0.5$:
```
cd gflownet-rl/bitseq
```
- $\alpha$-DB
```
python bitseq/run.py --objective db --k 8 --learning_rate 0.002 --alpha 0.5 --sampling_seed 0 --grad_clip_norm 20 --eval_every 500 --n_test_steps 1000
```
- $\alpha$-TB
```
python bitseq/run.py --objective tb --k 8 --learning_rate 0.002 --alpha 0.5 --sampling_seed 0 --grad_clip_norm 20 --eval_every 500 --n_test_steps 1000
```
- $\alpha$-SubTB
```
python bitseq/run.py --objective subtb --k 8 --learning_rate 0.002 --alpha 0.5 --sampling_seed 0 --grad_clip_norm 20 --eval_every 500 --n_test_steps 1000
```

### Molecule Generation
We implement our method with DB, SubTB and TB based on the  [open-source code](https://github.com/d-tiapkin/gflownet-rl/tree/main) of the paper [Generative Flow Networks as Entropy-Regularized RL](https://arxiv.org/abs/2310.12934), with FL-DB and FL-SubTB based on the implementation in [open-source code](https://github.com/ling-pan/FL-GFN/tree/main) of the paper [Better Training of GFlowNets with Local Credit and Incomplete Trajectories](https://openreview.net/pdf?id=beHp3L9KXc). Please refer to these two codebases for detailed documentation. We add the requirement for wandb installation.

Our implementation of $\alpha$-GFNs for this experiment is in the file gflownet-rl/mols/gflownet.py

Here's an example for experiments with $\alpha=0.5$:
```
cd gflownet-rl/mols/
```
- $\alpha$-DB
```
python gflownet.py --objective db --alpha 0.5 --sampling_seed 0 --random_action_prob 0.05 --clip_grad 0
```
- $\alpha$-SubTB
```
python gflownet.py --objective subtb --alpha 0.5 --sampling_seed 0 --random_action_prob 0.05 --clip_grad 0
```

- $\alpha$-TB
```
python gflownet.py --objective tb --alpha 0.5 --sampling_seed 0 --random_action_prob 0.05 --clip_grad 0
```

- $\alpha$-FL-DB
```
python gflownet.py --objective db --fl 1 --alpha 0.5 --sampling_seed 0 --random_action_prob 0.05 --clip_grad 0
```
- $\alpha$-FL-SubTB
```
python gflownet.py --objective subtb --fl 1 --alpha 0.5 --sampling_seed 0 --random_action_prob 0.05 --clip_grad 0
```