[Re] Bandit Theory and Thompson Sampling-guided Directed Evolution for Sequence Optimization

Luka Žontar

Keywords: Optimization, bandit learning, Thompson sampling, regret, evolution, reproducibility

TL;DR: This paper is a reproduction of the Bandit Theory and Thompson Sampling-guided Directed Evolution for Sequence Optimization paper by Yuan. We confirm the main claims and explain what had to be implemented in order to reproduce this work.

Abstract:

The paper presents a novel DE approach using Thompson Sampling and Bandit theory, TS-DE. Our reproducibility study aims to confirm the 5 main claims of the original paper, including sublinear Bayesian regret, improved performance compared to basic DE, robustness to mutation rate changes, initial diversification, and the concentration of the population to the optimal value in later iterations, and iterative distribution shift towards optimal population fitness. Finally, we provide a reproducible environment to support the main claims of the original paper along with the source code of the proposed approach and all experiments, comprehensive documentation, and unit tests.

No code was available beforehand for this article, thus we re-implemented the proposed approach by meticulously following the comprehensive explanations of the process in the original article. The experiments were run on a personal computer.

We managed to reproduce all the experiments supporting the main claims of the original article. Additionally, we add uncertainty quantification to the results as we believe this is a crucial part to confirm any of the claims. Finally, we present the exploration-exploitation trade-off experiment in a more robust manner leveraging the nucleotide diversity metric to gain additional insight into how the proposed algorithm works.

With comprehensive explanations in the original article, it was relatively easy to rewrite the main concepts from pseudo-code to Python code. Experiments were clearly explained with all the necessary hyperparameters.

Since no source code was available, every detail missing from the original article resulted in additional research and trial and error experimentation. To conclude, we recommend several improvements to the authors of the studied paper that could additionally improve the quality of their outstanding contribution. Some of the recommendations include better explanations of how the optimal solution is calculated, on which population the PCA is fitted, how to use $\theta^*$ and $\Tilde{\theta}$, and lastly improved documentation on how the basic DE was implemented.

We contacted the original authors on multiple occasions during the development of our reproducibility study but got no response.

Paper Url: https://neurips.cc/Conferences/2022/ScheduleMultitrack?event=53784

Paper Review Url: https://openreview.net/forum?id=drVX99PekKf

Paper Venue: NeurIPS 2022

Confirmation: The report follows the ReScience latex style guides as in the Reproducibility Report Template (https://paperswithcode.com/rc2022/registration)., The report contains the Reproducibility Summary in the first page.

Journal: ReScience Volume 9 Issue 2 Article 7

Doi: https://www.doi.org/10.5281/zenodo.8173664

Code: https://archive.softwareheritage.org/swh:1:dir:7cffa7d64bb51bac56c43f657e2e3746a2539915