NucleoBench: A Large-Scale Benchmark of Neural Nucleic Acid Design Algorithms

Joel Shor, Erik Strand, Cory Y McLean

Keywords: Nucleic Acid Design, Sequence Design, benchmark, bio

TL;DR: NucleoBench is a new, large-scale benchmark that compares various nucleic acid design algorithms across diverse biological tasks, proposing AdaBeam as a superior-performing hybrid algorithm that addresses limitations of existing methods.

Abstract: One outstanding open problem with high therapeutic value is how to design nucleic acid sequences with specific properties. Even just the 5' UTR sequence admits $2 \times 10^{120}$ possibilities, making exhaustive exploration impossible. Although the field has focused on developing high-quality predictive models, techniques for generating sequences with desired properties are often not well benchmarked. Lack of benchmarking hinders production of the best molecules from high-quality models, and slows improvements to design algorithms. In this work, we performed the first large-scale comparison of modern sequence design algorithms across 16 biological tasks (such as transcription factor binding and gene expression) and 9 design algorithms. Our benchmark, NucleoBench, compares design algorithms on the same tasks and start sequences across more than 400K experiments, allowing us to derive unique modeling insights on the importance of using gradient information, the role of randomness, scaling properties, and reasonable starting hyperparameters on new problems. We use these insights to present a novel hybrid design algorithm, AdaBeam, that outperforms existing algorithms on 11 of 16 tasks and demonstrates superior scaling properties on long sequences and large predictors. Our benchmark and algorithms are freely \href{https://github.com/move37-labs/nucleobench}{available online}.

Submission Number: 95