Prompting Toxicity: Analyzing Biosafety Risks in Genomic Language Models

Akshay Murthy, Mengmeng Zhang, Shanmukhi Kannamangalam, Benjamin Liu, Kevin Zhu

Keywords: Genomic Language Models, Toxicity, Biosecurity, Foundation Models

TL;DR: We analyze how susceptible genomic language models are to generating toxic DNA sequences to advocate for more regulation supporting gLMs.

Abstract: Genomic language models (gLMs) have transformed biomedical research by enabling large-scale generation and analysis of DNA sequences. Evo, a powerful gLM trained across multiple species, was designed to uncover patterns that link genetic variation to traits and disease risk. However, its generative capabilities raise biosafety concerns: given minimal input, Evo can produce sequences resembling those found in harmful biological agents. In this study, we analyze Evo's susceptibility to generating toxic outputs. Using a curated dataset of experimentally validated toxic bacterial sequences, we prompt Evo with partial contexts and evaluate its completions using ToxinPred3 and ToxinPred2. While reconstruction fidelity improves with longer prompts, we observe that toxic protein predictions double in the presence of prompt context. These findings highlight a pressing need to assess and regulate the use of genomic foundation models in laboratory and clinical settings, where malicious intent can lead to harmful generation.

Submission Number: 39