Evolutionary Curriculum Learning for Biological Sequence Modeling

Richard Yuxuan Zhu, Kento Nishi

Keywords: sequence modeling, VAE, curriculum learning, AI for biology, variational autoencoders, protein, RNA

TL;DR: Evolutionary Curriculum Learning improves biological sequence modeling through an expanding evolutionary neighborhood training schedule, yielding performance gains in protein variant effect prediction and RNA sequence generation.

Abstract: Variational autoencoders (VAEs) trained on multiple sequence alignments (MSAs) have emerged as powerful generative models for biological sequences, with applications ranging from disease variant prediction to functional RNA design. However, standard biological VAE training treats all sequences as exchangeable, ignoring the rich evolutionary structure that organizes homologous sequences from evolutionarily close to highly divergent. We propose Evolutionary Curriculum Learning (ECL), a training strategy that exploits this structure by progressively exposing the model to sequences of increasing evolutionary distance from sampled anchors, following a power-law expansion schedule. Applied to two architecturally distinct VAE models and two biological domains—protein variant effect prediction with EVE and RNA family sequence generation with RfamGen—ECL consistently improves downstream task performance: ClinVar variant classification AUROC increases from 0.842 to 1.000 for PTEN and from 0.972 to 0.986 for p53, while mean covariance model bit scores for generated RNA sequences improve by 1.9-7.1% across three diverse RNA families. These results suggest that evolutionary distance is a broadly effective inductive bias for ordering the training curriculum for biological sequence modeling.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 214