SHIVER: Somatic Hypermutation Informed Vocabulary Encoder Representations

ICML 2025 Workshop FM4LS Submission20 Authors

Keywords: protein language models, representation learning

TL;DR: Mutation-Aware BERT-style language model for human memory B-Cell (MBC) receptor sequences.

Abstract: Somatic hypermutations (SHMs) acquired during affinity maturation of memory B cell receptors (mBCRs) carry important immunological signals, but remain challenging for protein language models (PLMs) to capture effectively. We introduce SHIVER, a mutation-aware antibody language model that treats each amino acid substitution as a distinct token, allowing the model to directly encode the context-dependent impact of SHMs. Trained on paired heavy and light chain sequences from human mBCR repertoires, SHIVER incorporates a tailored vocabulary, mutation subsampling strategy, and partial masking scheme to better model the dynamics of affinity maturation. We evaluate SHIVER on the task of predicting mBCR binding to influenza antigens and find that it outperforms both general and antibody-specific PLMs using a simple logistic head. Our results suggest that explicitly modeling SHMs improves biological relevance and generalization of learned representations.

Submission Number: 20