Detecting Sparse Colorectal Cancer Signals from Multi-Modal Cell-Free DNA Representations Using Modern Hopfield Attention

Michael Widrich, Anooj Patel, Elisabeth Rumetshofer, Peter Ulz, Kaitlyn Coil, Thomas Royce, Cheng-Ho Jimmy Lin, Richard Bourgon, Anindita Dutta

Keywords: DNA Foundation Model Embeddings, Multi-modal Representations, High Dimensional and Multi-resolution Signal, Early Cancer Detection, Clinical Diagnostics, Deep Learning for Genomics, Attention Mechanism, Multiple Instance Learning, Modern Hopfield Networks, Associative Memory, Interpretability, Liquid Biopsy, Cell-free DNA, DNA Methylation, Circulating Tumor DNA, Next-Generation Sequencing

TL;DR: We propose Fragment-Level Deep Learning (FLDL), an extreme-scale multiple instance learning framework utilizing HyenaDNA-augmented multi-modal cell-free DNA representations and Modern Hopfield attention to detect sparse colorectal cancer signals

Abstract:

Next-generation sequencing-based early cancer detection from cell-free DNA (cfDNA) presents an extreme-scale multiple instance learning challenge: identifying rare tumor signals amidst millions of instances per sample (with witness rates as low as $<$0.0001\%). It also provides a testbed for translating DNA foundation model embeddings to a clinically important supervised learning task. We propose Fragment-Level Deep Learning (FLDL), an end-to-end deep learning framework that combines multi-modal fragment features, including a
HyenaDNA-derived sequence embedding, with Modern Hopfield Networks to perform dense associative retrieval over the massive cfDNA instance space. Using held-out real-world clinical and challenging contrived test sets, we compare FLDL’s performance to a state-of-the-art machine learning model and to a deep learning model without attention (max pooling). Our results demonstrate that only the attention-based FLDL model outperforms the machine learning model, in spite of a modest training set size ($n = 4,394$). FLDL also scales effectively with sample size and with the number of instances per sample while offering useful biological insights via attention weights and learned sample representations. This work establishes a new frontier for foundation-model-augmented cfDNA representations and highly scalable attention-based deep learning in clinical cfDNA diagnostics.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 32