EpiAttend: A transformer model of gene regulation combining single cell epigenomes with DNA sequence

Russell Li, Heng Xu, Eran A Mukamel

Keywords: Epigenome, Attention, Transformer, Single cell

TL;DR: A neural network model combining DNA sequence and cell type-specific epigenetic tracks over large (115kb) windows can learn to predict gene expression.

Abstract: Understanding cell type specific gene expression regulation requires models that integrate information across long genomic distances, such as enhancer-gene interactions spanning many tens of kilobases. Neural network models using deep convolutions and self-attention have achieved highly accurate prediction of cell type specific gene expression and other functional genomics measurements based on DNA sequence in local windows\citep{avsec,basenji2}. By contrast, leading models for linking enhancers with target genes take advantage of cell type specific epigenomes\citep{abc}. Here, we propose a framework for combining DNA sequence with epigenetic data from single cell sequencing within a neural network to predict cell type specific functional readouts such as mRNA expression. This approach has the potential to identify long-range gene-regulatory interactions, linking enhancers with genes based on both the epigenome and DNA sequence binding motifs.