Sequence-to-sequence modeling for Temporal Reconstruction of Cellular Events
Keywords: self-supervised learning, single cell RNAseq, generative modeling, temporal sequence modeling
TL;DR: We propose a generative model for temporal single cell generation.
Abstract: Single-cell omics technologies capture molecular snapshots of cells, while most
biological processes unfold over time. Accurately predicting single-cell gene ex-
pression at unmeasured time points enhances our understanding of these processes,
reducing costs and experimental effort by enabling the interpolation and extrap-
olation of observed data. This helps study continuous development, response to
perturbations, and disease progression. To address this problem, we propose an
encoder-decoder transformer architecture for Temporal Reconstruction of Cellular
Events (TRACE). TRACE models gene expression generation as a sequence-to-
sequence generation task by learning to transform a sequence of genes from a
source condition (e.g., previous time) into a sequence of genes in a target condition
(e.g., next time point). TRACE decoder learns to generate gene tokens of the target
condition by iteratively unmaking tokens in the target sequence, overcoming the dis-
cordance between autoregressive modeling and the non-sequential nature of gene
expression data. We evaluate TRACE both quantitatively and qualitatively on three
datasets, covering a range of tasks and biological scenarios. TRACE outperforms
existing models in generalizing across in-distribution and out-of-distribution tasks
for temporal prediction. Furthermore, we demonstrate the biological relevance of
the cell embeddings learned by TRACE by delineating activation-dependent cell
stages in immune cells, measured across multiple time points. Our findings suggest
that TRACE can enhance in silico hypothesis generation, improving our under-
standing and prediction of cellular changes over time. This ultimately facilitates
disease understanding and supports the design of cost-effective experiments for
biological discovery.
Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 11664
Loading