Multimodal single cell data integration challenge: results and lessons learned

Christopher Lance, Tianyu Liu, Fabian J Theis

Abstract:

Biology has become a data-intensive science. Recent technological advances in singlecell genomics have enabled the measurement of multiple facets of cellular state, producing
datasets with millions of single-cell observations. While these data hold great promise
for understanding molecular mechanisms in health and disease, analysis challenges arising
from sparsity, technical and biological variability, and high dimensionality of the data
hinder the derivation of such mechanistic insights. To promote the innovation of algorithms
for analysis of multimodal single-cell data, we organized a competition at NeurIPS 2021
applying the Common Task Framework to multimodal single-cell data integration. For
this competition we generated the first multimodal benchmarking dataset for single-cell
biology and defined three tasks in this domain: prediction of missing modalities, aligning
modalities, and learning a joint representation across modalities. We further specified
evaluation metrics and developed a cloud-based algorithm evaluation pipeline. Using this
setup, 280 competitors submitted over 2600 proposed solutions within a 3 month period,
showcasing substantial innovation especially in the modality alignment task. Here, we
present the results, describe trends of well performing approaches, and discuss challenges
associated with running the competition.