Benchmarking diffusion models for predicting perturbed cellular responses

Zijun Song, Changwen Zheng, Jiangmeng Li, Linhai Xie, Yujia Xiang

Keywords: Perturbation prediction, Diffusion models, Benchmarking, Single-cell transcriptomics, Generative models, Deep learning, Computational biology

TL;DR: This benchmark reveals that the VAE-based model scGen generally outperforms diffusion models in predicting cellular perturbation responses, although diffusion models show superior robustness in noisy data conditions.

Abstract: Predicting cellular responses to perturbations is important for understanding biological processes. Although several benchmarks exist, the performance of diffusion models in perturbation prediction has not been systematically studied. Here, we compare the VAE-based model scGen against five diffusion models to assess their suitability for perturbation prediction. Our benchmark covers the prediction of known perturbations, unseen conditions, and stress-test evaluations. The results show that scGen outperforms diffusion models in most settings, while diffusion models demonstrate robustness to noisy data. We also find that encoder design strongly influences model stability and that evaluation metrics can lead to different conclusions. Diffusion models capture the responses of differentially expressed genes, but perform less well on non-DE genes. Overall, our study provides insights into the strengths and limitations of diffusion models, informing their future application in perturbation prediction.

Submission Number: 18