ZoomLDM: Latent Diffusion Model for multi-scale image generation

Srikar Yellapragada, Alexandros Graikos, Kostas Triaridis, Prateek Prasanna, Rajarsi R. Gupta, Joel Saltz, Dimitris Samaras

Keywords: Vision and Learning, AI for Health

Abstract:

Diffusion models have revolutionized image generation, yet several challenges restrict their application to large-image domains, such as digital pathology and satellite imagery. Given that it is infeasible to directly train a model on 'whole' images from domains with potential gigapixel sizes, diffusion-based generative methods have focused on synthesizing small, fixed-size patches extracted from these images. However, generating small patches has limited applicability since patch-based models fail to capture the global structures and wider context of large images, which can be crucial for synthesizing (semantically) accurate samples. To overcome this limitation, we present ZoomLDM, a diffusion model tailored for generating images across multiple scales. Central to our approach is a novel magnification-aware conditioning mechanism that utilizes self-supervised learning (SSL) embeddings and allows the diffusion model to synthesize images at different 'zoom' levels, i.e., fixed-size patches extracted from large images at varying scales. ZoomLDM synthesizes coherent histopathology images that remain contextually accurate and detailed at different zoom levels, achieving state-of-the-art image generation quality across all scales and excelling in the data-scarce setting of generating thumbnails of entire large images. The multi-scale nature of ZoomLDM unlocks additional capabilities in large image generation, enabling computationally tractable and globally coherent image synthesis up to $4096 \times 4096$ pixels and $4\times$ super-resolution. Additionally, multi-scale features extracted from ZoomLDM are highly effective in multiple instance learning experiments. Code is available at https://github.com/cvlab-stonybrook/ZoomLDM.

**Accepted to The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2025**

Yellapragada, Srikar, et al. "ZoomLDM: Latent Diffusion Model for multi-scale image generation." arXiv preprint arXiv:2411.16969 (2024).

**Links:** [CVPR](https://cvpr.thecvf.com/virtual/2025/poster/35153), [pdf](https://arxiv.org/pdf/2411.16969)

**Keywords:** Vision and Learning, AI for Health

Submission Number: 130