Seeing the Unseen: How EMoE Unveils Bias in Text-to-Image Diffusion Models

Lucas Berry, Axel Brando, Wei-Di Chang, Juan Camilo Gamboa Higuera, David Meger

Keywords: Uncertainty Quantification, Text-to-Image Modeling, Ensembles

TL;DR: This paper presents EMoE, a framework for efficient epistemic uncertainty estimation in text-to-image diffusion models, identifying biases in linguistic representation while improving image quality assessment

Abstract: Estimating uncertainty in text-to-image diffusion models is challenging due to their large parameter counts (often exceeding 100 million) and operation in complex, high-dimensional spaces with virtually infinite input possibilities. In this paper, we propose EMoE, a novel framework for efficiently estimating epistemic uncertainty in diffusion models. EMoE leverages pre-trained networks without requiring additional training, enabling direct uncertainty estimation from a prompt. We introduce a novel latent space within the diffusion process that captures model uncertainty better during the first denoising step than existing methods. Experimental results on the COCO dataset demonstrate EMoE's effectiveness, showing a strong correlation between uncertainty and image quality. Additionally, EMoE identifies under-sampled languages and regions with higher uncertainty, revealing hidden biases related to linguistic representation. This capability demonstrates the relevance of EMoE as a tool for addressing fairness and accountability in AI-generated content.

Primary Area: foundation or frontier models, including LLMs

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 3396