POLYOMINOGEN: A Controlled Testbed for Understanding Memorization and Compositional Generalization in Conditional Diffusion Models

Aishi Huang

Keywords: Compositional generalization, diffusion models, controlled benchmark, generative model evaluation, few-shot transfer

Abstract: Deep generative models can produce samples that appear novel, yet in natural-image domains it is often unclear whether such samples reflect memorization, interpolation, or systematic generalization beyond the observed training support. We introduce POLYOMINOGEN, a controlled testbed for studying memorization and compositional generalization in conditional diffusion models. Each image is rendered from an exact symbolic tuple specifying shape, color, orientation, and position, enabling train–test splits in which specific attribute combinations or geometric transformations are withheld by construction. Across three U-Net DDPM runs, generated objects remain largely valid under held-out conditions, while tuple accuracy drops substantially, particularly for withheld geometric transformations. Pretrained initialization also outperforms training from scratch across all tested few-shot settings, and the novel-valid held-out metric remains stable across a range of nearest-neighbor thresholds. POLYOMINOGEN provides a lightweight diagnostic framework for probing when generative models memorize, when they generalize, and when they fail under controlled distribution shift.

Submission Number: 181