Flexible Conditional Generation with Stochastically Factorized Autoregressive Models
Abstract: Deep autoregressive generative models have demonstrated promising results in unconditional generation tasks for structured data, such as images. However, their effectiveness in conditional generation remains relatively underexplored. Recent autoregressive diffusion models efficiently amortize across all possible generation orders by defining uniform permutation orders or Markov chains with absorbing states, allowing them to parameterize any conditional distribution between data elements. Despite this flexibility, these models often struggle to conditionally generate content accurately during testing if the masking mechanism deviates from training assumptions. To address this limitation, we propose a novel deep generative model that leverages intrinsic data properties alongside self-supervision principles. Our approach extends established autoregressive frameworks by probabilistically modeling per-element generation as a mixture of semi-supervised mechanisms. This design provides a robust framework for conditional generation across diverse masking patterns. Furthermore, we hypothesize that the ability to model any conditional distribution makes these models particularly well-suited for data acquisition tasks, where collecting data to maximize predictive accuracy is critical. We propose a novel method for active data acquisition using autoregressive diffusion models, demonstrating promising results. Experimental evaluations show significant improvements in both simplicity and accuracy for conditional generation tasks, outperforming conventional methods that rely on random permutations or simultaneous generation of all dimensions
Submission Length: Regular submission (no more than 12 pages of main content)
Assigned Action Editor: ~Hsuan-Tien_Lin1
Submission Number: 3989
Loading