On Statistical Rates of Conditional Diffusion Transformer: Approximation and Estimation

ICLR 2025 Conference Submission5826 Authors

Keywords: Conditional Diffusion Transformer, Statistical Rates, Approximation, Estimation

TL;DR: We investigate the approximation and estimation rates of Conditional Diffusion Transformers (DiTs) with classifier-free guidance.

Abstract:

We investigate the approximation and estimation rates of conditional diffusion transformers (DiTs) with classifier-free guidance.
We present a comprehensive analysis for ``in-context'' conditional DiTs under four common data assumptions.
We show that both conditional DiTs and their latent variants lead to the minimax optimality of unconditional DiTs under identified settings.
Specifically, we discretize the input domains into infinitesimal grids and then perform a term-by-term Taylor expansion on the conditional diffusion score function under Hölder smooth data assumption.
This enables fine-grained use of transformers' universal approximation through a more detailed piecewise constant approximation, and hence obtains tighter bounds.
Additionally, we extend our analysis to the latent setting under the linear latent subspace assumption.
We not only show that latent conditional DiTs achieve lower bounds than conditional DiTs both in approximation and estimation, but also show the minimax optimality of latent unconditional DiTs.
Our findings establish statistical limits for conditional and unconditional DiTs, and offer
practical guidance toward developing more efficient and accurate DiT models.

Primary Area: generative models

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5826