Revisiting Block-wise Interactions of MMDiT for Training-free Improved Synthesis

Binglei Li, Mengping Yang, Zhiyu Tan, Junping Zhang, Hao Li

Keywords: Diffusion Models, Training-free Synthesis, Image Generation and Editing

TL;DR: We analyze block-wise mechanisms in MMDiT diffusion models, showing how semantics and details emerge, and propose training-free strategies that improve alignment, editing, and efficiency without quality loss.

Abstract:

Recent breakthroughs of transformer-based diffusion models, particularly with Multimodal Diffusion Transformers (MMDiT) driven models like FLUX and Qwen Image, have facilitated thrilling experiences in text-to-image generation and editing. To understand the internal mechanism of MMDiT-based models, existing methods tried to analysze the effect of specific components like positional encoding and attention layers. Yet, a comprehensive understanding of how different blocks and their interactions with textual conditions contribute to the synthesis process remains elusive. In this paper, we first develop a systematic pipeline to comprehensively investigate each block's functionality by removing, disabling and strengthening textual hidden-states at corresponding blocks. Our analysis reveals that 1) semantic information appears in earlier blocks and finer details are rendered in later blocks, 2) removing specific blocks is usually less disruptive than disabling text conditions, and 3) enhancing textual conditions in selective blocks improves semantic attributes.
Building on these observations, we further propose novel training-free strategies for improved text alignment, precise editing, and acceleration. Extensive experiments demonstrated that our method outperforms various baselines and remains flexible across text-to-image generation, image editing, and inference acceleration. Our method improves T2I-Combench from 56.92\% to 63.00\% and GenEval from 66.42\% to 71.63\% on SD3.5, without sacrificing synthesis quality. These results advance understanding of MMDiT models and provide valuable insights to unlock new possibilities for further improvements.

Primary Area: generative models

Submission Number: 5637