DVD-Quant: Data-free Video Diffusion Transformers Quantization

Zhiteng Li, Hanxuan Li, Junyi Wu, Kai Liu, Haotong Qin, Linghe Kong, Guihai Chen, Yulun Zhang, Xiaokang Yang

Keywords: video generation models; post-training quantization

TL;DR: DVD-Quant is a novel Data-free quantization framework for Video DiTs

Abstract:

Diffusion Transformers (DiTs) have emerged as the state-of-the-art architecture for video generation, yet their computational and memory demands hinder practical deployment. While post-training quantization (PTQ) presents a promising approach to accelerate Video DiT models, existing methods suffer from two critical limitations: (1) dependence on computation-heavy and inflexible calibration procedures, and (2) considerable performance deterioration after quantization.
To address these challenges, we propose DVD-Quant, a novel Data-free quantization framework for Video DiTs. Our approach integrates three key innovations:
(1) Bounded-init Grid Refinement (BGR) and
(2) Auto-scaling Rotated Quantization (ARQ) for calibration data-free quantization error reduction, as well as
(3) $\delta$-Guided Bit Switching ($\delta$-GBS) for adaptive bit-width allocation.
Extensive experiments across multiple video generation benchmarks demonstrate that DVD-Quant achieves an approximately 2$\times$ speedup over full-precision baselines on advanced DiT models while maintaining visual fidelity. Notably, DVD-Quant is the first to enable W4A4 PTQ for Video DiTs without compromising video quality. Code and models will be released to facilitate future research.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 11882