TCG: Taming CFG for Flow Matching Models via Moment Matching and Adaptive Clipping

Abstract

Classifier-free guidance (CFG) is a fundamental technique for flow-based models, significantly enhancing visual quality and prompt adherence. However, the guidance scale is typically tuned empirically due to instability at higher values, which often induces visual artifacts and mode collapse. This paper investigates the underlying mechanisms driving this instability and proposes an effective solution. Our analysis reveals that high CFG scales induce a detrimental distribution shift in the velocity prediction, damaging the generation fidelity. To address this, we introduce TCG, a novel plug-and-play method comprising two key components: (1) Moment Matching (MM), which stabilizes the velocity distribution by aligning its first two moments (mean and variance), thereby preventing mode collapse; and (2) Adaptive Clipping (AdapC), which dynamically constrains the guidance update term from both temporal and spatial perspectives to ensure smooth and stable sampling. As a result, our method enables robust and high-quality generation across a wide range of guidance scales. Extensive experiments on diverse text-to-image and text-to-video benchmarks validate that our method outperforms both standard CFG and its state-of-the-art variants.

Comparisons of Text-to-Image Generation

Comparisons of Text-to-Video Generation

Wan2.2 5B Visualization

A couple in formal evening wear going home get caught in a heavy downpour with umbrellas, animated style
A person is doing laundry
A person is driving car
A person is playing harp
A person is push up
A person is sharpening knives
A person is washing hands
A steam train moving on a mountainside
Motion colour drop in water, ink swirling in water, colourful ink in water, abstraction fancy dream cloud of ink.
a dog running happily
a sheep taking a peaceful walk

Wan2.2 A14B Visualization

An astronaut flying in space, in super slow motion
A person is shooting goal (soccer)
a cat and a dog
a donut and a suitcase
a giraffe running to join a herd of its kind
a horse
a shark is swimming in the ocean, surrealism style
a white cat
a wine glass and a chair
desert