On the Guidance of Flow Matching

Ruiqi Feng, Chenglei Yu, Wenhao Deng, Peiyan Hu, Tailin Wu

TL;DR: We introduce the first framework for general flow matching guidance, from which new guidance methods are derived and many classical guidance methods are covered as special cases.

Abstract: Flow matching has shown state-of-the-art performance in various generative tasks, ranging from image generation to decision-making, where generation under energy guidance (abbreviated as guidance in the following) is pivotal. However, the guidance of flow matching is more general than and thus substantially different from that of its predecessor, diffusion models. Therefore, the challenge in guidance for general flow matching remains largely underexplored. In this paper, we propose the first framework of general guidance for flow matching. From this framework, we derive a family of guidance techniques that can be applied to general flow matching. These include a new training-free asymptotically exact guidance, novel training losses for training-based guidance, and two classes of approximate guidance that cover classical gradient guidance methods as special cases. We theoretically investigate these different methods to give a practical guideline for choosing suitable methods in different scenarios. Experiments on synthetic datasets, image inverse problems, and offline reinforcement learning demonstrate the effectiveness of our proposed guidance methods and verify the correctness of our flow matching guidance framework. Code to reproduce the experiments can be found at https://github.com/AI4Science-WestlakeU/flow_guidance.

Lay Summary: We have developed a new method to help computers create things like images and make decisions more effectively. Our work focuses on improving how these computer programs follow instructions or "guidance" to get better results. While earlier methods used one type of approach, our method is more flexible and can handle a wider range of tasks. We created new ways for these computer programs to follow "guidance", and we carefully studied when and why to use each method. We tested our ideas in different situations, like creating images and controlling robots. Our experiments show our new methods work well, and we provide code for others to try them.

Link To Code: https://github.com/AI4Science-WestlakeU/flow_guidance

Primary Area: Deep Learning->Generative Models and Autoencoders

Keywords: flow matching, guided generation, generative modeling

Submission Number: 3557