Training-free linear image inverses via flows

Published: 20 Jun 2024, Last Modified: 17 Sept 2024Accepted by TMLREveryoneRevisionsBibTeXCC BY 4.0
Abstract: Solving inverse problems without any training involves using a pretrained generative model and making appropriate modifications to the generation process to avoid finetuning of the generative model. While recent methods have explored the use of diffusion models, they still require the manual tuning of many hyperparameters for different inverse problems. In this work, we propose a training-free method for solving linear inverse problems by using pretrained flow models, leveraging the simplicity and efficiency of Flow Matching models, using theoretically-justified weighting schemes, and thereby significantly reducing the amount of manual tuning. In particular, we draw inspiration from two main sources: adopting prior gradient correction methods to the flow regime, and a solver scheme based on conditional Optimal Transport paths. As pretrained diffusion models are widely accessible, we also show how to practically adapt diffusion models for our method. Empirically, our approach requires no problem-specific tuning across an extensive suite of noisy linear inverse problems on high-dimensional datasets, ImageNet-64/128 and AFHQ-256, and we observe that our flow-based method for solving inverse problems improves upon closely-related diffusion-based methods in most settings.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: Clarification about problem setup (non-blind setting) in abstract and contributions.
Video: https://cmu.zoom.us/rec/share/fF4aI575CVN6vwdeT8O21nkK3HpZyb_KXaLC3-MV-xqc1CGPfWtvsnC2xUlL33LU.QwynotqXrA5Qilg9
Assigned Action Editor: ~Grigorios_Chrysos1
Submission Number: 2361
Loading