pcaGAN: Improving Posterior-Sampling cGANs via Principal Component Regularization

Matthew C Bendel, Rizwan Ahmad, Philip Schniter

Keywords: Image recovery, inverse problems, MRI, posterior sampling, GAN

TL;DR: For image-recovery problems, we propose a fast and accurate posterior sampler by regularizing a cGAN to enforce correctness in the K principal components of the posterior covariance matrix, and in both the trace-covariance and conditional mean.

Abstract: In ill-posed imaging inverse problems, there can exist many hypotheses that fit both the observed measurements and prior knowledge of the true image. Rather than returning just one hypothesis of that image, posterior samplers aim to explore the full solution space by generating many probable hypotheses, which can later be used to quantify uncertainty or construct recoveries that appropriately navigate the perception/distortion trade-off. In this work, we propose a fast and accurate posterior-sampling conditional generative adversarial network (cGAN) that, through a novel form of regularization, aims for correctness in the posterior mean as well as the trace and K principal components of the posterior covariance matrix. Numerical experiments demonstrate that our method outperforms competitors in a wide range of ill-posed imaging inverse problems.

Primary Area: Machine vision

Flagged For Ethics Review: true

Submission Number: 18974