Missing Data Imputation by Reducing Mutual Information with Rectified Flows

Jiahao Yu, Qizhen Ying, Leyang Wang, Ziyue Jiang, Song Liu

Keywords: missing data, imputation, rectified flow, generative model

TL;DR: We propose a missing data imputation method using rectified flow by following mutual information minimization criterion.

Abstract:

This paper introduces a novel iterative method for missing data imputation that sequentially reduces the mutual information between data and the corresponding missingness mask. Inspired by GAN-based approaches that train generators to decrease the predictability of missingness patterns, our method explicitly targets this reduction in mutual information.
Specifically, our algorithm iteratively minimizes the KL divergence between the joint distribution of the imputed data and missingness mask, and the product of their marginals from the previous iteration.
We show that the optimal imputation under this framework can be achieved by solving an ODE whose velocity field minimizes a rectified flow training objective.
We further illustrate that some existing imputation techniques can be interpreted as approximate special cases of our mutual-information-reducing framework. Comprehensive experiments on synthetic and real-world datasets validate the efficacy of our proposed approach, demonstrating its superior imputation performance. Our implementation is available at \url{https://github.com/yujhml/MIRI-Imputation}.

Primary Area: General machine learning (supervised, unsupervised, online, active, etc.)

Submission Number: 24289