MissDiff: Training Diffusion Models on Tabular Data with Missing Values

Yidong Ouyang, Liyan Xie, Chongxuan Li, Guang Cheng

Primary Area: generative models

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Generative model, Tabular data, Missing data, Diffusion model

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract:

The diffusion model has shown remarkable performance in modeling data distributions and synthesizing data. However, the vanilla diffusion model requires complete or fully observed training data. Incomplete data is a common issue in various real-world applications, including healthcare and finance, particularly when dealing with tabular datasets. This work considers learning from data with missing values and generating synthetic complete data, beyond missing value imputations. The main challenge for this setting is that two-stage inference frameworks, the "impute-then-generate" pipeline or the ``generate-then-impute'' pipeline, are either biased or computationally expensive. To address this challenge, we present a unified and principled diffusion-based framework. Our method models the score of complete data distribution by denoising score matching on data with missing values. We prove that the proposed method can recover the score of the complete data distribution, and the proposed training objective serves as an upper bound for the negative likelihood of observed data. In the presence of incomplete training data, the proposed method can be used for synthetic data generation, as well as missing value imputations based on the learned generative model.
Extensive experiments on imputation tasks together with generation tasks demonstrate that our proposed framework outperforms existing state-of-the-art approaches on multiple tabular datasets.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 4832