Dequantified Diffusion-Schrödinger Bridge for Density Ratio Estimation

Wei Chen, Shigui Li, Jiacheng Li, Junmei Yang, John Paisley, Delu Zeng

TL;DR: Introduce D³RE, a dequantified framework combining diffusion/Schrödinger bridge interpolants with optimal transport to resolve the density-chasm problem. Theoretically guarantees lower asymptotic error.

Abstract:

Density ratio estimation is fundamental to tasks involving f-divergences, yet existing methods often fail under significantly different distributions or inadequately overlapping supports --- the density-chasm and the support-chasm problems.
Additionally, prior approaches yield divergent time scores near boundaries, leading to instability.
We design $\textbf{D}^3\textbf{RE}$, a unified framework for robust, stable and efficient density ratio estimation.
We propose the dequantified diffusion bridge interpolant (DDBI), which expands support coverage and stabilizes time scores via diffusion bridges and Gaussian dequantization.
Building on DDBI, the proposed dequantified Schr{\"o}dinger bridge interpolant (DSBI) incorporates optimal transport to solve the Schr{\"o}dinger bridge problem, enhancing accuracy and efficiency.
Our method offers uniform approximation and bounded time scores in theory, and outperforms baselines empirically in mutual information and density estimation tasks.

Lay Summary:

In machine learning, it's often important to understand how two probability distributions differ — for example, when comparing real-world data with a model's predictions. A key tool for this is density ratio estimation, which compares the likelihoods of data points under two different distributions. However, existing methods often fail when the distributions are very different or don’t overlap well, leading to inaccurate or unstable results.

We developed a new method called D³RE that can estimate these differences more robustly, stably, and efficiently. At the heart of our approach is a technique that uses noise and simulated particle movement — like watching smoke spread in the air — to better connect and compare the datasets. We also incorporate ideas from optimal transport, which helps find the most efficient way to shift one distribution to match another.

Our method not only avoids the pitfalls of previous approaches but also achieves better accuracy on tasks like estimating mutual information and learning probability models. This makes it a valuable tool for improving reliability in many machine learning applications.

Link To Code: https://github.com/Hoemr/Dequantified-Diffusion-Bridge-Density-Ratio-Estimation.git

Primary Area: Probabilistic Methods->Everything Else

Keywords: diffusion-bridge, optimal transport rearrangement, asymptotic estimation, density-ratio estimation

Submission Number: 9490