$\alpha$-Divergence Loss Function for Neural Density Ratio Estimation

Yoshiaki Kitazawa

Keywords: density ratio estimation, variational divergence optimization, $\alpha$-divergence, Kullback–Leibler divergence, and $f$-divergence.

TL;DR: This study proposes a novel loss function for probability density estimation using a variational representation of the alpha divergence.

Abstract:

Density ratio estimation (DRE) is a fundamental machine learning technique for capturing relationships between two probability distributions. State-of-the-art DRE methods estimate the density ratio using neural networks trained with loss functions derived from variational representations of $f$-divergence.
However, existing methods face optimization challenges, such as overfitting due to lower-unbounded loss functions, biased mini-batch gradients, vanishing training loss gradients, and high sample requirements for Kullback-Leibler (KL) divergence loss functions.
To address these issues, we focus on $\alpha$-divergence, which provides a suitable variational representation of $f$-divergence.
Subsequently, a novel loss function for DRE, the $\alpha$-divergence loss function ($\alpha$-Div), is derived.
$\alpha$-Div is concise but offers stable and effective optimization for DRE.
The boundedness of $\alpha$-divergence provides the potential for successful DRE with data exhibiting high KL-divergence.
Our numerical experiments demonstrate the effectiveness in optimization using $\alpha$-Div.
However, the experiments also show that the proposed loss function offers no significant advantage over the KL-divergence loss function in terms of RMSE for DRE. This indicates that the accuracy of DRE is
primarily determined by the amount of KL-divergence in the data and is less dependent on $\alpha$-divergence.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 1919