FMMI: Flow Matching Mutual Information Estimation

Ivan Butakov, Alexander Semenenko, Valeriia Kirova, Ivan Oseledets, Alexey Frolov

Keywords: flow matching, information theory, mutual information

TL;DR: Flow matching is leveraged for a direct mutual information estimation

Abstract: We introduce a novel Mutual Information (MI) estimator that fundamentally reframes the discriminative approach. Instead of training a classifier to discriminate between joint and marginal distributions, we learn a normalizing flow that transforms one into the other. This technique produces a computationally efficient and precise MI estimate that scales well to high dimensions and across a wide range of ground-truth MI values.

Submission Number: 147