High-Dimensional Dataset Entropy Estimation via Lossy Compression

Ivan Butakov, Sofia Malanchuk, Anna Neopryatnaya, Alexander Tolmachev, Kirill Andreev, Stanislav Kruglik, Evgeny Marshakov, Alexey Frolov

Abstract: This paper proposes a method for estimating a high-dimensional dataset entropy based on compression via autoencoder. We give theoretical proof of such estimation relevance and illustrate it by experiments with synthetic data and the MNIST digits dataset.