How to Train Your FALCON: Learning Log-Concave Densities with Energy-Based Neural Networks

Alexander Lin, Demba E. Ba

Keywords: log-concave, density estimation, energy-based models, computational efficiency

Abstract: A classic problem within statistics is log-concave density (LCD) estimation, which asks for the best log-concave density that maximizes the probability of observing some input data points. Due to the non-parametric nature of this problem, current algorithms are too computationally demanding to work beyond a few (i.e. ten) dimensions. We introduce a new approach that employs energy-based neural networks to convert the non-parametric problem into a parametric one over the network weights, enabling scalable LCD estimation in high dimensions. By leveraging deep learning infrastructure (e.g. GPUs), our method can learn LCDs up to thousands of times faster than existing approaches, while requiring hundreds of times fewer parameters. We further show that our method learns informative LCDs on a real protein expression dataset with 77 dimensions, which is beyond the capabilities of current LCD estimation algorithms.