Autoencoding Random Forests

Binh Duc Vu, Jan Kapar, Marvin N. Wright, David Watson

Keywords: Random Forests, Compression, Kernel Methods, Constrained Optimization, Nearest Neighbors

TL;DR: We propose a random forest autoencoder with applications in tabular, image and genomic data.

Abstract:

We propose a principled method for autoencoding with random forests. Our strategy builds on foundational results from nonparametric statistics and spectral graph theory to learn a low-dimensional embedding of the model that optimally represents relationships in the data.
We provide exact and approximate solutions to the decoding problem via constrained optimization, split relabeling, and nearest neighbors regression. These methods effectively invert the compression pipeline, establishing a map from the embedding space back to the input space using splits learned by the ensemble's constituent trees. The resulting decoders are universally consistent under common regularity assumptions. The procedure works with supervised or unsupervised models, providing a window into conditional or joint distributions. We demonstrate various applications of this autoencoder, including powerful new tools for visualization, compression, clustering, and denoising. Experiments illustrate the ease and utility of our method in a wide range of settings, including tabular, image, and genomic data.

Primary Area: General machine learning (supervised, unsupervised, online, active, etc.)

Submission Number: 15849