Efficiently Learning Fourier Sparse Set Functions

Andisheh Amrollahi, Amir Zandieh, Michael Kapralov, Andreas Krause

Abstract: Learning set functions is a key challenge arising in many domains, ranging from sketching graphs to black-box optimization with discrete parameters. In this paper we consider the problem of efficiently learning set functions that are sparse (say $k$-sparse) in the Fourier domain. This is a wide class, that includes graph and hypergraph cut functions, decision trees and more. Our central contribution is the first algorithm that allows learning functions whose Fourier support only contains low degree (say degree $d=o(n)$) polynomials using information theoretically optimal $O(k d \log n)$ sample complexity and runtime $O( n k \log k + d k \log k \log n)$. This implies that sparse graphs with $k$ edges can, for the first time, be learned from $O(k \log n)$ observations of cut values and in linear time in the number of vertices. Our algorithm also can efficiently learn decision trees of depth at most $d$ and $k$ nodes from $O(k d \log n)$ evaluations. The algorithm exploits techniques from the sparse Fourier transform literature and is easily implementable. Lastly, we also develop an efficient robust version of our algorithm and prove $\ell_2/\ell_2$ approximation guarantees without any statistical assumptions on the noise.

Code Link: https://github.com/andisheh94/Efficiently-Learning-Fourier-Sparse-Set-Functions

CMT Num: 8659