Not-So-Random Features

Brian Bullins, Cyril Zhang, Yi Zhang

Abstract: We propose a principled method for kernel learning, which relies on a Fourier-analytic characterization of translation-invariant or rotation-invariant kernels. Our method produces a sequence of feature maps, iteratively refining the SVM margin. We provide rigorous guarantees for optimality and generalization, interpreting our algorithm as online equilibrium-finding dynamics in a certain two-player min-max game. Evaluations on synthetic and real-world datasets demonstrate scalability and consistent improvements over related random features-based methods.

TL;DR: A simple and practical algorithm for learning a margin-maximizing translation-invariant or spherically symmetric kernel from training data, using tools from Fourier analysis and regret minimization.

Keywords: kernel learning, random features, online learning

Code: [ yz-ignescent/Not-So-Random-Features](https://github.com/yz-ignescent/Not-So-Random-Features)

Data: [CIFAR-10](https://paperswithcode.com/dataset/cifar-10), [MNIST](https://paperswithcode.com/dataset/mnist)

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:1710.10230/code)