Data-Driven Random Feature Selection for Deep Kernel Learning with Kernel Alignment

Di Gao, Xiaoyu Kang, Weiqing Huang, Chonghui Zheng

Abstract: Deep kernel learning (DKL) powerfully merges kernel methods with deep neural networks, yet prominent approaches based on random features (RF) necessitate manual selection of feature counts, often impairing efficiency and kernel approximation quality. We introduce DKL-IRF (Deep Kernel Learning with Implicit Random Features), a novel framework that addresses this limitation. DKL-IRF employs kernel alignment techniques to adaptively select an optimized subset of implicit random Fourier features (RFFs) that accurately represent the underlying data distribution and the target kernel function. Furthermore, it incorporates a learning mechanism to assign adaptive weights to these features within the deep architecture. This data-driven selection process prunes uninformative features, significantly boosting computational efficiency without sacrificing predictive power. Comprehensive evaluations on synthetic and real-world benchmark datasets confirm that DKL-IRF achieves state-of-the-art or comparable performance relative to existing DKL methods, while offering markedly improved algorithmic efficiency.

External IDs:dblp:conf/icic/GaoKHZ25