ParK: Sound and Efficient Kernel Ridge Regression by Feature Space PartitionsDownload PDF

21 May 2021, 20:51 (edited 03 Feb 2022)NeurIPS 2021 PosterReaders: Everyone
  • Keywords: Large-scale kernel methods, kernel ridge regression, random projections, partitions
  • TL;DR: We introduce a new large-scale algorithm to solve kernel ridge regression.
  • Abstract: We introduce ParK, a new large-scale solver for kernel ridge regression. Our approach combines partitioning with random projections and iterative optimization to reduce space and time complexity while provably maintaining the same statistical accuracy. In particular, constructing suitable partitions directly in the feature space rather than in the input space, we promote orthogonality between the local estimators, thus ensuring that key quantities such as local effective dimension and bias remain under control. We characterize the statistical-computational tradeoff of our model, and demonstrate the effectiveness of our method by numerical experiments on large-scale datasets.
  • Supplementary Material: pdf
  • Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.
13 Replies

Loading