Forster Decomposition and Learning Halfspaces with NoiseDownload PDF

21 May 2021, 20:51 (edited 26 Oct 2021)NeurIPS 2021 SpotlightReaders: Everyone
  • Keywords: learning theory, Forster transform, halfspaces, Massart noise
  • TL;DR: First efficient learning algorithm for Massart halfspaces with sample complexity independent of the bit complexity of the examples.
  • Abstract: A Forster transform is an operation that turns a multivariate distribution into one with good anti-concentration properties. While a Forster transform does not always exist, we show that any distribution can be efficiently decomposed as a disjoint mixture of few distributions for which a Forster transform exists and can be computed efficiently. As the main application of this result, we obtain the first polynomial-time algorithm for distribution-independent PAC learning of halfspaces in the Massart noise model with strongly polynomial sample complexity, i.e., independent of the bit complexity of the examples. Previous algorithms for this learning problem incurred sample complexity scaling polynomially with the bit complexity, even though such a dependence is not information-theoretically necessary.
  • Supplementary Material: pdf
  • Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.
12 Replies