Efficient Testable Learning of Halfspaces with Adversarial Label Noise

Ilias Diakonikolas, Daniel Kane, Vasilis Kontonis, Sihan Liu, Nikos Zarifis

Keywords: Machine Learning

TL;DR: We give an efficient active tester-learner pair for halfspaces in the testable learning framework defined by Rubinfeld and Vasilyan (2022).

Abstract:

We give the first polynomial-time algorithm for the testable learning
of halfspaces in the presence of adversarial label noise under the Gaussian distribution. In the recently introduced testable learning
model, one is required to produce a tester-learner such that if the data passes the tester, then one can trust the output of the robust learner on the data. Our tester-learner runs in time $\text{poly}(d/\epsilon)$ and outputs a halfspace with misclassification error $O(\text{opt})+\epsilon$, where $\text{opt}$ is the 0-1 error of the best fitting halfspace. At a technical level, our algorithm employs an iterative soft localization technique enhanced with appropriate testers to ensure that the data distribution is sufficiently similar to a Gaussian. Finally, our algorithm can be readily adapted to yield an efficient and testable active learner requiring only $d ~ \text{polylog}(1/\epsilon)$ labeled examples.

Submission Number: 5453