Active Learning Polynomial Threshold FunctionsDownload PDF

Published: 31 Oct 2022, Last Modified: 01 Oct 2022NeurIPS 2022 AcceptReaders: Everyone
Keywords: Statistical Learning Theory, Active Learning, Polynomial Threshold Functions, Enriched Queries
Abstract: We initiate the study of active learning polynomial threshold functions (PTFs). While traditional lower bounds imply that even univariate quadratics cannot be non-trivially actively learned, we show that allowing the learner basic access to the derivatives of the underlying classifier circumvents this issue and leads to a computationally efficient algorithm for active learning degree-$d$ univariate PTFs in $\tilde{O}(d^3\log(1/\varepsilon\delta))$ queries. We extend this result to the batch active setting, providing a smooth transition between query complexity and rounds of adaptivity, and also provide near-optimal algorithms for active learning PTFs in several average case settings. Finally, we prove that access to derivatives is insufficient for active learning multivariate PTFs, even those of just two variables.
TL;DR: We study active learning polynomial threshold functions where the learner may query the sign of underlying derivatives.
Supplementary Material: pdf
15 Replies