Symmetries of Functional Processes under Label Noise

Abhra Chaudhuri, Pedro Gomes

Keywords: Symmetry, Effective Function Class, Model Complexity, Label Noise

TL;DR: The effective, data-induced function class of DNNs exhibits symmetries under label noise with the true concept as the invariant, which can be asymptotically recovered with monotonically shrinking uncertainty via finite time / capacity approximations.

Abstract: Detecting label noise is backed by a vast empirical literature, yet when detection can be trusted, and how its reliability scales with computation, remain poorly understood. We introduce \emph{functional processes} --- aggregation mechanisms over estimates of the effective, data-induced function class. Functional processes exhibit symmetries under label noise whereby they asymptotically recover the clean concept $c(\mathrm{\mathbf{x}})$, i.e., the solution gradient flow would have learned under no label noise. This is achieved through the avoidance of complexity barriers --- the requirement that zero-loss separators of noisy instances be of strictly higher complexity than their clean counterparts --- surfacing $c$ as a common invariant; predictions concentrate around it with shrinking uncertainty. This yields arbitrarily precise noise detection, with finite-time SGD guarantees and an $\mathcal{O}(1/\sqrt{n})$ finite-capacity rate. We empirically validate the predicted complexity signatures, asymptotic convergence, and variance collapse on standard benchmarks.

Paper Type: Long (8 pages)

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 59