Rethinking supervised learning: insights from biological learning and from calling it by its name

Alex Hernández-García

Keywords: supervised learning, self-supervised learning, biological learning, generalization

TL;DR: We discuss some of the overambitious promises of the deep learning hype, namely that models should be able to generalise from a few examples without supervision. We discuss these claims under the light of biological learning and learning theory.

Abstract: The renaissance of artificial neural networks was catalysed by the success of classification models, tagged by the community with the broader term supervised learning. The extraordinary results gave rise to a hype loaded with ambitious promises and overstatements. Soon the community realised that the success owed much to the availability of thousands of labelled examples. And supervised learning went, for many, from glory to shame: Some criticised deep learning as a whole and others proclaimed that the way forward had to be alternatives to supervised learning: predictive, unsupervised, semi-supervised and, more recently, self-supervised learning. However, these seem all brand names, rather than actual categories of a theoretically grounded taxonomy. Moreover, the call to banish supervised learning was motivated by the questionable claim that humans learn with little or no supervision and are capable of robust out-of-distribution generalisation. Here, we review insights about learning and supervision in nature, revisit the notion that learning and generalization are not possible without supervision or inductive biases and argue that we will make better progress if we just call it by its name.

Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/rethinking-supervised-learning-insights-from/code)