Rethinking supervised learning: insights from biological learning and from calling it by its name

Alex Hernández-García

Keywords: deep learning, supervised learning, self-supervised learning, semi-supervised learning, unsupervised learning, biological learning

TL;DR: We discuss some of the overambitious promises and misconceptions of the deep learning hype around the role of supervision, and that have led to brand names such as self-supervised learning, by revisiting insights from biological learning.

Abstract: The renaissance of artificial neural networks was catalysed by the success of classification models, tagged by the community with the broader term supervised learning. The extraordinary results gave rise to a hype loaded with ambitious promises and overstatements. Soon the community realised that the success owed much to the availability of thousands of labelled examples. And supervised learning went, for many, from glory to shame. Some criticised deep learning as a whole and others proclaimed that the way forward had to be "alternatives" to supervised learning: predictive, unsupervised, semi-supervised and, more recently, self-supervised learning. However, these seem all brand names, rather than actual categories of a theoretically grounded taxonomy. Moreover, the call to banish supervised learning was motivated by the questionable claim that humans learn with little or no supervision. Here, we review insights about learning and supervision in nature, revisit the notion that learning is not possible without supervision and argue that we will make better progress if we just call it by its name.