Dissecting Supervised Constrastive Learning

Florian Graf, Christoph D. Hofer, Marc Niethammer, Roland Kwitt

Abstract: Minimizing cross-entropy over the softmax scores of a linear map composed with a high-capacity encoder is arguably the most popular choice for training neural networks on supervised learning tasks....