One-Class SVM-guided Negative Sampling for Enhanced Contrastive Learning

Dhruv Jain, Tsiry Mayet, Romain HÉRAULT, Romain MODZELEWSKI

Keywords: Contrastive Learning, Self Supervised Learning, One-Class SVM, Deep Learning

Abstract:

Recent studies on contrastive learning have emphasized carefully sampling and mixing negative samples.
This study introduces a novel and improved approach for generating synthetic negatives.
We propose a new method using One-Class Support Vector Machine (OCSVM) to guide in the selection process before mixing named as **Mixing OCSVM negatives (MiOC)**.
Our results show that our approach creates more meaningful embeddings, which lead to better classification performance.
We implement our method using publicly available datasets (Imagenet100, Cifar10, Cifar100, Cinic10, and STL10). We observed that MiOC exhibit favorable performance compared to state-of-the-art methods across these datasets.
By presenting a novel approach, this study emphasizes the exploration of alternative mixing techniques that expand the sampling space beyond the conventional confines of hard negatives produced by the ranking of the dot product.

Submission Number: 33