Token-Aware Representation Augmentation for Fine-Grained Semi-Supervised Learning

Hongyang He, Yan Zhong, Xinyuan Song, Daizong Liu, Victor Sanchez

Keywords: Semi-supervised learning, FixMatch, consistency regularization, token-aware masking, token-level augmentation, high-confidence token suppression, feature diversity

TL;DR: We introduce TA-FixMatch, a semi-supervised learning framework that improves FixMatch by operating at the token representation level.

Abstract: FixMatch is a widely adopted semi-supervised learning (SSL) framework that relies on consistency regularization between weakly and strongly augmented versions of unlabeled data. In the case of image classification, its reliance on indiscriminate image-level augmentations often leads to overfitting on early confident predictions while neglecting semantically rich but underexplored features. In this work, we introduce Token-Aware FixMatch (TA-FixMatch), a novel SSL framework that operates at the token representation level to enhance feature diversity and generalization. Specifically, we propose a token-aware masking strategy that identifies and softly suppresses the most influential tokens contributing to high-confidence predictions; and a structured token-level augmentation pipeline that perturbs, reorganizes, and semantically enriches the remaining tokens. These representation-level augmentations guide the model to attend to alternative evidence and discover complementary features, which is particularly beneficial in fine-grained classification tasks. Extensive experiments on standard (CIFAR-100, STL-10) and fine-grained (CUB-200-2011, NABirds, Stanford Cars) benchmarks demonstrate that TA-FixMatch outperforms existing state-of-the-art SSL methods under low-label regimes.

Submission Number: 72