When Confidence Fails: Revisiting Pseudo-Label Selection in Semi-supervised Semantic Segmentation

pan liu, Jinshi Liu

Abstract: While significant advances exist in pseudo-label generation for semi-supervised semantic segmentation, pseudolabel selection remains understudied. Existing methods typically use fixed confidence thresholds to retain highconfidence predictions as pseudo-labels. However, these methods cannot cope with network overconfidence tendency, where correct and incorrect predictions overlap significantly in high-confidence regions, making separation challenging and amplifying model cognitive bias. Meanwhile, the direct discarding of low-confidence predictions disrupts spatial-semantic continuity, causing critical context loss. We propose Confidence Separable Learning (CSL) to address these limitations. CSL formulates pseudo-label selection as a convex optimization problem within the confidence distribution feature space, establishing sample-specific decision boundaries to distinguish reliable from unreliable predictions. Additionally, CSL introduces random masking of reliable pixels to guide the network in learning contextual relationships from lowreliability regions, thereby mitigating the adverse effects of discarding uncertain predictions. Extensive experimental results on the Pascal, Cityscapes, and COCO benchmarks show that CSL performs favorably against state-ofthe-art methods. Code and model weights are available at:https://github.com/PanLiuCSU/CSL.