Learning Explicit Semantic-Spatial Synergy for Weakly Supervised Referring Image Segmentation

ICLR 2026 Conference Submission17472 Authors

Keywords: Weakly Supervised Referring Image Segmentation, vision language alignment

Abstract: Weakly Supervised Referring Image Segmentation (WSRIS) aims to segment target objects specified by natural language expressions using only image-text pairs. While recent advances have improved semantic grounding, existing methods still lack explicit mechanisms to incorporate spatial understanding, preventing them from achieving semantic–spatial synergy and leaving a large gap to fully supervised approaches. To address this limitation, we propose ES³Net, a novel framework that Explicitly learns Semantic-Spatial Synergy from the following three perspectives: First, we propose an Explicit Spatial Enhancement Module that integrates mask-grounded semantic features with object-centric 3D coordinates derived from readily obtained depth. This produces embeddings where spatial geometry is semantically anchored, enabling accurate localization of position-sensitive expressions. Second, a Language Consistency Module is proposed to enforce consistent alignment across diverse expressions referring to the same instance, improving robustness to linguistic variations. Finally, we introduce a Confidence-Aware Dense Distillation strategy that transforms high-confidence grounding predictions into pseudo labels, allowing a lightweight student–teacher RIS model to be trained for stable learning and efficient inference. Extensive experiments on RefCOCO, RefCOCO+, and RefCOCOg demonstrate that ES³Net establishes new state-of-the-art performance, underscoring the importance of explicit semantic–spatial synergy in advancing WSRIS.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 17472