Abstract: Highlights•A Label-Aware Attention module (LAA) learns label-aware feature affinity using a contrastive loss.•LAA enhances labelwise feature discriminativeness via affinity-guided feature reconstruction.•A Multi-scale Feature Boosting module (MFB) integrates scale context for feature encoding.•An encoder–decoder learning framework combines LAA and MFB for robust medical segmentation.