A Bayesian Approach to Segmentation with Noisy Labels via Spatially Correlated Distributions

Ryu Tadokoro, Tsukasa Takagi, Shin-ichi Maeda

Abstract:

In semantic segmentation, the accuracy of models heavily depends on the high-quality annotations. However, in many practical scenarios, such as medical imaging and remote sensing, obtaining true annotations is not straightforward and usually requires significant human labor.
Relying on human labor often introduces annotation errors, including mislabeling, omissions, and inconsistency between annotators.
In the case of remote sensing, differences in procurement time can lead to misaligned ground-truth annotations.
These label errors are not independently distributed, and instead usually appear in spatially connected regions where adjacent pixels are more likely to share the same errors.
To address these issues, we propose an approximate Bayesian estimation based on a probabilistic model that assumes training data include label errors, incorporating the tendency for these errors to occur with spatial correlations between adjacent pixels.
However, Bayesian inference for such spatially correlated discrete variables is notoriously intractable. To overcome this fundamental challenge, we introduce a novel class of probabilistic models, which we term the \textbf{ELBO-Computable Correlated Discrete Distribution (ECCD)}. By representing the discrete dependencies through a continuous latent Gaussian field with a Kac-Murdock-Szeg\"{o} (KMS) structured covariance, our framework enables scalable and efficient variational inference for problems previously considered computationally prohibitive.
Through experiments on multiple segmentation tasks, we confirm that leveraging the spatial correlation of label errors improves robustness.
Notably, in specific tasks such as lung segmentation, the proposed method achieves performance comparable to training with clean labels under moderate noise levels.

Submission Type: Long submission (more than 12 pages of main content)

Changes Since Last Submission:

We have made the following revisions compared to the original submission:

1. Clarification of optimization behavior under severe noise:
We added a detailed discussion in Section 5.3.1 regarding the degradation observed in the clavicle segmentation task under severe noise. We interpret this behavior as being related to the EM-like feedback mechanism in ECCD and clarify that the issue is likely due to optimization dynamics rather than a fundamental limitation of the model.

2. Supplementary analysis on warmup initialization:
We added a new supplementary experiment (Appendix C.2) to investigate the effect of initialization using a warmup strategy. We explicitly present this as a supplementary analysis rather than a conclusive validation. The results suggest that better initialization may stabilize ELBO optimization, although improvements in Dice score are modest.

3. Improved explanation of the probabilistic model:
We refined the explanation of the latent Gaussian formulation and the induced discrete distribution (Appendix B.1), clarifying its relation to logit-Gaussian random fields and emphasizing the role of higher-order dependencies.

4. Improved presentation and consistency:
- Improved wording throughout the manuscript to avoid overly strong claims and ensure a more precise interpretation of empirical results.
- Fixed minor grammatical issues and notation inconsistencies.

5. Additional experimental details:
We expanded the appendix with additional experimental results, including noise-type-wise evaluations and clean-label experiments, to provide a more comprehensive evaluation of the proposed method.

Code: https://github.com/pfnet-research/Bayesian_SpatialCorr

Assigned Action Editor: ~Chenyu_You1

Submission Number: 6447