Impact of uncertainty maps on manual editing of rectal cancer segmentation in radiotherapy

Federica Carmen Maruccio, Rita Simoes, Fokie Cnossen, Christian Jamtheim Gustafsson, Sanne Conijn, Alice Couwenberg, Suzan Gerrets - van Noord, Inge de Jong, Vivian van Pelt, Lisa Wiersema, Joëlle E. van Aalst, Jan-Jakob Sonke, Charlotte L. Brouwer, Tomas M. Janssen

Keywords: segmentation, uncertainty maps, manual editing

TL;DR: Uncertainty maps did not significantly influence clinicians’ editing behaviour for high-quality contours, with human factors playing a much larger role in influencing editing time and amount.

Abstract:

Uncertainty maps provide a quantitative and visual representation of the estimated confidence of Deep Learning (DL) models in contouring predictions and have been proposed to improve clinicians' efficiency during manual review. However, uncertainty maps are not currently integrated into clinical workflows, and evidence on their actual benefit in clinical decision-making remains limited. This study investigates the impact of simulated uncertainty maps on clinicians' behaviour during manual editing of high-quality clinical target volume (CTV) contours in rectal cancer radiotherapy. An inter-observer variability (IOV) dataset of 10 patients was used to simulate high-quality DL uncertainty maps and contours. Six clinicians edited the contours across two editing sessions, with and without uncertainty maps. For each session, editing time, editing amount, questionnaire responses, and interview feedback were collected to assess the impact both quantitatively and qualitatively.
Editing time and editing amount were comparable with and without uncertainty maps, while both measures decreased significantly in the second editing session, indicating a learning effect from task repetition. Qualitative feedback showed that clinicians’ decisions were shaped more by human factors, such as workload, mood, memory and anchoring biases, than by the uncertainty maps. Moreover, the study revealed low clinician trust in the uncertainty maps, which were used primarily for confirmation rather than decision-making. The findings suggest that the value of uncertainty maps may be limited for high-quality contours and highlight the need to investigate their relevance for different use cases.

Primary Subject Area: Uncertainty Estimation

Secondary Subject Area: Segmentation

Registration Requirement: Yes

Reproducibility: Released upon acceptance

Visa & Travel: No

Read CFP & Author Instructions: Yes

Originality Policy: Yes

Single-blind & Not Under Review Elsewhere: Yes

LLM Policy: Yes

Submission Number: 29