Clinical Measurements with Calibrated Instance-Dependent Confidence Interval
Keywords: confidence interval, uncertainty calibration, conformal prediction
TL;DR: We propose an instance based calibration method for a confidence interval of a network prediction
Abstract: Reporting meaningful confidence intervals for the predictions of a regression neural network is critical in medical imaging applications since clinical decisions are based on network predictions. We expect to ob-
tain larger intervals for difficult examples and smaller ones for easier examples to predict. A recently proposed calibration procedure suggests predicting the mean and the variance and scaling the variance on a validation set. Another calibration approach is based on applying conformal prediction to quantile regression. We show that assuming a Gaussian distribution to predict the variance followed by a non-parametric Con-
formal Prediction technique to scale the estimated variance is the most effective way of achieving a small confidence interval with a coverage guarantee. We report extensive experimental results on various medical
imaging datasets and network architectures.
Primary Subject Area: Uncertainty Estimation
Secondary Subject Area: Interpretability and Explainable AI
Paper Type: Both
Registration Requirement: Yes
Reproducibility: https://anonymous.4open.science/r/Calibrated-Instance-Dependent-95F9
Submission Number: 164
Loading