Leveraging shared feature representation in cross-domain alignment of decision thresholds for electronic health records data.

Elena Gal, Anshul Thakur, Soheila Molaei, Andrew Soltan, David A. Clifton

Track: long paper (up to 8 pages)

Keywords: Threshold setting, Calibration, Generative Learning, Domain Adaptation, ML for Healthcare

TL;DR: We propose a generative learning based method for improving calibration and cross-domain alignment of decision thresholds for electronic health records data

Abstract: The real-world deployment of clinical machine learning models requires adaptability to distributional shifts caused by variations in the patient population and data acquisition mechanisms. However, distributional shifts are known to significantly affect the raw probability scores output by deep learning models and thus compromise performance on clinically important metrics when a threshold must be chosen to generate the final output. We propose a generative learning-based method for threshold setting that utilises unlabelled samples from the target distribution to learn shared feature representation, reduce the distance between domains and improve cross-domain alignment of output probabilities. We demonstrate that the proposed method improves the alignment of decision thresholds for several clinical tasks on real-world electronic health record (EHR) data and derive theoretical bounds on calibration error. Our approach doesn't require ground-truth labels for target data, facilitating its use in applications.

Submission Number: 98