ComformalMOS: Uncertainty-Aware MOS Prediction with Conformal Intervals and Ordinal Modeling

ICLR 2026 Conference Submission12897 Authors

Keywords: Mean Opinion Score (MOS) prediction; Speech quality assessment; Uncertainty estimation; Conformal prediction

Abstract: Accurately predicting human Mean Opinion Scores (MOS) is essential for evaluating synthetic speech quality in text-to-speech (TTS) and voice conversion (VC) systems. Existing MOS prediction models focus on point estimates and often overlook uncertainty, reducing model selection and deployment reliability. Recent work has sought to address uncertainty estimation using probabilistic losses but lacks formal coverage guarantees. Addressing this limitation, we introduce ComformalMOS, a framework that augments MOS prediction with conformal prediction-based interval estimation to provide statistically valid prediction intervals with guaranteed coverage under exchangeability assumptions, alongside conventional point estimates. During training, ordinal-aware modeling of the MOS score converts one-hot labels into a soft distribution using a Gaussian kernel. By explicitly modeling the ordinal structure of MOS labels, our approach produces reliable uncertainty estimates when softmax-based confidence scores become overconfident on out-of-distribution speech, ensuring that the resulting intervals respect the ordering of MOS scores. We evaluate our method on both point-prediction quality and uncertainty quality. Experiments on BVCC datasets demonstrate that ComformalMOS maintains competitive point prediction performance (MSE = 0.08) while providing prediction intervals with empirically validated coverage rates. This dual capability enhances model reliability for deployment in production TTS and VC systems, where uncertainty quantification is critical.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Submission Number: 12897