Word-Sequence Entropy: Towards uncertainty estimation in free-form medical question answering applications and beyond

Zhiyuan Wang, Jinhao Duan, Chenxi Yuan, Qingyu Chen, Tianlong Chen, Yue Zhang, Ren Wang, Xiaoshuang Shi, Kaidi Xu

Abstract: Highlights•We devise Word-Sequence Entropy for uncertainty analysis in medical query-answering.•We investigate the issue of generative inequality in medical responses.•We capture and highlight keywords and reliable sequences based on semantic relevance.•We resample based on the calibrated uncertainty.