Medical Intervention Duration Estimation Using Language-enhanced Transformer Encoder with Medical Prompts

Yucheng Ruan, Xiang Lan, Daniel J. Tan, Hairil Rizal Abdullah, Mengling Feng

Abstract: Objective: This study focuses on estimating the duration of medical interventions using electronic health records (EHRs) in clinical decision support. Most existing models were designed for structured tabular data only and often suffer from data corruption problem. The unstructured clinical free-text data that provides valuable insights and is more resistant to data corruption is often overlooked. The objective of this research is to develop a multimodal deep learning framework that integrates different data modalities from EHRs, thereby fully utilizing the predictive capability of EHRs for medical intervention estimation. Materials and Methods: A novel prompt-based tabular transformer encoder framework is proposed for medical intervention duration estimation based on multimodal EHR data. The framework leverages a pre-trained sentence encoder with medical prompts to harmonize language representations of various clinical data modalities, which a tabular transformer encoder is developed to further explore. Results: The developed model demonstrates superior performance compared to the baselines in two EHR datasets. Furthermore, the model exhibits resilience to data corruption in EHRs, with the RMSE curve increasing gradually with higher corruption rates. Discussion: Other than the predictive effectiveness and robustness of the proposed framework, the ablation study highlights the significance of critical components, such as medical prompts, free-text information, and the pre-trained sentence encoder, all contributing to the model's predictive ability. Conclusion: This research presents a promising pathway to enhance medical intervention estimation by incorporating diverse data modalities from language perspective, ultimately bolstering the reliability of deep learning models in clinical care.