ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission

Kexin Huang, Jaan Altosaar, Rajesh Ranganath

Abstract:

Clinical notes contain information about patients that goes beyond structured data like lab
values and medications. However, clinical notes have been underused relative to structured
data, because notes are high-dimensional and sparse. This work develops and evaluates
representations of clinical notes using bidirectional transformers (ClinicalBert). ClinicalBert uncovers high-quality relationships between medical concepts as judged by humans. ClinicalBert outperforms baselines on 30-day hospital readmission prediction using
both discharge summaries and the first few days of notes in the intensive care unit. Code
and model parameters are available.1