HEART: Learning better representation of EHR data with a heterogeneous relation-aware transformer

Tinglin Huang, Syed Asad Rizvi, Rohan Krishna Thakur, Vimig Socrates, Meili Gupta, David van Dijk, R. Andrew Taylor, Rex Ying

Abstract: Highlights•HEART is a relation-aware transformer for EHR, achieving SOTA on two datasets and five tasks.•HEART includes an expressive transformer and two pretraining objectives tailored for EHR.•Learned diagnosis and medication representations capture code system hierarchy.