SVMPT: A Hybrid Approach to Sparse and Irregular Clinical Data Learning with Selective Variable-wise Message Passing and Transformer

Rongqin Chen, Dan Wu, Leong Hou U, Ye Li

Abstract: Electronic Health Records (EHRs) contain a wide range of patient data, presenting both opportunities and challenges for analysis. The complexity and sparsity of these multivariate time series data, characterised by high dimensionality and non-uniform sampling, pose difficulties for conventional time series techniques. To address these challenges, we present SVMPT, a novel two-stage machine learning model that combines selective variable-wise message passing with a Transformer architecture. SVMPT uses attention mechanisms to dynamically update data representations, capture inter-variable dependencies, and synthesise missing values. SVMPT delivers state-of-the-art results in the PhysioNet Mortality Prediction Challenge 2012 and the Sepsis Early Prediction Challenge 2019, demonstrating its effectiveness in processing irregular and sparse EHR data for downstream tasks. The introduction of SVMPT contributes to the development of advanced machine learning techniques for complex EHR data analysis.