Addressing Challenges in Healthcare Big Data Analytics

Santosh Tirunagari, Senthilkumar Mohan, David Windridge, Yashaswini Balla

Abstract: The exponential growth of healthcare data poses significant challenges for clinical researchers who strive to identify meaningful patterns and correlations. The complexity of this data arises from its high dimensionality, sparsity, inaccuracy, incompleteness, longitudinality, and heterogeneity. While conventional pattern recognition algorithms can partially address issues related to high dimensionality, sparsity, inaccuracy, and longitudinality, the problems of incompleteness and heterogeneity remain a persistent challenge, particularly when analyzing electronic health records (EHRs). EHRs often encompass diverse data types, such as clinical notes (text), blood pressure readings (longitudinal numerical data), MR scans (images), and DCE-MRIs (longitudinal video data), and may only include a subset of data for each patient at any given time interval. To tackle these challenges, we propose a kernel-based framework as the most suitable approach for handling heterogeneous data formats by representing them as matrices of equal terms. Our research endeavours to develop methodologies within this framework to construct a decision support system (DSS). To achieve this, we advocate for the incorporation of preprocessing mechanisms to address the challenges of incompleteness and heterogeneity prior to integration into the kernel framework.