Handling missing data in medical questionnaires using tensor decompositions

Justin Dauwels, Lalit Garg, Arul Earnest, Khai Pang Leong

Abstract: Questionnaires are often used to understand the quality of life of patients, treatment and disease burden and to obtain their feedback on the provided health care. However, a common problem with questionnaires is missing data. Some level of missing data is common and unavoidable. For example, patients may elect to leave one or more items unanswered either inadvertently or because they feel inhibited in responding to items dealing with a sensitive topic. Such missing data may lead to biased parameter estimates and inflated errors. In this paper, we propose an innovative collaborative filtering technique to complete missing data in medical questionnaires. The proposed technique is based on canonical tensor decomposition (CANDECOMP) and parallel factor decomposition (PARAFAC). It is very fast and effective especially with repeated medical questionnaires. To assess the different algorithms and our methods, we used SLEQOL questionnaires (“systemic lupus erythematosus-specific quality-of-life instrument”) completed by one hundred patients from TTSH and hospitals in China and Vietnam. Our results demonstrate that the tensor decomposition based method provides significant improvement on many existing methods and overcome their limitations in terms of various statistical measures.