Learning Representations from Medical Text for Effective Diagnoses and Knowledge Discovery

Anonymous

Abstract: Discovering knowledge and effectively predicting target events are two main goals of medical data mining. However, few models can achieve them simultaneously. In this study, we investigated the possibility of discovering knowledge and predicting diagnosis at once via raw medical text. We proposed the Enhanced Neural Topic Model (ENTM), a variant of the neural topic model, to learn interpretable representations. We introduced the auxiliary loss set to improve the effectiveness of learned representations. Then, we used learned representations to train a softmax regression model to predict target events. As each element in representations learned by ENTM has an explicit semantic meaning, weights in softmax regression represent knowledge of whether an element is a significant factor in predicting diagnosis. We adopted two independent medical text datasets to evaluate our ENTM model. Results indicate that our model obtained better performance compared to the latest pretrained neural language models. Meanwhile, analysis of model parameters indicates our model can discover reliable knowledge from data.

Paper Type: long