Knowledge-Augmented Deep Learning and its Applications: A Survey

Zijun Cui, Tian Gao, Kartik Talamadupula, Qiang Ji

Abstract: Deep learning models, though having achieved great success in many different fields over the past years, are usually data-hungry, fail to perform well on unseen samples, and lack interpretability. Different kinds of prior knowledge often exists in the target domain, and their use can alleviate the deficiencies with deep learning. To better mimic the behavior of human brains, different advanced methods have been proposed to identify domain knowledge and integrate it into deep models for data-efficient, generalizable, and interpretable deep learning, which we refer to as knowledge-augmented deep learning (KADL). In this survey, we define the concept of KADL and introduce its three major tasks, i.e., knowledge identification, knowledge representation, and knowledge integration. Different from existing surveys that are focused on a specific type of knowledge, we provide a broad and complete taxonomy of domain knowledge and its representations. Based on our taxonomy, we provide a systematic review of existing techniques, different from existing works that survey integration approaches agnostic to the taxonomy of knowledge. This survey subsumes existing works and offers a bird’s-eye view of research in the general area of KADL. The thorough and critical reviews of numerous papers help not only understand current progress but also identify future directions for the research on KADL.