Evidence-Aware Entropy Decomposition For Active Deep Learning

Weishi Shi, Xujiang Zhao, Feng Chen, Qi Yu

Keywords: active learning, entropy decomposition, uncertainty

TL;DR: An evidence-aware entropy decomposition approach for active deep learning using multiple sources of uncertainty

Abstract: We present a novel multi-source uncertainty prediction approach that enables deep learning (DL) models to be actively trained with much less labeled data. By leveraging the second-order uncertainty representation provided by subjective logic (SL), we conduct evidence-based theoretical analysis and formally decompose the predicted entropy over multiple classes into two distinct sources of uncertainty: vacuity and dissonance, caused by lack of evidence and conflict of strong evidence, respectively. The evidence based entropy decomposition provides deeper insights on the nature of uncertainty, which can help effectively explore a large and high-dimensional unlabeled data space. We develop a novel loss function that augments DL based evidence prediction with uncertainty anchor sample identification through kernel density estimation (KDE). The accurately estimated multiple sources of uncertainty are systematically integrated and dynamically balanced using a data sampling function for label-efficient active deep learning (ADL). Experiments conducted over both synthetic and real data and comparison with competitive AL methods demonstrate the effectiveness of the proposed ADL model.

Code: https://drive.google.com/drive/folders/1imwnOahh8HtHK_g_HSTb4TCxZ7YG04ay