An Adaptive Hoeffding Tree Model Based on Differential Entropy and Relative Entropy for Concept Drift Detection

Wenzheng Liu, Xiang Li, Yongtong Gu, Jin Li, Xiaoyong Tang, Ronghui Cao, Tan Deng, Xiaofeng Wang

Abstract: The concept drift detection algorithm can timely respond to and adjust the model by monitoring changes in data distribution over time. However, the dynamically adjusted ensemble model may still retain some components with weak adaptability. These components are involved in subsequent training and testing phases, leads to a significant decrease in classification performance. To solve these problems, this paper proposes an Adaptive Hoeffding Tree Model Based on Differential Entropy and Relative Entropy (AHT-DERE) for concept drift detection. It adopts a two-step strategy: a) A differential entropy-based drift detection method, which calculates the information entropy of the two most recently arrived data samples, and quantifies the difference between data distributions by subtracting the entropy values. This measurement serves as the criteria for determining the occurrence of concept drift. b) A relative entropy-based dynamic adjustment method, which utilizes the relative entropy similarity between the fitted and true distributions of the current data samples. This method selects well-adapted components for each round of incremental updates to improve the resilience of the ensemble model to concept drift. Compared to advanced algorithms, experimental results show that in two sets of experiments, the classification performance of AHT-DERE achieved an average improvement of 6.36% and 5.94% on seven publicly available real-world and synthetic datasets, respectively. The maximum improvement reached 13.55% and 10.82%, respectively.

External IDs:dblp:conf/ijcnn/LiuLGLTCDW24