Complex heterogeneity learning: A theoretical and empirical study

Pei Yang, Qi Tan, Jingrui He

Abstract: Highlights • A graph based hybrid approach to model task relatedness, view consistency, and bag instance correlations in a principled framework. • An iterative learning algorithm to solve the non convex non smooth optimization problem. • Theoretical analysis in terms of Rademacher complexity showing the improvement of generalization performance by jointly modeling triple heterogeneity. • Experimental results on various data sets demonstrating the effectiveness of the proposed algorithm. Abstract Data heterogeneity such as task heterogeneity, view heterogeneity, and instance heterogeneity often co-exist in many real-world applications including insider threat detection, traffic prediction, brain image analysis, quality control in manufacturing processes, etc. However, most of the existing techniques might not take fully advantage of the rich heterogeneity. To address this problem, we propose a novel graph-based approach named M3 to simultaneously model triple heterogeneity in a principled framework. The main idea is to employ the hybrid graphs to jointly model the task relatedness, view consistency, and bag-instance correlation by enhancing the labeling consistency between nearby nodes on the graphs. Furthermore, we analyze the generalization performance of the proposed method based on Rademacher complexity, which sheds light on the benefits of jointly modeling multiple types of heterogeneity. The resulting optimization problem is challenging since the objective function is non-smooth and non-convex. We propose an iterative algorithm based on block coordinate descent and bundle method to solve the problem. Experimental results on various datasets demonstrate the effectiveness of the proposed method.