Imbalanced multi-instance multi-label learning via tensor product-based semantic fusion

Xinyue Zhang, Tingjin Luo

Abstract: With powerful expressiveness of multi-instance multi-label learning (MIML) for objects with multiple semantics and its great flexibility for complex object structures, MIML has been widely applied to various applications. In practical MML tasks, the naturally skewed label distribution and label interdependence bring up the label imbalance issue and decrease model performance, which is rarely studied. To solve these problems, we propose an imbalanced multi-instance multi-label learning method via tensor product-based semantic fusion (IMIML-TPSF) to deal with label interdependence and label distribution imbalance simultaneously. Specifically, to reduce the effect of label interdependence, it models similarity between the query object and object sets of different label classes for similarity-structural features. To alleviate disturbance caused by the imbalanced label distribution, it establishes the ensemble model for imbalanced distribution features. Subsequently, IMIML-TPSF fuses two types of features by tensor product and generates the new feature vector, which can preserve the original and interactive feature information for each bag. Based on such features with rich semantics, it trains the robust generalized linear classification model and further captures label interdependence. Extensive experimental results on several datasets validate the effectiveness of IMIML-TPSF against state-of-the-art methods.