Imbalanced Multi-instance Multi-label Learning via Coding Ensemble and Adaptive Thresholds

Xinyue Zhang, Tingjin Luo, liuyueying, Chenping Hou

Abstract: Multi-instance multi-label learning (MIML), which deals with objects with complex structures and multiple semantics, plays a crucial role in various fields. In practice, the naturally skewed label distribution and label dependence contribute to the issue of label imbalance in MIML, which is crucial but rarely studied. Most existing MIML methods often produce biased models due to the ignorance of inter-class variations in imbalanced data. To address this issue, we propose a novel imbalanced multi-instance multi-label learning method named IMIMLC, based on the error-correcting coding ensemble and an adaptive threshold strategy. Specifically, we design a feature embedding method to extract the structural information of each object via Fisher vectors and eliminate inexact supervision. Subsequently, to alleviate the disturbance caused by the imbalanced distribution, a novel ensemble model is constructed by concatenating the error-correcting codes of randomly selected subtasks. Meanwhile, IMIMLC trains binary base classifiers on small-scale data blocks partitioned by our codes to enhance their diversity and then learns more reliable results to improve model robustness for the imbalance issue. Furthermore, IMIMLC adaptively learns thresholds for each individual label by margin maximization, preventing inaccurate predictions caused by the semantic discrepancy across many labels and their unbalanced ratios. Finally, extensive experimental results on various datasets validate the effectiveness of IMIMLC against state-of-the-art approaches.

Primary Subject Area: [Content] Multimodal Fusion

Secondary Subject Area: [Content] Media Interpretation, [Systems] Data Systems Management and Indexing

Relevance To Conference: Our work proposes an efficient pattern recognition and data mining algorithm that can process multimedia data, such as images, text, and audio, from real-world applications. In new problem scenarios, we explore new solutions to improve the performance of multimedia data management and practice.

Submission Number: 888