Core structure-guided multi-modal classification via Monte Carlo Tree Search

Guoqing Liu, Yuhua Qian, Xinyan Liang, Pinhan Fu

Abstract: Multi-Modal Classification (MMC) effectively integrates multiple data sources, for which the choice of fusion strategy is crucial for classification performance. Current research includes expertise-based and search-based techniques, among which Neural Architecture Search (NAS) approaches stand out, but face significant time and resource consumption issues. To address this issue, CSG-NAS narrows the search scope by using the core structure-guided search, which greatly improves search performance and efficiency. However, CSG-NAS only considers the complementarity between high-quality views when acquiring the core structures, leading to an incomplete narrowed search space. In this paper, we combine Monte Carlo Tree Search (MCTS) with Core Structure-Guided search to propose an efficient and credible MMC algorithm, MCTS-CSG. It includes the core structures acquisition module composed of learning and searching phases, and the optimal structure search module composed of evolving phase. Specifically, the learning phase partitions and ranks the entire space by learning the node regressor, the search phase is based on the MCTS method to sample the structure in the most promising subspace while avoiding falling into a local optimum. The core structures containing complementarity and optimality are obtained after cyclic execution of the learning and searching phases. In the evolving phase, high-quality search domains are constructed around the core structures, which are defined as areas likely to contain high-performance fusion structures, and the Evolutionary Algorithm (EA) is used to find the optimal model structure. Meanwhile, a knowledge inheritance strategy is introduced to improve search speed. The experimental results show that the MCTS-CSG algorithm performs well in the MMC task with the best known search efficiency and performance.

External IDs:dblp:journals/mlc/LiuQLF25