Efficient extended-graph multi-matrices subspace learning: Toward meta-similarity and efficiency promotion

Maoye Ren, Zhe Wang, Xinhai Yu

Abstract: Multi-view learning (MVL) has become a hot topic due to it can view a problem from different perspectives and settle it comprehensively. The consensus and complementarity principles provide important guidances for MVL. A category of MVL methods, called multi-matrices learning (MML), can excavate additional geometry information from each sample, and shows superior performance. However, existing MVL models primarily follow two types of sample relationships, each for one principle, whereas, there are actually three types of relationships. In this paper, we propose an efficient extended-graph multi-matrices subspace learning (EEGMMSL) approach based on MML, incorporating three types of relationships guaranteeing these two principles. The diagonal terms of our extended-graph can maintain the intro-view sample-similarity-relationships, embed samples of each view to a easier-separated-subspace, realizing the complementarity principle. The off-diagonal items of our extended-graph sustain the cross-view sample-similarity-relationships, and keep these embedded subspaces consensus, guaranteeing the consensus principle from sample level. By further maintaining the meta-similarity in our extended-graph, it can rectify the relationships in these subspaces, so that guarantees consensus principle from the feature level. Our approach can unify most of the relation-based MML methods and is more flexible in solving different problems. Focusing on the efficiency, we also proposed an new M-separate optimization algorithm to more efficiently optimize our objective function, instead of its traditional heuristic-gradient-descent optimization. This greatly reduces the computational complexity of MML. Experiments show the superior performance of EEGMMSL, and the remarkable time advantage of the M-separate optimization algorithm. The ideas in our paper can also be extended to other MVL methods.