Quantifying Information in Similarity Matrices for Improved Representation Learning

Zelong Zhang, Zhuhui Han, Jieting Wang, Feijiang Li

Abstract: Quantifying the informativeness of the similarity matrix has been successfully applied in kernel width selection, dimension size selection, and so on. Traditionally, the informational content is defined by calculating the distance between a matrix and non informative matrices. However, this method has limitations when dealing with adjacency matrices with the same marginal distribution but different structures, as it assigns the same distance values to these matrices. To address this issue, we introduce a new distance metric based on the adjacency matrix generated by label vectors, which accurately reflects the correct similarity structure. Through experimental analysis, we have demonstrated that the proposed distance metric can effectively distinguish matrices with different adjacency structures. This feature is crucial for capturing data diversity as it provides a stable measure of sample similarity for classification tasks. In graph neural networks (GNNs), reconstruction loss is crucial for model performance. To verify the effectiveness of the proposed metric, we apply it as a loss function in the training process of the GNN model. Extensive experimental results have shown that the proposed loss function can achieve higher accuracy compared to traditional loss functions.

External IDs:dblp:conf/ijcnn/ZhangHWL25