Optimized Graph Structures for Calibrating Graph Neural Networks with Out-of-Distribution Nodes

TMLR Paper4614 Authors

Abstract: Graph neural networks~(GNNs) achieve remarkable success in tasks such as node classification, link prediction, and graph classification. However, despite their effectiveness, the reliability of the GNN's prediction remains a major concern. particularly when the graphs contain out-of-distribution~(OOD) nodes. Up to now, the calibration of GNNs in the presence of OOD nodes is still largely under-explored. Our empirical studies reveal that the calibration issue becomes significantly more complex when OOD nodes are present, and existing calibration methods prove to be less effective in this scenario. Recently, graph structure learning~(GSL), a family of data-centric learning approaches, has proved to be effective in mitigating the adverse effects of the noisy information in the graph topology by optimizing the graph structure alongside with GNN training. However, current GSL methods do not explicitly address the calibration issue in graphs with OOD nodes. To tackle the this challenge, we propose a novel framework called \underline{G}raph \underline{C}alibration via \underline{S}tructure \underline{O}ptimization~(GCSO) to calibrate GNNs against OOD nodes. Our empirical findings suggest that manually reducing the weight of edges connecting in-distribution~(ID) nodes and OOD nodes could effectively mitigate the calibration issue. However, identifying these edges and determining their appropriate weights is challenging, as the distribution of OOD nodes is unknown. To address it, we propose a novel framework to calibrate GNNs against OOD nodes. In our method we first develop an iterative edge-sampling mechanism to capture the topological information of the graph and formulate it as the Markov Decision Process~(MDP). Then, we leverage the actor-critic method to dynamically adjust the edge weights and assess their impact on target nodes. Additionally, we design a specialized reward signal to guide the policy function toward an optimal graph structure that minimizes the negative influence of OOD nodes. Note that our modified graph structure could be seamlessly integrated with existing temperature scaling-based calibration techniques for further improvement. Experimental results on benchmark datasets demonstrate that our method can effectively reduce the expected calibration error~(ECE) while maintaining comparable accuracy in GNNs. And our approach outperforms strong baseline methods, The anonymous GitHub repository for the code is available at \url{https://anonymous.4open.science/r/calibration-7F61}.

Submission Length: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: Jiangchao Yao

Submission Number: 4614