Optimized Graph Structures for Calibrating Graph Neural Networks with Out-of-Distribution Nodes

Abstract: Graph neural networks (GNNs) have achieved remarkable success in tasks such as node classification, link prediction, and graph classification. However, despite their effectiveness, the reliability of GNN predictions remains a major concern, particularly when graphs contain out-of-distribution (OOD) nodes. To date, the calibration of GNNs in the presence of OOD nodes remains largely underexplored. Our empirical studies reveal that the calibration problem becomes significantly more complex in the presence of OOD nodes, and existing calibration methods are notably less effective in such scenarios. Recently, graph structure learning~(GSL), a family of data-centric learning approaches, has shown promise in mitigating the adverse effects of noisy information in graph topology by jointly optimizing the graph structure and GNN training. However, current GSL methods do not explicitly address the calibration challenges posed by OOD nodes. To tackle this challenge, we propose a novel framework called Graph Calibration via Structure Optimization~(GCSO) to calibrate GNNs in the presence of OOD nodes. Our empirical findings suggest that reducing the weights of edges connecting in-distribution~(ID) and OOD nodes can effectively alleviate the calibration issue. However, identifying such edges and determining their appropriate weights is challenging due to the unknown distribution of OOD nodes. To address this, GCSO introduces an iterative edge-sampling mechanism that captures the topological information of the graph and formulates the structure learning process as a Markov Decision Process (MDP). We then leverage an actor–critic method to dynamically adjust edge weights and evaluate their impact on target node predictions. Additionally, we design a tailored reward signal to guide the policy function toward an optimal adaptive graph structure that minimizes the influence of OOD nodes. Notably, the optimized graph structure can be seamlessly integrated with existing temperature scaling–based calibration techniques for further performance gains. Experimental results on benchmark datasets demonstrate that our method significantly reduces the expected calibration error while maintaining competitive accuracy.

Submission Type: Regular submission (no more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=tmZxjo07SB&referrer=%5BAuthor%20Console%5D(%2Fgroup%3Fid%3DTMLR%2FAuthors%23your-submissions)

Changes Since Last Submission:

1. We correct typos and grammatical errors from the previous submission to improve readability and overall presentation quality.

2. We further clarify several key claims concerning the motivation and methodology.

3. We provide a more detailed analysis of the computational complexity and demonstrate that our method is scalable to large datasets.

4. We conduct additional sensitivity analyses to further evaluate our method.

5. We conduct additional experiments under different OOD settings to validate the generalizability and effectiveness of our method.

6. We optimize our method to achieve statistically significant improvements over the baselines.

7. Similar to our integration with post-hoc calibration methods, we also evaluate the data-centric baseline DCGC combined with post-hoc calibration to demonstrate the advantages of our approach.

Assigned Action Editor: ~Hongfu_Liu2

Submission Number: 8384