Uncertainty-Informed Meta Pseudo Labeling for Surrogate Modeling with Limited Labeled Data

Xingyu Ren, Pengwei Liu, Pengkai Wang, Guanyu Chen, Qinxin Wu, Dong Ni

Keywords: Surrogate modeling, Semi-supervised regression, Meta pseudo label

TL;DR: We propose a model-agnostic UMPL framework that leverages teacher-student uncertainty to refine pseudo labels and improve generalization with unlabeled data.

Abstract:

Deep neural networks, particularly neural operators, provide an efficient alternative to costly simulations in surrogate modeling. However, their performance is often constrained by the need for large-scale labeled datasets, which are costly and challenging to acquire in many scientific domains.
Semi-supervised learning reduces label reliance by leveraging unlabeled data yet remains vulnerable to noisy pseudo-labels that mislead training and undermine robustness.
To address these challenges, we propose a novel framework, Uncertainty-Informed Meta Pseudo Labeling (UMPL).
The core mechenism is to refine pseudo-label quality through uncertainty-informed feedback signals. Specifically, the teacher model generates pseudo labels via epistemic uncertainty, while the student model learns from these labels and provides feedback based on aleatoric uncertainty.
This interplay forms a meta-learning loop where enhanced generalization and improved pseudo-label quality reinforce each other, enabling the student model to achieve more stable uncertainty estimation and leading to more robust training.
Notably, This framework is model-agnostic and can be seamlessly integrated into various neural architectures, facilitating effective exploitation of unlabeled data to enhance generalization in distribution shifts and out-of-distribution scenarios.
Extensive evaluations of four models across seven tasks covering steady state and transient prediction problems demonstrate that UMPL consistently outperforms the best existing semi-supervised regression methods. When using only 10% of the fully supervised training data, UMPL achieves a 14.18% improvement, highlighting its strong effectiveness under limited supervision. Our codes are available at https://github.com/small-dumpling/UMPL.

Primary Area: General machine learning (supervised, unsupervised, online, active, etc.)

Submission Number: 10749