Self Iterative Label Refinement via Robust Unlabeled Learning

Hikaru Asano, Tadashi Kozuno, Yukino Baba

Keywords: Large Language Models, Weakly Supervised Learning, LLM Self-Refinement, Low Resource Domain

TL;DR: We developed an iterative weakly supervised pipeline to refine LLM-generated pseudo-labels, consistently outperforming original LLMs and existing self-refinement methods across diverse datasets, while effectively supporting LLM safety alignment.

Abstract: Recent advances in large language models (LLMs) have yielded impressive performance on various tasks, yet they often depend on high-quality feedback that can be costly. Self-refinement methods attempt to leverage LLMs' internal evaluation mechanisms with minimal human supervision; however, these approaches frequently suffer from inherent biases and overconfidence, especially in domains where the models lack sufficient internal knowledge, resulting in performance degradation. As an initial step toward enhancing self-refinement for broader applications, we introduce an iterative refinement pipeline that employs the Unlabeled-Unlabeled learning framework to improve LLM-generated pseudo-labels for classification tasks. By exploiting two unlabeled datasets with differing positive class ratios, our approach iteratively denoises and refines the initial pseudo-labels, thereby mitigating the adverse effects of internal biases with minimal human supervision. Evaluations on diverse datasets, including low-resource language corpora, patent classifications, and protein structure categorizations, demonstrate that our method consistently outperforms both initial LLM's classification performance and the self-refinement approaches by cutting-edge models (e.g., GPT-4o and DeepSeek-R1). Moreover, we experimentally confirm that our refined classifier facilitates effective post-training alignment for safety in LLMs and demonstrate successful self-refinement in generative tasks as well. Our code is available at https://github.com/HikaruAsano/self-iterative-label-refinement.

Primary Area: General machine learning (supervised, unsupervised, online, active, etc.)

Submission Number: 24864