MaPLe: Marker-guided Partial Labeling

ACL ARR 2026 January Submission7278 Authors

Keywords: LLM-as-a-Evaluators, Marker-guided Partial Labeling, Prompt Masking, Reasoning Path Guidance, Chinese-L2 dataset

Abstract: In recent years, using large language models (LLMs) as evaluators has emerged as a new evaluation paradigm. However, when reasoning processes are complex, models often struggle to determine appropriate analysis directions, and unguided evaluation may lead to erroneous judgments. To address this, we propose a novel training strategy called MaPLe (Marker-guided Partial Labeling). This method explicitly triggers the model's implicit reasoning paths by randomly masking prompt information, thereby guiding the reasoning direction and enhancing evaluation accuracy. To validate the method's cross-lingual and multi-scenario adaptability, we constructed an automatic question-answering scoring chinese dataset for second language learners, Chinese-L2. Experimental results demonstrate that MaPLe achieves superior performance across multiple benchmarks and exhibits strong generalization capabilities in cross-lingual and multi-scenario data environments.Our method and related resources are released at {\url{https://anonymous.4open.science/r/MaPLe1-60D6}}

Paper Type: Long

Research Area: NLP Applications

Research Area Keywords: LLM-as-a-Evaluators, Automatic scoring

Contribution Types: Model analysis & interpretability, Data resources, Data analysis

Languages Studied: English, Chinese, Portuguese

Submission Number: 7278