Beyond Word Boundaries: A Hebrew Coreference Benchmark and an Evaluation Protocol for Morphologically Complex Text

ACL ARR 2026 January Submission2624 Authors

Keywords: Coreference Resolution, Morphology, Morphologically Rich Languages, Hebrew

Abstract: Coreference Resolution (CR) is a fundamental NLP task, critical for long-form tasks as information extraction, summarization, and many business applications. However, CR methods originally designed for English struggle with Morphologically Rich Languages (MRLs), where mention boundaries often do not align with word boundaries, and a single token may consist of multiple anaphors. CR modeling and evaluation practices standardly assumes that, as in English, words and mentions mostly align, but this assumption breaks down for MRLs, creating a significant gap, particularly in the face of LLMs' raw text processing and end-to-end tasks. To both assess and address this gap, we introduce KibutzR, the first comprehensive CR dataset for Modern Hebrew, an MRL rich with complex words and pronominal clitics. We deliver an annotated dataset that identifies references at word, multi-word, and sub-word levels, and propose an evaluation protocol that directly addresses word/morpheme boundary discrepancies. Our experiments show that contemporary LLMs perform significantly worse on Hebrew than on English, and performance degrades on raw unsegmented text, highlighting the unique challenges posed for CR by morphology. Critically, we show an inverse performance-trend in Hebrew relative to English, where smaller encoders perform far better than the much larger neural decoders, leaving ample space for investigation and improvement. This work lays the ground for advancing coreference resolution in MRLs, towards robust NLP models for such languages.

Paper Type: Long

Research Area: Resources and Evaluation

Research Area Keywords: corpus creation; benchmarking; language resources; evaluation methodologies; evaluation; datasets for low resource languages;

Contribution Types: Data resources

Languages Studied: Hebrew

Submission Number: 2624