Real, Fake, or Manipulated? Detecting Machine-Influenced Text

ACL ARR 2025 May Submission1755 Authors

Abstract: Large Language Model (LLMs) can be used to write or modify documents, presenting a challenge for understanding the intent behind their use. For example, benign uses may involve using LLM on human-written document to improve its grammar or to translate it into another language. However, a document entirely produced by a LLM may be more likely used to spread misinformation than simple translation (\eg, from use by malicious actors or simply by hallucinating). Prior works on Machine Generated Text (MGT) detection task mostly focus on simply identifying a document has human or machine written, ignoring these more fine-grained uses. In this paper, we introduce HERO, which learns to separate text samples of varying lengths from four primary types: human-written, machine-generated, machine polished, and machine-translated. HERO accomplishes this by combining predictions from length-specialist models that have been trained with Subcategory Guidance. Specifically, for categories that are easily confused (\eg, the different source languages), our Subcategory Guidance module encourages separation of the fine-grained categories, boosting performance. Extensive experiments across five LLMs and six domains demonstrate the benefits of our HERO approach, where we outperform the state-of-the-art by 2.5-3 mAP on average.

Paper Type: Long

Research Area: Ethics, Bias, and Fairness

Research Area Keywords: fine-grained machine generated text detection, defending against misinformation, machine translated text

Languages Studied: English, Chinese, Russian, Spanish, French

Submission Number: 1755