Revision History for Training-free LLM-generated Text...

Camera Ready Revision Edit by Authors

  • 18 Feb 2025, 07:31 Coordinated Universal Time
  • Title: Training-free LLM-generated Text Detection by Mining Token Probability Sequences
  • Authors: Yihuai Xu, Yongwei Wang, Yifei Bi, Huangsen Cao, Zhouhan Lin, Yu Zhao, Fei Wu
  • Authorids: Yihuai Xu, Yongwei Wang, Yifei Bi, Huangsen Cao, Zhouhan Lin, Yu Zhao, Fei Wu
  • Keywords: Fake text detection, training-free, detection
  • TLDR: We propose a novel and effective training-free method for detecting LLM-generated text by mining token probability sequences.
  • Abstract:

    Large language models (LLMs) have demonstrated remarkable capabilities in generating high-quality texts across diverse domains. However, the potential misuse of LLMs has raised significant concerns, underscoring the urgent need for reliable detection of LLM-generated texts. Conventional training-based detectors often struggle with generalization, particularly in cross-domain and cross-model scenarios. In contrast, training-free methods, which focus on inherent discrepancies through carefully designed statistical features, offer improved generalization and interpretability. Despite this, existing training-free detection methods typically rely on global text sequence statistics, neglecting the modeling of local discriminative features, thereby limiting their detection efficacy. In this work, we introduce a novel training-free detector, termed \textbf{Lastde}\footnote{The code and data are released at \url{https://github.com/TrustMedia-zju/Lastde_Detector}.} that synergizes local and global statistics for enhanced detection. For the first time, we introduce time series analysis to LLM-generated text detection, capturing the temporal dynamics of token probability sequences. By integrating these local statistics with global ones, our detector reveals significant disparities between human and LLM-generated texts. We also propose an efficient alternative, \textbf{Lastde++} to enable real-time detection. Extensive experiments on six datasets involving cross-domain, cross-model, and cross-lingual detection scenarios, under both white-box and black-box settings, demonstrated that our method consistently achieves state-of-the-art performance. Furthermore, our approach exhibits greater robustness against paraphrasing attacks compared to existing baseline methods.

  • PDF: pdf
  • Supplementary Material: zip
  • Primary Area: alignment, fairness, safety, privacy, and societal considerations

    Edit Info


    Readers: Everyone
    Writers: ICLR 2025 Conference, ICLR 2025 Conference Submission6494 Authors
    Signatures: ICLR 2025 Conference Submission6494 Authors

    Camera Ready Revision Edit by Authors

    • 18 Feb 2025, 06:59 Coordinated Universal Time
    • Title: Training-free LLM-generated Text Detection by Mining Token Probability Sequences
    • Authors: Yihuai Xu, Yongwei Wang, Yifei Bi, Huangsen Cao, Zhouhan Lin, Yu Zhao, Fei Wu
    • Authorids: Yihuai Xu, Yongwei Wang, Yifei Bi, Huangsen Cao, Zhouhan Lin, Yu Zhao, Fei Wu
    • Keywords: Fake text detection, training-free, detection
    • TLDR: We propose a novel and effective training-free method for detecting LLM-generated text by mining token probability sequences.
    • Abstract:

      Large language models (LLMs) have demonstrated remarkable capabilities in generating high-quality texts across diverse domains. However, the potential misuse of LLMs has raised significant concerns, underscoring the urgent need for reliable detection of LLM-generated texts. Conventional training-based detectors often struggle with generalization, particularly in cross-domain and cross-model scenarios. In contrast, training-free methods, which focus on inherent discrepancies through carefully designed statistical features, offer improved generalization and interpretability. Despite this, existing training-free detection methods typically rely on global text sequence statistics, neglecting the modeling of local discriminative features, thereby limiting their detection efficacy. In this work, we introduce a novel training-free detector, termed \textbf{Lastde} that synergizes local and global statistics for enhanced detection. For the first time, we introduce time series analysis to LLM-generated text detection, capturing the temporal dynamics of token probability sequences. By integrating these local statistics with global ones, our detector reveals significant disparities between human and LLM-generated texts. We also propose an efficient alternative, \textbf{Lastde++} to enable real-time detection. Extensive experiments on six datasets involving cross-domain, cross-model, and cross-lingual detection scenarios, under both white-box and black-box settings, demonstrated that our method consistently achieves state-of-the-art performance. Furthermore, our approach exhibits greater robustness against paraphrasing attacks compared to existing baseline methods. {Our codes are available at \url{https://anonymous.4open.science/r/Lastde-5DBC} anonymously}.

    • PDF: pdf
    • Supplementary Material: zip
    • Primary Area: alignment, fairness, safety, privacy, and societal considerations

      Edit Info


      Readers: Everyone
      Writers: ICLR 2025 Conference, ICLR 2025 Conference Submission6494 Authors
      Signatures: ICLR 2025 Conference Submission6494 Authors

      Rebuttal Revision Edit by Authors

      • 27 Nov 2024, 09:03 Coordinated Universal Time
      • Abstract:

        Large language models (LLMs) have demonstrated remarkable capabilities in generating high-quality texts across diverse domains. However, the potential misuse of LLMs has raised significant concerns, underscoring the urgent need for reliable detection of LLM-generated texts. Conventional training-based detectors often struggle with generalization, particularly in cross-domain and cross-model scenarios. In contrast, training-free methods, which focus on inherent discrepancies through carefully designed statistical features, offer improved generalization and interpretability. Despite this, existing training-free detection methods typically rely on global text sequence statistics, neglecting the modeling of local discriminative features, thereby limiting their detection efficacy. In this work, we introduce a novel training-free detector, termed \textbf{Lastde} that synergizes local and global statistics for enhanced detection. For the first time, we introduce time series analysis to LLM-generated text detection, capturing the temporal dynamics of token probability sequences. By integrating these local statistics with global ones, our detector reveals significant disparities between human and LLM-generated texts. We also propose an efficient alternative, \textbf{Lastde++} to enable real-time detection. Extensive experiments on six datasets involving cross-domain, cross-model, and cross-lingual detection scenarios, under both white-box and black-box settings, demonstrated that our method consistently achieves state-of-the-art performance. Furthermore, our approach exhibits greater robustness against paraphrasing attacks compared to existing baseline methods. {Our codes are available at \url{https://anonymous.4open.science/r/Lastde-5DBC} anonymously}.

      • PDF: pdf
      • Supplementary Material: zip

        Edit Info


        Readers: Everyone
        Writers: ICLR 2025 Conference, ICLR 2025 Conference Submission6494 Authors
        Signatures: ICLR 2025 Conference Submission6494 Authors

        Rebuttal Revision Edit by Authors

        • 26 Nov 2024, 15:18 Coordinated Universal Time
        • Abstract:

          Large language models (LLMs) have demonstrated remarkable capabilities in generating high-quality texts across diverse domains. However, the potential misuse of LLMs has raised significant concerns, underscoring the urgent need for reliable detection of LLM-generated texts. Conventional training-based detectors often struggle with generalization, particularly in cross-domain and cross-model scenarios. In contrast, training-free methods, which focus on inherent discrepancies through carefully designed statistical features, offer improved generalization and interpretability. Despite this, existing training-free detection methods typically rely on global text sequence statistics, neglecting the modeling of local discriminative features, thereby limiting their detection efficacy. In this work, we introduce a novel training-free detector, termed \textbf{Lastde} that synergizes local and global statistics for enhanced detection. For the first time, we introduce time series analysis to LLM-generated text detection, capturing the temporal dynamics of token probability sequences. By integrating these local statistics with global ones, our detector reveals significant disparities between human and LLM-generated texts. We also propose an efficient alternative, \textbf{Lastde++} to enable real-time detection. Extensive experiments on six datasets involving cross-domain, cross-model, and cross-lingual detection scenarios, under both white-box and black-box settings, demonstrated that our method consistently achieves state-of-the-art performance. Furthermore, our approach exhibits greater robustness against paraphrasing attacks compared to existing baseline methods. {Our codes are available at \url{https://anonymous.4open.science/r/Lastde-5DBC} anonymously}.

        • PDF: pdf
        • Supplementary Material: zip

          Edit Info


          Readers: Everyone
          Writers: ICLR 2025 Conference, ICLR 2025 Conference Submission6494 Authors
          Signatures: ICLR 2025 Conference Submission6494 Authors

          Rebuttal Revision Edit by Authors

          • 24 Nov 2024, 11:57 Coordinated Universal Time
          • Abstract:

            Large language models (LLMs) have demonstrated remarkable capabilities in generating high-quality texts across diverse domains. However, the potential misuse of LLMs has raised significant concerns, underscoring the urgent need for reliable detection of LLM-generated texts. Conventional training-based detectors often struggle with generalization, particularly in cross-domain and cross-model scenarios. In contrast, training-free methods, which focus on inherent discrepancies through carefully designed statistical features, offer improved generalization and interpretability. Despite this, existing training-free detection methods typically rely on global text sequence statistics, neglecting the modeling of local discriminative features, thereby limiting their detection efficacy. In this work, we introduce a novel training-free detector, termed \textbf{Lastde} that synergizes local and global statistics for enhanced detection. For the first time, we introduce time series analysis to LLM-generated text detection, capturing the temporal dynamics of token probability sequences. By integrating these local statistics with global ones, our detector reveals significant disparities between human and LLM-generated texts. We also propose an efficient alternative, \textbf{Lastde++} to enable real-time detection. Extensive experiments on six datasets involving cross-domain, cross-model, and cross-lingual detection scenarios, under both white-box and black-box settings, demonstrated that our method consistently achieves state-of-the-art performance. Furthermore, our approach exhibits greater robustness against paraphrasing attacks compared to existing baseline methods. {Our codes are available at \url{https://anonymous.4open.science/r/Lastde-5DBC} anonymously}.

          • PDF: pdf
          • Supplementary Material: zip

            Edit Info


            Readers: Everyone
            Writers: ICLR 2025 Conference, ICLR 2025 Conference Submission6494 Authors
            Signatures: ICLR 2025 Conference Submission6494 Authors