RN-F: A Novel Approach for Mitigating Contaminated Data in Large Language Models

Le Vu Anh, Dinh Duc Nha Nguyen, Phi Long Nguyen, Keshav Sood

Keywords: anomaly detection, data poisoning, quantization, quantization residuals

TL;DR: We introduce RN-F, a gradient-free method that uses quantization noise to detect contaminated data in LLMs with state-of-the-art accuracy and minimal compute cost.

Abstract: Large Language Models (LLMs) have become foundational in modern artificial intelligence, powering a wide range of applications from code generation and virtual assistants to scientific research and enterprise automation. However, concerns about data contamination, where test data overlaps with training data, have raised serious questions about the reliability of these applications. Despite awareness of this issue, existing methods fall short in effectively identifying or mitigating contamination. In this paper, we propose Residual-Noise Fingerprinting (RN-F), a novel framework for detecting contaminated data in LLMs. RN-F is a single-pass, gradient-free detection method that leverages residual signal patterns without introducing additional floating-point operations. Our approach is lightweight, model-agnostic, and efficient. We evaluate RN-F on multiple LLMs across various contaminated datasets and show that it consistently outperforms existing state-of-the-art methods, achieving performance improvements of up to 11.1% in contamination detection metrics.

Submission Number: 15