Machine Text Detectors are Membership Inference Attacks

Ryuto Koike, Liam Dugan, Masahiro Kaneko, Chris Callison-Burch, Naoaki Okazaki

Keywords: Membership inference, Machine-generated text detection

Abstract: In this work, we theoretically and empirically demonstrate the transferability, i.e., how well a method originally developed for one task performs on the other, between membership inference attacks (MIAs) and machine-generated text detection. We prove that the asymptotically optimal metric is identical for both tasks, unify existing methods under this metric, and hypothesize that how well a method approximates it shapes its transferability. Our large-scale experiments show a strong rank correlation ($\rho \approx 0.7$) in cross-task performance, and notably, a machine text detector achieves the strongest performance among evaluated methods on both tasks. To facilitate cross-task development and fair evaluation, we introduce MINT, a unified evaluation suite implementing 15 recent methods from both tasks.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 76