JUMP: Single-Pass Membership Inference on Fine-Tuned Diffusion Language Models

Yeachan Jun, Albert No

Keywords: membership inference, diffusion language models

Abstract:

Membership inference attacks (MIAs) test whether a candidate example appeared in a model's training data.
We study MIAs for fine-tuned discrete diffusion language models (dLLMs), where membership means inclusion in the target model's fine-tuning set.
Unlike autoregressive language models, dLLMs allow an attacker to choose arbitrary mask sets and obtain token distributions for all masked positions in parallel.
The prior dLLM attack, SAMA, follows a natural loss-mimicking strategy by averaging reconstruction signals over many randomly sampled masks, but it uses the any-order interface only as randomization and requires many target/reference queries.
We propose \textsc{JUMP} (\emph{Joint Uncertainty-Guided Mask Probing}), a single-pass scoring attack that exploits both distinctive properties of dLLMs:
any-order decodability is used to select low-reference-confidence positions, and parallel decodability is used to score all selected positions through one joint masked query per model.
\textsc{JUMP} masks the selected positions jointly and computes a clipped target/reference reconstruction-gap statistic.
On fine-tuned LLaDA-8B-Base across six MIMIR domains, \textsc{JUMP} improves mean ROC-AUC from $0.82$ to $0.90$ over SAMA and substantially improves low-FPR detection, while requiring only one selector pass and one scoring pass through each of the target and reference models.

Submission Number: 83