Evolving and Detecting Multi-Turn Deception using Geometric Signatures

Surender Suresh Kumar, Mary Cummings

Keywords: Large Language Models, Multi-Turn Deception, Prompt Optimization, Explainable AI, Evolutionary Algorithms, Synthetic Data, Geometric Signatures

TL;DR: We investigate geometric signature of multi-turn deception in embeddings; we generate synthetic data, validate with a human study, and show detection

Abstract: Safety defenses for large language models (LLMs) are typically trained and evaluated on single-turn prompts, yet real attacks often unfold as indirect, multi-turn probing. To defend against this more nuanced form of deception, we present a unified pipeline that generates realistic multi-turn deceptive question sets via multi-objective genetic prompt optimization with co-evolving mutation operators. We validate this data set through a human study, which also revealed that early generations yielded the most convincing deception and practical constraints such as adherence filtering and ordering effects. Using this data, we were able to detect deceptive attempts to access prohibited information using simple, explainable geometric signals in embedding space coupled with a tiny feed-forward classifier. Three geometric features (angular coverage, distance ratio, and linearity) augmented with pairwise similarity statistics led to a compact predictive model that achieved consistently high recall (0.89) across base, reworded, and truncated (three-turn) scenarios, with test-time F1 ranging from 0.74-0.86. The results support a central hypothesis that multi-turn deceptive intent leaves a stable geometric footprint that enables lightweight, transparent screening without expensive end-to-end training. We further discuss responsible uses, limitations, and paths toward larger, more diverse human-evaluated datasets.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 23021