Conversation as Belief Revision: GreedySAT Revision for Global Logical Consistency in Multi-Turn LLM Dialogues

Sanjan Baitalik, Rajashik Datta, Amit Kumar Das, Sruti Das Choudhury

Keywords: Logical Consistency, Belief Revision, Multi-Turn Dialogue, Large Language Models, Neuro-Symbolic Reasoning, Chain-of-Thought Prompting, Propositional Logic

Abstract:

Large language models (LLMs) are increasingly deployed as interactive assistants, yet their responses are often evaluated in isolation rather than as components of an evolving belief state. Recent benchmarks reveal that even state-of-the-art LLMs frequently violate basic logical consistency, especially under negation, multi-step entailment, or adversarial question sequences. We argue that multi-turn dialogue with an LLM should be viewed as a process of constructing and revising an explicit theory of the world.

We propose GreedySAT Revision, a lightweight, backend-agnostic framework that treats an LLM as a black-box generator of propositional commitments, wrapped by an external symbolic solver maintaining a globally consistent belief state. At each turn, the LLM proposes an answer to a query about a synthetic world; we map it to a propositional literal and tentatively add it to the current theory. A SAT/SMT-based checker verifies satisfiability against world rules and, if needed, performs minimal belief revision by retracting prior commitments. We instantiate this with API-only models---OpenAI gpt-4.1-mini and Gemini 2.5 Flash---without finetuning, evaluating on synthetic multi-turn logical dialogues under random and stress-test query schedules.

On gpt-4.1-mini, our solver-augmented system eliminates all inconsistent final belief states in adversarial ``stress'' settings (from $5/120$ to $0/120$ dialogues under direct prompting, and $3/120$ to $0/120$ under chain-of-thought), preserving task accuracy within $0.3$ points. Across conditions, it incurs only $12$ and $10$ retractions, respectively---about $0.08$--$0.10$ per dialogue. On Gemini 2.5 Flash, the belief-state interpreter boosts raw per-turn accuracy (e.g., from $0.32$ to $0.49$ and $0.29$ to $0.41$ in random and stress direct settings), with the solver rarely intervening. These results show that explicit, solver-checked global belief states provide strong logical guarantees for interactive LLMs without large or GPU-intensive models, paving the way for trustworthy neuro-symbolic multi-turn reasoning.

Submission Number: 113