Entropy Jurisprudence: Auditing Procedural Fidelity in LLM Normative Reasoning

CHEN XIWEI

Track: tiny / short paper (up to 4 pages)

Keywords: logical reasoning, procedural consistency, normative reasoning, LLM evaluation, reasoning stability, contradiction detection

TL;DR: LLMs with identical outcome accuracy exhibit dramatically different procedural fidelity when following formal rules, revealing failure modes invisible to standard evaluation.

Abstract: Outcome-correct but procedurally inconsistent reasoning poses deployment risks for LLM-based agents. We introduce Entropy Jurisprudence, a procedural audit framework testing whether LLMs faithfully execute formal normative rules. Using a minimal harm formula ($E = H \times R$), we measure parameter stability across 720 trials on six models. Results reveal a strong empirical alignment-reasoning tension: instruction-faithful models (Qwen3) execute rules reliably but may follow harmful logic; prior-dominant models (Gemma3) maintain safety but ignore parameters entirely (97.5% Guilty); context-sensitive models (Llama3) reconcile conflicts through scale hallucination—generating out-of-distribution numeric values (RI=328). Notably, all models achieve identical ETHICS-style accuracy (50%) while exhibiting dramatically different procedural fidelity, demonstrating that outcome-based evaluation alone is insufficient. Our framework provides a minimal methodology for auditing procedural fidelity before deploying LLM-based agents with irreversible real-world action capabilities.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Presenter: ~CHEN_XIWEI1

Format: No, the presenting author is unable to, or unlikely to be able to, attend in person.

Submission Number: 1