Beyond Rationalization: Criteria and Guidelines for Algorithmic Reasoning Traces in LLM Logical Reasoning

Karun Thankachan, Prateek Kohli

Track: tiny / short paper (up to 4 pages)

Keywords: chain-of-thought prompting, Neuro-symbolic AI, Symbolic reasoning, Linguistic rationalization, Logical reasoning

TL;DR: Logical reasoning must be separated as lingusitic rationalizion and algorithmic reasoning, and paper represents 4 criteria to distinguish that will help provide more structure to research in the field

Abstract: Chain-of-thought (CoT) prompting is now a standard way to elicit “reasoning” from large language models, but recent work shows that CoT can hurt accuracy on some pattern-based in-context learning tasks and often produces explanations that look reasonable but do not match how the model actually made its decision. At the same time, symbolic and neuro-symbolic approaches such as Faithful CoT, SymbCoT, and Logic-LM connect natural language traces to executable formalisms and obtain higher accuracy and verifiable faithfulness on logical benchmarks. In this \textit{position paper}, we argue that for logical reasoning it is important to separate linguistic rationalization, where CoT mainly describes an opaque pattern-matching process, from algorithmic reasoning, where the trace corresponds to a concrete computation that a solver can run. We propose four simple criteria for treating a CoT trace as algorithmic reasoning in logic-focused tasks and we suggest a three-condition evaluation protocol that compares direct answering, free-form CoT, and symbolic or neuro-symbolic CoT, together with lightweight checks of faithfulness and computational cost. The goal is to provide concrete guidance on when CoT should be treated only as a narrative aid and when logical reasoning research should instead require solver-backed, verifiable reasoning traces.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Submission Number: 171