Cognitive Loop: Reversible Hierarchical Markov Chain for Bidirectional Self-Verifying Reasoning

Jia-Chen Zhang, Zheng Zhou, Yu-Jie Xiong, Chun-Ming Xia, Xihe Qiu

Keywords: Large language models, Chain of thought, Markov chain

TL;DR: We propose a novel Chain-of-Thought framework based on Reversible Hierarchical Markov Chains, termed Cognitive Loop of Thought (CLOT), and a backward reasoning dataset CLOT-Instruct.

Abstract: Multi-step Chain-of-Thought (CoT) has significantly enhanced the mathematical reasoning capabilities of large language models by leveraging clear reasoning steps and task-specific logical structures. However, with the widespread adoption of Long CoT, the number of reasoning steps often exceeds the system's manageable limits. To address this, existing approaches attempt to reduce redundancy in KV Cache by introducing Markov chain-like reasoning structures, thereby improving inference efficiency. Nonetheless, such Markov chain-based reasoning methods introduce two critical issues: Lack of memory and Limited backward reasoning capability. To address these limitations, we propose a novel Chain-of-Thought framework based on Reversible Hierarchical Markov Chains, termed Cognitive Loop of Thought (CLoT), and a backward reasoning dataset CLoT-Instruct. In CLoT, the original problem is decomposed into sub-problems with hierarchical dependencies and modeled as a hierarchical Markov chain based on the number of dependencies. Humans typically revisit and verify their reasoning steps after reaching a conclusion to avoid errors. Inspired by this cognitive behavior, we introduce a similar backward verification mechanism at each layer. Moreover, when all higher-level (multi-dependency) sub-problems are verified as correct, we prune the remaining lower-level (fewer-dependency) sub-problems. CLoT effectively mitigates error propagation along the reasoning path and enhances the robustness of the entire reasoning process. We conduct experiments on four mathematical reasoning benchmarks, demonstrating the effectiveness of CLoT. Notably, on the AddSub dataset, when applied to the GPT-4o-mini model, CLoT achieves an accuracy of 99.0\% , outperforming traditional CoT and CoT-SC by 4.1\% and 2.9\%. Our code is publicly available at: https://anonymous.4open.science/r/CLoT-7EBD.

Primary Area: interpretability and explainable AI

Submission Number: 7119