Factored Representation for Neuro-Symbolic AI

ICLR 2026 Conference Submission19319 Authors

Keywords: reasoning, knowledge representation, large language models, neuro-symbolic

TL;DR: Imposing reasoning constraints on an LLM to use a structured representation tends to conflict with its learned reasoning patterns but also shows the potential to improve model's reasoning performance.

Abstract: Large language models (LLMs) have demonstrated remarkable capabilities across a wide range of tasks, yet their internal decision-making processes remain largely opaque, posing a significant challenge to their trustworthiness in complex reasoning scenarios. We explore the hypothesis that compelling a pre-trained LLM to maintain a structured internal state using a formal, symbolic representation can enhance interpretability without degrading its reasoning performance. To investigate this, we employ two primary techniques: prompt fine-tuning and parameter-efficient fine-tuning (PEFT) using LoRA. Thus, we prompt a variety of LLMs to articulate their reasoning steps using various structured formalisms, including basic semantic triples, lists of attribute-value pairs, and first-order logic. We also fine-tune a pre-trained LLM on a structured representation that the LLM is subsequently prompted to use as an internal representation during reasoning to solve a task. Our results demonstrate that while state-of-the-art models struggle to generate consistently structured reasoning, their core reasoning capabilities remain largely intact. This suggests that the LLM's reasoning mechanism is not necessarily fully aligned with its generative capabilities. However, this result shows the potential for specialized models capable of performing complex reasoning while providing verifiable chains of thought.

Primary Area: neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)

Submission Number: 19319