Boldly Propose, Carefully Verify: LLMs in Agents Should Be Used Only For Abduction

Eneko Sabaté Iturgaiz, Victor Gimenez-Abalos, Adrián Tormos, Oriol Miro-Lopez-Feliu, Sergio Alvarez-Napagao

Keywords: Abduction, Inference to the best explanation, LLM-based agents, Scientific discovery, Learning, Agentic AI, Knowledge representation, Knowledge Transfer, Cognitive Architecture

TL;DR: We propose the usage of LLMs to guess why the environment behaved unexpectedly with respect to knowledge, and propose goals to verify whether the guess was correct, interfacing with a simple means-ends reasoner. We test this in the game Baba Is You.

Abstract: The appearance of Large Language Models (LLMs) has transformed the field of agents with new architectures and inference mechanisms. These systems, however, struggle with capabilities such as rationality or coherent behaviour. Stand-alone LLM-based architectures are not capable of reliably performing planning to produce rational behaviour. On the other hand, traditional agents have long-standing knowledge to tackle these capabilities, but struggle with tasks requiring autonomous abstractions and knowledge abduction, making some domains unattainable: these agents do not have full reasoning capabilities due to failing at abduction. While abduction is computationally difficult, LLMs excel at producing probable hypotheses without many requirements. Moreover, verifying these hypotheses via deductive knowledge and interaction with the surroundings is within the capacity of traditional agents, and the feedback can establish a virtuous loop with abduction mechanisms. In this paper, we propose to work toward hybrid systems integrating two stand-alone LLM-based abductive modules: an Abductive Reasoner Module to generate hypotheses based on detected discrepancies between agent beliefs and environment behaviour, and an Experiment Designer Module to generate goals testing these hypotheses. These modules could be integrated with cognitive architectures using means-ends reasoning to fulfil goals (tests and system-goals). We illustrate this perspective through a practical case study in Baba Is You, a complex rule-learning environment, empirically showing that abductive-deductive separation is viable even with minimal agent design.

Journal Edition Interest: Yes

Submission Number: 52