KPI-Chain: Multi-Agent Planning with Entity-Based Task Chaining for Reliable Recovery

ICLR 2026 Conference Submission13987 Authors

Keywords: multi-agent systems, large language models, automated planning, failure recovery, entity extraction, key performance indicators, task chaining, JSON-path memory, Model Context Protocol, agent reliability

Abstract:

Large language model (LLM) agents and multi-agent systems promise flexible
problem-solving, but they remain brittle: plans often fail silently, and
existing approaches lack mechanisms for reliable recovery. We propose a
generic multi-agent planning framework called KPI-Chain with a novel plan
design that integrates per-task key performance indicators (KPIs) based on
entity extraction. In our formulation, each task—whether a tool call or a
reasoning step—is associated with a set of expected entities extracted from
its output. These entities are typed, and they serve both to determine task
success and to populate the input parameters of subsequent dependent tasks
retrieved from a task registry. If the KPI is not met, the system automatically
triggers replanning with failure feedback, enabling reliable recovery
from failure. To support this design, we introduce a JSON-path memory
representation for structured, queryable, and type-aware state tracking. We
integrate with Model Context Protocol (MCP) servers for standardized tool
access and use chain-of-thought prompting for reasoning tasks. Across 5
challenging benchmarks, our KPI-Chain framework achieves higher success
rates compared to existing agent architectures including ReAct and Planand-
Execute. These results suggest that KPI-driven planning with typed,
entity-based task chaining provides a foundation for building more reliable
and adaptive multi-agent systems.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 13987