CausalPlan: Empowering Efficient LLM Multi-Agent Coordination Through Causality-Driven Planning

TMLR Paper8657 Authors

Abstract: Large language model (LLM) agents often generate causally invalid plans in multi-agent coordination tasks due to reliance on spurious statistical correlations rather than grounded causal reasoning, leading to poor task performance. We propose CausalPlan, a framework that enforces causal consistency in LLM planning by embedding learned structural knowledge directly into the decoding process. CausalPlan first extracts a Structural Causal Action (SCA) model, which learns a policy-level causal graph from agent trajectories to capture how prior actions and current environment states influence future decisions. The learned SCA guides planning by reweighting candidate action tokens during generation and providing grounded alternatives when causal violations are detected. By embedding causal knowledge, CausalPlan constrains planning to causal-consistent behaviors under the learned causal model without requiring fine-tuning. We evaluated CausalPlan on the Overcooked-AI benchmark across five multi-agent tasks and four LLMs: Gemma-7B, Llama-8B, Qwen-14B and Llama-70B. Experimental results show that CausalPlan consistently reduces causally invalid actions and improves task completion in both AI-AI and human-AI collaboration settings, outperforming strong LLMs and reinforcement learning baselines. Our findings demonstrate that causality-driven planning is essential for deploying efficient, interpretable, and robust multi-agent systems.

Submission Type: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~bo_han2

Submission Number: 8657