JudgeFlow: Agentic Workflow Optimization via Block Judge

ICLR 2026 Conference Submission15919 Authors

Keywords: LLM-agent, Agentic Workflows, Multi-agent System

TL;DR: We propose JudgeFlow which incorporates a judge module to identify problematic block, guiding the LLM optimizer.

Abstract: Optimizing LLM-based agentic workflows is challenging for scaling AI capabilities, some current methods rely on coarse, end-to-end evaluation signals and lack fine-grained signals on where to refine, often resulting in inefficient or low-impact modifications. To address these limitations, we propose JudgeFlow, an Evaluation-Judge-Optimization-Update pipeline. We incorporate reusable and configurable logic blocks into agentic workflows, capturing fundamental forms of logic. On top of this abstraction, we design a dedicated Judge module that inspects execution traces, particularly failed runs, and assigns rank-based responsibility scores to problematic blocks. These fine-grained diagnostic signals are then leveraged by an LLM-based optimizer, which focuses modifications on the most problematic block in the workflow. Our approach improves sample efficiency, enhances interpretability through block-level diagnostics, and provides a scalable foundation for automating increasingly complex agentic workflows. We evaluate JudgeFlow on mathematical reasoning and code generation benchmarks, and the results demonstrate that JudgeFlow achieves superior performance and optimization efficiency compared to existing methods.

Primary Area: applications to robotics, autonomy, planning

Submission Number: 15919