Agent Systems with Harness Engineering

Xinyu Tang, Han Peng, Guoxin Chen, Yuze Shi, Zitao Su, Peiyu Liu, Xin Zhao, Yawen Li, Zhe Xue

Abstract: Developing effective agent systems has been a long-standing frontier in artificial intelligence. The recent emergence of LLM-based agents marks a paradigm shift, yet even highly capable models deployed without adequate system support exhibit systematic failures in long-horizon settings, including loss of intermediate goals, incorrect tool invocation, and inability to incorporate environmental feedback. These failures reflect not gaps in model training but a structural mismatch between the single-turn generative interface of LLMs and the stateful, iterative nature of real-world problem solving. In this paper, we formalize the surrounding infrastructure as the harness and introduce harness engineering as the joint optimization of both the harness and the model it supports. The relationship between the two is inherently synergistic: a well-designed harness unlocks latent model potential through structured memory, error recovery, and adaptive context management, while a capable model enables the harness to implement sophisticated workflows that would be infeasible with a weaker reasoner. We present a structured taxonomy of harness components, spanning agent workflows, memory systems, skill libraries, and multi-agent orchestration, and analyze how each contributes to system-level performance, distilling design principles and architectural trade-offs from current practice. We further review optimization strategies from both the scaffold side and the model side, and summarize evaluation benchmarks across software engineering, deep research, tool use, computer use, and scientific discovery. By shifting focus from what agents can accomplish to how the surrounding infrastructure enables them to do so, this paper aims to serve as a practical reference for building reliable, scalable, and controllable agent systems through principled harness engineering.