Bootstrapping LLM Agents via Verification

Anonymous

Abstract: We present a self-training method that allows language model-based agents to improve performance without distilling proprietary models. Existing self-verification methods struggle to validate function signatures defined in agent prompts. A common failure is the verifier hallucinating non-existent constraints on function calls due to interference between model knowledge and examples in prompts. To address this, we devise a neural-symbolic verification system that prioritizes language models for validating solution completeness and pertinence while delegating fact checks to a symbolic system. We propose bootstrap-by-verification learning which combines massive agent trajectory sampling with our verification for self-training. Experiments on spreadsheet and web browsing benchmarks show the method's effectiveness.

Paper Type: long

Research Area: NLP Applications

Languages Studied: English