Diffusion-based Behavior Cloning in Multi-Agent Games via Dynamic Guidance

ICLR 2026 Conference Submission20695 Authors

Keywords: imitation learning, multi- agent task, diffusion, behavior cloning

Abstract: In offline multi-agent imitation learning, agents are constrained to learn from static datasets without interaction, which poses challenges in generalizing across diverse behaviors. Behavior Cloning (BC), a widely used approach, models conditional actions from local observations but lacks robustness under behavioral variability. Recent diffusion-based policies have been introduced to capture diverse action distributions. However, in multi-agent environments, their iterative denoising process can accumulate errors in interactive settings, degrading performance under shifting opponent behaviors. To address these challenges, we propose Diffusion Dynamic Guidance Imitation Learning (DDGIL), a diffusion-based framework built on classifier-free guidance (CFG), which balances conditional and unconditional denoising predictions. Unlike prior methods with fixed weighting, DDGIL introduces a dynamic guidance mechanism that adaptively adjusts the weight at each denoising step, enhancing stability across different agent strategies. Empirical evaluations on competitive and cooperative benchmarks show that DDGIL achieves reliable performance. In high-fidelity sports simulations, it reproduces action strategies that closely resemble expert demonstrations while maintaining robustness against diverse opponents.

Primary Area: reinforcement learning

Submission Number: 20695