L2E: Learning to Exploit Your Opponent

Zhe Wu, Kai Li, Enmin Zhao, Hang Xu, Haobo Fu, QIANG FU, Bo An, Junliang Xing

Keywords: Opponent Modeling, Learning to Learn

Abstract: Opponent modeling is essential to exploit sub-optimal opponents in strategic interactions. Most previous works focus on building explicit models to directly predict the opponents' styles or strategies, which require a large amount of data to train the model and lack adaptability to unknown opponents. In this work, we propose a novel Learning to Exploit (L2E) framework for implicit opponent modeling. L2E acquires the ability to exploit opponents by a few interactions with different opponents during training, thus can adapt to new opponents with unknown styles during testing quickly. We propose a novel opponent strategy generation algorithm that produces effective opponents for training automatically. We evaluate L2E on two poker games, a grid-world soccer environment, and a high-dimensional simulated robot environment, which are very challenging benchmarks for opponent modeling. Comprehensive experimental results indicate that L2E quickly adapts to diverse styles of unknown opponents.

One-sentence Summary: We propose a framework to quickly exploit sub-optimal opponents without building explicit opponent models.