Automatic Difficulty Balance in Two-Player Games with Deep Reinforcement Learning

Simão Reis, Rita Novais, Luís Paulo Reis, Nuno Lau

Abstract: Regardless of the goal of a game, it should be a pleasant and fun experience for its players. For some games to be enjoyable, the level of difficulty must be carefully calibrated, otherwise, players will feel bored or frustrated. Multiplayer scenarios in particular, where one player’s satisfaction might not translate to the enjoyment of other players and poses extra challenges in balancing the difficulty. The performance of one player is relative to the opponent, versus single-player scenarios where we can fully control the environment. We propose an AI automation framework for difficulty balancing in two-player games, where balancing is seen as a Reinforcement Learning task. A Game Master (GM) agent learns how to use handicap game mechanics, signaled by a reward function that evaluates a weighted combination of aesthetic criteria that encourages dramatization and allows a player in the lead to go back and a player in the rear to catch up, creating the desired rubber banding effect that balances out skill gaps. The quality of the games with the trained GM embedded is examined by measuring the same aesthetic criteria on the resulting games, and by analyzing the resulting changes in the game.