Bilevel Entropy based Mechanism Design for Balancing Meta in Video Games

Sumedh Pendurkar, Chris Chow, Luo Jie, Guni Sharon

Abstract:

We address a mechanism design problem where the goal of the
designer is to maximize the entropy of a player’s mixed strategy at
a Nash equilibrium. This objective is of special relevance to video
games where game designers wish to diversify the players’ inter-
action with the game. To solve this design problem, we propose a
bi-level alternating optimization technique that (1) approximates
the mixed strategy Nash equilibrium using a Nash Monte-Carlo
reinforcement learning approach and (2) applies a gradient-free op-
timization technique (Covariance-Matrix Adaptation Evolutionary
Strategy) to maximize the entropy of the mixed strategy obtained in
level (1). The experimental results show that our approach achieves
comparable results to the state-of-the-art approach on three bench-
mark domains “Rock-Paper-Scissors-Fire-Water”, “Workshop War-
fare” and “Pokemon Video Game Championship”. Next, we show
that, unlike previous state-of-the-art approaches, the computational
complexity of our proposed approach scales significantly better in
larger combinatorial strategy spaces.