CandyRL: A Hybrid Reinforcement Learning Model for Gameplay

Sara Karimi, Sahar Asadi, Francesco Lorenzo, Amir Hossein Payberah

Abstract: Although Reinforcement Learning (RL) is becoming increasingly popular in gameplay, making a generalized RL model is still challenging. This paper presents CandyRL, a generalized RL solution for match-3 games, particularly Candy Crush Friends Saga. CandyRL rewards RL agents not only for achieving the game objectives but also for learning some intrinsic basic skills, which are not directly related to the game objectives. CandyRL also includes a hybrid model that combines several pre-trained agents to determine the actions. We propose two approaches to determine the weights of the pre-trained agents in the hybrid model to reflect their importance by using (i) a heuristic method and (ii) an RL-based approach. We show that the hybrid model outperforms all the pre-trained agents used as its building blocks through the experiments. Moreover, the RL-based approach of learning weights in the hybrid model shows better generalization than the other approaches as it performs better on unseen new game levels.