Evaluating a Procedural Content Orchestrator Gameplay Data and Classifying User Profiles

Leonardo Tórtoro Pereira, Tyago Yuji Teoi, Claudio Fabiano Motta Toledo

Abstract: Background: Content Orchestration is a novel research field focused on coordinating distinct types of algorithmically generated game content. Purpose: Thus, the lack of research in this area hinders the analysis of gameplay data and player profiling in games with orchestrated content. Methods: This paper is an extension of a work that collected and analyzed gameplay logs of 15 players who played 119 game sections of 12 different dungeons of a top-down action game. The game’s Levels, Rules, and Narrative content were orchestrated and adapted to player profiles defined from a pre-test questionnaire. PCA and clustering techniques were used to highlight relevant gameplay metrics for distinguishing play styles. In this extension, we used the gameplay data alone to train classifiers with and without data augmentation to predict a user’s profile, measuring the accuracy, precision, recall and f1-score with a train-test split and a 5-fold cross-validation for a more robust accuracy. We also implemented data augmentation on our gameplay metrics sample. Results: We identified, through the previous work, two components of PCA explaining a total of 65% of data variability, containing data such as Lock Usage Rate, Enemy Kill Rate, Map Completion, and Completed Immersion Quests. We also found game difficulty as an important level component for impact clustering. Through data augmentation, we achieved novel results, such as a mean accuracyof almost 95%, measured with a 5-fold cross-validation, for the Histogram-based Gradient Boosting classifier when predicting a player’s profile based on their gameplay data, even with our small sample size. Conclusion: Our work guides developers and researchers to choose relevant gameplay metrics to determine players’ play styles. Our extended results suggest that we can predict player’s profiles through gameplay metrics and data augmentation, even for small samples. More studies are needed to validate our findings, with a larger and more diverse player-base.