Joint Quantum Reinforcement Learning and Stabilized Control for Spatio-Temporal Coordination in Metaverse

Soohyun Park, Jaehyun Chung, Chanyoung Park, Soyi Jung, Minseok Choi, Sungrae Cho, Joongheon Kim

Abstract: In order to build realistic metaverse systems, enabling high synchronization between physical-space and virtual meta-space is essentially required. For this purpose, this paper proposes a novel system-wide coordination algorithm for high synchronization under characteristics (i.e., highly realistic meta-space construction under the constraints of physical-space). The proposed algorithm consists of the following three stages. The first stage is quantum multi-agent reinforcement learning (QMARL)-based scheduling for low-delay temporal-synchronization using differentiated age-of-information (AoI) during data gathering in physical-space by observers for meta-space construction. This is beneficial for scalability according to action dimension reduction in reinforcement learning computation. The second stage is for creating virtual contents under delay constraints in meta-space based on the gathered data. When rendering regions that have received more user attention, avatar-popularity is considered for spatio-synchronization. Thus, a stabilized control mechanism is designed for time-average reality quality maximization for each region. The last stage is for caching based on avatar-popularity and AoI which can be helpful in constructing low-delay realistic meta-space. Furthermore, the concept of AoI is divided into two separate sub-concepts of physical AoI and virtual AoI such that the AoI in virtual meta-space can be thoroughly implemented.