FovOptix: Human Vision-Compatible Video Encoding and Adaptive Streaming in VR Cloud Gaming

Ahmad Alhilal, Ze Wu, Yuk Hang TSUI, Pan Hui

Abstract: VR cloud gaming enables users to play high-end VR games on lightweight devices by offloading rendering tasks to cloud servers. Despite video compression, high-definition video streaming requires substantial data transfer rates. Foveated rendering (FR) and video encoding (FVE) leverage the non-uniform perception of the human visual system to reduce computing and bandwidth demand. They enhance visual quality in central gaze regions and reduce it in the periphery. However, bandwidth variation may hinder the provision of smooth VR gaming experiences. We present FovOptix, a system that combines FR with adaptive FVE to deliver video stream at a lower yet adaptive bitrate while not compromising the perceived video quality. FovOptix is based on a game-agnostic open-source to ensure reproducibility and compatibility with various games. We evaluate FovOptix against benchmarks using 5G mobile network traces. FovOptix achieves a latency reduction of 3% compared to the Google standard and a significant +100% reduction compared to other solutions. Additionally, it enhances the visual quality within the player's region of interest. Consequently, FovOptix attains the highest playability and gaming scores while minimizing the severity of motion sickness. FovOptix thus offers smooth and accessible VR cloud gaming for a wider range of players.