Lumos: Optimizing Live 360-degree Video Upstreaming via Spatial-Temporal Integrated Neural Enhancement
Abstract: As VR devices become increasingly prevalent, live 360-degree video has surged in popularity. However, current live 360-degree video systems heavily rely on uplink bandwidth to deliver high-quality live videos. Recent advancements in neural-enhanced streaming offer a promising solution to this limitation by leveraging server-side computation to conserve bandwidth. Nevertheless, these methods have primarily concentrated on neural enhancement within a single domain (either spatial or temporal), which may not adeptly adapt to diverse video scenarios and fluctuating bandwidth conditions. In this paper, we propose Lumos, a novel spatial-temporal integrated neural-enhanced live 360-degree video streaming system. To accommodate varied video scenarios, we devise a real-time Neural-enhanced Quality Prediction (NQP) model to predict the neural-enhanced quality for different video contents. To cope with varying bandwidth conditions, we design a Content-aware Bitrate Allocator, which dynamically allocates bitrates and selects an appropriate neural enhancement configuration based on the current bandwidth. Moreover, Lumos employs online learning to improve prediction performance and adjust resource utilization to optimize user quality of experience (QoE). Experimental results demonstrate that Lumos surpasses state-of-the-art neural-enhanced systems with an improvement of up to 0.022 in terms of SSIM, translating to an 8.2%-8.5% enhancement in QoE for live stream viewers.
Primary Subject Area: [Systems] Transport and Delivery
Secondary Subject Area: [Experience] Multimedia Applications, [Systems] Systems and Middleware, [Experience] Interactions and Quality of Experience
Relevance To Conference: Our work focuses on optimizing the upstreaming of live 360-degree video streaming, which is part of the multimedia streaming delivery pipeline and coincides with the theme--"Multimedia Systems". In our work, To tackle the constrained upload bandwidth and insufficient computational resources of the uploading client, we integrate spatial-temporal neural enhancement techniquea into the proposed system to enhance the quality of live 360-degree videos. The significance of our work coincides with the goals of the ACM Multimedia Conference, which proposes a new perspective of delivering multimedia content and pushes the boundaries of immersive media experiences in the meteverse system.
Submission Number: 3327
Loading