FSVFG: Towards Immersive Full-Scene Volumetric Video Streaming with Adaptive Feature Grid
Abstract: Full-scene volumetric video streaming, an emerging technology providing immersive viewing experiences via the Internet, is receiving increasing attention from both the academic and industrial communities. Considering the vast amount of full-scene volumetric data to be streamed and the limited bandwidth on the internet, achieving adaptive full-scene volumetric video streaming over the internet presents a significant challenge. Inspired by the advantages offered by neural fields, especially the feature grid method, we propose FSVFG, a novel full-scene volumetric video streaming system integrated feature grids as the representation of volumetric content. FSVFG employs an incremental training approach for feature grids and stores the features and residuals between adjacent grids as frames. To support adaptive streaming, we delve into the data structure and rendering processes of feature grids and propose bandwidth adaptation mechanisms. The mechanisms involve a coarse ray-marching for the selection of features and residuals to be sent, and achieve variable bitrate streaming by Level-of-Detail (LoD) and residual filtering. Based on these mechanisms, FSVFG achieves adaptive streaming by adaptively balancing the transmission of feature and residual according to the available bandwidth. Our preliminary results demonstrate the effectiveness of FSVFG, demonstrating its ability to improve visual quality and reduce bandwidth requirements of full-scene volumetric video streaming.
Primary Subject Area: [Systems] Transport and Delivery
Relevance To Conference: This work makes a contribution to the field of multimedia and multimodal processing by addressing the challenge of streaming full-scene volumetric video over the Internet. The proposed FSVFG system, which integrates feature grids as the representation of volumetric content, is a solution to the problem of transmitting large amounts of volumetric data with limited bandwidth. The proposed method utilizes incremental frozen training for frame compression and achieves rate adaptation by ray-aware feature selection and filtering. Our work aligns closely with the research direction of the ACM Multimedia conference. Our paper cites and builds upon previous works presented at this conference, demonstrating the continuity and progression of the research in this domain.
Submission Number: 877
Loading