Group-aware Parameter-efficient Updating for Content-Adaptive Neural Video Compression

Zhenghao Chen, Luping Zhou, Zhihao Hu, Dong Xu

Abstract: Content-adaptive compression is crucial for enhancing the adaptability of the pre-trained neural codec for various contents. Although these methods have been very practical in neural image compression (NIC), their application in neural video compression (NVC) is still limited due to two main aspects: 1), video compression relies heavily on temporal redundancy, therefore updating just one or a few frames can lead to significant errors accumulating over time; 2), NVC frameworks are generally more complex, with many large components that are not easy to update quickly during encoding. To address the previously mentioned challenges, we have developed a content-adaptive NVC technique called Group-aware Parameter-Efficient Updating (GPU). Initially, to minimize error accumulation, we adopt a group-aware approach for updating encoder parameters. This involves adopting a patch-based Group of Pictures (GoP) training strategy to segment a video into patch-based GoPs, which will be updated to facilitate a globally optimized domain-transferable solution. Subsequently, we introduce a parameter-efficient delta-tuning strategy, which is achieved by integrating several light-weight adapters into each coding component of the encoding process by both serial and parallel configuration. Such architecture-agnostic modules stimulate the components with large parameters, thereby reducing both the update cost and the encoding time. We incorporate our GPU into the latest NVC framework and conduct comprehensive experiments, whose results showcase outstanding video compression efficiency across four video benchmarks and adaptability of one medical image benchmark.

Primary Subject Area: [Systems] Transport and Delivery

Secondary Subject Area: [Experience] Multimedia Applications

Relevance To Conference: Video coding is a crucial technique for the transport and delivery of multimedia content. Developing such a system presents challenges but has attracted significant attention from multimedia researchers. In this submission, we investigate innovative methods for content-adaptive NVC. We propose a novel strategy, termed GPU, notable for its patch-based GoP updating mechanism and encoder-side adaptor modules, tailored for diverse configurations. Our extensive experiments validate our GPU's effectiveness and adaptability, demonstrating its integration with contemporary NVC framework for the compression of both standard videos and specialized medical MRI sequences. Remarkably, it consistently surpasses state-of-the-art video compression algorithms, such as H.266/VVC and DCVC_DC, thereby establishing a formidable benchmark for both video and MRI compression. These achievements underscore our GPU approach as a robust and efficient baseline for content-adaptive NVC methods. As the landscape of NVC evolves to incorporate more intricate techniques, our architecture-agnostic updating strategy is anticipated to increase, offering a solution to reduce computational requirements while maximizing efficiency. Moreover, such efficient content-adaptive NVC is poised to broaden the scope of NVC technologies, enabling them to address more extensive applications of complex content types beyond standard video. This research heralds new avenues for the compression of more diverse content.

Submission Number: 202