An image to tailor: I-Frame Domain Adaptation in Neural Video Compression

Alberto Presta, Gabriele Spadaro, Attilio Fiandrotti, Marco Grangetto

Keywords: Neural Image Compression, Neural video compression, Domain adaptation, adapters

TL;DR: This work exploits convolutional adapters (CADs) inserted in the I-frame decoder to perform domain adaptation on neural video compression models.

Abstract:

Neural video compression (NVC) models recently outperformed traditional methods.
They typically include an I-Frame codec for Intra-Frames and a P-Frame codec for P-frames. However, their performance may be far from optimal with data outside the training set. We propose domain adaptation (DA) in NVC using lightweight convolutional adapters inserted in the I-Frame decoder of a pre-trained NVC model, which are then fine-tuned.
These adapters shift knowledge to a specific domain without altering the architecture or causing catastrophic forgetting. They enhance compression for both I-frames and P-frames while using minimal parameters with respect to the entire architecture, improving NVC robustness.

Submission Number: 21