MLP Embedded Inverse Tone Mapping

Panjun Liu, Jiacheng Li, Lizhi Wang, Zheng-Jun Zha, Zhiwei Xiong

Abstract: The advent of High Dynamic Range/Wide Color Gamut (HDR/WCG) display technology has made significant progress in providing exceptional richness and vibrancy for the human visual experience. However, the widespread adoption of HDR/WCG images is hindered by their substantial storage requirements, imposing significant bandwidth challenges during distribution. Besides, HDR/WCG images are often tone-mapped into Standard Dynamic Range (SDR) versions for compatibility, necessitating the usage of inverse Tone Mapping (iTM) techniques to reconstruct their original representation. In this work, we propose a meta-transfer learning framework for practical HDR/WCG media transmission by embedding image-wise metadata into their SDR counterparts for later iTM reconstruction. Specifically, we devise a meta-learning strategy to pre-train a lightweight multilayer perceptron (MLP) model that maps SDR pixels to HDR/WCG ones on an external dataset, resulting in a domain-wise iTM model. Subsequently, for the transfer learning process of each HDR/WCG image, we present a spatial-aware online mining mechanism to select challenging training pairs to adapt the meta-trained model to an image-wise iTM model. Finally, the adapted MLP, embedded as metadata, is transmitted alongside the SDR image, facilitating the reconstruction of the original image on HDR/WCG displays. We conduct extensive experiments and evaluate the proposed framework with diverse metrics. Compared with existing solutions, our framework shows superior performance in fidelity (up to 3dB gain in perceptual-uniform PSNR), minimal latency (1.2s for adaptation and 2ms for reconstruction of a 4K image), and negligible overhead (40KB).

Primary Subject Area: [Content] Media Interpretation

Secondary Subject Area: [Systems] Systems and Middleware

Relevance To Conference: Inverse tone mapping (iTM) is a significant topic in multimedia, enabling the conversion of visual media from standard dynamic range (SDR) to High Dynamic Range (HDR) and Wide Color Gamut (WCG), thereby enhancing its quality and human visual experience on HDRTV. Our work proposes an inverse tone mapping method for HDR/WCG media transmission by embedding an image-wise iTM model as metadata in the SDR version, showing significant advantages in both performance and efficiency over existing solutions.

Submission Number: 964