Conditional Hallucinations for Image Compression
Keywords: Learned image compression, compression, alignment
Abstract: In lossy image compression, models face the challenge of either hallucinating details or generating out-of-distribution samples due to the information bottleneck.
This implies that at times, introducing hallucinations is necessary to generate in-distribution samples.
The optimal level of hallucination varies depending on image content, as humans are sensitive to small changes that alter the semantic meaning.
We propose a novel compression method that dynamically balances the degree of hallucination based on content.
We collect data and train a model to predict user preferences on hallucinations.
By using this prediction to adjust the perceptual weight in the reconstruction loss, we develop a \textbf{Con}ditionally \textbf{Ha}llucinating compression model (\textbf{ConHa}) that outperforms state-of-the-art image compression methods.
Code and images are available at \href{https://polybox.ethz.ch/index.php/s/owS1k5JYs4KD4TA}{https://polybox.ethz.ch/index.php/s/owS1k5JYs4KD4TA}.
Submission Number: 27
Loading