AlphaVAE: Unified End-to-End RGBA Image Reconstruction and Generation with Alpha-Aware Representation Learning

Zile Wang, Hao Yu, Jiabo Zhan, Chun Yuan

Keywords: Variational Autoencoder, Latent Diffusion, Image Generation, Transparent Image

Abstract:

Recent advances in latent diffusion models have achieved remarkable results in high-fidelity RGB image synthesis by leveraging pretrained VAEs to compress and reconstruct pixel data at low computational cost. However, the generation of transparent or layered content (RGBA images) remains largely unexplored, partly due to the absence of dedicated evaluation metrics. In this work, we introduce a new evaluation metric for RGBA images that adapts standard RGB measures to four-channel data via alpha blending over canonical backgrounds. We further propose AlphaVAE, a unified end-to-end RGBA VAE that extends a pretrained RGB VAE by incorporating a dedicated alpha channel. The model is trained with a composite objective that combines alpha-blended pixel reconstruction, patch-level fidelity, perceptual consistency, and dual KL divergence constraints to ensure latent fidelity across both RGB and alpha representations. Our RGBA VAE, trained on only 8K images in contrast to 1M used by prior methods, achieves a +4.9 dB improvement in PSNR and a +3.2\% increase in SSIM over LayerDiffuse in reconstruction. It also enables superior transparent image generation when fine-tuned within a latent diffusion framework. Our code, data, and models are released on https://anonymous.4open.science/r/AlphaVAE-0DB8
for reproducibility.

Primary Area: generative models

Submission Number: 1405