Random Projection-Induced Gaussian Latent Features for Arbitrary Style Transfer

TMLR Paper5913 Authors

Abstract: The style transfer technique centered around mean and variance, widely recognized as AdaIN, is the foundation of current style transfer approaches. This technique assumes that the features designated for transfer follow Gaussian distributions. However, this assumption is often difficult to meet in practice, as the features typically exhibit sparse distributions due to the significant spatial correlation inherent in natural images. To address this challenge, we propose initially projecting the sparse features into lower dimensions via random projection, and then performing style transfer on these projections. Statistically, the projections will be closer to Gaussian distributions, thereby better aligning with AdaIN's requirements and enhancing transfer performance. With the stylized projections, we can further reconstruct them back to the original feature space by leveraging compressed sensing theory, thereby obtaining the stylized features. The entire process constitutes a projection-stylization-reconstruction module, which can be seamlessly integrated into AdaIN without necessitating network retraining. Furthermore, our proposed module can also be incorporated into the recently introduced style transfer technique based on cumulative distribution functions, known as EFDM, which faces limitations when there are substantial differences in sparsity levels between content and style features. By projecting both types of features into dense, Gaussian distributions, random projection can reduce their sparsity disparity, ultimately improving performance. Experiments demonstrate that the performance improvements mentioned can be achieved on existing state-of-the-art approaches.

Submission Length: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Francesco_Locatello1

Submission Number: 5913