Generalizable Style Transfer for Implicit Neural Represenation

Taehong Moon, Jaeho Moon, Wonyong Seo

Abstract: Implicit Neural Representation (INR) has revolutionized scene representation using neural networks, offering high-quality rendering and memory efficiency. Leveraging a neural network as a continuous function, INR learns to render images through reconstruction loss, including RGB images for 3D scenes using multi-view images. In this project, we tackle the problem of style transfer for implicit neural representations. Existing methods for INR style transfer rely on test-time operations using fixed network parameters, leading to limited diversity in stylized images. To overcome these limitations, we propose an enhanced style transfer approach for INR. Our framework introduces generalizable INRs for style transfer, incorporating a ViT-based hypernetwork to predict instance-specific features for an MLP. By leveraging Adaptive Instance Normalization (AdaIN), we fuse content and style features, providing the MLP with the necessary inputs. We evaluate the effectiveness of our generalizable style transfer INR framework on the ImageNette dataset, demonstrating its ability to produce high-quality stylized images.