Weight Space Representation Learning via Neural Field Adaptation

Zhuoqian Yang, Mathieu Salzmann, Sabine Süsstrunk

Keywords: Weight Space Learning, Neural Fields, LoRA, Representation Learning

TL;DR: We show neural field weights can serve as data representations. Fine-tuning a shared base model with multiplicative LoRA plus asymmetric masking induces semantic structure and yields higher-quality weight-space generation on 2D and 3D data.

Abstract: In this work, we investigate the potential of weights to serve as effective representations, focusing on neural fields. Our key insight is that constraining the optimization space through a pre-trained base model and multiplicative low-rank adaptation (mLoRA) can induce structure in weight space. Across reconstruction, generation, and analysis tasks on 2D and 3D data, we find that mLoRA weights achieve high representation quality while exhibiting distinctiveness and semantic structure. When used with latent diffusion models, mLoRA weights enable higher-quality generation than existing weight-space methods.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 5