Recursive Self-Similarity in Deep Weight Spaces of Neural Architectures: A Fractal and Coarse Geometry Perspective

Ambarish Moharil, Indika Kumara, Majid Mohammadi, Damian Andrew Tamburri, Willem-Jan van den Heuvel

Published: 05 Mar 2025, Last Modified: 05 Mar 2025ICLR 2025 Workshop Weight Space Learning PosterEveryoneRevisionsBibTeXCC BY 4.0
Track: long paper (up to 8 pages)
Keywords: Fractal Geometry, Coarse Group Action, Deep Weight Spaces, Discrete Scale Invariance (DSI)
TL;DR: We present a coarse group action over deep weight spaces which fractalizes neural weight matrices/tensors into self-similar structures preserving invariance to structural scaling, permutations as well as intertwiner group relations.
Submission Number: 14