Stable Low-Rank Julia Networks: Fractal Dynamics for Parameter-Efficient Polynomial Learning

Keywords: Low-Rank Approximations, Polynomial Neural Networks, Julia Sets, Fractal Dynamics, Parameter-Efficient Learning

TL;DR: Stable Low-Rank Julia Networks (Julia-PNN) harness damped complex fractal dynamics to implicitly expand the effective rank of ultra-compressed seed matrices, enabling highly expressive and parameter-efficient polynomial learning.

Abstract: Standard polynomial neural networks excel at capturing highly non-linear feature interactions but are notoriously plagued by exponential parameter explosion. Conversely, while standard low-rank approximations like Singular Value Decomposition (SVD) drastically curtail parameter footprints, they impose severe dimensional bottlenecks that choke the mathematical expressivity of the weight space. To resolve this dichotomy, we introduce Stable Low-Rank Julia Networks (Julia-PNN), a novel architecture that intertwines the weight space of a polynomial network with the infinitely intricate complex dynamics of Julia sets ($Z_{n+1} = Z_n^2 + C$). By initializing weight tensors as ultra-compressed low-rank seed matrices and evolving them through a dynamically damped complex fractal recurrence, Julia-PNN implicitly triggers an explosive expansion of the effective rank while strictly preserving a minimal memory footprint. Comprehensive empirical evaluations across Fashion-MNIST, MNIST, and CIFAR-10 reveal that Julia-PNN outperforms six rigorous parameter-matched baselines, including linear Low-Rank factorization, CP-Decomposed PNNs, TT-Decomposed PNNs, Low-Rank Quadratic networks, and Hypernetworks. Crucially, parameter scaling laws demonstrate that under extreme compression regimes where traditional factorizations suffer catastrophic degradation, Julia-PNN maintains robust representational capacity. Our theoretical framework demonstrates that the novel residual damping mechanism serves as a pivotal heuristic, helping to mitigate chaotic fractal divergence and locally stabilize gradient flow.

Submission Number: 25