Toward a Type-Theoretic Framework for Linear Mode Connectivity: Univalence and Path-Finding in Weight Spaces

Şuayp Talha Kocabay, Kerem Yalçın, Talha Rüzgar Akkuş, Erik Hillbom

Keywords: Homotopy Type Theory, Univalence Axiom, Linear Mode Connectivity, Permutation Alignment, Model Merging

TL;DR: We use the Univalence Axiom from HoTT to show that permutation-aligning two neural networks is equivalent to constructing a continuous low-loss path between them, giving Linear Mode Connectivity a formal type-theoretic foundation.

Abstract: Linear Mode Connectivity (LMC) reveals that independently trained networks can be connected by low-loss linear paths after permutation alignment, yet why discrete symmetry matching implies continuous topological connectivity remains theoretically unresolved. We propose a framework grounded in Homotopy Type Theory (HoTT), conjecturing that neural symmetry orbits as Higher Inductive Types allow the Univalence Axiom to lift permutation equivalences into continuous identity paths under cohesive semantics. Multi-trial experiments on MNIST MLPs demonstrate consistent barrier collapse (peak loss reduced by $>$98\%), and ResNet20 experiments on CIFAR-10 show that activation renormalization (REPAIR) is necessary to realize low-loss paths in architectures with batch normalization. We position this univalent framework as a principled foundation for formal verification of model merging algorithms.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 16