Beyond Pairwise: Diagnosing Higher-Order Merge Failures via Hodge Decomposition

Dongzhe Zheng, Christine Allen-Blanchette

Keywords: Model Merging, Mode Connectivity, Alignment

Abstract: Model merging implicitly assumes pairwise mergeability extends to higher-order combinations. On 24 weight-matched PlainCNNs (CIFAR-10), this fails: over 90\% of pairwise-mergeable triples are not jointly mergeable ($\beta_1$ peaks at 122). Hodge decomposition of the merge-barrier signal separates incompatibility into gradient, curl, and harmonic components. Star-topology alignment leaves 25.5\% of energy in the harmonic component; adjusting merge weights within the fixed alignment cannot reduce it. Re-aligning along a minimum spanning tree weighted by harmonic energy drives this to 1.8\% and collapses $\beta_1$ from 75 to 5, at a cost of 15\% higher total barrier. The composition moves out of the harmonic component into gradient and curl, which existing merging strategies already address. Whether these patterns hold at larger scales remains open.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 20