Why Equivariant Networks Lose Information: Invariant Rings and the Role of Aggregation

TMLR Paper6979 Authors

Abstract: Equivariant neural networks exhibit well-documented expressivity limitations: rotation-equivariant networks collapse directional information to radial features, and matrix-equivariant networks show rank degeneracy. We provide an algebraic account of these phenomena using two complementary frameworks: classical (compact-group) invariant theory and Sato–Kimura prehomogeneous vector space (PVS) theory, which handles non-compact actions admitting an open dense orbit. This allows us to organize a body of recent results — encompassing geometric GNN completeness, body-order, sparse-graph rigidity, stabilizer obstructions, and tensor-product selection rules — under a common algebraic frame. Escape from these constraints comes from aggregation: the transition from a single fiber V to the product V^n enriches the invariant ring with cross-invariants, and this transition rather than network depth is what enables discrimination of geometric relationships. We use this frame to analyze modern architectures — SchNet, PaiNN, DimeNet, MACE, HEGNN, GotenNet — through their body-order, the largest n for which they access the invariant ring of V^n. Illustrative experiments verify the predicted SO(3)-vs-O(3) and aggregation-vs-fiber-only gaps.

Submission Type: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Wenbing_Huang1

Submission Number: 6979