$(MPO)^2$: Multivariate Polynomial Optimization based on Matrix Product Operators

ICLR 2026 Conference Submission19044 Authors

Keywords: tensor networks, polynomial, regresson

TL;DR: Multivariate Polynomial Optimization compressing parameters and linear transformation in the feature space as tensor networks.

Abstract:

Central to machine learning is the ability to perform universal function approximation and learn complex input-output relationships from limited numbers of observations.
Suitable Multivariate Polynomial Optimization can in theory provide universal function approximations. However, the coefficients of the polynomial regression model grows exponentially in the polynomial degree. To reduce exponential growth tensor factorizations of the associated weight tensor have been explored, including the canonical polyadic decomposition (CPD) and tensor train (TT) decompositions.
Whereas CPD has expressive power proportional to rank and current TT formulations are feature order dependent with each input feature associated to a specific factorization block.
Furthermore, these procedures account for redundancies sub-optimally in the weight tensor.
We presently explore multivariate polynomial optimization of matrix product operator (MPO) structures forming the (MPO)$^2$.
Notably, the (MPO)$^2$ defines a flexible framework that naturally combines MPO polynomial weight tensors with MPO feature embeddings.
The (MPO)$^2$ consequently produces an expressive yet compact representation of multivariate polynomials that is feature order independent and explicitly accounts for symmetries in the weight tensor.
On a series of regression and classification problems we observe that the proposed (MPO)$^2$ provides superior performance when compared to existing tensor decomposition based multivariate polynomial regression approaches even outperforming conventional universal function approximation procedures on some datasets.
The (MPO)$^2$ provides an expressive and versatile alternative to deep learning for universal function approximation with simple and efficient inference using second order methods.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 19044