Tensor Network Structure Search Via Canonical Dimension Tree Enumeration

TMLR Paper8877 Authors

Abstract:

Tensor networks provide a powerful framework for compressing multi-dimensional data. The optimal tensor network structure for a given data tensor depends on both data characteristics and specific optimality criteria, making tensor network structure search a challenging problem. Existing solutions typically rely on sampling and compressing numerous candidate structures; these procedures are computationally expensive and therefore limiting for practical applications. We address this challenge by decoupling topology enumeration from rank assignment search. We first represent the search space using canonical dimension trees, a hierarchical structure that encodes potential network topology through nested index partitions. This representation inherently rules out redundant and suboptimal topologies by construction. To eliminate the assessment bottleneck, we introduce a mechanism powered by the precomputation of a singular value map. By archiving the singular values of all feasible tensor matricizations, we transform the evaluation of any candidate dimension tree into a constraint-solving problem. This allows us to solve for the near-optimal ranks and calculate the dimension tree's cost via simple metadata lookup, bypassing the on-the-fly tensor decompositions for all but the most promising candidates. Experimental results show that our approach improves search speed by up to $10\times$ and achieves compression ratios $1.5\times$ to $3\times$ better than state-of-the-art. Notably, our approach scales to larger tensors that are unattainable by prior work.
Furthermore, the discovered topologies generalize well to similar data; they achieve compression ratios up to $2.4\times$ better than generic structures, while maintaining a search time of approximately $110$ seconds for 6D tensors of 1--2 GB disk size.

Submission Type: Long submission (more than 12 pages of main content)

Assigned Action Editor: ~Grigorios_Chrysos1

Submission Number: 8877