Fast Matrix Multiplication Without Tears: A Constraint Programming Approach

Arnaud Deza, Elias Boutros Khalil, Pashootan Vaezipoor, Chang Liu

Abstract:

It is known that the multiplication of an N ×M matrix with an M ×P matrix can be performed using
fewer multiplications than what the naive NMP approach suggests. The most famous instance of
this is Strassen’s algorithm for multiplying 2×2 matrices in 7 instead of 8 multiplications. This gives
rise to the constraint satisfaction problem of fast matrix multiplication, where a set of R < NMP
multiplication terms must be chosen and combined such that they satisfy correctness constraints on
the output matrix. Despite its highly combinatorial nature, this problem has not been exhaustively
examined from that perspective, as evidenced for example by the recent deep reinforcement learning
approach of AlphaTensor. In this work, we propose a simple yet novel Constraint Programming
approach to find algorithms for fast matrix multiplication or provide proof of infeasibility otherwise.
We propose a set of symmetry-breaking constraints and valid inequalities that are particularly helpful
in proving infeasibility. On the feasible side, we find that exploiting solver performance variability
in conjunction with a sparsity-based problem decomposition enables finding solutions for larger
(feasible) instances of fast matrix multiplication. Our experimental results using CP Optimizer
demonstrate that we can find fast matrix multiplication algorithms for matrices up to 3 × 3 with
R = 23 in a short amount of time.