Multi-ResNets for Low Rank Preconditioning in Constrained Optimization

Keywords: constrained optimization, low-rank preconditioning, multi-resolution neural networks, predict-complete-correct, hierarchical architectures, lexicographic constraints, AC optimal power flow

Abstract: We propose Multi-ResNets for optimization problems that encode inductive biases toward low-rank constraint set approximations. By leveraging compressed representations and embedding a priori structure, the method enables effective low-rank preconditioning, improving convergence and reducing sensitivity to local minima. This hierarchical formulation supports greedy refinement, where low-rank solutions initialize and guide optimization in the full space. Empirically, Multi-ResNets achieve $2$–$6\times$ reductions in primary constraint violation and improved convergence across convex and non-convex benchmarks.

Submission Number: 120