L-SR1: Learned Symmetric-Rank-One Preconditioning

ICLR 2026 Conference Submission13524 Authors

Keywords: Learned Optimization, Quasi-Newton, Monocular Human Mesh Reconstruction

TL;DR: L-SR1 is a lightweight learned second-order optimizer that accelerates convergence by augmenting SR1 with data-driven preconditioning, without needing annotated data or fine-tuning.

Abstract:

End-to-end deep learning has achieved impressive results but remains limited by its reliance on large labeled datasets, poor generalization to unseen scenarios, and growing computational demands. In contrast, classical optimization methods are data-efficient and lightweight but often suffer from slow convergence. While learned optimizers offer a promising fusion of both worlds, most focus on first-order methods, leaving learned second-order approaches largely unexplored.

We propose a novel learned second-order optimizer that introduces a trainable preconditioning unit to enhance the classical Symmetric-Rank-One (SR1) algorithm. This unit generates data-driven vectors used to construct positive semi-definite rank-one matrices, aligned with the secant constraint via a learned projection. Our method is evaluated through analytic experiments and on the real-world task of Monocular Human Mesh Recovery (HMR), where it outperforms existing learned optimization-based approaches. On the HMR task, it surpasses a fully-trained baseline using only 10% of the training data, underscoring its data efficiency. Featuring a lightweight model and requiring no annotated data or fine-tuning, our method offers strong generalization and is well-suited for integration into broader optimization-based frameworks.

Primary Area: optimization

Submission Number: 13524