Higher-order singular-value derivatives of real rectangular matrices

Róisín Luo, James McDermott, Colm O'Riordan

Abstract: Higher-order derivatives of singular values in real rectangular matrices arise naturally in both numerical simulation and theoretical analysis, with applications in areas such as statistical physics and optimization in deep learning. Deriving closed-form expressions beyond first order has remained a difficult problem within classical matrix analysis, and no general framework has been available. To address this gap, we present an operator-theoretic framework that extends Kato's analytic perturbation theory from self-adjoint operators to real rectangular matrices, thereby yielding general $n$-th order Fr\'echet derivatives of singular values. As a special case, we obtain a closed-form Kronecker-product representation of the singular-value Hessian, not previously found in the literature. This framework bridges abstract perturbation theory with matrix analysis and provides a systematic tool for higher-order spectral analysis.