Accelerating variable projection algorithm by Aitken technique

Min Xu, Fenggen Lin, Guang-Yong Chen, Min Gan

Abstract: Separable nonlinear models (SNLMs) adopt a linear combination of nonlinear functions, which is often used in the field of system identification, machine learning and signal processing. In this paper, we studied the performance of gradient-based algorithms in identifying separable nonlinear models. We put forward a gradient descent-based variable projection (GD-VP) algorithm which taking advantage of the particular structure of SNLM. In each iteration, the algorithm eliminates the linear parameters of the model, then updates the nonlinear parameters through the gradient descent (GD) algorithm. To improve the convergence rate of GD algorithm, an accelerated GD-VP algorithm is derived by employing the Aitken acceleration technology. Numerical experiments shows the efficiency of the proposed algorithm.