Linear Asymptotic Convergence of Anderson Acceleration: Fixed-Point Analysis

Hans De Sterck, Yunhui He

Abstract: We study the asymptotic convergence of AA, i.e., Anderson acceleration (AA) with window size for accelerating fixed-point methods , . Convergence acceleration by AA has been widely observed but is not well understood. We consider the case where the fixed-point iteration function is differentiable and the convergence of the fixed-point method itself is root-linear. We identify numerically several conspicuous properties of AA convergence: First, AA sequences converge root-linearly, but the root-linear convergence factor depends strongly on the initial condition. Second, the AA acceleration coefficients do not converge but oscillate as converges to . To shed light on these observations, we write the AA iteration as an augmented fixed-point iteration , , and analyze the continuity and differentiability properties of and . We find that the vector of acceleration coefficients is not continuous at the fixed point . However, we show that, despite the discontinuity of , the iteration function is Lipschitz continuous and directionally differentiable at for AA(1), and we generalize this to AA with for most cases. Furthermore, we find that is not differentiable at . We then discuss how these theoretical findings relate to the observed convergence behavior of AA. The discontinuity of at allows to oscillate as converges to , and the nondifferentiability of allows AA sequences to converge with root-linear convergence factors that strongly depend on the initial condition. Additional numerical results illustrate our findings for several linear and nonlinear fixed-point iterations and for various values of the window size .