Randomized Block-Diagonal Preconditioning for Parallel Learning

Celestine Mendler-Dünner, Aurélien Lucchi

Abstract: We study preconditioned gradient-based optimization methods where the preconditioning matrix has block-diagonal form. Such a structural constraint comes with the advantage that the update computati...