A Quantitative Analysis of the Effect of Batch Normalization on Gradient Descent

Yongqiang Cai, Qianxiao Li, Zuowei Shen

Abstract: Despite its empirical success and recent theoretical progress, there generally lacks a quantitative analysis of the effect of batch normalization (BN) on the convergence and stability of gradient d...