Accelerating training of batch normalization: A manifold perspective

Mingyang Yi

Abstract: Batch normalization (BN) has become a critical component across diverse deep neural networks. The network with BN is invariant to positively linear re-scale transformation, which makes there exist ...