Path-BN: Towards effective batch normalization in the Path Space for ReLU networks

Xufang Luo, Qi Meng, Wei Chen, Yunhong Wang, Tie-Yan Liu

Abstract: Neural networks with ReLU activation functions (abbrev. ReLU Networks), have demonstrated their success in many applications. Recently, researchers noticed that ReLU networks are positively scale-i...