On spectral bias reduction of multi-scale neural networks for regression problems

Bo Wang, Heng Yuan, Lizuo Liu, Wenzhong Zhang, Wei Cai

Abstract: Highlights•Presents a diffusion equation model in the frequency domain for the evolution of the training error of multiscale neural networks for approximations.•The frequency-dependent diffusion coefficients in the model clearly show the effect of multiple scales on reducing the spectral bias.•The derived error diffusion model correctly predicts the error evolution of neural network trainings in practice.