Shallow and Deep Networks are Near-Optimal Approximators of Korobov FunctionsDownload PDF

Anonymous

Sep 29, 2021 (edited Nov 22, 2021)ICLR 2022 Conference Blind SubmissionReaders: Everyone
  • Abstract: In this paper, we analyze the number of neurons and training parameters that a neural network needs to approximate multivariate functions of bounded second mixed derivatives --- Korobov functions. We prove upper bounds on these quantities for shallow and deep neural networks, drastically lessening the curse of dimensionality. Our bounds hold for general activation functions, including ReLU. We further prove that these bounds nearly match the minimal number of parameters any continuous function approximator needs to approximate Korobov functions, showing that neural networks are near-optimal function approximators.
  • One-sentence Summary: We analyze the number of neurons and training parameters that a neural network needs to approximate multivariate functions of bounded second mixed derivatives
16 Replies

Loading