The Learning Dynamcis of a Universal Approximator

Ansgar Heinrich Ludolf West, David Saad, Ian T. Nabney

Abstract: The learning properties of a universal approximator, a normalized committee machine with adjustable biases, are studied for on-line back-propagation learning. Within a statistical mechanics frame(cid:173) work, numerical studies show that this model has features which do not exist in previously studied two-layer network models with(cid:173) out adjustable biases, e.g., attractive suboptimal symmetric phases even for realizable cases and noiseless data.