Ladder Polynomial Neural Networks

Li-Ping Liu, Ruiyuan Gu, Xiaozhe Hu

TL;DR: This paper proposes LPNN, which is a new type of polynomial neural networks that can have arbitrary polynomial order.

Abstract:

The underlying functions of polynomial neural networks are polynomial functions. These networks are shown to have nice theoretical properties by previous analysis, but they are actually hard to train when their polynomial orders are high. In this work, we devise a new type of activations and then create the Ladder Polynomial Neural Network (LPNN). This new network
can be trained with generic optimization algorithms. With a feedforward structure, it can also be combined with deep learning techniques such as batch normalization and dropout. Furthermore, an LPNN provides good control of its polynomial order because its polynomial order increases by 1 with each of its hidden layers. In our empirical study, deep LPNN models achieve good performances in a series of regression and classification tasks.

Code: https://anonymous.4open.science/r/ba53e832-0151-48f2-bc45-3746390b72e4/

Keywords: polynomial neural networks