AlgoNet: $C^\infty$ Smooth Algorithmic Neural Networks

Felix Petersen, Christian Borgelt, Oliver Deussen

TL;DR: Integrating classical algorithms into neural networks.

Abstract:

Artificial neural networks have revolutionized many areas of computer science in recent years, providing solutions to a number of previously unsolved problems.
On the other hand, for many problems, classic algorithms exist, which typically exceed the accuracy and stability of neural networks.
To combine these two concepts, we present a new kind of neural networks—algorithmic neural networks (AlgoNets).
These networks integrate smooth versions of classic algorithms into the topology of neural networks.
A forward AlgoNet includes algorithmic layers into existing architectures to enhance performance and explainability while a backward AlgoNet enables solving inverse problems without or with only weak supervision.
In addition, we present the algonet package, a PyTorch based library that includes, inter alia, a smoothly evaluated programming language, a smooth 3D mesh renderer, and smooth sorting algorithms.

Keywords: Algorithms, Smoothness, Differentiable, Inverse Problems, Adversarial Training, Neural Networks, Deep Learning, Differentiable Renderer, 3D Mesh, Turing-completeness, Library