Reproducibility of "Augmented Neural ODEs"

Henry Ho, Luca Zarow, Eric Zimmermann

Abstract: The Augmented Neural ODE (ANODE) is an extension of Neural Ordinary Differential Equations (NODE) that aim to represent and model continuous spaces. This class of networks are based on discrete problem modeling that Residual Networks accomplished. The purpose of this paper is to analyze the validity of ANODE in order to ensure that the proposed space augmentation method does in fact alleviate constrained network flows associated to current NODE model. This is accomplished by comparing baseline experiments on two image classification datasets, MNIST and CIFAR-10. MNIST is ran with an extensive hyperparameter search in order to validate the robustness of the model. ANODE provides improved accuracies for both datasets. For the MNIST dataset, the accuracy for ANODE is 97.25 ± 0.543%, and for NODE is 95.57 ± 0.323%, while for the CIFAR-10 dataset, the accuracy for ANODE is 52.42 ± 1.19%, and for NODE is 51.72 ± 0.571%. ANODE provides far fewer function evaluations compared to NODE. ANODE's improvement over NODE is also found over a wide variety of hyperparamters. With these ablations, we conclude that the model proposed by Dupont et al. [2019] is robust and is a significant improvement over previous models.

Track: Ablation

NeurIPS Paper Id: https://openreview.net/forum?id=BylEPErxUS&noteId=rylDAJBJiH