Ablation Study of Neural Network Based General Temporal Point Processes Model

Geng Shao, Gloria Wan, Rebecca Wang

Abstract: In this project, we did the ablation study of the paper "Fully Neural Network based Model for General Temporal Point Processes" by Takahiro Omi from the2019 NeurIPS accepted papers. This paper discusses Recurrent Neural Network(RNN) based models developed for point processes. RNN based models usually assume a specific functional form for the time course of the intensity function of a point process. This paper proposed a novel model where the time evolution of the conditional intensity function is formulated based on a neural network rather than assuming a specific functional form. We confirmed that the conclusions of the paper are supported. This new model overall achieves superior performances compared to the previous state-of-the-art methods for synthetic datasets. However because of the nature of the neural network where it requires a training process, it has a higher variance and highly depends on the process used to generate the data. Typically it performs better when the process is more complex, specifically stationary renewal, non-stationary-renewal, and Hawkes Processes. We show that tanh is a better activation function for the Neural Network and the 64 units per hidden layer setting in this paper performs worse compared to 16 units and 32units, reducing the layer does not only increase the accuracy but also decreases the computational power required. Thus, we purposed ways to optimize the model and improve the result.

Track: Ablation

NeurIPS Paper Id: https://openreview.net/forum?id=SJeeIESeLH