The Impact of Aggregation Window Width on Properties of Contextual Neural Networks with Constant Field of Attention

Miroslava Mikusová, Antonin Fuchs, Marcin Jodlowiec, Erik Dawid Burnell, Krzysztof Wolk

Abstract: Artificial Neural Networks are quickly developing machine learning models with numerous applications. This includes also contextual neural networks (CxNNs). In this paper it is analyzed how the width of the aggregation window of the Constant Field of Attention (CFA) aggregation function changes the basic properties of the CxNN models, with the focus on classification accuracy and activity of hidden connections. Both aspects were analyzed in the custom H2O environment for real-life microarray data of ALL-AML leukemia gene expression and selected benchmarks hosted by the UCI ML repository. The results of presented study confirmed that increased length of the aggregation window of the CFA function can lead to similar or better classification accuracy than in the case of the Sigma-if aggregation. On the other hand, the activity of hidden connections in the majority of analyzed data sets resulted considerably higher. Presented analysis can be used as a guideline in further research on contextual neural networks.