Dynamic depth-width optimization for capsule graph convolutional network

Chuliang Weng

Abstract:

Authors: Shangwei WU , Yingtong XIONG , Chuliang WENG

Encouraged by the success of Convolutional Neural Networks (CNNs), many studies [1], known as Graph Convolutional Networks (GCNs), borrowed the idea of convolution and redefined it for graph data. In graph-level classification tasks, Classic GCN methods [2,3] generate graph embeddings based on the learned node embeddings which consider each node’s representation as multiple independent scalar features. However, they neglect the detailed mutual relations among different node features such as position, direction, and connection. Inspired by CapsNet [4] which encodes each feature of an image as a vector (a capsule), CapsGNN [5] extracts multi-scale node features from different convolutional layers in the form of capsules. However, CapsGNN uses a static model structure to conduct training, which inherently restricts its representation ability on different datasets.
In this paper, we propose Dynamic Depth-Width Optimization for Capsule Graph Convolutional Network (DynaCGCN) to explore the optimal depth-width setting on each dataset. Specifically, we leverage Reinforcement Learning (RL) to design an efficient online assistant module for evaluating different changes to depth (number of convolutional layers, denoted by D) and width (number of capsule channels in each layer, denoted by W). Differing from a typical RL-based Neural Architecture Search (NAS) task that evaluates different model structures separately, we move the RL procedure into only one full training and choose one action (i.e., one alteration to D and W) at one time in a sliding epoch window, according to not only the accuracy results on the validation set but also the reduction rate of training loss.