Spatial-Hierarchical Graph Neural Network with Dynamic Structure Learning for Histological Image Classification

Wentai Hou, Helong Huang, Qiong Peng, Rongshan Yu, Lequan Yu, Liansheng Wang

Abstract: Graph neural network (GNN) has achieved tremendous success in histological image classification, as it can explicitly model the notion and interaction of different biological entities (e.g., cell, tissue and etc.). However, the potential of GNN has not been fully unleashed for histological image analysis due to (1) the fixed design mode of graph structure and (2) the insufficient interactions between multi-level entities. In this paper, we proposed a novel spatial-hierarchical GNN framework (SHGNN) equipped with a dynamic structure learning (DSL) module for effective histological image classification. Compared with traditional GNNs, the proposed framework has two compelling characteristics. First, the DSL module integrates the positional attribute and semantic representation of entities to learn the adjacency relationship of them during the training process. Second, the proposed SHGNN can extract rich and discriminative features by mining the spatial features of different entities via graph convolutions and aggregating the semantic of multi-level entities via a vision transformer (ViT) based interaction mechanism. We evaluate the proposed framework on our collected colorectal cancer staging (CRCS) dataset and the public breast carcinoma subtyping (BRACS) dataset. Experimental results demonstrate that our proposed method yield superior classification results compared to state-of-the-arts.