Revisiting Feature Propagation and Aggregation in Polyp Segmentation

Yanzhou Su, Yiqing Shen, Jin Ye, Junjun He, Jian Cheng

Abstract: Accurate segmentation of polyps is a crucial step in the efficient diagnosis of colorectal cancer during screening procedures. The prevalent UNet-like encoder-decoder frameworks are commonly employed, due to their capability of capturing multi-scale contextual information efficiently. However, two major limitations hinder the network from achieving effective feature propagation and aggregation. Firstly, the skip connection only transmits a single scale feature to the decoder, which can result in limited feature representation. Secondly, the features are transmitted without any information filter, which is inefficient for performing feature fusion at the decoder. To address these limitations, we propose a novel feature enhancement network that leverages feature propagation enhancement and feature aggregation enhancement modules for more efficient feature fusion and multi-scale feature propagation. Specifically, the feature propagation enhancement module transmits all encoder-extracted feature maps from the encoder to the decoder, while the feature aggregation enhancement module performs feature fusion with gate mechanisms, allowing for more effective information filtering. The multi-scale feature aggregation module provides rich multi-scale semantic information to the decoder, further enhancing the network’s performance. Extensive evaluations on five datasets demonstrate the effectiveness of our method, particularly on challenging datasets such as CVC-ColonDB and ETIS, where it can outperform the previous state-of-the-art models by a significant margin (3%) in terms of mIoU and mDice.