Digestive Organ Recognition in Video Capsule Endoscopy Based on Temporal Segmentation Network

Yejee Shin, Taejoon Eo, Hyeongseop Rha, Dong Jun Oh, Geonhui Son, Jiwoong An, You Jin Kim, Dosik Hwang, Yun Jeong Lim

Abstract: The interpretation of video capsule endoscopy (VCE) usually takes more than an hour, which can be a tedious process for clinicians. To shorten the reading time of VCE, algorithms that automatically detect lesions in the small bowel are being actively developed, however, it is still necessary for clinicians to manually mark anatomic transition points in VCE. Therefore, anatomical temporal segmentation must first be performed automatically at the full-length VCE level for the fully automated reading. This study aims to develop an automated organ recognition method in VCE based on a temporal segmentation network. For temporal locating and classifying organs including the stomach, small bowel, and colon in long untrimmed videos, we use MS-TCN++ model containing temporal convolution layers. To improve temporal segmentation performance, a hybrid model of two state-of-the-art feature extraction models (i.e., TimeSformer and I3D) is used. Extensive experiments showed the effectiveness of the proposed method in capturing long-range dependencies and recognizing temporal segments of organs. For training and validation of the proposed model, the dataset of 200 patients (100 normal and 100 abnormal VCE) was used. For the test set of 40 patients (20 normal and 20 abnormal VCE), the proposed method showed accuracy of 96.15, F1-score@{50,75,90} of {96.17, 93.61, 86.80}, and segmental edit distance of 95.83 in the three-class classification of organs including the stomach, small bowel, and colon in the full-length VCE.