Enhancing Cell Detection in Histopathology Images: A ViT-Based U-Net Approach

Zhaoyang Li, Wangkai Li, Huayu Mai, Tianzhu Zhang, Zhiwei Xiong

Abstract: Cell detection in histology images is a pivotal and fundamental task within the field of computational pathology. Recent advancements have led to the introduction of the OCELOT dataset, which offers annotated images featuring overlapping cell and tissue structures derived from diverse organs. The significance of OCELOT dataset lies in its provision of valuable insights into the intricate relationship between the surrounding tissue structures and individual cells. Based on the OCELOT dataset, We propose a ViT-based U-Net (Cell-Tissue-ViT) in a unified deep model via an encoder-decoder structure for robust cell detection, simultaneously leveraging tissue and cell information. Specifically, we adopt the pretrained ViT encoder of the large-scale pre-trained Segment Anything Model(SAM) as our backbone to enhance the feature extraction capability of the model and adopt LoRA to fine-tune the backbone, intending to enhance its suitability for our specific task. Our approach achieves highly promising results in cell detection on the OCELOT dataset, with an F1-detection score of 0.7558, as indicated by the preliminary results on the validation set. What’s more, we achieved 1st place on the official test set. The code is available in https://github.com/Lzy-dot/OCELOT2023.git.