ST-CellSeg: Cell segmentation for imaging-based spatial transcriptomics using multi-scale manifold learning

Youcheng Li, Leann Lac, Qian Liu, Pingzhao Hu

Abstract: Author summary Spatial transcriptomics data is a type of biological data that describes gene expression patterns in the context of tissue or cell spatial arrangement. Traditional transcriptomics studies the gene expression of a group of cells or a tissue sample as a whole, revealing which genes are active or inactive in that sample. Spatial transcriptomics, on the other hand, is a recent technology that can maintain the spatial information of where these genes are expressed inside the tissue. These methods provide a more accurate description of tissue and cell subcellular architecture, allowing for a better understanding of physical and biochemical interactions between cells. Precise cell identification is critical because it can aid in the discovery of unusual cell types, particularly in cancer research. Traditional clustering approaches, on the other hand, frequently fail to account for spatial information. The issue in bioinformatics is thus to diversify cell segmentation approaches in spatial transcriptomic analysis. To that purpose, we develop a cell segmentation technique for spatial transcriptomic data that uses distance metrics to better define the spatial transcriptomics distribution. The experimental results reveal that this algorithm outperforms the popular cell segmentation algorithms and performs faster under the same conditions.