Crossseg: Cross-Scene Few-Shot Aerial Segmentation Using Probabilistic Prototypes

Yu Zhang, Muhammad Usman Rafique, Nathan Jacobs

Abstract: In this work, we propose a novel framework called CrossSeg that addresses the task of few-shot semantic segmentation for different aerial imagery. Conventional semantic segmentation approaches struggle to generalize well to unseen object categories, making them a significant limitation for modern intelligent systems, especially those deployed in realistic real-time settings, such as unmanned aerial vehicles (UAVs). CrossSeg overcomes this limitation and generalizes well in a cross-scene setting with only a few labeled samples. Unlike traditional methods that use a set of fixed prototypes for each class, CrossSeg utilizes high-quality probabilistic prototypes that can not only represent different semantic classes but also handle significant variations in different scenes. Experiments show that our approach significantly improves upon conventional few-shot segmentation baselines and does not require extensive tuning.