UAVDB: Point-Guided Masks for UAV Detection and Segmentation

Yu-Hsi Chen

Keywords: UAV detection, weak supervision, point-guided annotation, instance segmentation

TL;DR: We present UAVDB, a benchmark for UAV detection and segmentation with point-guided PIC and SAM2, enabling accurate multi-task labels without manual effort and providing strong baselines.

Abstract: The widespread use of Unmanned Aerial Vehicles (UAVs) in surveillance, security, and airspace monitoring demands accurate and scalable detection methods. Progress, however, is limited by the lack of large-scale, high-resolution datasets with precise yet cost-efficient annotations. To address these challenges, we present UAVDB, a benchmark dataset for UAV detection and segmentation, built through a point-guided weak supervision pipeline. UAVDB leverages trajectory point annotations and RGB video frames from a multi-view drone tracking dataset captured by fixed cameras. We introduce Patch Intensity Convergence (PIC), a lightweight annotation method that converts trajectory points into high-fidelity bounding boxes, eliminating manual labeling while maintaining accurate spatial localization. From these boxes, we further derive instance segmentation masks using SAM2, enabling rich multi-task annotations with minimal supervision. UAVDB captures UAVs across diverse scales, ranging from clearly visible objects to nearly single-pixel instances, under challenging conditions. Additionally, PIC is lightweight and readily pluggable into other point-guided scenarios, making it easy to scale up dataset generation across various domains. We quantitatively show that PIC outperforms existing annotation techniques in IoU accuracy and efficiency. Finally, we benchmark several state-of-the-art (SOTA) YOLO detectors on UAVDB, establishing strong baselines for future research. UAVDB and all associated tools will be publicly released to accelerate point-guided detection and segmentation research.

Primary Area: datasets and benchmarks

Submission Number: 4252