Debris segmentation using post-hurricane aerial imagery

Kooshan Amini, Yuhao Liu, Jamie Padgett, Guha Balakrishnan, Ashok Veeraraghavan

Abstract: Timely and accurate detection of hurricane debris is critical for effective disaster response and community resilience. While post‐disaster aerial imagery is readily available, robust debris segmentation solutions applicable across multiple disaster regions remain limited. Since environmental and imaging conditions vary by region and alter debris's visual signatures, developing a debris segmentation model suitable for different hazard scenarios is challenging, especially given the scarcity of training data. This study addresses these challenges by relying on vast pretrained knowledge from the foundation vision model Contrastive Language‐Image Pre‐Training (CLIP) and fine‐tuning a lightweight decoder on labeled debris imagery, thereby achieving robust performance in segmenting non‐vegetative debris within complex urban backgrounds with a relatively small, high‐quality dataset. To enable fine‐tuning, this work introduces an open‐source dataset comprising approximately 1200 manually annotated aerial RGB images of structural debris from Hurricanes Ian, Ida, and Ike. To mitigate human biases and enhance data quality, labels from multiple annotators are strategically aggregated into consensus annotations. The resulting fine‐tuned model, named CLIPSeg‐debris, achieves a Dice score of 0.86 on data from Hurricane Ida—a disaster event entirely excluded during training—outperforming baseline models. This approach thereby enables broader adoption across multiple hazard scenarios and effectively adapts to various backgrounds, sensor resolutions, and debris conditions.