Where are the Whales: A Human-in-the-loop Detection Method for Identifying Whales in High-resolution Satellite Imagery

Caleb Robinson, Kimberly T. Goetz, Christin B. Khan, Meredith Sackett, Kathleen Leonard, Rahul M Dodhia, Juan M Lavista Ferres

Keywords: whales, anomaly detection, labeling interface, satellite imagery, VHR imagery

TL;DR: We use simple anomaly detection methods to identify "interesting points" in high resolution satellite imagery over the ocean, create an interface allowing experts to annotate each point, and run a case study with these experts to identify whales.

Abstract: Monitoring whales and other marine mammals is critical for conservation efforts, particularly as many populations face increasing anthropogenic pressure and the effects of climate change. While prior work has shown that whales can be identified in very high-resolution (VHR) satellite imagery, large-scale automated detection remains challenging due to a lack of annotated imagery, variability in image quality and environmental conditions, and the cost of building robust machine learning pipelines over massive remote sensing archives. In this work, we present a semi-automated approach for surfacing possible whale detections in VHR imagery using a simple statistical anomaly detection method that identifies spatial outliers, i.e. "interesting points". We pair this detector with a web-based labeling interface designed to enable experts to quickly annotate the interesting points. We evaluate our system on three benchmark scenes with known whale annotations and achieve recalls of 90.3% to 96.4% while greatly reducing the search area. Our method does not rely on labeled training data and offers a scalable first step toward future machine-assisted marine mammal monitoring from space. We have open sourced the entire pipeline at https://github.com/microsoft/whales.

Submission Number: 4