Zero-Shot Insect Detection via Weak Language Supervision

Benjamin Feuer, Ameya Joshi, Minsu Cho, Kewal Jani, Shivani Chiranjeevi, ZI Kang Deng, Aditya Balu, Asheesh K Singh, Soumik Sarkar, Nirav Merchant, Arti Singh, Baskar Ganapathysubramanian, Chinmay Hegde

Keywords: insect detection, Detic, vision-language models, inaturalist

TL;DR: Labeling insect images with bounding boxes is cumbersome; instead, we just use Detic with generic prompts to get excellent zero-shot results.

Abstract: Open source image datasets collected via citizen science platforms (such as iNaturalist) can pave the way for the development of powerful AI models for insect detection and classification. However, traditional supervised learning methods require labeled data, and manual annotation of these raw datasets with useful labels (such as bounding boxes) can be extremely laborious, expensive, and error-prone. In this paper, we show that recent advances in vision-language models enable highly accurate zero-shot detection of insects in a variety of challenging environs. Our contributions are twofold: a) We curate the Insecta rank class of iNaturalist to form a new benchmark dataset of approximately 6M images consisting of 2526 agriculturally important species (both pests and beneficial insects). b) Using a vision-language object detection method coupled with weak language supervision, we are able to automatically annotate images in this dataset with bounding box information localizing the insect within each image. Our method succeeds in detection of diverse insect species present in a wide variety of backgrounds, producing high-quality bounding boxes in a zero-shot manner with no additional training cost.