Identification of Non-Plant Elements in Herbarium Images Using YOLO

Youcef Sklab, Hanane Ariouat, Edi Prifti, Jean-Daniel Zucker, Eric Chenin

Abstract: Herbarium scans are an essential resource for studying plant adaptation to climate change and responses to various environmental factors. Automation of functional trait characterization and measurement from these images is crucial for scaling-up and advancing research in plant taxonomy while improving biodiversity documentation. Traditionally, the extraction of these data is conducted manually, a labor-intensive and error-prone process. Nonetheless, in order to derive meaningful data and knowledge, the automated analysis of these images presents significant challenges due to variations in scale, complex backgrounds, and differences in specimen color, shape, and orientation. Advanced computer vision and text mining techniques offer the potential to unlock extensive potential in the field of biodiversity by facilitating the rapid extraction of text- and trait-based information from digital herbarium images. In this study we developed multiple object detection models utilizing a series of YOLO inspired algorithms and digitized collection images from the major Museums of Natural History in France. These models were trained on 722 manually annotated images and validated on a subset of 80 images, to ef fectively identify six non-plant component types in the digital specimen images, namely envelope, stamp, barcode, textbox, color palette, and scale bar. These models can be used to identify herbarium specimens that contain any of these components for detailed analysis (such as text boxes) or to recognize specimens that lack these elements, facilitating the enhancement of collections. Additionally, determining the frequency of such non-plant components can promt new studies. The overall performance analysis indicates that the YOLOv7-ag model surpasses other models.