Evaluating Self-Supervised Foundation Models in Holographic Imaging

Silas Dietler, Yanick Zeder, Elias Graf, Kilian Koch, Andreas Schwendimann, Tommaso Bendinelli

Keywords: Self-supervised training, DinoV2, Computer Vision, Holographic Images

TL;DR: We assess the capabilities of the self-supervised method DINOv2 on the task of bioaerosol particle classification using holographic images

Abstract:

DINOv2, a large self-supervised computer vision foundation model, has achieved impressive performance on downstream tasks like classification, segmentation, and depth estimation. This success suggests the idea that universal features can be extracted through large-scale pre-training. However, its applicability beyond natural image domains remains relatively unexplored. This study aims to contribute in this direction, by exploring the potential of DINOv2 for a niche but important task: pollen classification based on holographic images.
Our findings reveal that features learned by the network in the natural image domain are not informative for this task. However, when DINOv2 is pre-trained on a pollen-specific dataset, it achieves superior performance compared to supervised methods, especially in scenarios with limited data. This superior performance opens doors for new applications such as online few-shot (bio)aerosol particle classification with holographic imaging.

Submission Number: 90