FungiTastic: A multi-modal dataset and benchmark for image categorization

Lukas Picek, Klara Janouskova, Milan Šulc, Jiri Matas

Keywords: image classification, fine grained visual categorization, multi-modal, open-set recognition, species identification, domain shift, segmentation, few-shot learning

Abstract:

We introduce a new, highly challenging benchmark and a dataset -- FungiTastic -- based on data continuously collected over a twenty-year span.
The dataset originates in fungal records labeled and curated by experts. It consists of about 350k multi-modal observations that include more than 650k photographs from 5k fine-grained categories and diverse
accompanying information, e.g., acquisition metadata, satellite images, and body part segmentation.
FungiTastic is the only benchmark that includes a test set with partially DNA-sequenced ground truth of unprecedented label reliability.
The benchmark is designed to support
(i) standard close-set classification,
(ii) open-set classification,
(iii) multi-modal classification,
(iv) few-shot learning,
(v) domain shift, and many more.
We provide baseline methods tailored for almost all the use-cases.
We provide a multitude of ready-to-use pre-trained models on HuggingFace and a framework for model training.
A comprehensive documentation describing the dataset features and the baselines are available at \href{https://sulc.github.io/DanishFungi2024/}{GitHub} and Kaggle.

Flagged For Ethics Review: true

Submission Number: 709