Diffusion-based free-viewpoint synthesis for dataset expansion and wildlife classification

Jess Tam, Hideo Saito

Keywords: biodiversity monitoring, conservation technology, camera trapping

TL;DR: We increased the size of our training dataset for wildlife classification by synthesizing new data using DimensionX, a diffusion-based free-viewpoint synthesis model.

Abstract: Processing wildlife imagery for conservation and management poses significant challenges, especially when limited data hinders the ability of classification models to extract sufficient features from each wildlife class. In this study, we propose the use of a 3D free-viewpoint image-to-video generative model to augment the dataset by synthesizing new images for fine-tuning classification models. Our results demonstrate a notable improvement in the F1-score, increasing from a baseline of 0.338 to 0.525. While the quality of the synthesized images can be further enhanced, particularly in terms of incorporating wildlife-specific semantics, the study highlights the potential of generative AI not only for media creation but also for advancing environmental monitoring applications, despite challenges such as high computational cost.

Submission Number: 24