SniffyArt: The Dataset of Smelling Persons

Mathias Zinnen, Azhar Hussian, Hang Tran, Prathmesh Madhu, Andreas Maier, Vincent Christlein

Abstract:

Smell gestures play a crucial role in the investigation of past smells
in the visual arts yet their automated recognition poses significant
challenges. This paper introduces the SniffyArt dataset, consisting
of 1941 individuals represented in 441 historical artworks. Each
person is annotated with a tightly fitting bounding box, 17 pose
keypoints, and a gesture label. By integrating these annotations, the
dataset enables the development of hybrid classification approaches
for smell gesture recognition. The dataset’s high-quality human
pose estimation keypoints are achieved through the merging of
five separate sets of keypoint annotations per person. The paper
also presents a baseline analysis, evaluating the performance of
representative algorithms for detection, keypoint estimation, and
classification tasks, showcasing the potential of combining keypoint
estimation with smell gesture classification. The SniffyArt dataset
lays a solid foundation for future research and the exploration of
multi-task approaches leveraging pose keypoints and person boxes
to advance human gesture and olfactory dimension analysis in
historical artworks.