Evaluating visual "common sense" using fine-grained classification and captioning tasks

Raghav Goyal, Farzaneh Mahdisoltani, Guillaume Berger, Waseem Gharbieh, Ingo Bax, Roland Memisevic

Keywords: video dataset, fine-grained understanding, common sense, video captioning

Abstract: We introduce the Something-something V2 dataset, which contains captions of finely-varying human-object interactions. We also discuss various baseline models, and show that neural networks show surprisingly strong performance on many of the very hard, detailed discrimination tasks associated with this dataset.