GraphCLIP: Image-graph contrastive learning for multimodal artwork classification

Raffaele Scaringi, Giuseppe Fiameni, Gennaro Vessio, Giovanna Castellano

Abstract: Highlights•We introduce GraphCLIP, a contrastive learning framework for artwork classification.•GraphCLIP combines visual data with contextual knowledge.•We achieve state-of-the-art performance on the ArtGraph<math><mrow is="true"><mi mathvariant="script" is="true">A</mi><mi is="true">r</mi><mi is="true">t</mi><mi mathvariant="script" is="true">G</mi><mi is="true">r</mi><mi is="true">a</mi><mi is="true">p</mi><mi is="true">h</mi></mrow></math> dataset.•We demonstrate robustness with unseen classes in distribution shift scenarios.•We provide visual and contextual explanations to enhance model interpretability.