VideoCLIP 2.0: An Interactive CLIP-Based Video Retrieval System for Novice Users at VBS2024

Thao-Nhu Nguyen, Le Minh Quang, Graham Healy, Binh T. Nguyen, Cathal Gurrin

Abstract: In this paper, we present an interactive video retrieval system named VideoCLIP 2.0 developed for the Video Browser Showdown in 2024. Building upon the foundation of the previous year’s system, VideoCLIP, this upgraded version incorporates several enhancements to support novice users in solving retrieval tasks quickly and effectively. Firstly, the revised system enables search using a variety of modalities, such as rich text, dominant colour, OCR, query-by-image, and now relevance feedback. Additionally a new keyframe selection technique and a new embedding model to replace the existing CLIP model have been employed. This new model aims to obtain richer visual representations in order to improve search performance in the live interactive challenge. Lastly, the user interface has been refined to enable quicker inspection and user-friendly navigation, particularly beneficial for novice users. In this paper we describe the updates to VideoCLIP.