Music database query with video by synesthesia observation

RuiDuo Yang, Michael S. Brown

Abstract: We propose a novel framework to query a music database by supplying an MPEG video sequence. The music and video are treated as time series data and the retrieval query is based on the similarity between corresponding features in the music and video sequence (e.g. the tempo in the music and the motion in the video). A similarity measure for matching these features is proposed to capture the synesthesia effect in music-video.