Library-based image coding

Nuno Vasconcelos, Andrew Lippman

Abstract: We explore a video coding algorithm that uses long term memory in addition to motion compensated prediction to build a model from video frames for coding. The long term memory, modeled on the cache used in operating systems and searched by a vector quantization formalism, holds recurring visual elements such as cyclically revealed information and actors that persist though several images. This video "library" exploits persistence to gain coding efficiency. Further, the existence of such a library can facilitate interactive and content-based operations on the video sequence such as browsing, searching by content, categorizing and enhancing the pictorial quality of the images.<