Temporal rate-distortion based optimal video summary generation

Zhu Li, Aggelos K. Katsaggelos, Bhavan Gandhi

Abstract: Video summary work originates from a viewing time constraint. A shorter version of the original video sequence is desirable in some applications. Clearly, a shorter version is necessary in applications where storage or bandwidth is limited. Our work is based on visual significance analysis of frames in a video sequence and a video temporal rate-distortion optimization framework. Temporal rate defines the number of frames allowable into the video summary, while temporal distortion is computed based on MPEG-7 metrics between mismatched frames. Several one-pass and two-pass algorithms are proposed along with discussions on experimental results.