Fast Optical Flow Extraction from Compressed Video

Sean I. Young, David Taubman

Abstract: We propose the fast optical flow extractor, a filtering scheme that recovers artifact-free optical flow fields from HEVC-compressed video. To extract accurate optical flow fields, we form a regularized optimization problem that considers the smoothness of the solution and the pixelwise confidence weights of the artifact-ridden HEVC motion field. Solving such an optimization problem is slow, so we first convert the problem into a confidence-weighted filtering task. We then use the extracted optical flow fields for fast synthetic refocus of video using camera motion. By leveraging the already-available HEVC motion information, we achieve a 10-fold speed-up in the running times compared to similar methods. Our fast optical flow extractor is useful since video signals encountered in real life are typically available already in coded formats. Our fast optical flow extractor is general, and works on motion fields of any video coder.