Autonomous Video Registration Using Sensor Model Parameter Adjustments

Richard W. Cannata, Steven G. Blask, John A. Van Workum, Mubarak Shah

Abstract: We outline a novel approach for near real-time video registration based on sensor model parameter adjustments and the application of a Kalman filter. The goal of our precision video registration (PVR) development is to register video with a reference image to provide accurate 3D geolocations. Our sensor-based 3D treatment is unique since most registration approaches employ only simple image-to-image mappings, such as affine transformations. In our approach, we explicitly model the projections between the 3D world and 2D images and perform registration in 3D with greater accuracy and fidelity. PVR performance results show significant accuracy improvement over unregistered frame geolocation, and the autonomously generated video mosaics appear smooth and seamless.