Integrating TPS, cylindrical projection, and plumb-line constraint for natural stitching of multiple images

Jiongli Gao, Jun Wu, Xuemei Zhao, Gang Xu

Abstract: This paper presents a novel approach for natural stitching multiple images by integrating thin-plate spline (TPS), cylindrical projection, and plumb-line constraint. Firstly, the homography estimated under plumb-line constraint is used to transform each image to keep the scene in the image upward as much as possible, so as to suppress the accumulation of image projection deformation and make the transformed images approximately available for cylindrical projection. Then, by introducing cylindrical projection into TPS as a global transformation, a multiple image alignment framework called cylindrical projection thin-plate spline (CP-TPS) is established to accurately align the transformed images. In this step, the virtual control points (VCP) are set in the non-overlapping area of images so that the CP-TPS can produce desired deformation in the final stitched image. Finally, a seam-line intersecting the significant structure in the aligned image is automatically generated by combining TPS, dynamic programming matching, and control points triangulation. In this step, the seam-line itself is used to estimate CP-TPS parameter. Experiments were conducted on four public image sets. The results show that the proposed approach can realize the natural stitching of public multiple image sets and has the best performance, compared with Autostitch, APAP, NISwGSP, ELA, and GES-GSP.