Real-Time Flexible Accelerator for Fisheye Image Correction Based on Zynq SoC

HoBin Oh, Gyu Hyeon Hwang, Jae Wook Jeon

Abstract: Fisheye cameras' broad field of view makes them ideal for a variety of applications, such as robotic navigation, drone vision, and car surround-view systems. Fisheye eyeglasses' intrinsic nonlinear distortion, however, makes it difficult to accurately interpret images and analyze them later. This research proposes a Zynq System on Chip (SoC)-based accelerator to dynamically restore fisheye lens image distortion in real time. The proposed system improves the limitations of conventional field programmable gate array (FPGA) designs that require fixed parameters and leverages the Python Productivity for Zynq (PYNQ) framework to enable dynamic parameter modification without reprogramming. Experimental results show that the proposed system processes images with a resolution of 960x640 at a speed of 135.7 frames per second (fps), achieving more than 20 times faster performance compared to software-based solutions. Additionally, it can maintain real-time performance even at Full HD resolution. We also evaluated the accuracy by comparing the fisheye distortion correction of four interpolation techniques provided by OpenCV with the quality indicators Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM). It is also flexible enough to allow the system to dynamically adapt to environmental changes and process images efficiently. This research provides a highly flexible, real-time fisheye distortion correction system for various industries such as robotics, automotive, and drones.