Kernelless Blind Inverse Imaging for Flat Meta-Optics Camera

Samuel Pinilla, Wenzhu Xing, SeyyedReza MiriRostami, Vladimir Katkovnik, Igor Shevkunov, Johannes E. Fröch, Arka Majumdar, Karen O. Egiazarian

Abstract: The race for color micro-cameras employing flat meta-optics instead of conventional refractive lenses has rapidly developed various end-to-end design frameworks. The metaoptics produce a specially designed spatial modulation of light wavefronts resulting in heavily blurred registered images. The optimal modulation is engineered to achieve advanced sharp imaging after computational data processing. The wavefront modulation and the image reconstruction are the fundamental micro-camera design problems with meta-optics. The popular convolution-based blurred image modeling (kernel-based) does not fit well with cameras with meta-optics. As a valuable alternative, we develop for image reconstruction the kernelless blind inverse imaging. This technique is based on a convolutional neural network. Its efficiency is demonstrated in the frame of the hardware-in-the-loop (HIL) joint optimization of meta-optics and image reconstruction software. The developed HIL setup allows us to overcome fundamental limitations of mismatch between theory-based and resultant experimental image formation problems of meta-optics. The resulting camera achieves high-quality full-color imaging for a 5 mm aperture optics with a focal length of 5 mm. We have observed a superior quality of the images captured by the developed hybrid meta-optical camera compared to the compound multi-lens optics of a commercial camera.