Deep Optics: Learning Cameras and Optical Computing Systems

Gordon Wetzstein, Hayato Ikoma, Christopher A. Metzler, Yifan Peng

Abstract: Neural networks and other advanced image processing algorithms excel in a wide variety of computer vision and imaging applications, but their high performance also comes at a high computational cost and their success is sometimes limited. Here, we review recent hybrid optical-digital strategies to computational imaging that outsource parts of the algorithm into the optical domain. Using such a co-design of optics and image processing, we can facilitate application-domain-specific cameras or compute parts of a convolutional neural network in optics. Optical computing happens at the speed of light and without any memory or power requirements, thereby opening new directions for intelligent imaging systems.