Physics-Based Learned Diffuser for Single-shot 3D Imaging

Eric Markley, Fanglin Linda Liu, Michael Kellman, Nick Antipa, Laura Waller

Keywords: End-to-End Design, Computational Imaging, Inverse Problems, Physics-Based Learning, Machine Learning, Fluorescence Microscopy

TL;DR: By coupling a differential fluorescence microscopy forward model with a physics-based differentiable reconstruction algorithm we are able to learn a diffuser for single-shot 3D imaging out performs previous approaches.

Abstract:

A diffuser in the Fourier space of an imaging system can encode 3D fluorescence intensity information in a single-shot 2D measurement, which is then recovered by a compressed sensing algorithm. Typically, the diffusers used in such systems are either off-the-shelf, heuristically designed, or merit function driven. In this work we use a differentiable forward model of single-shot 3D microscopy
in conjunction with an invertible and differentiable reconstruction algorithm, ISTA-Net+, to jointly optimize both the diffuser surface shape and the reconstruction parameters. By choosing a differentiable and invertible reconstruction method, we enable the use of memory-efficient backpropagation to trade off storage with a reasonable increase in compute time, in order to fit an unrolled network containing a large-scale 3D volume into a single GPU’s memory. We validate our method on 2D and 3D single-shot imaging, where our learned diffuser demonstrates improved reconstruction quality compared to previous heuristic designs.