Comparison of super-resolution and noise reduction for passive single-photon imaging

Martin Laurenzis, Trevor Seets, Emmanuel Bacher, Atul Ingle, Andreas Velten

Abstract:

Single-photon sensitive image sensors have recently gained popularity in passive
imaging applications where the goal is to capture photon flux (brightness) values of different
scene points in the presence of challenging lighting conditions and scene motion. Recent
work has shown that high-speed bursts of single-photon timestamp information captured using
a single-photon avalanche diode camera can be used to estimate and correct for scene motion
thereby improving signal-to-noise ratio and reducing motion blur artifacts. We perform a comparison of various design choices in the processing pipeline used for noise reduction, motion
compensation, and upsampling of single-photon timestamp frames. We consider various
pixelwise noise reduction techniques in combination with state-of-the-art deep neural network
upscaling algorithms to super-resolve intensity images formed with single-photon timestamp
data. We explore the trade space of motion blur and signal noise in various scenes with different
motion content. Using real data captured with a hardware prototype, we achieved superresolution reconstruction at frame rates up to 65.8 kHz (native sampling rate of the sensor) and
captured videos of fast-moving objects. The best reconstruction is obtained with the motion
compensation approach, which achieves a structural similarity (SSIM) of about 0.67 for fastmoving rigid objects. We are able to reconstruct subpixel resolution. These results show the
relative superiority of our motion compensation compared to other approaches that do not exceed
an SSIM of 0.5.