Learning from Irregularly Sampled Data with Deep Nadaraya–Watson Kernel Regression Networks (NWNet): Application to Endomicroscopy Image Reconstruction

Agnieszka Barbara Szczotka, Daniele Ravi, Dzhoshkun Ismail Shakir, Stephen P Pereira, Tom Vercauteren

Abstract: Probe-based Confocal Laser Endomicroscopy (pCLE) enables more accurate diagnosis via optical biopsy. pCLE probes relay on of fibres bundles, which generate irregularly sampled signals. Current pCLE reconstruction is based on interpolating irregular signals onto an over-sampled Cartesian grid, using a sub-optimal Delaunay triangulation based linear interpolation scheme. High-quality reconstruction with improved information representation should be possible with the use of Deep Convolutional Neural Networks (CNNs). However, classical CNNs are limited to take as an input only Cartesian images, not irregular data. In this work, we propose to embed Nadaraya-Watson (NW) kernel regression into the CNN framework as a novel trainable CNN layer that allows for processing of irregularly sampled data represented as sparse data on a Cartesian grid. We design a new NWNet architecture in conjunction with examplar-based super-resolution CNN, which allows reconstructing high-quality pCLE images from the irregularly sampled input data. Models were trained on a database of 8806 images from 238 pCLE video sequences. The results were validated through an image quality assessment based on a composition of the following metrics: PSNR, SSIM, GCF. Our analysis indicates that the proposed solution unlocks the potential of CNNs for sparse data processing. NW layer is the main contribution of our end-to-end model performing pCLE image reconstruction directly from sparse imaging input to high-resolution cartesian images. Our method outperforms the reconstruction method in current clinical use.

Keywords: Nadaraya–Watson Kernel Regression, sparse data, Endomicroscopy Image Reconstruction

Author Affiliation: Wellcome / EPSRC Centre for Interventional and Surgical Sciences, UCL Institute for Liver and Digestive Health at University College London