Neural Point Processes for Pixel-wise Regression

Chengzhi Shi, Gözde Özcan, Miquel Sirera Perelló, Yuanyuan Li, Nina Iftikhar Shamsi, Stratis Ioannidis

TL;DR: We introduce a novel approach that combines pin-based Gaussian Processes with neural networks to exploit spatial correlations between pixels on sparsely annotated 2D images.

Abstract: We study pixel-wise regression problems with sparsely annotated images. Traditional regression methods based on mean squared error emphasize pixels with labels, leading to distorted predictions in unlabeled areas. To address this limitation, we introduce Neural Point Processes, a novel approach that combines 2D Gaussian Processes with neural networks to leverage spatial correlations between sparse labels on images. This approach offers two key advantages: it imposes smoothness constraints on the model output and enables conditional predictions when sparse labels are available at inference time. Empirical results on synthetic and real-world datasets demonstrate a substantial improvement in mean-squared error and $R^2$ scores, outperforming standard regression techniques. On the real-world dataset COWC, we achieve an $R^2$ of $0.769$ with $81$ out of $40,000$ ($0.2$%) points labeled, while standard regression loss (MSE) results in an $R^2$ of $0.060$.

Submission Number: 540