Per-channel autoregressive linear prediction padding in tiled CNN processing of 2D spatial data

Olli Niemitalo, Otto Rosenberg, Nathaniel Narra, Olli Koskela, Iivari Kunttu

Keywords: CNN, padding, linear prediction, spatial data

TL;DR: Linear prediction used as a padding method in a super-resolution CNN reduces error in tile edges compared to zero and replication padding.

Abstract: We present linear prediction as a differentiable padding method that has no trainable parameters. For each channel, a stochastic autoregressive linear model is fitted to the data by minimizing its noise terms in the least-squares sense. The data is iteratively padded with conditional expected values of the autoregressive model. We trained the convolutional RVSR super-resolution model from scratch on satellite image data, using different padding methods. The simplest variant of linear prediction padding reduced the mean square super-resolution error by ~2% at the image edges, compared to zero and replication padding, with a ~25% increase in inference time. Linear prediction padding better approximated satellite image data and RVSR feature map data. With zero padding, RVSR appeared to use more of its capacity to compensate for the higher approximation error. Cropping the RVSR output by a few pixels reduced the super-resolution error and suppressed the impact of the choice of padding method, favoring fast zero and replication padding.

Primary Area: Deep learning (e.g., architectures, generative models, optimization for deep networks, foundation models, LLMs)

Submission Number: 11609