Pixel Deconvolutional Networks

Hongyang Gao, Hao Yuan, Zhengyang Wang, Shuiwang Ji

Abstract:

Deconvolutional layers have been widely used in a variety of deep
models for up-sampling, including encoder-decoder networks for
semantic segmentation and deep generative models for unsupervised
learning. One of the key limitations of deconvolutional operations
is that they result in the so-called checkerboard problem. This is
caused by the fact that no direct relationship exists among adjacent
pixels on the output feature map. To address this problem, we
propose the pixel deconvolutional layer (PixelDCL) to establish
direct relationships among adjacent pixels on the up-sampled feature
map. Our method is based on a fresh interpretation of the regular
deconvolution operation. The resulting PixelDCL can be used to
replace any deconvolutional layer in a plug-and-play manner without
compromising the fully trainable capabilities of original models.
The proposed PixelDCL may result in slight decrease in efficiency,
but this can be overcome by an implementation trick. Experimental
results on semantic segmentation demonstrate that PixelDCL can
consider spatial features such as edges and shapes and yields more
accurate segmentation outputs than deconvolutional layers. When used
in image generation tasks, our PixelDCL can largely overcome the
checkerboard problem suffered by regular deconvolution operations.

TL;DR: Solve checkerboard problem in Deconvolutional layer by building dependencies between pixels

Keywords: Deep Learning, Deconvolutional Layer, Pixel CNN

Code: [ divelab/PixelDCN](https://github.com/divelab/PixelDCN) + [ 3 community implementations](https://paperswithcode.com/paper/?openreview=B1spAqUp-)

Data: [COCO](https://paperswithcode.com/dataset/coco), [CelebA](https://paperswithcode.com/dataset/celeba)

Community Implementations: [ 4 code implementations](https://www.catalyzex.com/paper/arxiv:1705.06820/code)