Fusion for Tile-based Deconvolution Layers

Min-Wu Jeong, Chae-Eun Rhee

Abstract: Recently, various deep learning accelerators are being studied through data flow structure improvement and memory access optimization. Among them, the encoder-decoder model is widely used in object detection and semantic segmentation showing good performance. However, due to the deconvolution operation that outputs a high-resolution feature map from the decoder, the memory access and computational complexity are higher than that of the existing encoder-only structure. Thus, it is a big obstacle to the implementation of encoder-decoder accelerators. Most of the previous studies have focused only on the encoder part. This paper attempts to apply the fusion approach, which was effective for the convolution layer of the encoder, to the deconvolution of the decoder and shows the possibility of reducing the processing time and hardware complexity.