Expanded Convolutional Neural Network Based Look-Up Tables for High Efficient Single-Image Super-Resolution

Kai Yin, Jie Shen

Abstract: Advanced mobile computing has led to a surge in the need for practical super-resolution (SR) techniques. The look-up table (LUT) based SR-LUT has pioneered a new avenue of research without needing hardware acceleration. Nevertheless, all preceding methods that drew inspiration from the SR-LUT framework invariably resort to interpolation and rotation techniques for diminishing the LUT size, thereby prolonging the inference time and contradicting the original objective of efficient SR. Recently, a study named EC-LUT proposed an expanded convolution method to avoid interpolation operations. However, the performance of EC-LUT regarding SR quality and LUT volume is unsatisfactory. To address these limitations, this paper proposes a novel expanded convolutional neural network (ECNN). Specifically, we further extend feature fusion to the feature channel dimension to enhance mapping ability. In addition, our approach reduces the number of single indexed pixels to just one, eliminating the need for rotation tricks and dramatically reducing the LUT size from the MB level to the KB level, thus improving cache hit rates. By leveraging these improvements, we can stack expanded convolutional layers to form an ECNN, with each layer convertible to LUTs during inference. Experiments show that our method improves the overall performance of the upper limit of LUT based methods. For example, under comparable SR quality conditions, our model achieves state-of-the-art performance in speed and LUT volume.

Primary Subject Area: [Content] Vision and Language

Secondary Subject Area: [Content] Vision and Language

Relevance To Conference: The ECNN model presented in this work significantly contributes to multimedia processing by enhancing the efficiency of single-image super-resolution. It achieves this through an innovative approach that reduces the size of Look-Up Tables, enabling faster processing speeds and improved computational efficiency without compromising on image quality. This advancement is particularly beneficial for multimedia applications where high-quality image processing is essential, such as in the production and distribution of high-resolution images and videos. The ECNN's efficiency also opens up new possibilities for handling high-dimensional multimodal data, thereby advancing the field of multimedia processing technology.

Submission Number: 3108