Supremo: Cloud-Assisted Low-Latency Super-Resolution in Mobile Devices

Juheon Yi, Seongwon Kim, Joongheon Kim, Sunghyun Choi

Abstract: We present <inline-formula><tex-math notation="LaTeX">${\sf Supremo}$</tex-math></inline-formula> , a cloud-assisted system for low-latency image super-resolution (SR) in mobile devices. As SR is extremely compute-intensive, we first further optimize state-of-the-art DNN to reduce the inference latency. Furthermore, we design a mobile-cloud cooperative execution pipeline composed of specialized data compression algorithms to minimize end-to-end latency with minimal image quality degradation. Finally, we extend <inline-formula><tex-math notation="LaTeX">${\sf Supremo}$</tex-math></inline-formula> to video applications by formulating a dynamic optimal control algorithm to design <inline-formula><tex-math notation="LaTeX">${\sf Supremo-Opt}$</tex-math></inline-formula> , which aims to maximize the impact of SR while satisfying latency and resource constraints under practical network conditions. <inline-formula><tex-math notation="LaTeX">${\sf Supremo}$</tex-math></inline-formula> upscales 360p image to 1080p in 122 ms, which is 43.68× faster than on-device GPU execution. Compared to cloud offloading-based solutions, <inline-formula><tex-math notation="LaTeX">${\sf Supremo}$</tex-math></inline-formula> reduces wireless network bandwidth consumption and end-to-end latency by 15.23× and 4.85× compared to baseline approach of sending and receiving whole images, and achieves 2.39 dB higher PSNR compared to using conventional JPEG to achieve similar data size compression. Furthermore, <inline-formula><tex-math notation="LaTeX">${\sf Supremo-Opt}$</tex-math></inline-formula> guarantees robust performance in practical scenarios.