ETNet: Error Transition Network for Arbitrary Style Transfer

Chunjin Song, Zhijie Wu, Yang Zhou, Minglun Gong, Hui Huang

Abstract: Numerous valuable efforts have been devoted to achieve arbitrary style transfer since the seminal work of Gatys et al. However, existing state-of-the-art approaches often generate insufficiently stylized results under challenging cases. We believe a fundamental reason is that these approaches try to generate the stylized result in a single shot and hence fail to fully satisfy the constraints on semantic structures in the content images and style patterns in the style images. Inspired by the works on error-correction, instead we propose a self-correcting model to predict what is wrong with the current stylized result and refine it iteratively. For each refinement, we transit the error features across both the spatial and scale domain and invert the processed features into a residual image, with a network we call Error Transition Network (ETNet). The proposed model improves over the state-of-the-art methods with better semantic structures and more adaptive style pattern details. Various qualitative and quantitative experiments show that the key concept of both progressive strategy and error-correction yield better results.

Code Link: https://github.com/zhijieW94/ETNet

CMT Num: 332