Arbitrary Style Transfer with Prototype-Based Channel Alignment

Yan Hong, Li Niu, Jianfu Zhang

Abstract: Style transfer aims to migrate the "style" from a style image to a content image. Despite the appealing results achieved by existing methods, few studies have considered the alignment of semantics or structures between the style image and the content image. To overcome this problem, we propose a novel network with two parallel branches: coarse-grained stylization branch and fine-grained decoration branch. In the stylization branch, we perform conventional AdaIN to produce globally stylized feature. In the decoration branch, we propose a ProtoType-based Channel Alignment module to align the channels between style feature and content feature, followed by Adaptive Group Transfer to produce locally stylized feature. Extensive experiments demonstrate that our proposed method outperforms state-of-the-art methods in terms of visual quality and efficiency.