Deep semantic space guided multi-scale neural style transfer

Jiachen Yu, Li Jin, Jiayi Chen, Youzi Xiao, Zhiqiang Tian, Xuguang Lan

Abstract: This paper mainly studies the Neural Style Transfer (NST) problem based on convolutional neural networks (CNN). Existing deep style migration algorithms do not mimic the styles to a reasonable position. To solve the problem, this paper proposes a multi-scale style transfer algorithm based on deep semantic matching. For purpose of guiding the correct migration of the style, we use the priori spatial segmentation and illumination information of the input image to integrate the deep semantic information. First, we find that spatial division and illumination analysis are two important visual understanding approaches for artists to make each painting decision. In order to simulate these two visual understanding approaches, this paper defines the DSS (deep semantic space), which contains spatial segmentation and contextual illumination information. The semantic exists in the form of CNN (convolution neural network) characteristic graph. Second, we propose a deep semantic loss function based on DSS matching and nearest neighbor search to optimize the effect of deep style migration. Third, we propose a multi-scale optimization strategy for improving the speed of our method. The experiments show that our method can reasonably synthesize images in spatial structures. The placement of each style is more reasonable and has a good visual aesthetic.