Multimodal Style Transfer via Graph Cuts

Yulun Zhang, Chen Fang, Yilin Wang, Zhaowen Wang, Zhe Lin, Yun Fu, Jimei Yang

Abstract:

An assumption widely used in recent neural style transfer methods is that image styles can be described by global
statics of deep features like Gram or covariance matrices.
Alternative approaches have represented styles by decomposing them into local pixel or neural patches. Despite the
recent progress, most existing methods treat the semantic
patterns of style image uniformly, resulting unpleasing results on complex styles. In this paper, we introduce a more
flexible and general universal style transfer technique: multimodal style transfer (MST). MST explicitly considers the
matching of semantic patterns in content and style images.
Specifically, the style image features are clustered into substyle components, which are matched with local content features under a graph cut formulation. A reconstruction network is trained to transfer each sub-style and render the final stylized result. We also generalize MST to improve some
existing methods. Extensive experiments demonstrate the
superior effectiveness, robustness, and flexibility of MST