Coloring anime line art videos with transformation region enhancement network

Ning Wang, Muyao Niu, Zhi Dou, Zhihui Wang, Zhiyong Wang, Zhaoyan Ming, Bin Liu, Haojie Li

Abstract: Highlights•We propose a multi-scale Transformation Region Enhancement Network (TRE-Net) to enhance the learning on geometric transformation regions.•We propose to locate geometric transformation regions with Multi-scale Euclidean Distance Difference (Multi-scale EDD) Map. To the best of our knowledge, the Multi-scale EDD Map is used for the first time in anime line art colorization. As a result, the network can adaptively leverage different strategies in different regions. Different from mask propagation algorithms for gray video colorization, our Multi-scale EDD Maps work well when dealing with line art sequences and act as robust guidance for our TRE-Net to improve the colorization quality and efficiency.•Feature Enhancement Module (FEM) and Attention loss are devised to generate color-aligned reference features by enhancing the feature learning on geometric transformation regions.•Extensive qualitative and quantitative experimental results demonstrate that our model can generate visually appealing colorized frames.