PCA-Based Knowledge Distillation Towards Lightweight and Content-Style Balanced Photorealistic Style Transfer

Tai-Yin Chiu, Danna Gurari

Abstract: Photorealistic style transfer is the task of transferring the style of a reference image to another image such that the result still seems like a plausible photo. Our work is inspired by the observation that existing models are slow due to their large sizes. We introduce PCA-based knowledge distillation to distill lightweight models and show it is motivated by theory. To our knowledge, this is the first knowledge distillation method for photorealistic style transfer. We conduct experiments to demonstrate its versatility for use with different backbone architectures, VGG and MobileNet, across six image resolutions. Results demonstrate that, compared to existing models, our top-performing model runs at speeds 5-20x faster using at most 1% of the parameters. Beyond these speed and size benefits, we also demonstrate that our distilled models achieve a better balance between stylization strength and content preservation than existing models.