MagicMask: A Real-time and High-fidelity Face Swapping Method Robust to Face Pose]{MagicMask: A Fast and High-fidelity Face Swapping Method Robust to Face Pose

Jongmin Yu, Anoushka Harit, Jiankang Deng, Shan Luo, Jinhong Yang, Zhongtian Sun

Abstract: Recent face-swapping methods excel under controlled conditions but often fail when presented with extreme facial poses. Diffusion-based approaches may be able to overcome these issues, but they still face significant computational costs. This paper introduces MagicMask, a novel face-swapping framework that robustly handles various poses in real time by fusing visual and geometric information. Our method incorporates explicit, identity-adapted geometric cues into the latent feature space via a multi-head attention mechanism. It employs an Adversarial Facial Silhouette Alignment (AFSA) loss to preserve detailed facial boundaries adapted to source identity. Comprehensive experiments on multiple benchmarks demonstrate that MagicMask competes with state-of-the-art methods under standard conditions and significantly outperforms them in extreme pose scenarios. The source code for the demonstration of MagicMask is attached as supplementary materials.

Submission Number: 248