MoViE: Mobile Diffusion for Video Editing

Recent progress in diffusion-based video editing techniques has shown remarkable potential and is being increasingly utilized in practical applications. However, these methods remain prohibitively expensive and particularly challenging to deploy on mobile devices. In this study, we introduce a series of optimizations that render mobile video editing feasible. Building upon the existing image editing model, we first optimize its architecture and incorporate a lightweight autoencoder. Subsequently, we propose a new classifier-free guidance distillation with multiple modalities, resulting in a 3× on-device speed-up. Finally, we reduce the number of sampling steps to one (10× speed-up) by introducing a novel adversarial distillation scheme which preserves the controllability of the editing process in contrast to previous arts. Collectively, these optimizations enable video editing at an impressive 12 frames per second on mobile devices, while maintaining high editing quality.