GeMS: Efficient Gaussian Splatting for Extreme Motion Blur

TMLR Paper4917 Authors

Abstract:

We introduce GeMS, a framework for 3D Gaussian Splatting designed to handle severely
motion-blurred images. State-of-the-art deblurring method for extreme motion blur, such
as ExBluRF, as well as Gaussian Splatting-based approaches like Deblur-GS, typically as-
sume access to corresponding sharp images for camera pose estimation and point cloud
generation, which is an unrealistic assumption. Additionally, methods relying on COLMAP
initialization, such as BAD-Gaussians, fail due to the lack of reliable feature correspon-
dences in cases of severe motion blur. To address these challenges, we propose GeMS, a
3D Gaussian Splatting (3DGS) framework that reconstructs scenes directly from extremely
motion-blurred images. GeMS integrates: (1) VGGSfM, a deep learning-based Structure
from Motion (SfM) pipeline which estimates camera poses and generates point clouds di-
rectly from severely motion-blurred images; (2) 3DGS-MCMC (Markov Chain Monte Carlo)
enables robust scene initialization by treating Gaussians as samples from an underlying prob-
ability distribution, eliminating heuristic densification and pruning strategies; and (3) Joint
optimization of camera motion trajectory and Gaussian parameters which ensures stable and
accurate reconstruction. While this pipeline produces reasonable reconstructions, extreme
motion blur can still introduce inaccuracies, especially when all input views are severely
blurred. To address this, we propose GeMS-E, which integrates a progressive refinement
step when event data is available. Specifically, we perform (4) Event-based Double Integral
(EDI) deblurring, which first restores deblurred images from motion-blurred inputs using
events. These deblurred images are then fed into the GeMS framework, leading to improved
pose estimation, point cloud generation, and hence overall reconstruction quality. Both
GeMS & GeMS-E achieve state-of-the-art performance on synthetic as well as real-world
datasets, demonstrating their effectiveness in handling extreme motion blur. To the best
of our knowledge, we are the first to effectively address this motion deblurring problem in
extreme blur scenarios within a 3D Gaussian Splatting framework directly from severely
motion blurred images.

Submission Length: Long submission (more than 12 pages of main content)

Assigned Action Editor: ~Ming-Hsuan_Yang1

Submission Number: 4917