Feed-Forward Bullet-Time Reconstruction of Dynamic Scenes from Monocular Videos

Anonymous Author(s)

Submission ID: papers_17189

What do we do?

Framework figure

BTimer: Given a monocular video as input, our method reconstructs a 3D Gaussian Splatting (3DGS) representation at any desired timestamp in a feed-forward fashion.

Technical Contributions

We present the first feed-forward reconstruction model for dynamic scenes using a bullet-time formulation;

BTimer reconstructs a bullet-time scene within 150ms while reaching state-of-the-art performance on both static and dynamic scene datasets, even compared with optimization-based approaches.

Method Framework

Framework figure

The model takes as input a sequence of context frames and their Plücker embeddings, along with the context timestamp and target (`bullet') timestamp embeddings. It then directly predicts the 3DGS representation at the bullet timestamp.

Results

Qualitative Results

Qualitative Results on SORA scenes. We pause the video at a frame and move the camera to make a bullet-time effect.

Qualitative Results on DAVIS dataset. Left is the input video, right is the video rendered from a novel camera trajectory.

Qualitative Results on DyCheck iPhone dataset. Left is the input video, right two are the videos rendered from novel camera trajectories.

Dynamic Novel View Synthesis Benchmark Results

Quantitative Comparison — Baseline results on NVIDIA Dynamic Scene Dataset.

Results on Static Scenes

Results on the Tanks & Temples static scenes benchmark.

Effect of the NTE Module

Visualization of learned deformation

We visualize the deformation leanred by our model. With only 4 images as input, our model can reconstruct smooth 3D motions.

Visualization of learned scene flow

We visualize the scene flow learned by our model. We color the Gaussians by the pixel positions they associate with.