A Survey on Future Frame Synthesis: Bridging Deterministic and Generative Approaches

TMLR Paper4528 Authors

Abstract: Future Frame Synthesis (FFS) focuses on generating future frame sequences conditioned on existing content. This survey provides a comprehensive review of existing research on FFS, covering commonly used datasets and representative algorithms. We discuss key challenges and trace the evolution of FFS in computer vision, particularly the shift from deterministic to generative approaches. Our taxonomy outlines major advances and methodological shifts, emphasizing the rising significance of generative models in producing realistic and diverse predictions. This survey offers a comprehensive analysis of current research and, moreover, suggests promising avenues for future exploration in this ever-changing domain.

Submission Length: Long submission (more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=QSUTq0plnW&referrer=%5BAuthor%20Console%5D(%2Fgroup%3Fid%3DTMLR%2FAuthors%23your-submissions)

Changes Since Last Submission:

We introduced a clearer taxonomy of FFS methods based on stochasticity, now explicitly summarized in Section 2.2.
We also introduced summary tables (e.g., Table 2, 3, and 4) that categorize representative models by architecture and key ideas.
We have incorporated several recent and previously missing works. This extension improves both academic coverage and connection to real-world progress.
We expanded the dataset section to discuss how dataset scale, diversity, and supervision modalities impact FFS performance and generalization.
The applications section has been extended with more specific discussions of how FFS techniques are applied in robotics, simulation, and creative industries.
We added an explicit subsection (2.3.3) on generalization challenges, detailing how training data affects synthesis quality and generalizability, especially in high-resolution and long-term synthesis scenarios.

Assigned Action Editor: ~Andreas_Lehrmann1

Submission Number: 4528