Real-Time View Synthesis Using Deep Learning

Sushobhan Ghosh

Abstract: Recent developments in deep learning have led to breakthroughs in rendering novel views from sparse input views of a scene. While the accuracy of these algorithms has improved dramatically, it has come at a huge computational cost. While developments in graphics hardware have ameliorated some of the computational burdens, deep learning-based algorithms are far from being used for real-time view synthesis on commodity hardware and further from being used in head-mounted displays for AR/VR applications. We analyze different deep learning methods employed for view synthesis and test their performance in terms of accuracy as well as runtime. We identify the accuracy as well as speed bottlenecks in different approaches and propose various solutions to solve them.