Generative Perception of Shape and Material from Differential Motion

We visualize the input video sequence, predicted surface normal and albedo (diffuse color) from a moving object.

(1) Objects with synthetic textures:

(2) Objects with original, artists' designed textures:

Visualization of the shiny/matte perception test designed by Hartung and Kersten. Note in the croissant shape example the static frames can look similar for both the shiny and diffuse material. Motion cues help to disambiguate the materials.

We apply temporal consistency guidance on the predictions of the overlapping frame to extend our model to longer video sequence (13 frames instead of the default 3 frames). This guided sampling method allows temporally consistent shape and material estimation. We compare with the shape prediction result from StableNormal, using their official video mode demo.