3D Scene Reconstruction and Parameter Inversion Using NeRF and Material Point Method
Keywords: novel view synthesis, granular material, parameter inversion
TL;DR: Inverse parameter with synthesized images data
Abstract: This study introduces a novel approach integrating Neural Radiance Fields (NeRF), the Material Point Method (MPM), and finite difference optimization for 3D scene reconstruction and parameter inversion in granular material simulations. Our method offers a new pathway for inferring material properties from visual data. We begin by simulating an actuator pushing sand at a constant velocity using MPM, treating these results as our experimental data. The initial state and the final state of material point positions are rendered in Blender to create highly realistic visualizations. We then capture multiple scene images from various angles, mimicking real-world multi-view photography. These images are the only known information about the experiment. NeRF is then employed to reconstruct the 3D scene from these multi-view images, using its ability to synthesize novel views and capture complex geometries. From the NeRF-reconstructed point cloud, we sample points to serve as material points in subsequent MPM simulations. This step is crucial as it allows us to initialize our simulations with a realistic representation of the initial state. Using MPM as the forward model, we apply the finite difference method to optimize actuator parameters such as force and velocity. The optimization process minimizes the multi-view image loss between MPM simulation results and the observed final state. This approach allows us to iteratively refine our simulation parameters to match the observed behavior of the granular material. Our results demonstrate accurate inversion of simulation parameters, offering a promising technique for inferring real-world material properties from visual observation alone.
Submission Number: 12
Loading