Plant Geometry Reconstruction From Field Data Using Neural Radiance Fields

Anushrut Jignasu, Ethan Herron, Talukder Zaki Jubery, James Afful, Aditya Balu, Baskar Ganapathysubramanian, Soumik Sarkar, Adarsh Krishnamurthy

Keywords: Neural Radiance Fields, Implicit Neural Surface Reconstruction, Digital Twins

Abstract: Real-time simulations of large-scale farming operations would provide farmers with data-driven and physics-consistent decision support. These real-time farming simulations could be accomplished using predictive digital twins. Predictive digital twins of biological entities allow for a virtual simulation of real-life processes for various environmental conditions, thus paving the way for a comprehensive understanding of various biological responses. One of the first steps in constructing a predictive digital twin is the 3D reconstruction of plant geometry. While traditional approaches for the reconstruction of plant geometry exist, they require a very expensive setup using a LIDAR or destructive imaging of the plant in a controlled environment. Neural approaches for 3D scene reconstruction have alleviated the data collection burden associated with traditional 3D reconstruction methods. In this work, we demonstrate the ability to generate a 3D reconstruction (mesh) of a maize plant by leveraging a recent work in 3D computer vision, Neural Radiance Fields (NeRFs), which uses data collected from a mobile phone camera. Our approach aims to generate high-resolution geometric models for several downstream tasks, such as developing a predictive digital twin.