Physics-Informed Autoencoder for Enhancing Data Quality to Improve the Forecasting Reliability of Carbon Dioxide Emissions from Agricultural Fields

Corentin Houpert, Muhammad Saad Zia, Ashiq Anjum, Noel Clancy, Jörg Kaduk, Heiko Balzter

Keywords: physics-informed machine learning, autoencoders, gap-fillling, net ecosystem exchange, noise, stochastic differential equation

TL;DR: We introduce a Physics-Informed Autoencoder in order to fill the gaps and improve the training of the Net Ecosystem Exchange measurements at farm scale.

Abstract: Missing values in measurements for carbon dioxide emissions on drained peatlands remains an open challenge for training forecasting techniques to achieve net zero. Existing methods struggle to model $\ce{CO_2}$ emissions to fill gaps at the field scale, especially in nighttime measurements. We propose novel Physics-Informed Autoencoders (PIAEs) for stochastic differential equations (SDEs), which combine the generative capabilities of Autoencoders with the reliability of physical models of Net Ecosystem Exchange (NEE) that quantify $\ce{CO_2}$ exchanges between the atmosphere and major carbon pools. Our method integrates an SDE describing the changes in NEE and associated uncertainties to fill gaps in the NEE measurements from eddy covariance (EC) flux towers. We define this SDE as a Wiener process with a deterministic drift term based on day and night time NEE physics models, and stochastic noise term. In the PIAE model, various sensor measurements are encoded into the latent space, and a set of deterministic decoders approximate the SDE parameters, and a probabilistic decoder predicts noise term. These are then used to predict the drift in NEE and thereby the optimal NEE forecast at the next time instance using the SDE. Finally, we use a loss function as a weighted sum of the Mean Squared Error (MSE) and Maximum Mean Discrepancy (MMD) between the measurements and the reconstructed samples and the associated noise and drift. PIAE outperforms the current state-of-the-art Random Forest Robust on predicting nighttime NEE measurements on various distribution-based and data-fitting metrics. We present a significant improvement in capturing temporal trends in the NEE at daily, weekly, monthly and quarterly scales.

Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8173