VIPER: Video-Informed PDE Extraction and Recovery

Farhat Shaikh, Ayan Banerjee, Sandeep Gupta

Keywords: video-based physics learning, PDE discovery, sparse regression, differentiable simulation, Liquid Time-Constant networks, implicit dynamics, computer vision

TL;DR: First framework to recover sparse PDE coefficients directly from RGB video, achieving up to 101x improvement on clean video and enabling coefficient recovery from 20% spatial coverage.

Abstract: Video captures physical phenomena everywhere, from smartphone recordings of ocean waves to thermal cameras in industrial processes, yet extracting the governing mathematical laws from such observations remains an open challenge at the intersection of computer vision and scientific discovery. Existing PDE discovery methods require direct access to spatiotemporal field measurements and rely on numerical differentiation, which amplifies noise at rate O((σ/Δx)^k) for k-th order derivatives and demands complete spatial coverage. We introduce VIPER (Video-Informed PDE Extraction and Recovery), the first framework to recover explicit sparse PDE coefficients directly from RGB video of spatiotemporal dynamics. VIPER extracts scalar fields via colormap inversion, encodes temporal dynamics through a Liquid Time-Constant (LTC) neural network, and validates coefficient estimates via a differentiable PDE solver that converts the noise-amplifying differentiation problem into a noise-attenuating integration problem. On five canonical PDEs (KdV, Burgers, Kuramoto-Sivashinsky, Schrodinger, NLS), VIPER achieves up to 101x improvement on clean video and 4-29x improvement under 5% noise on four PDEs. Critically, VIPER recovers coefficients from as little as 20% spatial coverage, a regime where derivative-based methods require interpolation that degrades accuracy, enabling interpretable digital twin construction from partial visual observations.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 23