Continual Learning of Physical Systems via Derivative Distillation

Claudia Gentili, Alessandro Trenta, Andrea Cossu, Davide Bacciu

Keywords: continual learning, physics-inspired, distillation

TL;DR: We point to physical systems as a new benchmark for continual learning. We show that distillation-based continual learning approaches acting on physical variables are effective in learning continually physical systems.

Abstract: Continual learning (CL) designs models that adapt to non-stationary data streams while preserving previously acquired knowledge. Moving away from current CL benchmarks and datasets, we turn our attention to scientific machine learning, and we study how CL can approach physical systems, which are inherently continuous in time and space and are naturally described by differential equations over spatio-temporal domains. We adopt Derivative Distillation, a distillation-based approach that leverages model derivatives as a compact representation of knowledge and integrates it within a physics-informed learning framework. Our results show that the Derivative Distillation enables stable adaptation with minimal forgetting. In some cases, the performance surpasses that of a model jointly trained on all data at once. These findings highlight physical systems as a promising and realistic benchmark for CL.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 72