Differentiating without Partial Evaluation

ICLR 2026 Conference Submission19551 Authors

Keywords: automatic differentiation, interpretable surrogate model

Abstract:

In the physical sciences, the gradient of a model is often simplified into a
compact form ideal for a given context to be interpretable and more efficient;
in fact, sometimes the efficiency of evaluation can be improved by an
asymptotic factor due to symmetries. To learn interpretable surrogate models
that accelerate physics simulations, a differentiation system capable of
compact and unevaluated gradient expressions is highly desirable. However,
standard symbolic and algorithmic differentiation both start by partially
evaluating the model. After this points, the gradients irreversibly become
blackboxes with potentially obscure performance ceilings. Based on the
observation that composition is one of two combinators that form a complete
basis, we complete the chain rule with a second rule that enables
differentiation without any form of evaluation. Using a prototype
implementation, we obtain compact gradient expressions for an MLP and a common
physics model that, historically, resisted algorithmic differentiation. Lastly,
we discuss the theoretical and practical limitations of our approach.

Primary Area: infrastructure, software libraries, hardware, systems, etc.

Submission Number: 19551