Derivative-Controlled Compact Surrogates for Predictable Sensitivity

TMLR Paper7308 Authors

Abstract: Compact neural models are frequently deployed as surrogates inside larger pipelines, where failures are driven less by raw accuracy than by instability and excessive sensitivity. This paper develops a derivative-controlled training approach for low-capacity models, treating derivatives as a primary interface for shaping behavior. We introduce a compact parameterization paired with a derivative-aware objective that discourages brittle sensitivity across depth. We evaluate the approach with property-driven tests---training stability, sensitivity diagnostics, and downstream settings where shape-consistent behavior matters---showing that derivative control can improve behavioral stability and gradient predictability while preserving useful predictive performance.

Submission Type: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

Section 2 (Derivatives as Behavioral Control): We have significantly expanded this section to provide a theoretical and practical motivation for focusing on intermediate derivative amplification. It now includes a detailed discussion on the distinction between training stability and input sensitivity, as well as the conceptual link to Physics-Informed Neural Networks (PINNs) as suggested.

Section 3 (Related Work): We have restructured this section to better position our work against recent literature (post-2020) and established baselines like Spectral Normalization and Sharpness-Aware Minimization (SAM).

Assigned Action Editor: ~Pin-Yu_Chen1

Submission Number: 7308