Nonparametric Teaching for Sequential Property Learners

ICLR 2026 Conference Submission15861 Authors

Keywords: Nonparametric Teaching, Sequential Property Learning, Functional Gradient Descent

Abstract: Determining the properties of sequence-structured data, e.g., the sentiment of a text, fundamentally requires learning the implicit relationship that maps sequences to their corresponding properties. This learning process is often expensive for sequential property learners like Recurrent Neural Networks (RNNs). To tackle this, we introduce a paradigm called **Re**current **N**eural **T**eaching (ReNT), which reinterprets the learning process through a novel nonparametric teaching lens. Specifically, the latter provides a theoretical framework for teaching implicitly defined (i.e., nonparametric) mappings via example selection. Such an implicit mapping is realized by a dense set of sequence-property pairs, with the ReNT teacher selecting a subset of them to facilitate faster convergence in RNN training. By analytically investigating the effect of sequence order on parameter-based gradient descent during training, and recasting the evolution of RNNs—driven by parameter updates—through functional gradient descent in nonparametric teaching, we reveal *for the first time* that teaching sequential property learners (i.e., RNNs) is consistent with teaching order-aware nonparametric learners. These new findings readily prompt ReNT to improve the learning efficiency of the sequential property learner, achieving substantial cuts in training time for sequence-level (-32.77% to -46.39%) and element-level (-36% to -39.17%) tasks, while still preserving its generalization performance.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 15861