Micro-Learning for Learning-Hard Problems

ICLR 2026 Conference Submission19284 Authors

Keywords: Learning-hard, Micro‑Learning, NP-complete, data imbalance, small-sample, deep learning

Abstract: Machine learning (ML) systems increasingly face high complexity data whose non-linear structure, noise, imbalance, or limited sample size thwart conventional models. We formalize this difficulty through the notion of Learning Hard Problems (LH-Ps), tasks that (i) defeat the vast majority of models, yet (ii) admit at least one high‑quality solution if the relevant domain knowledge is appropriately incorporated during training. To address LH‑Ps we introduce Micro‑Learning (MiL), a principled framework that constructs instance‑specific traininglets: small, knowledge‑fused subsets of the training data with demonstrably low complexity—and infers a deterministic local model for each that collectively form a global predictor. We prove that the decision version of optimal traininglet selection is NP‑complete, establishing a strong theoretical foundation for MiL. MiL dramatically reduces overfitting risk by eliminating irrelevant or noisy samples, while retaining interpretability and reproducibility through deterministic optimization in a Reproducing Kernel Hilbert Space. Experiments in benchmark domains, from music information retrieval to medical proteomics, show that MiL solves LH-Ps successfully and outperforms deep learning (DL) and classical baselines, especially on imbalanced or small-sample datasets, with negligible overfitting. Beyond an effective algorithm, our work provides (i) the first formal definition and characterization of LH‑Ps, (ii) a Learning‑Hard Index (LHI) to quantify task difficulty pre‑training, and (iii) theoretical guarantees on traininglet optimality and complexity. Together, these contributions enrich modern learning theory and offer a practical path to reliable AI in challenging regimes.

Primary Area: learning theory

Submission Number: 19284