A Pattern Language for Machine Learning Tasks

Benjamin Rodatz, Ian Fan, Tuomas Laakkonen, Neil John Ortega, Thomas Hoffmann, Vincent Wang

Keywords: Category Theory, Interpretability, Compositionality, Machine Learning Theory, Formal Methods, Behaviour Modelling, Objective Function Analysis, Diagrammatic Reasoning

TL;DR: mathematical framework (with empirical confirmation) for machine learning tasks, enhancing predictability and interpretability.

Abstract:

We formalise the essential data of objective functions as equality constraints on composites of learners. We call these constraints "tasks", and we investigate the idealised view that such tasks determine model behaviours. We develop a flowchart-like graphical mathematics for tasks that allows us to;
(1) design and optimise desired behaviours model-agnostically;
(2) offer a unified perspective of approaches in machine learning across domains;
(3) import insights from theoretical computer science into practical machine learning.
As a proof-of-concept of the potential practical impact of our theoretical framework, we exhibit and implement a novel "manipulator" task that minimally edits input data to have a desired attribute. Our model-agnostic approach achieves this without the need for custom architectures, adversarial training, random sampling, or interventions on the data, hence enabling capable, small-scale, and training-stable models.

Primary Area: other topics in machine learning (i.e., none of the above)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 6952