Algorithm Design for Learned Algorithms

Avi Schwarzschild, Sean Michael McLeish, Arpit Bansal, Gabriel Diaz, Alex Stein, Aakash Chandnani, Aniruddha Saha, Richard Baraniuk, Long Tran-Thanh, Jonas Geiping, Tom Goldstein

Keywords: Algorithmic reasoning

TL;DR: When learning algorithms with neural networks, we can control the speed, accuracy, and generality.

Abstract:

Neural networks can learn known algorithms from data even when only trained with input/output pairs and no supervision over the intermediate steps.
This means that with labelled examples, we can potentially learn new algorithmic approaches.
Engineers designing new algorithms are often faced with trade offs like efficiency versus accuracy or generality and specificity.
We show that when learning algorithms from data, the same controls exist and we explore how model hyperparameters control the accuracy, efficiency, and generality of the resulting algorithm.
Our analysis covers learned approaches to computing prefix sums, solving mazes, and filling Sudoku puzzles.
As these domains have existing fast and accurate solvers, they serve as a test-bed for our analysis.
Finally, we extend our analysis to learning algorithms for constraint satisfiability -- an NP-Complete problem.

Primary Area: general machine learning (i.e., none of the above)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8179