SALSA-CLRS: A Sparse and Scalable Benchmark for Algorithmic Reasoning

Julian Minder, Florian Grötschla, Joël Mathys, Roger Wattenhofer

Keywords: algorithmic reasoning, benchmark, generalisation, scalability

TL;DR: We introduce an extension to the CLRS algorithmic learning benchmark, prioritizing scalability and the utilization of sparse representations.

Abstract:

We introduce an extension to the CLRS algorithmic learning benchmark, prioritizing scalability and the utilization of sparse representations. Many algorithms in CLRS require global memory or information exchange, mirrored in its execution model, which constructs fully connected graphs (not sparse) based on the underlying problem. Despite CLRS’s aim of assessing how effectively learned algorithms can generalize to larger instances, the existing execution model becomes a significant constraint due to its demanding memory requirements and runtime
(hard to scale). Simultaneously, certain algorithms do not demand a fully connected graph; these algorithms, primarily distributed in nature, align closely with the message-passing paradigm employed by Graph Neural Networks. Hence, we propose SALSA-CLRS, an extension of the current CLRS benchmark specifically with scalability and sparseness in mind. Our approach includes adapted algorithms from the original CLRS benchmark and introduces new problems from distributed and randomized algorithms. Moreover, we perform a thorough empirical evaluation of our benchmark. Code is publicly available at https://github.com/jkminder/salsa-clrs.

Submission Type: Extended abstract (max 4 main pages).

Agreement: Check this if you are okay with being contacted to participate in an anonymous survey.

Software: https://github.com/jkminder/salsa-clrs

Submission Number: 114