Verbalized Algorithms

Supriya Lall, Christian Farrell, Hari Pathanjaly, Marko Pavic, Sarvesh Chezhian, Masataro Asai

Keywords: algorithmic reasoning, generative computing

TL;DR: Run trusty algorithms from the 80's, but *remastered* for the modern LLM era.

Abstract:

Instead of querying LLMs in a one-shot manner and hoping to get the right answer for a reasoning task,
we propose a paradigm we call \emph{verbalized algorithms} (VAs),
which leverage classical algorithms with established theoretical understanding.
VAs decompose a task into simple elementary operations on natural language strings
that they should be able to answer reliably, and limit the scope of LLMs to only those simple tasks.
For example, for sorting a series of natural language strings,
\emph{verbalized sorting} uses an LLM as a binary comparison oracle
in a known and well-analyzed sorting algorithm (e.g., bitonic sorting network).
We demonstrate the effectiveness of this approach
on sorting and clustering tasks.

Submission Number: 27