Zero-Shot Mathematical Problem Solving with Large Language Models via Multi-Agent Conversation Programming

Vivian Keating

Track: Innovations in AI for Education (Day 1)

Paper Length: short-paper (2 pages + references)

Keywords: Machine Learning, LLM, Multi-agent, Conversation Programming

Abstract: This research explores the application of conversation programming and multi-agent collaboration techniques to enhance the zero-shot, mathematical problem solving of Large Language Models (LLMs), with a specific focus on GPT-4. We compare various math solver strategies: a straightforward system prompt (Raw), a single-agent with Python access, and a multi-agent solution (Aurek), vs a single agent solution explored in prior work. Here two agents, 'Aurek' and 'Besh', work together through conversation programming and the social dynamics between them is central to achieving accurate final results. In the context of using the GPT-4 and without providing additional mathematical knowledge, we achieve a state-of-the-art 62.69\% accuracy for zero-shot solution generation on a representative subset of the MATH dataset.

Submission Number: 14