The Road to Generalizable Neuro-Symbolic Learning Should be Paved with Foundation Models

Adam Stein, Aaditya Naik, Neelay Velingker, Mayur Naik, Eric Wong

Keywords: neuro-symbolic, foundation models, llms, generalization, interpretability, code, robustness

Abstract:

Neuro-symbolic learning was proposed to address challenges with training neural networks for complex reasoning tasks with the added benefits of interpretability, reliability, and efficiency.
Neuro-symbolic learning methods traditionally train neural models in conjunction with symbolic programs but they face significant challenges that limit them to simplistic problems.
On the other hand, purely-neural foundation models now reach state-of-the-art performance through prompting rather than training, but they are often unreliable and lack interpretability.
Supplementing foundation models with symbolic programs, which we call neuro-symbolic prompting, provides a way to use these models for complex reasoning tasks.
Doing so raises the question: What role does specialized model training as part of neuro-symbolic have in the age of foundation models?
To explore this question, we highlight three pitfalls of traditional neuro-symbolic learning with respect to the compute, data, and programs leading to generalization problems.
This position paper argues that
foundation models enable generalizable neuro-symbolic solutions,
offering a path towards achieving the original goals of neuro-symbolic learning without the downsides of training from scratch.

Submission Number: 675