Human-like Few-Shot Learning via Bayesian Reasoning over Natural Language

Kevin Ellis

Keywords: Cognitive science, Bayesian, Language model, Induction, Psychology, Reasoning

TL;DR: We build a model of human concept learning by integrating language models with probabilistic reasoning

Abstract:

A core tension in models of concept learning is that the model must carefully balance the tractability of inference against the expressivity of the hypothesis class. Humans, however, can efficiently learn a broad range of concepts.
We introduce a model of inductive learning that seeks to be human-like in that sense.
It implements a Bayesian reasoning process where a language model first proposes candidate hypotheses expressed in natural language, which are then re-weighed by a prior and a likelihood.
By estimating the prior from human data, we can predict human judgments on learning problems involving numbers and sets, spanning concepts that are generative, discriminative, propositional, and higher-order.

Submission Number: 8750