Complex exploration dynamics from simple heuristics in a collective learning environment

Sabina Sloman, Robert L. Goldstone, Cleotilde Gonzalez

Abstract: Effective problem solving requires both exploration and exploitation. We analyze data from a group problem-solving taskto gain insight into how people use information from past experiences and from others to achieve explore-exploit trade-offs incomplex environments. The behavior we observe is consistent with the use of simple, reinforcement-based heuristics. Participants increase exploration immediately after experiencing a low payoff, and decrease exploration immediately after experiencing a high or improved payoff. We suggest that whether an outcome is perceived as “high” or “low” is a dynamic function of the outcome information available to participants. The degree to which the distribution of observed information reflects the true range of possible outcomes plays an important role in determining whether or not this heuristic is adaptive in a given environment.