A model for learning based on the joint estimation of stochasticity and volatility

Payam Piray, Nathaniel D. Daw

Abstract: Previous research has stressed the importance of uncertainty for controlling the speed of learning, and how such control depends on the learner inferring the noise properties of the environment, especially volatility: the speed of change. However, learning rates are jointly determined by the comparison between volatility and a second factor, moment-to-moment stochasticity. Yet much previous research has focused on simplified cases corresponding to estimation of either factor alone. Here, we introduce a learning model, in which both factors are learned simultaneously from experience, and use the model to simulate human and animal data across many seemingly disparate neuroscientific and behavioral phenomena. By considering the full problem of joint estimation, we highlight a set of previously unappreciated issues, arising from the mutual interdependence of inference about volatility and stochasticity. This interdependence complicates and enriches the interpretation of previous results, such as pathological learning in individuals with anxiety and following amygdala damage. Human learning depends on opposing effects of two noise factors: volatility and stochasticity. Here the authors present a model of learning that shows how and why joint estimation of these factors is important for understanding healthy and pathological learning.

External IDs:doi:10.1038/s41467-021-26731-9