Learning environment-specific learning rates

Jonas Simoens, Tom Verguts, Senne Braem

Abstract: Author summary People constantly have to make decisions, such as what to wear for the day, or which pizza to order at a restaurant. Fortunately, people can learn from past decisions to inform future ones. However, environments may be unstable: The best pizza today is not necessarily the best pizza tomorrow. The chef may have had a bad day, in which case no learning needs to take place, or the restaurant may have changed chefs, in which case learning needs to restart from scratch. For this reason, it pays off to learn the instabilities of different environments: Which pizza is best may change more often in one restaurant than in another (e.g., because chefs change more quickly in one restaurant than in another). Formally, environmental instability determines how strongly one should update the expected value of an option (e.g., a pizza) based on novel information, often referred to as the learning rate. We thus investigated if people can learn different learning rates for different environments. We demonstrated that they can: Participants randomly and quickly alternated between a stable and an unstable environment, and they learned to use higher learning rates in the unstable than in the stable environment.