Effects of Stress and Genotype on Exploration-Exploitation Dynamics in Reinforcement Learning

Stress and genetic background regulate different aspects of behavioral learning through the action of stress hormones and neuromodulators. Similarly, in reinforcement learning (RL) models, exploitation-exploration factor and other meta-parameters control learning dynamics and performance. We found that many different measures of animal learning and performance can be reproduced by simple RL models using dynamic control of the meta-parameters. To study the effects of stress and genotype, we carried out 5-hole-box light conditioning and Morris water maze experiments with 2 different genetic strains of mice, exposing them to different stressors. Then, we used RL models to simulate their behavior. For each experimental session, we estimated a set of model meta-parameters that produced the best fit between the model and the animal performance. Exploration-exploitation factors had similar characteristic dynamics for the two simulated experiments, and there were statistically significant differences between different genetic strains and stress conditions.

Author: Gedi Lukšys, École Polytechnique Fédérale De Lausanne