Sarah Starosta - Dopamine and the algorithmic basis of foraging decisions

The Future of Dopamine - ViDA Symposium - Nov 19, 2020

Sarah Starosta
Kepecs Lab, Washington University School of Medicine St. Louis

Dopamine and the algorithmic basis of foraging decisions

We are continually confronted with decisions about whether to stay engaged with the current option or to switch to a new one. These decisions include when to give up waiting in line or when to settle with a partner for life. These decisions have been extensively studied as foraging decisions, yet little is known about the underlying neural basis.
Here, we studied these decisions and its neural correlate in mice facing the decision when to leave depleting reward sources. To explore the choice strategy and its neural correlates, we implemented several reward manipulations and performed optical recordings dopamine neuron activity in the Ventral Tegmental Area (VTA). We observed, unexpectedly, that mice tend to leave a depleting source earlier after a higher than expected reward. Critically, this observation allowed us to distinguish between different theoretical models because only one that implemented a decision rule where animals compare the next expected reward to the average of the previous rewards predicted this result. Additionally, we show that this decision rule may be learned via a reinforcement learning (RL) paradigm called R-learning, but is not consistent with classical V-, or Q-learning paradigms. On a neuronal level, we observed that dopaminergic signaling in the VTA best correlates with the Reward Prediction Error (RPE) of R-learning, pointing to a potential learning mechanism that optimizes stay-or-leave choices.
Overall, our work offers an algorithmic decision rule and neuronal implementation for an ethologically relevant behavior based on qualitative different model predictions.