Neurostimulation Reveals Context-Dependent Arbitration Between Model-Based and Model-Free Reinforcement Learning
While it is established that humans use model-based (MB) and model-free (MF) reinforcement learning in a complementary fashion, much less is known about how the brain determines which of these systems should control behavior at any given moment. Here we provide causal evidence for a neural mechanism that acts as a context-dependent arbitrator between both systems. We applied excitatory and inhibitory transcranial direct current stimulation over a region of the left ventrolateral prefrontal cortex previously found to encode the reliability of both learning systems. The opposing neural interventions resulted in a bidirectional shift of control between MB and MF learning. Stimulation also affected the sensitivity of the arbitration mechanism itself, as it changed how often subjects switched between the dominant system over time. Both of these effects depended on varying task contexts that either favored MB or MF control, indicating that this arbitration mechanism is not context-invariant but flexibly incorporates information about current environmental demands.
Additional Information© The Author(s) 2019. Published by Oxford University Press. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model). Received: 01 February 2018; Revision Received: 23 January 2019; Accepted: 28 January 2019; Published: 19 March 2019. Funding: Swiss National Science Foundation with the "Sinergia" grant "Neuroeconomics of value-based decision making" (141965) and the Samsung Research Funding Center of Samsung Electronics (SRFC-TC1603-06). We want to thank Rafael Polania for assistance in applying the direct current stimulation. Conflict of Interest: None declared.
Supplemental Material - bhz019_supplementary_task_instructions.pdf