Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published November 2019 | Supplemental Material
Journal Article Open

Neurostimulation Reveals Context-Dependent Arbitration Between Model-Based and Model-Free Reinforcement Learning


While it is established that humans use model-based (MB) and model-free (MF) reinforcement learning in a complementary fashion, much less is known about how the brain determines which of these systems should control behavior at any given moment. Here we provide causal evidence for a neural mechanism that acts as a context-dependent arbitrator between both systems. We applied excitatory and inhibitory transcranial direct current stimulation over a region of the left ventrolateral prefrontal cortex previously found to encode the reliability of both learning systems. The opposing neural interventions resulted in a bidirectional shift of control between MB and MF learning. Stimulation also affected the sensitivity of the arbitration mechanism itself, as it changed how often subjects switched between the dominant system over time. Both of these effects depended on varying task contexts that either favored MB or MF control, indicating that this arbitration mechanism is not context-invariant but flexibly incorporates information about current environmental demands.

Additional Information

© The Author(s) 2019. Published by Oxford University Press. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model). Received: 01 February 2018; Revision Received: 23 January 2019; Accepted: 28 January 2019; Published: 19 March 2019. Funding: Swiss National Science Foundation with the "Sinergia" grant "Neuroeconomics of value-based decision making" (141965) and the Samsung Research Funding Center of Samsung Electronics (SRFC-TC1603-06). We want to thank Rafael Polania for assistance in applying the direct current stimulation. Conflict of Interest: None declared.

Attached Files

Supplemental Material - bhz019_supplementary_task_instructions.pdf


Files (978.8 kB)
Name Size Download all
978.8 kB Preview Download

Additional details

August 19, 2023
October 20, 2023