CaltechAUTHORS
  A Caltech Library Service

Neural Computations Underlying Arbitration between Model-Based and Model-free Learning

Lee, Sang Wan and Shimojo, Shinsuke and O’Doherty, John P. (2014) Neural Computations Underlying Arbitration between Model-Based and Model-free Learning. Neuron, 81 (3). pp. 687-699. ISSN 0896-6273. http://resolver.caltech.edu/CaltechAUTHORS:20140210-091325729

[img]
Preview
PDF - Supplemental Material
See Usage Policy.

2197Kb

Use this Persistent URL to link to this item: http://resolver.caltech.edu/CaltechAUTHORS:20140210-091325729

Abstract

There is accumulating neural evidence to support the existence of two distinct systems for guiding action selection, a deliberative "model-based" and a reflexive "model-free" system. However, little is known about how the brain determines which of these systems controls behavior at one moment in time. We provide evidence for an arbitration mechanism that allocates the degree of control over behavior by model-based and model-free systems as a function of the reliability of their respective predictions. We show that the inferior lateral prefrontal and frontopolar cortex encode both reliability signals and the output of a comparison between those signals, implicating these regions in the arbitration process. Moreover, connectivity between these regions and model-free valuation areas is negatively modulated by the degree of model-based control in the arbitrator, suggesting that arbitration may work through modulation of the model-free valuation system when the arbitrator deems that the model-based system should drive behavior.


Item Type:Article
Related URLs:
URLURL TypeDescription
http://dx.doi.org/10.1016/j.neuron.2013.11.028DOIArticle
http://www.sciencedirect.com/science/article/pii/S0896627313011252PublisherArticle
http://www.cell.com/neuron/abstract/S0896-6273(13)01125-2PublisherArticle
http://dx.doi.org/10.1016/j.neuron.2014.01.030Related ItemReferenced Article
Additional Information:© 2014 Elsevier Inc. Accepted: November 25, 2013; Published: February 5, 2014. Supplemental Information includes five figures, five tables, and Supplemental Experimental Procedures and can be found with this article online at http://dx. doi.org/10.1016/j.neuron.2013.11.028. We thank Mimi Liljeholm and Peter Bossaerts for suggestions and insightful comments and Ralph Lee for his assistance. This work was funded by NIH grant DA033077-01 (supported by OppNet, NIH’s Basic Behavioral and Social Science Opportunity Network) to J.P.O.D., by funds from the Gordon and Betty foundation to J.P.O.D., by grants from JST-CREST to S.S., and by the Caltech-Tamagawa gCOE to S.S. and J.P.O.D.
Funders:
Funding AgencyGrant Number
NIHDA033077-01
Gordon and Betty Moore FoundationUNSPECIFIED
JST-CRESTUNSPECIFIED
Caltech-Tamagawa gCOEUNSPECIFIED
Record Number:CaltechAUTHORS:20140210-091325729
Persistent URL:http://resolver.caltech.edu/CaltechAUTHORS:20140210-091325729
Official Citation:Lee, Sang W., Shimojo, S., & O Doherty, John P. (2014). Neural Computations Underlying Arbitration between Model-Based and Model-free Learning. Neuron, 81(3), 687-699. doi: 10.1016/j.neuron.2013.11.028.
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:43734
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:10 Feb 2014 23:30
Last Modified:10 Feb 2014 23:30

Repository Staff Only: item control page