CaltechAUTHORS
  A Caltech Library Service

Human Dorsal Striatal Activity during Choice Discriminates Reinforcement Learning Behavior from the Gambler’s Fallacy

Jessup, Ryan K. and O'Doherty, John P. (2011) Human Dorsal Striatal Activity during Choice Discriminates Reinforcement Learning Behavior from the Gambler’s Fallacy. Journal of Neuroscience, 31 (17). pp. 6296-6304. ISSN 0270-6474. PMCID PMC6622656. https://resolver.caltech.edu/CaltechAUTHORS:20110519-152522041

[img]
Preview
PDF - Published Version
Creative Commons Attribution Non-commercial Share Alike.

833Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20110519-152522041

Abstract

Reinforcement learning theory has generated substantial interest in neurobiology, particularly because of the resemblance between phasic dopamine and reward prediction errors. Actor–critic theories have been adapted to account for the functions of the striatum, with parts of the dorsal striatum equated to the actor. Here, we specifically test whether the human dorsal striatum—as predicted by an actor–critic instantiation—is used on a trial-to-trial basis at the time of choice to choose in accordance with reinforcement learning theory, as opposed to a competing strategy: the gambler's fallacy. Using a partial-brain functional magnetic resonance imaging scanning protocol focused on the striatum and other ventral brain areas, we found that the dorsal striatum is more active when choosing consistent with reinforcement learning compared with the competing strategy. Moreover, an overlapping area of dorsal striatum along with the ventral striatum was found to be correlated with reward prediction errors at the time of outcome, as predicted by the actor–critic framework. These findings suggest that the same region of dorsal striatum involved in learning stimulus–response associations may contribute to the control of behavior during choice, thereby using those learned associations. Intriguingly, neither reinforcement learning nor the gambler's fallacy conformed to the optimal choice strategy on the specific decision-making task we used. Thus, the dorsal striatum may contribute to the control of behavior according to reinforcement learning even when the prescriptions of such an algorithm are suboptimal in terms of maximizing future rewards.


Item Type:Article
Related URLs:
URLURL TypeDescription
https://doi.org/10.1523/JNEUROSCI.6421-10.2011DOIArticle
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6622656PubMed CentralArticle
ORCID:
AuthorORCID
O'Doherty, John P.0000-0003-0016-3531
Additional Information:© 2011 The Authors. Received Dec. 9, 2010; revised Feb. 7, 2011; accepted March 4, 2011. This work was funded by Science Foundation Ireland Grant 08/IN.1/B1844 to J.P.O. We thank Donal Cahill for his help recruiting participants and collecting data, and Charlotte Prevost for advice on the manuscript. Author contributions: R.K.J. and J.P.O. designed research; R.K.J. performed research; R.K.J. analyzed data; R.K.J. and J.P.O. wrote the paper.
Funders:
Funding AgencyGrant Number
Science Foundation, Ireland08/IN1/B1854
Issue or Number:17
PubMed Central ID:PMC6622656
Record Number:CaltechAUTHORS:20110519-152522041
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20110519-152522041
Official Citation:Ryan K. Jessup and John P. O'Doherty Human Dorsal Striatal Activity during Choice Discriminates Reinforcement Learning Behavior from the Gambler's Fallacy The Journal of Neuroscience, 27 April 2011, 31(17):6296-6304; doi:10.1523/JNEUROSCI.6421-10.2011
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:23740
Collection:CaltechAUTHORS
Deposited By: Ruth Sustaita
Deposited On:19 May 2011 22:55
Last Modified:03 Oct 2019 02:49

Repository Staff Only: item control page