CaltechAUTHORS
  A Caltech Library Service

Neural signature of fictive learning signals in a sequential investment task

Lohrenz, Terry and McCabe, Kevin and Camerer, Colin F. and Montague, P. Read (2007) Neural signature of fictive learning signals in a sequential investment task. Proceedings of the National Academy of Sciences of the United States of America, 104 (22). pp. 9493-9498. ISSN 0027-8424. PMCID PMC1876162. https://resolver.caltech.edu/CaltechAUTHORS:LOHpnas07

[img]
Preview
PDF - Published Version
See Usage Policy.

969Kb
[img] PDF (Supplementary Figure 7) - Supplemental Material
See Usage Policy.

673Kb
[img]
Preview
PDF (Supplementary Figure 8) - Supplemental Material
See Usage Policy.

53Kb
[img]
Preview
PDF (Supplementary Figure 9) - Supplemental Material
See Usage Policy.

27Kb
[img]
Preview
PDF (Data Set) - Supplemental Material
See Usage Policy.

174Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:LOHpnas07

Abstract

Reinforcement learning models now provide principled guides for a wide range of reward learning experiments in animals and humans. One key learning (error) signal in these models is experiential and reports ongoing temporal differences between expected and experienced reward. However, these same abstract learning models also accommodate the existence of another class of learning signal that takes the form of a fictive error encoding ongoing differences between experienced returns and returns that "could-have-been-experienced" if decisions had been different. These observations suggest the hypothesis that, for all real-world learning tasks, one should expect the presence of both experiential and fictive learning signals. Motivated by this possibility, we used a sequential investment game and fMRI to probe ongoing brain responses to both experiential and fictive learning signals generated throughout the game. Using a large cohort of subjects (n = 54), we report that fictive learning signals strongly predict changes in subjects' investment behavior and correlate with fMRI signals measured in dopaminoceptive structures known to be involved in valuation and choice.


Item Type:Article
Related URLs:
URLURL TypeDescription
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1876162/PubMed CentralArticle
ORCID:
AuthorORCID
Camerer, Colin F.0000-0003-4049-1871
Additional Information:© 2007 by The National Academy of Sciences of the USA Edited by Dale Purves, Duke University Medical Center, Durham, NC, and approved April 13, 2007 (received for review October 6, 2006) This article is a PNAS Direct Submission. Freely available online through the PNAS open access option. We thank B. King-Casas, P. Chiu, and X. Cui for comments on this manuscript; D. Eagleman for stimulating discussions; the Hyperscan Development Team at Baylor College of Medicine for software implementation [NEMO (www.hnl.bcm.tmc.edu/nemo)]; N. Apple, K. Pfeiffer, J. McGee, C. Bracero, X. Cui [xjView (http://people.hnl.bcm.tmc.edu/cuixu/xjView)], and P. Baldwin for technical assistance; and the three anonymous referees for their comments. This work was supported by National Institute on Drug Abuse Grant DA11723 (to P.R.M.), National Institute of Neurological Disorders and Stroke Grant NS045790 (to P.R.M.), and the Kane Family Foundation (P.R.M.). P.R.M. was also supported by the Institute for Advanced Study (Princeton, NJ) for part of the work contained in this article. Author contributions: T.L., K.M., and P.R.M. designed research; T.L. performed research; T.L. and P.R.M. analyzed data; and T.L., K.M., C.F.C., and P.R.M. wrote the paper. Conflict of interest statement: T.L. is Executive Vice President and Director of Research for Computational Management, Inc. This article contains supporting information online at www.pnas.org/cgi/content/full/0608842104/DC1.
Funders:
Funding AgencyGrant Number
NIHDA11723
NIHNS045790
Kane Family FoundationUNSPECIFIED
Institute for Advanced StudyUNSPECIFIED
Subject Keywords:counterfactual signals; decision-making; neuroeconomics; reinforcement learning
Issue or Number:22
PubMed Central ID:PMC1876162
Record Number:CaltechAUTHORS:LOHpnas07
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:LOHpnas07
Alternative URL:http://dx.doi.org/10.1073/pnas.0608842104
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:8292
Collection:CaltechAUTHORS
Deposited By: Archive Administrator
Deposited On:02 Aug 2007
Last Modified:02 Oct 2019 23:49

Repository Staff Only: item control page