CaltechAUTHORS
  A Caltech Library Service

Cortical substrates for exploratory decisions in humans

Daw, Nathaniel D. and O'Doherty, John P. and Dayan, Peter and Seymour, Ben and Dolan, Raymond J. (2006) Cortical substrates for exploratory decisions in humans. Nature, 441 (7095). pp. 876-879. ISSN 0028-0836. PMCID PMC2635947. https://resolver.caltech.edu/CaltechAUTHORS:20181024-100030520

[img] PDF - Accepted Version
See Usage Policy.

919kB
[img] PDF (This file contains Supplementary Methods, Supplementary Discussion and Supplementary Tables 1–5) - Supplemental Material
See Usage Policy.

380kB

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20181024-100030520

Abstract

Decision making in an uncertain environment poses a conflict between the opposing demands of gathering and exploiting information. In a classic illustration of this ‘exploration–exploitation’ dilemma, a gambler choosing between multiple slot machines balances the desire to select what seems, on the basis of accumulated experience, the richest option, against the desire to choose a less familiar option that might turn out more advantageous (and thereby provide information for improving future decisions). Far from representing idle curiosity, such exploration is often critical for organisms to discover how best to harvest resources such as food and water. In appetitive choice, substantial experimental evidence, underpinned by computational reinforcement learning (RL) theory, indicates that a dopaminergic, striatal and medial prefrontal network mediates learning to exploit. In contrast, although exploration has been well studied from both theoretical and ethological perspectives, its neural substrates are much less clear. Here we show, in a gambling task, that human subjects' choices can be characterized by a computationally well-regarded strategy for addressing the explore/exploit dilemma. Furthermore, using this characterization to classify decisions as exploratory or exploitative, we employ functional magnetic resonance imaging to show that the frontopolar cortex and intraparietal sulcus are preferentially active during exploratory decisions. In contrast, regions of striatum and ventromedial prefrontal cortex exhibit activity characteristic of an involvement in value-based exploitative decision making. The results suggest a model of action selection under uncertainty that involves switching between exploratory and exploitative behavioural modes, and provide a computationally precise characterization of the contribution of key decision-related brain systems to each of these functions.


Item Type:Article
Related URLs:
URLURL TypeDescription
https://doi.org/10.1038/nature04766DOIArticle
https://rdcu.be/9ZmuPublisherFree ReadCube access
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2635947PubMed CentralArticle
ORCID:
AuthorORCID
O'Doherty, John P.0000-0003-0016-3531
Dayan, Peter0000-0003-3476-1839
Dolan, Raymond J.0000-0001-9356-761X
Additional Information:© 2006 Nature Publishing Group. Received 07 February 2006; Accepted 30 March 2006; Published 15 June 2006. We thank J. Li, S. McClure, B. King-Casas and P. R. Montague for sharing their unpublished data on exploration, and Y. Niv, Z. Gharamani and C. Camerer for discussions. Funding was from a Royal Society USA Research Fellowship (N.D.), the Gatsby Foundation (N.D., P.D.), the EU BIBA project (N.D., P.D.), and a Wellcome Trust Programme Grant (J.O.D., R.D.). Nathaniel D. Daw & John P. O'Doherty: These authors contributed equally to this work. The authors declare no competing financial interests.
Funders:
Funding AgencyGrant Number
Royal SocietyUNSPECIFIED
Gatsby FoundationUNSPECIFIED
European UnionUNSPECIFIED
Wellcome TrustUNSPECIFIED
Issue or Number:7095
PubMed Central ID:PMC2635947
Record Number:CaltechAUTHORS:20181024-100030520
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20181024-100030520
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:90391
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:24 Oct 2018 18:28
Last Modified:25 Nov 2020 00:52

Repository Staff Only: item control page