CaltechAUTHORS
  A Caltech Library Service

Risk, Unexpected Uncertainty, and Estimation Uncertainty: Bayesian Learning in Unstable Settings

Payzan-LeNestour, Elise and Bossaerts, Peter (2011) Risk, Unexpected Uncertainty, and Estimation Uncertainty: Bayesian Learning in Unstable Settings. PLoS Computational Biology, 70 (1). Art. No. e1001048. ISSN 1553-734X. https://resolver.caltech.edu/CaltechAUTHORS:20110301-094909173

[img]
Preview
PDF - Published Version
Creative Commons Attribution.

771Kb
[img]
Preview
PDF (Figure S1) - Supplemental Material
Creative Commons Attribution.

14Kb
[img]
Preview
PDF (Figure S2) - Supplemental Material
Creative Commons Attribution.

18Kb
[img]
Preview
PDF (Figure S3) - Supplemental Material
Creative Commons Attribution.

19Kb
[img]
Preview
PDF (Figure S4) - Supplemental Material
Creative Commons Attribution.

19Kb
[img]
Preview
PDF (Text S1) - Supplemental Material
Creative Commons Attribution.

190Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20110301-094909173

Abstract

Recently, evidence has emerged that humans approach learning using Bayesian updating rather than (model-free) reinforcement algorithms in a six-arm restless bandit problem. Here, we investigate what this implies for human appreciation of uncertainty. In our task, a Bayesian learner distinguishes three equally salient levels of uncertainty. First, the Bayesian perceives irreducible uncertainty or risk: even knowing the payoff probabilities of a given arm, the outcome remains uncertain. Second, there is (parameter) estimation uncertainty or ambiguity: payoff probabilities are unknown and need to be estimated. Third, the outcome probabilities of the arms change: the sudden jumps are referred to as unexpected uncertainty. We document how the three levels of uncertainty evolved during the course of our experiment and how it affected the learning rate. We then zoom in on estimation uncertainty, which has been suggested to be a driving force in exploration, in spite of evidence of widespread aversion to ambiguity. Our data corroborate the latter. We discuss neural evidence that foreshadowed the ability of humans to distinguish between the three levels of uncertainty. Finally, we investigate the boundaries of human capacity to implement Bayesian learning. We repeat the experiment with different instructions, reflecting varying levels of structural uncertainty. Under this fourth notion of uncertainty, choices were no better explained by Bayesian updating than by (model-free) reinforcement learning. Exit questionnaires revealed that participants remained unaware of the presence of unexpected uncertainty and failed to acquire the right model with which to implement Bayesian updating.


Item Type:Article
Related URLs:
URLURL TypeDescription
http://dx.doi.org/10.1371/journal.pcbi.1001048 DOIUNSPECIFIED
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3024253/?report=abstract&tool=pmcentrezPublisherUNSPECIFIED
ORCID:
AuthorORCID
Bossaerts, Peter0000-0003-2308-2603
Additional Information:© 2011 Payzan-LeNestour, Bossaerts. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Received November 2, 2009; Accepted December 2, 2010; Published January 20, 2011. Editor: Tim Behrens, John Radcliffe Hospital, United Kingdom. Funding: The authors acknowledge financial support from the Swiss Finance Institute and from NCCR FINRISK of the Swiss National Science Foundation. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript application. We are grateful to Chen Feng for programming the board game application. Author Contributions: Conceived and designed the experiments: EP-LN. Performed the experiments: EP-LN. Analyzed the data: EP-LN. Wrote the paper: EPLN PB. Supervised EP-LN: PB.
Funders:
Funding AgencyGrant Number
Swiss Finance InstituteUNSPECIFIED
Swiss National Science FoundationUNSPECIFIED
Issue or Number:1
Record Number:CaltechAUTHORS:20110301-094909173
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20110301-094909173
Official Citation:Payzan-LeNestour E, Bossaerts P (2011) Risk, Unexpected Uncertainty, and Estimation Uncertainty: Bayesian Learning in Unstable Settings. PLoS Comput Biol 7(1): e1001048. doi:10.1371/journal.pcbi.1001048
Usage Policy:This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
ID Code:22571
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:03 Mar 2011 17:48
Last Modified:03 Oct 2019 02:38

Repository Staff Only: item control page