A Caltech Library Service

Neural computations underlying inverse reinforcement learning in the human brain

Collette, Sven and Pauli, Wolfgang M. and Bossaerts, Peter and O'Doherty, John (2017) Neural computations underlying inverse reinforcement learning in the human brain. eLife, 6 . Art. No. e29718. ISSN 2050-084X. PMCID PMC5662289.

[img] PDF - Published Version
Creative Commons Attribution.

[img] PDF (Transparent reporting form) - Supplemental Material
Creative Commons Attribution.


Use this Persistent URL to link to this item:


In inverse reinforcement learning an observer infers the reward distribution available for actions in the environment solely through observing the actions implemented by another agent. To address whether this computational process is implemented in the human brain, participants underwent fMRI while learning about slot machines yielding hidden preferred and non-preferred food outcomes with varying probabilities, through observing the repeated slot choices of agents with similar and dissimilar food preferences. Using formal model comparison, we found that participants implemented inverse RL as opposed to a simple imitation strategy, in which the actions of the other agent are copied instead of inferring the underlying reward structure of the decision problem. Our computational fMRI analysis revealed that anterior dorsomedial prefrontal cortex encoded inferences about action-values within the value space of the agent as opposed to that of the observer, demonstrating that inverse RL is an abstract cognitive process divorceable from the values and concerns of the observer him/herself.

Item Type:Article
Related URLs:
URLURL TypeDescription CentralArticle
Collette, Sven0000-0002-0234-1867
Bossaerts, Peter0000-0003-2308-2603
Additional Information:© 2017 Copyright Collette et al. This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited. Received: 19 June 2017; Accepted: 11 October 2017; Published: 30 October 2017. Data availability: The full anonymized dataset from this study is available in the NDAR data repository under the collection ID 2417. Summary information on the data (e.g. additional details about the experiment such as picture files or exact timings of stimuli) is available on the NDA home-page without the need for an NDA account. To request access to detailed human subjects data, you must be sponsored by an NIH recognized institution with a Federalwide Assurance and have a research related need to access NDA data. Further information as to how to request access can be found here The fMRI activation maps are available at neurovault ( This work was supported by the NIMH Caltech Conte Center for the Neurobiology of Social Decision Making (JPO). We thank Tim Armstrong and Lynn K Paul for support with the participant recruitment, and Ralph E Lee and Julian M Tyszka for assistance with the experiments. The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication. The authors declare that no competing interests exist. Author contributions: Sven Collette, Conceptualization, Data curation, Software, Formal analysis, Validation, Investigation, Visualization, Methodology, Writing—original draft, Writing—review and editing; Wolfgang M Pauli, Investigation, Writing—review and editing; Peter Bossaerts, Resources, Software, Methodology, Writing—original draft, Writing—review and editing; John O’Doherty, Conceptualization, Supervision, Funding acquisition, Methodology, Writing—original draft, Writing—review and editing.
Funding AgencyGrant Number
PubMed Central ID:PMC5662289
Record Number:CaltechAUTHORS:20171108-131218516
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:83067
Deposited By: Tony Diaz
Deposited On:08 Nov 2017 21:22
Last Modified:03 Oct 2019 19:01

Repository Staff Only: item control page