The involvement of model-based but not model-free learning signals during observational reward learning in the absence of choice
A major open question is whether computational strategies thought to be used during experiential learning, specifically model-based and model-free reinforcement learning, also support observational learning. Furthermore, the question of how observational learning occurs when observers must learn about the value of options from observing outcomes in the absence of choice has not been addressed. In the present study we used a multi-armed bandit task that encouraged human participants to employ both experiential and observational learning while they underwent functional magnetic resonance imaging (fMRI). We found evidence for the presence of model-based learning signals during both observational and experiential learning in the intraparietal sulcus. However, unlike during experiential learning, model-free learning signals in the ventral striatum were not detectable during this form of observational learning. These results provide insight into the flexibility of the model-based learning system, implicating this system in learning during observation as well as from direct experience, and further suggest that the model-free reinforcement learning system may be less flexible with regard to its involvement in observational learning.
Additional Information© 2016 the American Physiological Society. Submitted 14 January 2016; accepted in final form 1 April 2016. We thank Sojo Joseph and Bridget Finnegan for assistance during data collection. This work was supported by grants from the Wellcome Trust (to J. P. O'Doherty) and the NIMH Caltech Conte Center for the Neurobiology of Social Decision Making (to J.P. O'Doherty). Author contributions: J.P.O. conceived and designed research; S.D. and A.D. collected data; S.D. analyzed data, prepared figures and drafted manuscript; S.D. and J.P.O. edited and revised manuscript; S.D, A.D. and J.P.O. approved final version of manuscript.
Accepted Version - emss-68486.pdf