CaltechAUTHORS
  A Caltech Library Service

Adding Prediction Risk to the Theory of Reward Learning

Preuschoff, Kerstin and Bossaerts, Peter (2007) Adding Prediction Risk to the Theory of Reward Learning. In: Reward and decision making in corticobasal ganglia networks. Annals of the New York Academy of Sciences. No.1104. New York Academy of Sciences , Boston, MA, pp. 135-146. ISBN 978-1-57331-674-3 . https://resolver.caltech.edu/CaltechAUTHORS:20101014-101918850

[img] PDF - Published Version
Restricted to Repository administrators only
See Usage Policy.

138Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20101014-101918850

Abstract

This article analyzesthe simple Rescorla–Wagner learning rule from the vantage point of least squares learning theory. In particular, it suggests how measures of risk, such as prediction risk, can be used to adjust the learning constant in reinforcement learning. It argues that prediction risk is most effectively incorporated by scaling the prediction errors. This way, the learning rate needs adjusting only when the covariance between optimal predictions and past (scaled) prediction errors changes. Evidence is discussed that suggests that the dopaminergic system in the (human and nonhuman) primate brain encodes prediction risk, and that prediction errors are indeed scaled with prediction risk (adaptive encoding).


Item Type:Book Section
Related URLs:
URLURL TypeDescription
http://dx.doi.org/10.1196/annals.1390.005DOIUNSPECIFIED
http://onlinelibrary.wiley.com/doi/10.1196/annals.1390.005/abstract;jsessionid=B5F79DD7B5E6013176560B56E5BFF912.d02t02PublisherUNSPECIFIED
ORCID:
AuthorORCID
Bossaerts, Peter0000-0003-2308-2603
Additional Information:© 2007 New York Academy of Sciences. Article first published online: 20 Jun. 2007. We thank John O’Doherty for detailed comments on an earlier draft, and Tim Behrens, Mark Walton, and Matthew Rushworth for further discussions on the link between uncertainty and the learning rate. Peter Bossaerts thanks the Swiss Finance Institute for financial support during his stay at the Université de Lausanne, where this article was written.
Funders:
Funding AgencyGrant Number
Swiss Finance InstituteUNSPECIFIED
Subject Keywords:reinforcement learning; learning rate; least squares learning; dopaminergic system; reward anticipation; prediction risk; uncertainty; adaptive encoding
Series Name:Annals of the New York Academy of Sciences
Issue or Number:1104
Record Number:CaltechAUTHORS:20101014-101918850
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20101014-101918850
Official Citation:PREUSCHOFF, K. and BOSSAERTS, P. (2007), Adding Prediction Risk to the Theory of Reward Learning. Annals of the New York Academy of Sciences, 1104: 135–146. doi: 10.1196/annals.1390.005
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:20430
Collection:CaltechAUTHORS
Deposited By: Jason Perez
Deposited On:14 Oct 2010 18:33
Last Modified:03 Oct 2019 02:10

Repository Staff Only: item control page