Pennartz, Cyriel M. A. (1995) The ascending neuromodulatory systems in learning by reinforcement: comparing computational conjectures with experimental findings. Brain Research Reviews, 21 (3). pp. 219-245. ISSN 0165-0173. doi:10.1016/0165-0173(95)00014-3. https://resolver.caltech.edu/CaltechAUTHORS:20170408-162304380
Full text is not posted in this repository. Consult Related URLs below.
Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20170408-162304380
Abstract
A central problem in cognitive neuroscience is how animals can manage to rapidly master complex sensorimotor tasks when the only sensory feedback they use to improve their performance is a simple reinforcing stimulus. Neural network theorists have constructed algorithms for reinforcement learning that can be used to solve a variety of biological problems and do not violate basic neurophysiological principles, in contrast to the back-propagation algorithm. A key assumption in these models is the existence of a reinforcement signal, which would be diffusively broadcast throughout one or several brain areas engaged in learning. This signal is further assumed to mediate up- and downward changes in synaptic efficacy by acting as a multiplicative factor in learning rules. The biological plausibility of these algorithms has been defended by the conjecture that the neuromodulators noradrenaline, acetylcholine or dopamine may form the neurochemical substrate of reinforcement signals. In this commentary, the predictions raised by this hypothesis are compared to anatomical, electrophysiological and behavioural findings. The experimental evidence does not support, and often argues against, a general reinforcement-encoding function of these neuromodulatory systems. Nevertheless, the broader concept of evaluative signalling between brain structures implied in learning appears to be reasonable and the available algorithms may open new avenues for constructing more realistic network architectures.
Item Type: | Article | ||||||
---|---|---|---|---|---|---|---|
Related URLs: |
| ||||||
Additional Information: | © 1996 Elsevier. Accepted 27 November 1995. I would like to thank J.J. Hopfield for his stimulating advice during the course of this research project and F.H. Lopes da Silva, J. van Pelt, A. van Ooijen for their critical reading of the manuscript. This project was supported by a Talent Fellowship of the Netherlands Organization for Scientific Research. | ||||||
Funders: |
| ||||||
Subject Keywords: | Acetylcholine; Dopamine; Long-term potentiation; Memory; Noradrenaline; Supervised learning; Synaptic plasticity; Temporal difference learning | ||||||
Issue or Number: | 3 | ||||||
DOI: | 10.1016/0165-0173(95)00014-3 | ||||||
Record Number: | CaltechAUTHORS:20170408-162304380 | ||||||
Persistent URL: | https://resolver.caltech.edu/CaltechAUTHORS:20170408-162304380 | ||||||
Official Citation: | Cyriel M.A Pennartz, The ascending neuromodulatory systems in learning by reinforcement: comparing computational conjectures with experimental findings, Brain Research Reviews, Volume 21, Issue 3, 1995, Pages 219-245, ISSN 0165-0173, http://dx.doi.org/10.1016/0165-0173(95)00014-3. (http://www.sciencedirect.com/science/article/pii/0165017395000143) | ||||||
Usage Policy: | No commercial reproduction, distribution, display or performance rights in this work are provided. | ||||||
ID Code: | 76174 | ||||||
Collection: | CaltechAUTHORS | ||||||
Deposited By: | 1Science Import | ||||||
Deposited On: | 26 Jul 2017 23:32 | ||||||
Last Modified: | 15 Nov 2021 16:58 |
Repository Staff Only: item control page