CaltechAUTHORS
  A Caltech Library Service

Deviation from the matching law reflects an optimal strategy involving learning over multiple timescales

Iigaya, Kiyohito and Ahmadian, Yashar and Sugrue, Leo P. and Corrado, Greg S. and Loewenstein, Yonatan and Newsome, William T. and Fusi, Stefano (2019) Deviation from the matching law reflects an optimal strategy involving learning over multiple timescales. Nature Communications, 10 . Art. No. 1466. ISSN 2041-1723. PMCID PMC6443814. https://resolver.caltech.edu/CaltechAUTHORS:20190405-084758735

[img] PDF - Published Version
Creative Commons Attribution.

1684Kb
[img] PDF - Supplemental Material
Creative Commons Attribution.

362Kb
[img] PDF (Peer Review File) - Supplemental Material
Creative Commons Attribution.

526Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20190405-084758735

Abstract

Behavior deviating from our normative expectations often appears irrational. For example, even though behavior following the so-called matching law can maximize reward in a stationary foraging task, actual behavior commonly deviates from matching. Such behavioral deviations are interpreted as a failure of the subject; however, here we instead suggest that they reflect an adaptive strategy, suitable for uncertain, non-stationary environments. To prove it, we analyzed the behavior of primates that perform a dynamic foraging task. In such nonstationary environment, learning on both fast and slow timescales is beneficial: fast learning allows the animal to react to sudden changes, at the price of large fluctuations (variance) in the estimates of task relevant variables. Slow learning reduces the fluctuations but costs a bias that causes systematic behavioral deviations. Our behavioral analysis shows that the animals solved this bias-variance tradeoff by combining learning on both fast and slow timescales, suggesting that learning on multiple timescales can be a biologically plausible mechanism for optimizing decisions under uncertainty.


Item Type:Article
Related URLs:
URLURL TypeDescription
https://doi.org/10.1038%2Fs41467-019-09388-3DOIArticle
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6443814PubMed CentralArticle
ORCID:
AuthorORCID
Iigaya, Kiyohito0000-0002-4748-8432
Sugrue, Leo P.0000-0001-7315-4519
Fusi, Stefano0000-0002-3035-6652
Additional Information:© 2019 The Author(s). This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/. Received 03 November 2017; Accepted 08 March 2019; Published 01 April 2019. Data availability: The data and code that support the findings of this study are available from the corresponding author upon reasonable request. We thank P. Dayan, L.F. Abbott, K.D. Miller, C. R. Gallistel, and K. Lloyd for fruitful discussions. Supported by NSF’s NeuroNex program award DBI-1707398, the Gatsby Charitable Foundation, the Simons Foundation, the Schwartz foundation, the Kavli foundation, the Japan Society for the Promotion of Science, the Israel Science Foundation (Grant No. 757/16), the National Eye Institute, and the Howard Hughes Medical Institute. Author Contributions: K.I., Y.A., Y.L., W.T.N., and S.F. conceived the current project. L.P.S., G.S.C., and W.T.N. have designed and run the original macaque experiment. K.I., Y.A., Y.L. and S.F. developed the theoretical models and analyzed the data, with inputs from L.P.S., G.S.C. and W.T.N.; All authors participated in writing the manuscript. The authors declare no competing interests.
Funders:
Funding AgencyGrant Number
NSFDBI-1707398
Gatsby Charitable FoundationUNSPECIFIED
Simons FoundationUNSPECIFIED
Schwartz FoundationUNSPECIFIED
Kavli FoundationUNSPECIFIED
Japan Society for the Promotion of Science (JSPS)UNSPECIFIED
Israel Science Foundation757/16
National Eye InstituteUNSPECIFIED
Howard Hughes Medical Institute (HHMI)UNSPECIFIED
NIHUNSPECIFIED
PubMed Central ID:PMC6443814
Record Number:CaltechAUTHORS:20190405-084758735
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20190405-084758735
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:94492
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:05 Apr 2019 17:02
Last Modified:03 Oct 2019 21:04

Repository Staff Only: item control page