CaltechAUTHORS
  A Caltech Library Service

Changepoint detection versus reinforcement learning: Separable neural substrates approximate different forms of Bayesian inference

Pauli, Wolfgang M. and Jones, Matt (2019) Changepoint detection versus reinforcement learning: Separable neural substrates approximate different forms of Bayesian inference. . (Unpublished) http://resolver.caltech.edu/CaltechAUTHORS:20190328-145236200

[img] PDF - Submitted Version
Creative Commons Attribution Non-commercial.

845Kb

Use this Persistent URL to link to this item: http://resolver.caltech.edu/CaltechAUTHORS:20190328-145236200

Abstract

Adaptive behavior in even the simplest decision-making tasks requires predicting future events in an environment that is generally nonstationary. As an inductive problem, this prediction requires a commitment to the statistical process underlying environmental change. This challenge can be formalized in a Bayesian framework as a question of choosing a generative model for the task dynamics. Previous learning models assume, implicitly or explicitly, that nonstationarity follows either a continuous diffusion process or a discrete changepoint process. Each approach is slow to adapt when its assumptions are violated. A new mixture of Bayesian experts framework proposes separable brain systems approximating inference under different assumptions regarding the statistical structure of the environment. This model explains data from a laboratory foraging task, in which rats experienced a change in reward contingencies after pharmacological disruption of dorsolateral (DLS) or dorsomedial striatum (DMS). The data and model suggest DLS learns under a diffusion prior whereas DMS learns under a changepoint prior. The combination of these two systems offers a new explanation for how the brain handles inference in an uncertain environment.


Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription
https://doi.org/10.1101/591818DOIDiscussion Paper
ORCID:
AuthorORCID
Pauli, Wolfgang M.0000-0002-0966-0254
Additional Information:The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC 4.0 International license. This research was supported by Air Force Office of Scientific Research grant FA9550-10-1-0177. Thanks for helpful discussions of this work go to the members of the computational cognitive neuroscience lab at the university of Colorado at Boulder, and members of the decision neuroscience lab at the California Institute of Technology.
Funders:
Funding AgencyGrant Number
Air Force Office of Scientific Research (AFOSR)FA9550-10-1-0177
Record Number:CaltechAUTHORS:20190328-145236200
Persistent URL:http://resolver.caltech.edu/CaltechAUTHORS:20190328-145236200
Official Citation:Changepoint detection versus reinforcement learning: Separable neural substrates approximate different forms of Bayesian inference Wolfgang M Pauli, Matt Jones bioRxiv 591818; doi: https://doi.org/10.1101/591818
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:94259
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:28 Mar 2019 22:25
Last Modified:28 Mar 2019 22:25

Repository Staff Only: item control page