CaltechAUTHORS
  A Caltech Library Service

Active inference and learning

Friston, Karl and FitzGerald, Thomas and Rigoli, Francesco and Schwartenbeck, Philipp and O’Doherty, John and Pezzulo, Giovanni (2016) Active inference and learning. Neuroscience and Biobehavioral Reviews, 68 . pp. 862-879. ISSN 0149-7634. http://resolver.caltech.edu/CaltechAUTHORS:20160715-071522556

[img] PDF - Published Version
Creative Commons Attribution Non-commercial No Derivatives.

5Mb

Use this Persistent URL to link to this item: http://resolver.caltech.edu/CaltechAUTHORS:20160715-071522556

Abstract

This paper offers an active inference account of choice behaviour and learning. It focuses on the distinction between goal-directed and habitual behaviour and how they contextualise each other. We show that habits emerge naturally (and autodidactically) from sequential policy optimisation when agents are equipped with state-action policies. In active inference, behaviour has explorative (epistemic) and exploitative (pragmatic) aspects that are sensitive to ambiguity and risk respectively, where epistemic (ambiguity-resolving) behaviour enables pragmatic (reward-seeking) behaviour and the subsequent emergence of habits. Although goal-directed and habitual policies are usually associated with model-based and model-free schemes, we find the more important distinction is between belief-free and belief-based schemes. The underlying (variational) belief updating provides a comprehensive (if metaphorical) process theory for several phenomena, including the transfer of dopamine responses, reversal learning, habit formation and devaluation. Finally, we show that active inference reduces to a classical (Bellman) scheme, in the absence of ambiguity.


Item Type:Article
Related URLs:
URLURL TypeDescription
http://dx.doi.org/10.1016/j.neubiorev.2016.06.022DOIArticle
http://www.sciencedirect.com/science/article/pii/S0149763416301336PublisherArticle
Additional Information:© 2016 The Authors. Under a Creative Commons license. Published by Elsevier Ltd. Received date: 5-3-2016; Revised date: 15-6-2016; Accepted date: 17-6-2016. Available online 29 June 2016. KJF is funded by the Wellcome Trust (Ref: 088130/Z/09/Z). Philipp Schwartenbeck is a recipient of a DOC Fellowship of the Austrian Academy of Sciences at the Centre for Cognitive Neuroscience; University of Salzburg. GP gratefully acknowledges support of HFSP (Young Investigator Grant RGY0088/2014).
Funders:
Funding AgencyGrant Number
Wellcome Trust088130/Z/09/Z
Österreichische Akademie der WissenschaftenUNSPECIFIED
Human Frontier Science ProgramRGY0088/2014
Subject Keywords:active inference; habit learning; Bayesian inference; goal-directed; free energy; information gain; Bayesian surprise; epistemic value; exploration; exploitation
Record Number:CaltechAUTHORS:20160715-071522556
Persistent URL:http://resolver.caltech.edu/CaltechAUTHORS:20160715-071522556
Official Citation:Karl Friston, Thomas FitzGerald, Francesco Rigoli, Philipp Schwartenbeck, John O’Doherty, Giovanni Pezzulo, Active inference and learning, Neuroscience & Biobehavioral Reviews, Volume 68, September 2016, Pages 862-879, ISSN 0149-7634, http://dx.doi.org/10.1016/j.neubiorev.2016.06.022. (http://www.sciencedirect.com/science/article/pii/S0149763416301336)
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:69039
Collection:CaltechAUTHORS
Deposited By: Ruth Sustaita
Deposited On:15 Jul 2016 16:31
Last Modified:11 Oct 2016 19:09

Repository Staff Only: item control page