Friston, Karl and FitzGerald, Thomas and Rigoli, Francesco and Schwartenbeck, Philipp and O'Doherty, John P. and Pezzulo, Giovanni (2016) Active inference and learning. Neuroscience and Biobehavioral Reviews, 68 . pp. 862-879. ISSN 0149-7634. PMCID PMC5167251. doi:10.1016/j.neubiorev.2016.06.022. https://resolver.caltech.edu/CaltechAUTHORS:20160715-071522556
![]() |
PDF
- Published Version
Creative Commons Attribution. 5MB |
Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20160715-071522556
Abstract
This paper offers an active inference account of choice behaviour and learning. It focuses on the distinction between goal-directed and habitual behaviour and how they contextualise each other. We show that habits emerge naturally (and autodidactically) from sequential policy optimisation when agents are equipped with state-action policies. In active inference, behaviour has explorative (epistemic) and exploitative (pragmatic) aspects that are sensitive to ambiguity and risk respectively, where epistemic (ambiguity-resolving) behaviour enables pragmatic (reward-seeking) behaviour and the subsequent emergence of habits. Although goal-directed and habitual policies are usually associated with model-based and model-free schemes, we find the more important distinction is between belief-free and belief-based schemes. The underlying (variational) belief updating provides a comprehensive (if metaphorical) process theory for several phenomena, including the transfer of dopamine responses, reversal learning, habit formation and devaluation. Finally, we show that active inference reduces to a classical (Bellman) scheme, in the absence of ambiguity.
Item Type: | Article | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
Related URLs: |
| |||||||||
ORCID: |
| |||||||||
Additional Information: | © 2016 The Authors. Published by Elsevier Under a Creative Commons license - Attribution 4.0 International (CC BY 4.0) Received date: 5-3-2016; Revised date: 15-6-2016; Accepted date: 17-6-2016. Available online 29 June 2016. KJF is funded by the Wellcome Trust (Ref: 088130/Z/09/Z). Philipp Schwartenbeck is a recipient of a DOC Fellowship of the Austrian Academy of Sciences at the Centre for Cognitive Neuroscience; University of Salzburg. GP gratefully acknowledges support of HFSP (Young Investigator Grant RGY0088/2014). | |||||||||
Funders: |
| |||||||||
Subject Keywords: | active inference; habit learning; Bayesian inference; goal-directed; free energy; information gain; Bayesian surprise; epistemic value; exploration; exploitation | |||||||||
PubMed Central ID: | PMC5167251 | |||||||||
DOI: | 10.1016/j.neubiorev.2016.06.022 | |||||||||
Record Number: | CaltechAUTHORS:20160715-071522556 | |||||||||
Persistent URL: | https://resolver.caltech.edu/CaltechAUTHORS:20160715-071522556 | |||||||||
Official Citation: | Karl Friston, Thomas FitzGerald, Francesco Rigoli, Philipp Schwartenbeck, John O’Doherty, Giovanni Pezzulo, Active inference and learning, Neuroscience & Biobehavioral Reviews, Volume 68, September 2016, Pages 862-879, ISSN 0149-7634, http://dx.doi.org/10.1016/j.neubiorev.2016.06.022. (http://www.sciencedirect.com/science/article/pii/S0149763416301336) | |||||||||
Usage Policy: | No commercial reproduction, distribution, display or performance rights in this work are provided. | |||||||||
ID Code: | 69039 | |||||||||
Collection: | CaltechAUTHORS | |||||||||
Deposited By: | Ruth Sustaita | |||||||||
Deposited On: | 15 Jul 2016 16:31 | |||||||||
Last Modified: | 11 Nov 2021 04:08 |
Repository Staff Only: item control page