A Caltech Library Service

Active inference and learning

Friston, Karl and FitzGerald, Thomas and Rigoli, Francesco and Schwartenbeck, Philipp and O'Doherty, John P. and Pezzulo, Giovanni (2016) Active inference and learning. Neuroscience and Biobehavioral Reviews, 68 . pp. 862-879. ISSN 0149-7634. PMCID PMC5167251. doi:10.1016/j.neubiorev.2016.06.022.

[img] PDF - Published Version
Creative Commons Attribution.


Use this Persistent URL to link to this item:


This paper offers an active inference account of choice behaviour and learning. It focuses on the distinction between goal-directed and habitual behaviour and how they contextualise each other. We show that habits emerge naturally (and autodidactically) from sequential policy optimisation when agents are equipped with state-action policies. In active inference, behaviour has explorative (epistemic) and exploitative (pragmatic) aspects that are sensitive to ambiguity and risk respectively, where epistemic (ambiguity-resolving) behaviour enables pragmatic (reward-seeking) behaviour and the subsequent emergence of habits. Although goal-directed and habitual policies are usually associated with model-based and model-free schemes, we find the more important distinction is between belief-free and belief-based schemes. The underlying (variational) belief updating provides a comprehensive (if metaphorical) process theory for several phenomena, including the transfer of dopamine responses, reversal learning, habit formation and devaluation. Finally, we show that active inference reduces to a classical (Bellman) scheme, in the absence of ambiguity.

Item Type:Article
Related URLs:
URLURL TypeDescription CentralArticle
O'Doherty, John P.0000-0003-0016-3531
Additional Information:© 2016 The Authors. Published by Elsevier Under a Creative Commons license - Attribution 4.0 International (CC BY 4.0) Received date: 5-3-2016; Revised date: 15-6-2016; Accepted date: 17-6-2016. Available online 29 June 2016. KJF is funded by the Wellcome Trust (Ref: 088130/Z/09/Z). Philipp Schwartenbeck is a recipient of a DOC Fellowship of the Austrian Academy of Sciences at the Centre for Cognitive Neuroscience; University of Salzburg. GP gratefully acknowledges support of HFSP (Young Investigator Grant RGY0088/2014).
Funding AgencyGrant Number
Wellcome Trust088130/Z/09/Z
Österreichische Akademie der WissenschaftenUNSPECIFIED
Human Frontier Science ProgramRGY0088/2014
Subject Keywords:active inference; habit learning; Bayesian inference; goal-directed; free energy; information gain; Bayesian surprise; epistemic value; exploration; exploitation
PubMed Central ID:PMC5167251
Record Number:CaltechAUTHORS:20160715-071522556
Persistent URL:
Official Citation:Karl Friston, Thomas FitzGerald, Francesco Rigoli, Philipp Schwartenbeck, John O’Doherty, Giovanni Pezzulo, Active inference and learning, Neuroscience & Biobehavioral Reviews, Volume 68, September 2016, Pages 862-879, ISSN 0149-7634, (
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:69039
Deposited By: Ruth Sustaita
Deposited On:15 Jul 2016 16:31
Last Modified:11 Nov 2021 04:08

Repository Staff Only: item control page