A Caltech Library Service

Computational approaches to habits in a model-free world

Pauli, Wolfgang M. and Cockburn, Jeffrey and Pool, Eva R. and Pérez, Omar D. and O'Doherty, John P. (2018) Computational approaches to habits in a model-free world. Current Opinion in Behavioral Sciences, 20 . pp. 104-109. ISSN 2352-1546. doi:10.1016/j.cobeha.2017.12.001.

Full text is not posted in this repository. Consult Related URLs below.

Use this Persistent URL to link to this item:


Model-free (MF) reinforcement learning (RL) algorithms account for a wealth of neuroscientific and behavioral data pertinent to habits; however, conspicuous disparities between model-predicted response patterns and experimental data have exposed the inadequacy of MF-RL to fully capture the domain of habitual behavior. We review several extensions to generic MF-RL algorithms that could narrow the gap between theory and empirical data. We discuss insights gained from extending RL algorithms to operate in complex environments with multidimensional continuous state spaces. We also review recent advances in hierarchical RL and their potential relevance to habits. Neurobiological evidence suggests that similar mechanisms for habitual learning and control may apply across diverse psychological domains.

Item Type:Article
Related URLs:
URLURL TypeDescription
Pauli, Wolfgang M.0000-0002-0966-0254
Cockburn, Jeffrey0000-0003-2004-8653
Pool, Eva R.0000-0001-5929-1007
Pérez, Omar D.0000-0002-4168-5435
O'Doherty, John P.0000-0003-0016-3531
Additional Information:© 2017 Elsevier Ltd. Available online 20 December 2017. This work was supported by NIDA-NIH R01 grant (1R01DA040011-01A1). The authors would like to thank all members of the O’Doherty Human Reward and Decision Making laboratory for intriguing discussions.
Funding AgencyGrant Number
Record Number:CaltechAUTHORS:20180622-083942345
Persistent URL:
Official Citation:Wolfgang M Pauli, Jeffrey Cockburn, Eva R Pool, Omar D Pérez, John P O’Doherty, Computational approaches to habits in a model-free world, Current Opinion in Behavioral Sciences, Volume 20, 2018, Pages 104-109, ISSN 2352-1546, (
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:87310
Deposited By: Tony Diaz
Deposited On:23 Jun 2018 15:43
Last Modified:13 Apr 2023 17:43

Repository Staff Only: item control page