A Caltech Library Service

Explaining face representation in the primate brain using different computational models

Chang, Le and Egger, Bernhard and Vetter, Thomas and Tsao, Doris Y. (2021) Explaining face representation in the primate brain using different computational models. Current Biology, 31 (13). pp. 2785-2795. ISSN 0960-9822. PMCID PMC8566016. doi:10.1016/j.cub.2021.04.014.

[img] PDF - Published Version
Creative Commons Attribution Non-commercial No Derivatives.

[img] PDF - Accepted Version
See Usage Policy.

[img] PDF (March 30, 2021) - Submitted Version
See Usage Policy.

[img] PDF (Figures S1–S5) - Supplemental Material
Creative Commons Attribution Non-commercial No Derivatives.


Use this Persistent URL to link to this item:


Understanding how the brain represents the identity of complex objects is a central challenge of visual neuroscience. The principles governing object processing have been extensively studied in the macaque face patch system, a sub-network of inferotemporal (IT) cortex specialized for face processing. A previous study reported that single face patch neurons encode axes of a generative model called the “active appearance” model, which transforms 50D feature vectors separately representing facial shape and facial texture into facial images. However, a systematic investigation comparing this model to other computational models, especially convolutional neural network models that have shown success in explaining neural responses in the ventral visual stream, has been lacking. Here, we recorded responses of cells in the most anterior face patch anterior medial (AM) to a large set of real face images and compared a large number of models for explaining neural responses. We found that the active appearance model better explained responses than any other model except CORnet-Z, a feedforward deep neural network trained on general object classification to classify non-face images, whose performance it tied on some face image sets and exceeded on others. Surprisingly, deep neural networks trained specifically on facial identification did not explain neural responses well. A major reason is that units in the network, unlike neurons, are less modulated by face-related factors unrelated to facial identification, such as illumination.

Item Type:Article
Related URLs:
URLURL TypeDescription CentralArticle Paper
Egger, Bernhard0000-0002-4736-2397
Tsao, Doris Y.0000-0003-1083-1919
Alternate Title:What computational model provides the best explanation of face representations in the primate brain?
Additional Information:© 2021 The Author(s). Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license ( Received 20 May 2020, Revised 22 March 2021, Accepted 8 April 2021, Available online 4 May 2021. This work was supported by NIH (EY030650-01), the Howard Hughes Medical Institute, and the Chen Center for Systems Neuroscience at Caltech. We are grateful to Nicole Schweers for help with animal training and MingPo Yang for help with implementing the CORnets. Author contributions: L.C. and D.Y.T. conceived the project and wrote the paper with the help of all other authors, L.C. performed the experiments and analyzed the data, D.Y.T. supervised the project, and B.E. and T.V. constructed the 3D morphable model used to compare with the neural data. The authors declare no competing interests.
Group:Tianqiao and Chrissy Chen Institute for Neuroscience
Funding AgencyGrant Number
Howard Hughes Medical Institute (HHMI)UNSPECIFIED
Tianqiao and Chrissy Chen Institute for NeuroscienceUNSPECIFIED
Subject Keywords:inferotemporal cortex; primate vision; face processing; neural coding; electrophysiology; computational model
Issue or Number:13
PubMed Central ID:PMC8566016
Record Number:CaltechAUTHORS:20200610-100834335
Persistent URL:
Official Citation:Le Chang, Bernhard Egger, Thomas Vetter, Doris Y. Tsao, Explaining face representation in the primate brain using different computational models, Current Biology, Volume 31, Issue 13, 2021, Pages 2785-2795.e4, ISSN 0960-9822,
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:103817
Deposited By: Tony Diaz
Deposited On:10 Jun 2020 17:47
Last Modified:22 Jul 2022 17:39

Repository Staff Only: item control page