A Caltech Library Service

Efficient prediction of trait judgments from faces using deep neural networks

Keleş, Ümit and Lin, Chujun and Adolphs, Ralph (2021) Efficient prediction of trait judgments from faces using deep neural networks. . (Unpublished)

[img] PDF - Submitted Version
Creative Commons Attribution.


Use this Persistent URL to link to this item:


Judgments of people from their faces are often invalid but influence many social decisions (e.g., legal sentencing), making them an important target for automated prediction. Direct training of deep convolutional neural networks (DCNNs) is difficult because of sparse human ratings, but features obtained from DCNNs pre-trained on other classifications (e.g., object recognition) can predict trait judgments within a given face database. However, it remains unknown if this latter approach generalizes across faces, raters, or traits. Here we directly compare three distinct types of face features, and test them across multiple out-of-sample datasets and traits. DCNNs pre-trained on face identification provided features that generalized the best, and models trained to predict a given trait also predicted several other traits. We demonstrate the flexibility, generalizability, and efficiency of using DCNN features to predict human trait judgments from faces, providing an easily scalable framework for automated prediction of human judgment.

Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription Paper Materials
Lin, Chujun0000-0002-7605-6508
Adolphs, Ralph0000-0002-8053-9692
Additional Information:License: CC-By Attribution 4.0 International. Created: January 11, 2021; Last edited: January 14, 2021. Funded in part by NSF grants BCS-1840756 and BCS-1845958, the Simons Foundation Collaboration on the Global Brain (542941), and the Carver Mead New Adventures Fund. Data availability: All data are from publicly available datasets which could be accessed via the links provided in the papers cited. Author contributions: U.K. and R.A. developed the study concept and designed the study; R.A. supervised the experiments and analyses; C.L. performed data collection; U.K. performed data analyses; all authors drafted, revised, and reviewed the manuscript, and approved the final manuscript for submission. The authors declare no competing interests.
Funding AgencyGrant Number
Simons Foundation542941
Carver Mead New Adventures FundUNSPECIFIED
Subject Keywords:social cognition; face perception; deep learning; neural networks
Record Number:CaltechAUTHORS:20210120-145142979
Persistent URL:
Official Citation:Keles, U., Lin, C., & Adolphs, R. (2021, January 12). Efficient prediction of trait judgments from faces using deep neural networks.
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:107603
Deposited By: Tony Diaz
Deposited On:20 Jan 2021 23:06
Last Modified:16 Nov 2021 19:04

Repository Staff Only: item control page