A Caltech Library Service

On Eliciting Beliefs in Strategic Games

Palfrey, Thomas R. and Wang, Stephanie W. (2009) On Eliciting Beliefs in Strategic Games. Journal of Economic Behavior & Organization, 71 (2). pp. 98-109. ISSN 0167-2681.

[img] PDF (sswp 1721 - Appendix (Sample Instructions)) - Supplemental Material
Restricted to Caltech community only
See Usage Policy.

[img] PDF (sswp 1271 - published) - Published Version
Restricted to Caltech community only
See Usage Policy.


Use this Persistent URL to link to this item:


Several recent studies in experimental economics have tried to measure beliefs of subjects engaged in strategic games with other subjects. Using data from one such study we conduct an experiment where our experienced subjects observe early rounds of strategy choices from that study and are given monetary incentives to report forecasts of choices in later rounds. We elicit beliefs using three different scoring rules: linear, logarithmic, and quadratic. We compare forecasts across the scoring rules and compare the forecasts of our trained observers to forecasts of the actual players in the original experiment. We find significant differences across scoring rules. The improper linear scoring rule produces forecasts closer to 0 and 1 than the proper rules, and these forecasts are poorly calibrated. The two proper scoring rules induce significantly different distributions of forecasts. We find that forecasts by observers under both proper scoring rules are significantly different from the forecasts of the actual players, in terms of accuracy, calibration, and the distribution of forecasts. We also find evidence for belief convergence among the observers.

Item Type:Article
Related URLs:
URLURL TypeDescription
Palfrey, Thomas R.0000-0003-0769-8109
Additional Information:© 2009 Elsevier. Received 12 January 2008. Received in revised form 23 March 2009. Accepted 25 March 2009. Available online 10 April 2009. We gratefully acknowledge the financial support of the National Science Foundation (SES-0617820), The Gordon and Betty Moore Foundation, the Princeton Laboratory for Experimental Social Science, and the Social Science Experimental Laboratory at Caltech. We are grateful for detailed comments and suggestions from an editor, two referees, Juan Carrillo, audience members at the 2007 meeting of the Public Choice Society in Amsterdam, and participants at the 2007 conference at the University of Exeter on Risk, Forecast, and Decision.
Group:Social Science Working Papers
Funding AgencyGrant Number
Gordon and Betty Moore FoundationUNSPECIFIED
Princeton Laboratory for Experimental Social Science (PLESS)UNSPECIFIED
Caltech Social Science Experimental LaboratoryUNSPECIFIED
Subject Keywords:Scoring rules; Experiment; Game theory; Forecasting; Beliefs
Other Numbering System:
Other Numbering System NameOther Numbering System ID
Social Science Working Paper1271R
Issue or Number:2
Record Number:CaltechAUTHORS:20160301-140744431
Persistent URL:
Official Citation:Thomas R. Palfrey, Stephanie W. Wang, On eliciting beliefs in strategic games, Journal of Economic Behavior & Organization, Volume 71, Issue 2, August 2009, Pages 98-109, ISSN 0167-2681, (
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:64926
Deposited By: Susan Vite
Deposited On:02 Mar 2016 21:42
Last Modified:22 Nov 2019 09:58

Repository Staff Only: item control page