CaltechAUTHORS
  A Caltech Library Service

Screening p-Hackers: Dissemination Noise as Bait

Echenique, Federico and He, Kevin (2022) Screening p-Hackers: Dissemination Noise as Bait. In: Proceedings of the 23rd ACM Conference on Economics and Computation. Association for Computing Machinery , New York, NY, p. 791. ISBN 978-1-4503-9150-4. https://resolver.caltech.edu/CaltechAUTHORS:20220707-170534070

[img] PDF - Published Version
See Usage Policy.

797kB
[img] PDF - Submitted Version
See Usage Policy.

755kB

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20220707-170534070

Abstract

We show that adding noise to data before making data public is effective at screening p-hacked findings: spurious explanations of the outcome variable produced by attempting multiple econometric specifications. Noise creates "baits'' that affect two types of researchers differently. Uninformed p-hackers who engage in data mining with no prior information about the true causal mechanism often fall for baits and report verifiably wrong results when evaluated with the original data. But informed researchers who start with an ex-ante hypothesis about the causal mechanism before seeing any data are minimally affected by noise. We characterize the optimal level of dissemination noise and highlight the relevant trade-offs in a simple theoretical model. Dissemination noise is a tool that statistical agencies (e.g., the US Census Bureau) currently use to protect privacy, and we show this existing practice can be repurposed to improve research credibility.


Item Type:Book Section
Related URLs:
URLURL TypeDescription
https://doi.org/10.1145/3490486.3538358DOIArticle
https://arxiv.org/abs/2103.09164arXivDiscussion Paper
ORCID:
AuthorORCID
Echenique, Federico0000-0002-1567-6770
He, Kevin0000-0001-5806-0370
Additional Information:© 2022 Copyright held by the owner/author(s). This research was made possible through the support of the Linde Institute. Echenique also thanks the NSF’s support through the grants SES-1558757 and CNS-1518941. We are grateful for comments from Sylvain Chassang, Eva Jin, Albert Ma, Pascal Michaillat, Marco Ottaviani, Nathan Yoder, and the audiences at the University of Pennsylvania, Caltech, Universidad de la República, Boston University, and ASSA 2022.
Funders:
Funding AgencyGrant Number
Linde Institute of Economic and Management ScienceUNSPECIFIED
NSFSES-1558757
NSFCNS-1518941
Subject Keywords:p-hacking, dissemination noise, screening
DOI:10.1145/3490486.3538358
Record Number:CaltechAUTHORS:20220707-170534070
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20220707-170534070
Official Citation:Federico Echenique and Kevin He. 2022. Screening p-Hackers: Dissemination Noise as Bait. In Proceedings of the 23rd ACM Conference on Economics and Computation (EC ’22), July 11–15, 2022, Boulder, CO, USA. ACM, New York, NY, USA, 1 page. https://doi.org/10.1145/3490486.3538358
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:115366
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:07 Jul 2022 19:41
Last Modified:29 Jul 2022 21:16

Repository Staff Only: item control page