A Caltech Library Service

Comparing Bayes factors and hierarchical inference for testing general relativity with gravitational waves

Isi, Maximiliano and Farr, Will M. and Chatziioannou, Katerina (2022) Comparing Bayes factors and hierarchical inference for testing general relativity with gravitational waves. Physical Review D, 106 (2). Art. No. 024048. ISSN 2470-0010. doi:10.1103/physrevd.106.024048.

[img] PDF - Published Version
See Usage Policy.

[img] PDF - Accepted Version
Creative Commons Attribution.


Use this Persistent URL to link to this item:


In the context of testing general relativity with gravitational waves, constraints obtained with multiple events are typically combined either through a hierarchical formalism or though a combined multiplicative Bayes factor. We show that the well-known dependence of Bayes factors on the analysis priors in regions of the parameter space without likelihood support can lead to strong confidence in favor of incorrect conclusions when one employs the multiplicative Bayes factor. Bayes factors O(1) are ambivalent as they depend sensitively on the analysis priors, which are rarely set in a principled way; additionally, combined Bayes factors > O(10³) can be obtained in favor of the incorrect conclusion depending on the analysis priors when many O(1) Bayes factors are multiplied, and specifically when the priors are much wider than the underlying population. The hierarchical analysis that instead infers the ensemble distribution of the individual beyond-general-relativity constraints does not suffer from this problem, and generically converges to favor the correct conclusion. Rather than a naive multiplication, a more reliable Bayes factor can be computed from the hierarchical analysis. We present a number of toy models showing that the practice of multiplying Bayes factors can lead to incorrect conclusions.

Item Type:Article
Related URLs:
URLURL TypeDescription Paper
Isi, Maximiliano0000-0001-8830-8672
Farr, Will M.0000-0003-1540-8562
Chatziioannou, Katerina0000-0002-5833-413X
Additional Information:© 2022 American Physical Society. (Received 8 May 2022; accepted 17 July 2022; published 26 July 2022) We are grateful to Tyson Littenberg for insightful feedback on this manuscript. During part of this work, M. I. was supported by NASA through the NASA Hubble Fellowship Grant No. HST-HF2-51410.001-A awarded by the Space Telescope Science Institute, which is operated by the Association of Universities for Research in Astronomy, Inc., for NASA, under Contract No. NAS5-26555. The Flatiron Institute is a division of the Simons Foundation, supported through the generosity of Marilyn and Jim Simons. We thank Gregorio Carullo for comments on non-nested models. This paper carries LIGO Document No. LIGO-P2200099. Software: matplotlib [56], julia [57], turing.jl [58], plots.jl [59,60].
Group:Astronomy Department, LIGO
Funding AgencyGrant Number
NASA Hubble FellowshipHST-HF2-51410.001-A
Simons FoundationUNSPECIFIED
Other Numbering System:
Other Numbering System NameOther Numbering System ID
LIGO DocumentP2200099
Issue or Number:2
Record Number:CaltechAUTHORS:20220726-998129000
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:115866
Deposited By: George Porter
Deposited On:27 Jul 2022 20:08
Last Modified:27 Jul 2022 20:08

Repository Staff Only: item control page