A Caltech Library Service

SpECTRE: A task-based discontinuous Galerkin code for relativistic astrophysics

Kidder, Lawrence E. and Field, Scott E. and Foucart, Francois and Schnetter, Erik and Teukolsky, Saul A. and Bohn, Andy and Deppe, Nils and Diener, Peter and Hébert, François and Lippuner, Jonas and Miller, Jonah and Ott, Christian D. and Scheel, Mark A. and Vincent, Trevor (2017) SpECTRE: A task-based discontinuous Galerkin code for relativistic astrophysics. Journal of Computational Physics, 335 . pp. 84-114. ISSN 0021-9991.

[img] PDF - Submitted Version
See Usage Policy.


Use this Persistent URL to link to this item:


We introduce a new relativistic astrophysics code, SpECTRE, that combines a discontinuous Galerkin method with a task-based parallelism model. SpECTRE's goal is to achieve more accurate solutions for challenging relativistic astrophysics problems such as core-collapse supernovae and binary neutron star mergers. The robustness of the discontinuous Galerkin method allows for the use of high-resolution shock capturing methods in regions where (relativistic) shocks are found, while exploiting high-order accuracy in smooth regions. A task-based parallelism model allows efficient use of the largest supercomputers for problems with a heterogeneous workload over disparate spatial and temporal scales. We argue that the locality and algorithmic structure of discontinuous Galerkin methods will exhibit good scalability within a task-based parallelism framework. We demonstrate the code on a wide variety of challenging benchmark problems in (non)-relativistic (magneto)-hydrodynamics. We demonstrate the code's scalability including its strong scaling on the NCSA Blue Waters supercomputer up to the machine's full capacity of 22,380 nodes using 671,400 threads.

Item Type:Article
Related URLs:
URLURL TypeDescription Paper
Lippuner, Jonas0000-0002-5936-3485
Ott, Christian D.0000-0003-4993-2055
Additional Information:© 2017 Elsevier Inc. Received 5 September 2016, Revised 21 December 2016, Accepted 29 December 2016, Available online 5 January 2017. We acknowledge helpful discussions with Sebastiano Bernuzzi, Bernd Brügmann, Marcus Bugner, Mani Chandra, Roland Haas, Daniel Hemberger, Jan Hesthaven, Cameron Hummels, Fraser Hutchison, Jonathan Lifflander, Geoffrey Lovelace, Phil Miller, Harald Pfeiffer, David Radice, and Christian Reisswig. We thank the anonymous referee for detailed comments and suggestions which helped to improve the manuscript. Support for F. Foucart was provided by NASA through Einstein Postdoctoral Fellowship grant numbered PF4-150122 awarded by the Chandra X-ray Center, which is operated by the Smithsonian Astrophysical Observatory for NASA under contract NAS8-03060. Authors at Cornell were partially supported by NSF under award Nos. TCAN AST-1333129 and PHY-1306125, and by the Sherman Fairchild Foundation. Authors at Caltech were partially supported by NSF under award Nos. TCAN AST-1333520, CAREER PHY-1151197, and PHY-1404569, and by the Sherman Fairchild Foundation. ES acknowledges support from NSF award OCI-0905046. ES and JM were supported by an NSERC Discovery grant. Research at the Perimeter Institute is supported by the Government of Canada through Industry Canada and by the Province of Ontario through the Ministry of Research & Innovation. Computations were performed on NSF/NCSA Blue Waters under allocation PRAC ACI-1440083, on the NSF XSEDE network under allocations TG-PHY100033 and TG-PHY990007, and on the Caltech compute cluster Zwicky (NSF MRI-R2 award No. PHY-0960291).
Funding AgencyGrant Number
NASA Einstein FellowshipPF4-150122
Sherman Fairchild FoundationUNSPECIFIED
Natural Sciences and Engineering Research Council of Canada (NSERC)UNSPECIFIED
Industry Canada UNSPECIFIED
Ontario Ministry of Research and InnovationUNSPECIFIED
Subject Keywords:Discontinuous Galerkin; Hydrodynamics; Magnetohydrodynamics; Task-based parallelism
Record Number:CaltechAUTHORS:20170427-142549072
Persistent URL:
Official Citation:Lawrence E. Kidder, Scott E. Field, Francois Foucart, Erik Schnetter, Saul A. Teukolsky, Andy Bohn, Nils Deppe, Peter Diener, François Hébert, Jonas Lippuner, Jonah Miller, Christian D. Ott, Mark A. Scheel, Trevor Vincent, SpECTRE: A task-based discontinuous Galerkin code for relativistic astrophysics, Journal of Computational Physics, Volume 335, 15 April 2017, Pages 84-114, ISSN 0021-9991, (
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:77016
Deposited By: Ruth Sustaita
Deposited On:27 Apr 2017 21:47
Last Modified:03 Oct 2019 17:52

Repository Staff Only: item control page