A Caltech Library Service

Scalable Adaptive Mantle Convection Simulation on Petascale Supercomputers

Burstedde, Carsten and Ghattas, Omar and Gurnis, Michael and Stadler, Georg and Tan, Eh and Tu, Tiankai and Wilcox, Lucas C. and Zhong, Shijie (2008) Scalable Adaptive Mantle Convection Simulation on Petascale Supercomputers. In: 2008 International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE , Piscataway, NJ, pp. 53-67. ISBN 978-1-4244-2834-2.

PDF - Published Version
See Usage Policy.


Use this Persistent URL to link to this item:


Mantle convection is the principal control on the thermal and geological evolution of the Earth. Mantle convection modeling involves solution of the mass, momentum, and energy equations for a viscous, creeping, incompressible non-Newtonian fluid at high Rayleigh and Peclet numbers. Our goal is to conduct global mantle convection simulations that can resolve faulted plate boundaries, down to 1 km scales. However, uniform resolution at these scales would result in meshes with a trillion elements, which would elude even sustained petaflops supercomputers. Thus parallel adaptive mesh refinement and coarsening (AMR) is essential. We present RHEA, a new generation mantle convection code designed to scale to hundreds of thousands of cores. RHEA is built on ALPS, a parallel octree-based adaptive mesh finite element library that provides new distributed data structures and parallel algorithms for dynamic coarsening, refinement, rebalancing, and repartitioning of the mesh. ALPS currently supports low order continuous Lagrange elements, and arbitrary order discontinuous Galerkin spectral elements, on octree meshes. A forest-ofoctrees implementation permits nearly arbitrary geometries to be accommodated. Using TACC’s 579 teraflops Ranger supercomputer, we demonstrate excellent weak and strong scalability of parallel AMR on up to 62,464 cores for problems with up to 12.4 billion elements. With RHEA’s adaptive capabilities, we have been able to reduce the number of elements by over three orders of magnitude, thus enabling us to simulate large-scale mantle convection with finest local resolution of 1.5 km.

Item Type:Book Section
Related URLs:
URLURL TypeDescription
Gurnis, Michael0000-0003-1704-597X
Additional Information:© 2008 IEEE. Date of Current Version: 25 August 2009. This work was partially supported by NSF’s PetaApps program (grants OCI-0749334, OCI-0749045, and OCI-0748898), NSF Earth Sciences (EAR-0426271), DOE Office of Science’s SciDAC program (grant DE-FC02-06ER25782), DOE NNSA’s PSAAP program (cooperative agreement DE-FC52-08NA28615), and NSF grants ATM-0326449, CCF-0427985, CNS-0540372, CNS-0619838, and DMS-0724746. We acknowledge many helpful discussions with hypre developers Rob Falgout and Ulrike Yang, and with George Biros. We thank TACC for their outstanding support, in particular Bill Barth, Tommy Minyard, Romy Schneider, and Karl Schulz.
Group:Seismological Laboratory
Funding AgencyGrant Number
NSF PetaAppsOCI-0749334
NSF PetaAppsOCI-0749045
NSF PetaAppsOCI-0748898
NSF Earth SciencesEAR-0426271
Department of Energy (DOE) Office of Science SciDAC ProgramDE-FC02-06ER25782
Department of Energy (DOE) NNSA PSAAP ProgramDE-FC52-08NA28615
Other Numbering System:
Other Numbering System NameOther Numbering System ID
INSPEC Accession Number10846494
Record Number:CaltechAUTHORS:20121114-092032390
Persistent URL:
Official Citation:Burstedde, C.; Ghattas, O.; Gurnis, M.; Stadler, G.; Eh Tan; Tiankai Tu; Wilcox, L.C.; Shijie Zhong; , "Scalable adaptive mantle convection simulation on petascale supercomputers," High Performance Computing, Networking, Storage and Analysis, 2008. SC 2008. International Conference for , vol., no., pp.1-15, 15-21 Nov. 2008 doi: 10.1109/SC.2008.5214248 URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:35452
Deposited By: Tony Diaz
Deposited On:14 Nov 2012 22:59
Last Modified:09 Nov 2021 23:15

Repository Staff Only: item control page