Block-structured adaptive mesh refinement (AMR) is a technique that can be used when solving partial differential equations to reduce the number of cells necessary to achieve the required accuracy in areas of interest. These areas (shock fronts, material interfaces, etc.) are recursively covered with finer mesh patches that are grouped into a hierarchy of refinement levels. Despite the potential for large savings in computational requirements and memory usage without a corresponding reduction in accuracy, AMR adds overhead in managing the mesh hierarchy, adding complex communication and data movement requirements to a simulation. In this paper, we describe the design and implementation of a resident GPU-based AMR library, including: the classes used to manage data on a mesh patch, the routines used for transferring data between GPUs on different nodes, and the data-parallel operators developed to coarsen and refine mesh data. We validate the performance and accuracy of our implementation using three test problems and two architectures: an 8 node cluster, and 4,196 nodes of Oak Ridge National Laboratory's Titan supercomputer. Our GPU-based AMR hydrodynamics code performs up to 4.87× faster than the CPU-based implementation, and is scalable on 4,196 K20x GPUs using a combination of MPI and CUDA.
|Title of host publication||Proceedings - 2015 44th International Annual Conference on Parallel Processing, ICPP 2015|
|Publisher||Institute of Electrical and Electronics Engineers (IEEE)|
|Number of pages||10|
|Publication status||Published - 8 Dec 2015|
|Event||44th International Conference on Parallel Processing, ICPP 2015 - Beijing, China|
Duration: 1 Sept 2015 → 4 Sept 2015
|Name||Proceedings of the International Conference on Parallel Processing|
|Conference||44th International Conference on Parallel Processing, ICPP 2015|
|Period||1/09/15 → 4/09/15|
Bibliographical notePublisher Copyright:
© 2015 IEEE.
- Adaptive mesh refinement
ASJC Scopus subject areas
- Hardware and Architecture