Search

The MFEM (Modular Finite Element Methods) library is a high-performance C++ library for finite element discretizations. MFEM supports numerous types of finite element methods and is the discretization engine powering many computational physics and engineering applications across a number of domains. This paper describes some of the recent research and development in MFEM, focusing on performance portability across leadership-class supercomputing facilities, including exascale supercomputers, as well as new capabilities and functionality, enabling a wider range of applications. Much of this work was undertaken as part of the Department of Energy's Exascale Computing Project (ECP) in collaboration with the Center for Efficient Exascale Discretizations (CEED). [ABSTRACT FROM AUTHOR]

Efficient exploitation of exascale architectures requires rethinking of the numerical algorithms used in many large-scale applications. These architectures favor algorithms that expose ultra fine-grain parallelism and maximize the ratio of floating point operations to energy intensive data movement. One of the few viable approaches to achieve high efficiency in the area of PDE discretizations on unstructured grids is to use matrix-free/partially assembled high-order finite element methods, since these methods can increase the accuracy and/or lower the computational time due to reduced data motion. In this paper we provide an overview of the research and development activities in the Center for Efficient Exascale Discretizations (CEED), a co-design center in the Exascale Computing Project that is focused on the development of next-generation discretization software and algorithms to enable a wide range of finite element applications to run efficiently on future hardware. CEED is a research partnership involving more than 30 computational scientists from two US national labs and five universities, including members of the Nek5000, MFEM, MAGMA and PETSc projects. We discuss the CEED co-design activities based on targeted benchmarks, miniapps and discretization libraries and our work on performance optimizations for large-scale GPU architectures. We also provide a broad overview of research and development activities in areas such as unstructured adaptive mesh refinement algorithms, matrix-free linear solvers, high-order data visualization, and list examples of collaborations with several ECP and external applications. [ABSTRACT FROM AUTHOR]

We describe a framework for controlling and improving the quality of high-order finite element meshes based on extensions of the Target-Matrix Optimization Paradigm (TMOP) of [P. Knupp, Eng. Comput., 28 (2012), pp. 419--429]. This approach allows high-order applications to have a very precise control over local mesh quality, while still improving the mesh globally. We address the adaption of various TMOP components to the settings of general isoparametric element mappings, including the mesh quality metric in 2D and 3D, the selection of sample points and the solution of the resulting mesh optimization problem. We also investigate additional practical concerns, such as tangential relaxation and restricting the deviation from the original mesh. The benefits of the new high-order TMOP algorithms are illustrated on a number of test problems and examples from a high-order arbitrary Lagrangian--Eulerian (ALE) application [BLAST: High-order curvilinear finite elements for shock hydrodynamics. [ABSTRACT FROM AUTHOR]

We present a new approach for multi-material arbitrary Lagrangian-Eulerian (ALE) hydrodynamics simulations based on high-order finite elements posed on high-order curvilinear meshes. The method builds on and extends our previous work in the Lagrangian [V. A. Dobrev, T. V. Kolev, and R. N. Rieben, SIAM J. Sci. Comput., 34 (2012), pp. B606-B641] and remap [R. W. Anderson et al., Internat. J. Numer. Methods Fluids, 77 (2015), pp. 249-273] phases of ALE, and depends critically on a functional perspective that enables subzonal physics and material modeling [V. A. Dobrev et al., Internat. J. Numer. Methods Fluids, 82 (2016), pp. 689-706]. Curvilinear mesh relaxation is based on node movement, which is determined through the solution of an elliptic equation. The remap phase is posed in terms of advecting state variables between two meshes over a fictitious time interval. The resulting advection equation is solved by a discontinuous Galerkin (DG) formulation, combined with a customized Flux Corrected Transport (FCT) type algorithm. Because conservative fields are remapped, additional synchronization steps are introduced to preserve bounds with respect to primal fields. These steps include modification of the low-order FCT solutions, def- inition of conservative FCT uxes based on primal field bounds, and monotone transitions between primal and conservative fields. This paper describes the mathematical formulation and properties of our approach and reports a number of numerical results from its implementation in the BLAST code [BLAST: High-order finite element Lagrangian hydrocode, http://www.llnl.gov/CASC/blast]. Additional details can be found in [R. W. Anderson et al., High-Order Multi-Material ALE Hy- drodynamics (Extended Version), Tech. report LLNL-JRNL-706339, Lawrence Livermore National Laboratory, Livermore, CA, 2016]. [ABSTRACT FROM AUTHOR]

We present a computational framework for high-performance tensor contractions on GPUs. High-performance is difficult to obtain using existing libraries, especially for many independent contractions where each contraction is very small, e.g., sub-vector/warp in size. However, using our framework to batch contractions plus application-specifics, we demonstrate close to peak performance results. In particular, to accelerate large scale tensor-formulated high-order finite element method (FEM) simulations, which is the main focus and motivation for this work, we represent contractions as tensor index reordering plus matrix-matrix multiplications (GEMMs). This is a key factor to achieve algorithmically many-fold acceleration (vs. not using it) due to possible reuse of data loaded in fast memory. In addition to using this context knowledge, we design tensor data-structures, tensor algebra interfaces, and new tensor contraction algorithms and implementations to achieve 90+% of a theoretically derived peak on GPUs. On a K40c GPU for contractions resulting in GEMMs on square matrices of size 8 for example, we are 2.8× faster than CUBLAS, and 8.5× faster than MKL on 16 cores of Intel Xeon ES-2670 (Sandy Bridge) 2.60GHz CPUs. Finally, we apply autotuning and code generation techniques to simplify tuning and provide an architecture-aware, user-friendly interface. [ABSTRACT FROM AUTHOR]

We have investigated the use of the adaptive high-order finite-element method (FEM) for geoelectromagnetic modeling. Because high-order FEM is challenging from the numerical and computational points of view, most published finite-element studies in geoelectromagnetics use the lowest order formulation. Solution of the resulting large system of linear equations poses the main practical challenge. We have developed a fully parallel and distributed robust and scalable linear solver based on the optimal block-diagonal and auxiliary space preconditioners. The solver was found to be efficient for high finite element orders, unstructured and nonconforming locally refined meshes, a wide range of frequencies, large conductivity contrasts, and number of degrees of freedom (DoFs). Furthermore, the presented linear solver is in essence algebraic; i.e., it acts on the matrix-vector level and thus requires no information about the discretization, boundary conditions, or physical source used, making it readily efficient for a wide range of electromagnetic modeling problems. To get accurate solutions at reduced computational cost, we have also implemented goal-oriented adaptive mesh refinement. The numerical tests indicated that if highly accurate modeling results were required, the high-order FEM in combination with the goal-oriented local mesh refinement required less computational time and DoFs than the lowest order adaptive FEM. [ABSTRACT FROM AUTHOR]

The numerical approximation of the Euler equations of gas dynamics in a moving Lagrangian frame is at the heart of many multiphysics simulation algorithms. In this paper, we present a general framework for high-order Lagrangian discretization of these compressible shock hydrodynamics equations using curvilinear finite elements. This method is an extension of the approach outlined in [Dobrev et al., Internat. J. Numer. Methods Fluids, 65 (2010), pp. 1295-1310] and can be formulated for any finite dimensional approximation of the kinematic and thermody-namic fields, including generic finite elements on two- and three-dimensional meshes with triangular, quadrilateral, tetrahedral, or hexahedral zones. We discretize the kinematic variables of position and velocity using a continuous high-order basis function expansion of arbitrary polynomial degree which is obtained via a corresponding high-order parametric mapping from a standard reference element. This enables the use of curvilinear zone geometry, higher-order approximations for fields within a zone, and a pointwise definition of mass conservation which we refer to as strong mass conservation. We discretize the internal energy using a piecewise discontinuous high-order basis function expansion which is also of arbitrary polynomial degree. This facilitates multimaterial hydrodynamics by treating material properties, such as equations of state and constitutive models, as piecewise discontinuous functions which vary within a zone. To satisfy the Rankine-Hugoniot jump conditions at a shock boundary and generate the appropriate entropy, we introduce a general tensor artificial viscosity which takes advantage of the high-order kinematic and thermodynamic information available in each zone. Finally, we apply a generic high-order time discretization process to the semidiscrete equations to develop the fully discrete numerical algorithm. Our method can be viewed as the high-order generalization of the so-called staggered-grid hydrodynamics (SGH) approach and we show that under specific low-order assumptions, we exactly recover the classical SGH method. We present numerical results from an extensive series of verification tests that demonstrate several important practical advantages of using high-order finite elements in this context. [ABSTRACT FROM AUTHOR]

In this paper we review a number of auxiliary space based preconditioners for the second order definite and semi-definite Maxwell problems discretized with the lowest order Nédélec finite elements. We discuss the parallel implementation of the most promising of these methods, the ones derived from the recent Hiptmair-Xu (HX) auxiliary space decomposition [Hiptmair and Xu, SIAM J. Numer. Anal., 45 (2007), pp. 2483-2509]. An extensive set of numerical experiments demonstrate the scalability of our implementation on large-scale H(curl) problems. [ABSTRACT FROM AUTHOR]

This paper is concerned with the pure displacement problem of planar linear elasticity. Our interest is focused on the locking-free FEM approximation of the problem in the case of almost incompressible material. Crouzeix-Raviart linear finite elements are implemented. An optimal order pure algebraic multiplicative two-level preconditioner for the related stiffness matrix is developed. The proposed construction is based on a proper hierarchical basis of the FEM space. It is important to note that the nodal FEM spaces corresponding to successive levels of mesh refinements are not nested for the non-conforming elements under consideration. Local spectral analysis is applied to determine the scaling parameter of the preconditioner as well as to estimate the related constant in the strengthened CBS inequality. The derived estimates are uniform with respect to the Poisson ratio ν ∈ (0, 0.5). A set of numerical tests is presented to illustrate the accuracy of the FEM approximation, and the convergence rate of the two-level PCG method. Copyright © 1999 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

MFEM is an open-source, lightweight, flexible and scalable C++ library for modular finite element methods that features arbitrary high-order finite element meshes and spaces, support for a wide variety of discretization approaches and emphasis on usability, portability, and high-performance computing efficiency. MFEM's goal is to provide application scientists with access to cutting-edge algorithms for high-order finite element meshing, discretizations and linear solvers, while enabling researchers to quickly and easily develop and test new algorithms in very general, fully unstructured, high-order, parallel and GPU-accelerated settings. In this paper we describe the underlying algorithms and finite element abstractions provided by MFEM, discuss the software implementation, and illustrate various applications of the library. [ABSTRACT FROM AUTHOR]