Search

Author(s): Godoy, WF; Podhorszki, N; Wang, R; Atkins, C; Eisenhauer, G; Gu, J; Davis, P; Choi, J; Germaschewski, K; Huck, K; Huebl, A; Kim, M; Kress, J; Kurc, T; Liu, Q; Logan, J; Mehta, K; Ostrouchov, G; Parashar, M; Poeschel, F; Pugmire, D; Suchyta, E; Takahashi, K; Thompson, N; Tsutsumi, S; Wan, L; Wolf, M; Wu, K; Klasky, S | Abstract: We present ADIOS 2, the latest version of the Adaptable Input Output (I/O) System. ADIOS 2 addresses scientific data management needs ranging from scalable I/O in supercomputers, to data analysis in personal computer and cloud systems. Version 2 introduces a unified application programming interface (API) that enables seamless data movement through files, wide-area-networks, and direct memory access, as well as high-level APIs for data analysis. The internal architecture provides a set of reusable and extendable components for managing data presentation and transport mechanisms for new applications. ADIOS 2 bindings are available in C++11, C, Fortran, Python, and Matlab and are currently used across different scientific communities. ADIOS 2 provides a communal framework to tackle data management challenges as we approach the exascale era of supercomputing.

The present study introduces a parallel Jacobian-free Newton Krylov (JFNK) general minimal residual (GMRES) solution for the discretized radiative transfer equation (RTE) in 3D, absorbing, emitting and scattering media. For the angular and spatial discretization of the RTE, the discrete ordinates method (DOM) and the finite volume method (FVM) including flux limiters are employed, respectively. Instead of forming and storing a large Jacobian matrix, JFNK methods allow for large memory savings as the required Jacobian-vector products are rather approximated by semiexact and numerical formulations, for which convergence and computational times are presented. Parallelization of the GMRES solution is introduced in a combined memory-shared/memory-distributed formulation that takes advantage of the fact that only large vector arrays remain in the JFNK process. Results are presented for 3D test cases including a simple homogeneous, isotropic medium and a more complex non-homogeneous, non-isothermal, absorbing-emitting and anisotropic scattering medium with collimated intensities. Additionally, convergence and stability of Gram-Schmidt and Householder orthogonalizations for the Arnoldi process in the parallel GMRES algorithms are discussed and analyzed. Overall, the introduction of JFNK methods results in a parallel, yet scalable to the tested 2048 processors, and memory affordable solution to 3D radiative transfer problems without compromising the accuracy and convergence of a Newton-like solution.

General-purpose computing on graphics processing units (GPGPU) is a recent technique that allows the parallel graphics processing unit (GPU) to accelerate calculations performed sequentially by the central processing unit (CPU). To introduce GPGPU to radiative transfer, the Gauss-Seidel solution of the well-known expressions for 1-D and 3-D homogeneous, isotropic media is selected as a test case. Different algorithms are introduced to balance memory and GPU-CPU communication, critical aspects of GPGPU. Results show that speed-ups of one to two orders of magnitude are obtained when compared to sequential solutions. The underlying value of GPGPU is its potential extension in radiative solvers (e.g., Monte Carlo, discrete ordinates) at a minimal learning curve.

The application of flux limiters to the discrete ordinates method (DOM), S"N, for radiative transfer calculations is discussed and analyzed for 3D enclosures for cases in which the intensities are strongly coupled to each other such as: radiative equilibrium and scattering media. A Newton-Krylov iterative method (GMRES) solves the final systems of linear equations along with a domain decomposition strategy for parallel computation using message passing libraries in a distributed memory system. Ray effects due to angular discretization and errors due to domain decomposition are minimized until small variations are introduced by these effects in order to focus on the influence of flux limiters on errors due to spatial discretization, known as numerical diffusion, smearing or false scattering. Results are presented for the DOM-integrated quantities such as heat flux, irradiation and emission. A variety of flux limiters are compared to ''exact'' solutions available in the literature, such as the integral solution of the RTE for pure absorbing-emitting media and isotropic scattering cases and a Monte Carlo solution for a forward scattering case. Additionally, a non-homogeneous 3D enclosure is included to extend the use of flux limiters to more practical cases. The overall balance of convergence, accuracy, speed and stability using flux limiters is shown to be superior compared to step schemes for any test case.