Search

Computation of piled-raft interaction effects based on boundary element method (BEM) achieves high accuracy. However, it's not preferred, due to the impractical requirement for huge computational time. In this paper, a simplified coding technique which is generally applicable to any BEM application is demonstrated for pile-soil-pile interaction problems. The proposed simplified technique is based on parallel computing to optimize computational performance. Three parallel computing approaches have been developed in order to take full advantage of the computer's power. The approaches are based on CPU Multithreading, GPU Multithreading, and a hybrid system that combines both. The approaches have been tested on five machines with different hardware (CPU, GPU, and Memory) configurations to analyze four practical examples with different load cases, geometric details, interaction methods, and cases of interactions. The parallelization approaches performance illustrated high efficiency with large-scale computations and the consumed time has been reduced by 95 % for both the GPU and hybrid system approaches and 90 % reduction for the multithreading on the CPU. The analysis results confirmed the validity of the proposed methods without affecting the accuracy of the numerical results. The strength of the proposed technique is in its simplicity and scalability to be used for any BEM code. [ABSTRACT FROM AUTHOR]

Earth system models (ESM) demand significant hardware resources and energy consumption to solve atmospheric chemistry processes. Recent studies have shown improved performance from running these models on GPU accelerators. Nonetheless, there is room for improvement in exploiting even more GPU resources. This study proposes an optimized distribution of the chemical solver's computational load on the GPU, named Block-cells. Additionally, we evaluate different configurations for distributing the computational load in an NVIDIA GPU. We use the linear solver from the Chemistry Across Multiple Phases (CAMP) framework as our test bed. An intermediate-complexity chemical mechanism under typical atmospheric conditions is used. Results demonstrate a 35× speedup compared to the single-CPU thread reference case. Even using the full resources of the node (40 physical cores) on the reference case, the Block-cells version outperforms them by 50%. The Block-cells approach shows promise in alleviating the computational burden of chemical solvers on GPU architectures. • Integration of a Biconjugate Gradient algorithm as GPU linear solver for chemistry in Earth System Models. • Development of a strategy to eliminate communication between GPU blocks (Block-cells approach). • Performance assessment of different thread block arrangements. • Up to 35x linear solver speedup over the base single-thread MPI. • The GPU Block-cells version of the linear solver is up to 50% faster than the multi-core MPI version (40 cores). [ABSTRACT FROM AUTHOR]

This work presents GALÆXI as a novel, energy-efficient flow solver for the simulation of compressible flows on unstructured hexahedral meshes leveraging the parallel computing power of modern Graphics Processing Units (GPUs). GALÆXI implements the high-order Discontinuous Galerkin Spectral Element Method (DGSEM) using shock capturing with a finite-volume subcell approach to ensure the stability of the high-order scheme near shocks. This work provides details on the general code design, the parallelization strategy, and the implementation approach for the compute kernels with a focus on the element local mappings between volume and surface data due to the unstructured mesh. The scheme is implemented using a pure distributed memory parallelization based on a domain decomposition, where each GPU handles a distinct region of the computational domain. On each GPU, the computations are assigned to different compute streams which allows to antedate the computation of quantities required for communication while performing local computations from other streams to hide the communication latency. This parallelization strategy allows for maximizing the use of available computational resources. This results in excellent strong scaling properties of GALÆXI up to 1024 GPUs if each GPU is assigned a minimum of one million degrees of freedom. To verify its implementation, a convergence study is performed that recovers the theoretical order of convergence of the implemented numerical schemes. Moreover, the solver is validated using both the incompressible and compressible formulation of the Taylor–Green-Vortex at a Mach number of 0.1 and 1.25, respectively. A mesh convergence study shows that the results converge to the high-fidelity reference solution and that the results match the original CPU implementation. Finally, GALÆXI is applied to a large-scale wall-resolved large eddy simulation of a linear cascade of the NASA Rotor 37. Here, the supersonic region and shocks at the leading edge are captured accurately and robustly by the implemented shock-capturing approach. It is demonstrated that GALÆXI requires less than half of the energy to carry out this simulation in comparison to the reference CPU implementation. This renders GALÆXI as a potent tool for accurate and efficient simulations of compressible flows in the realm of exascale computing and the associated new HPC architectures. [ABSTRACT FROM AUTHOR]

The Direct Simulation Monte Carlo (DSMC) method was widely used to simulate low density gas flows with large Knudsen numbers. However, DSMC encounters limitations in the regime of lower Knudsen numbers (Kn < 0. 05). In such cases, approaches from classical computational fluid dynamics (CFD) relying on the continuum assumption are preferred, offering accurate solutions at acceptable computational costs. In experiments aimed at imaging aerosolized nanoparticles in vacuo a wide range of Knudsen numbers occur, which motivated the present study on the analysis of the advantages and drawbacks of DSMC and CFD simulations of rarefied flows in terms of accuracy and computational effort. Furthermore, the potential of hybrid methods is evaluated. For this purpose, DSMC and CFD simulations of the flow inside a convergent–divergent nozzle (internal expanding flow) and the flow around a conical body (external shock generating flow) were carried out. CFD simulations utilize the software OpenFOAM and the DSMC solution is obtained using the software SPARTA. The results of these simulation techniques are evaluated by comparing them with experimental data (1), evaluating the time-to-solution (2) and the energy consumption (3), and assessing the feasibility of hybrid CFD-DSMC approaches (4). • Accuracy and performance evaluation of flow predictions using CFD and DSMC. • Low density internal expanding flow and external shock-generating flow. • Sensitivity study of the DSMC simulations concerning various parameters. • Strong/weak scaling studies analyzing the comput. speedup and energy consumption. • Exploitation of an optimal parametric setup for hybrid DSMC/CFD simulations. [ABSTRACT FROM AUTHOR]

Computational Fluid Dynamics (CFD) traditionally relies on long-standing numerical simulation strategies for the Navier–Stokes equations. Recently, interest in data-driven hybrid CFD solvers has spiked, leveraging pre-computed datasets to enhance various weak links inside existing solvers, such as closure models, under-resolved physics, or even to guide numerical resolution strategies. Running these hybrid solvers, notably in High Performance Computing (HPC) environments, presents specific challenges. In particular, context-aware deep learning (e.g. Convolutional (CNN) and Graph (GNN) Neural Networks) is promising for this task, but requires passing data representations between the physics solver and the neural network. In relevant industrial configurations, CFD meshes can be Cartesian but highly irregular, or unstructured, both of which do not match the pixel/voxel structure needed to run CNNs. In addition, discrepancies in programming language and libraries are common between CFD and machine learning applications. This work explores the many challenges of running a parallel hybrid solver in an HPC context, through the coupling of the AVBP CFD solver with neural networks in turbulent combustion and wall friction modeling applications. The knowledge gained is showcased in this article, as well as assembled in an actionable open-source library. • Strategies to associate traditional CFD solvers with data-driven approaches. • High-performance coupling of massively parallel CFD solver to distributed context-aware deep neural networks. • Exploitation of hybrid CPU/GPU architectures, with CFD computations of CPU and deep learning inference on GPU. [ABSTRACT FROM AUTHOR]

Direct numerical simulation (DNS) of fluid flow problems has been one of the most important applications of high-performance computing (HPC) in the last decades. For example, turbulent flows require the simultaneous resolution of multiple spatial and temporal scales as all scales are coupled, resulting in very large simulations with enormous degrees of freedom. Another example is reactive flows, which typically result in a large system of coupled differential equations and multiple transport equations that must be solved simultaneously. In addition, many flows exhibit chaotic behavior, meaning that only statistical ensembles of results can be compared, further increasing the computational time. In this work, a combined HPC/deep learning (DL) workflow is presented that drastically reduces the overall computational time required while still providing acceptable accuracy. Traditionally, all the simulations required to compute ensemble statistics are performed using expensive DNS. The idea behind the combined HPC/DL workflow is to reduce the number of expensive DNSs by developing a DL-assisted large-eddy simulation (LES) approach that uses a sophisticated DL network, called PIESRGAN, as a subfilter model for all unclosed terms and is accurate enough to substitute DNSs. The remaining DNSs are thus used in two ways: first, as data contributing to the ensemble statistics, and second, as data used to train the DL network. It was found that in many cases two remaining DNSs are sufficient for training the LES approach. The cost of the DL-supported LES is usually more than one order of magnitude cheaper than the DNS, which drastically speeds up the workflow, even considering the overhead for training the DL network. • Presentation of a novel modeling approach based on AI super-resolution with GANs. • Very good prediction quality (a priori and a posteriori). • Description of HPC aspects of modeling approach for state-of-the-art supercomputers. • Application of AI super-resolution to quantify cycle-to-cycle variations. • Description of a highly efficient new evaluation approach. [ABSTRACT FROM AUTHOR]

Previous implementation of the full kinetic particle-in-cell (PIC) simulation on the Graphical Processing Unit (GPU) device has shown advantages compared to traditional Central Processing Unit (CPU) computing. However, conventional PIC simulations using GPU computing have faced limitations in terms of low performance when the simulation box size or the number of particles per cell increases. In this study, we boost computing efficiency by designing novel schemes of kernels with the combination of numerical and technical aspects. The time expenses of the global data transferring and duplicate data fetching processes have been massively reduced by the utilization of the on-chip memory. The reduction treatment on the particles, the graded current computing scheme, and the 2.5D thread launch strategy are designed to accelerate the simulation iterations. The new scheme can reach up to about 5.5 times the acceleration ratio than the old one and attain the highest to 734 times faster than the program using CPUs. Our new scheme can realize large-scale PIC simulations with both decent performances on the legacy GPUs and the increasing trend of the acceleration ratio on the lasted GPUs. [ABSTRACT FROM AUTHOR]

This paper considers discovering frequent itemsets in transactional databases and addresses the time complexity problem by using high performance computing (HPC). Three HPC versions of the Single Scan (SS) algorithm are proposed. The first one (GSS) implements SS on a GPU (Graphics Processing Unit) architecture using an efficient mapping between thread blocks and the input data. The second approach (CSS) implements SS on a cluster architecture by scheduling independent jobs to workers in a cluster. The third, (CGSS) accelerates the frequent itemset mining process by using multiple cluster nodes equipped with GPUs. Moreover, three partitioning strategies are proposed to reduce GPU thread divergence and cluster load imbalance. Results show that CGSS outperforms SS, GSS, and CSS in terms of speedup. Specifically, CGSS provides up to a 350 times speedup for low minimum support values on large datasets. GCSS demonstrably outperforms the state-of-the-art HPC-based algorithms on big databases. [ABSTRACT FROM AUTHOR]

Scalable and efficient numerical simulations continue to gain importance, as computation is firmly established as the third pillar of discovery, alongside theory and experiment. Meanwhile, the performance of computing hardware grows through increasingly heterogeneous parallelism, enabling simulations of ever more complex models. However, efficiently implementing scalable codes on heterogeneous, distributed hardware systems becomes the bottleneck. This bottleneck can be alleviated by intermediate software layers that provide higher-level abstractions closer to the problem domain, reducing development times and allowing computational scientists to focus. Here, we present OpenFPM, an open and scalable framework that provides an abstraction layer for numerical simulations using particles and/or meshes. OpenFPM provides transparent and scalable infrastructure for shared-memory and distributed-memory implementations of particles-only and hybrid particle-mesh simulations of both discrete and continuous models, as well as non-simulation codes. This infrastructure is complemented with frequently used numerical routines, as well as interfaces to third-party libraries. We present the architecture and design of OpenFPM, detail the underlying abstractions, and benchmark the framework in applications ranging from Smoothed-Particle Hydrodynamics (SPH) to Molecular Dynamics (MD), Discrete Element Methods (DEM), Vortex Methods, stencil codes (finite differences), and high-dimensional Monte Carlo sampling (CMA-ES), comparing it to the current state of the art and to existing software frameworks. Program Title: OpenFPM Program Files doi: http://dx.doi.org/10.17632/4yrp8nbm7c.1 Licensing provisions: GPLv3 Programming language: C++ Nature of problem: Writing numerical simulation programs that use meshes, particles, or any combination of the two typically requires long development times, in particular if the code is to scale efficiently on parallel distributed-memory computers. The long development times incur high financial and project-time costs and often lead to sub-optimal program performance as shortcuts are taken. Yet, a large portion of the functionality is common across programs and could be automated or provided as reusable software components, leading to large savings in project costs and potentially improved software performance. Solution method: OpenFPM provides a scalable, highly efficient software platform for numerical simulations using meshes, particles, or any combination of the two on parallel computers. It is based on a well-known set of abstract data types and operators that suffice to express any such simulation, regardless of the application domain. OpenFPM provides reusable, tested, and internally parallelized software components that reduce development times and make parallel computing accessible to computational scientists without extensive knowledge in parallel programming. Additional comments including restrictions and unusual features: OpenFPM is a software library based on which users can implement their simulation codes at a fraction of the development cost. All parallelization and memory handling is transparently done by the library. As its main innovation, OpenFPM makes use of C++ Template Meta Programming in order to enable simulations in arbitrary-dimensional spaces, distribution of arbitrary user-defined C++ objects, and compile-time code optimization and targeting for specific hardware platforms. OpenFPM-based simulations can directly output VTK files for visualization of results and HDF5 files for data archiving. [ABSTRACT FROM AUTHOR]

Abstract Feltor is a modular and free scientific software package. It allows developing platform independent code that runs on a variety of parallel computer architectures ranging from laptop CPUs to multi-GPU distributed memory systems. Feltor consists of both a numerical library and a collection of application codes built on top of the library. Its main targets are two- and three-dimensional drift- and gyro-fluid simulations with discontinuous Galerkin methods as the main numerical discretization technique. We observe that numerical simulations of a recently developed gyro-fluid model produce non-deterministic results in parallel computations. First, we show how we restore accuracy and bitwise reproducibility algorithmically and programmatically. In particular, we adopt an implementation of the exactly rounded dot product based on long accumulators, which avoids accuracy losses especially in parallel applications. However, reproducibility and accuracy alone fail to indicate correct simulation behavior. In fact, in the physical model slightly different initial conditions lead to vastly different end states. This behavior translates to its numerical representation. Pointwise convergence, even in principle, becomes impossible for long simulation times. We briefly discuss alternative methods to ensure the correctness of results like the convergence of reduced physical quantities of interest, ensemble simulations, invariants or reduced simulation times. In a second part, we explore important performance tuning considerations. We identify latency and memory bandwidth as the main performance indicators of our routines. Based on these, we propose a parallel performance model that predicts the execution time of algorithms implemented in Feltor and test our model on a selection of parallel hardware architectures. We are able to predict the execution time with a relative error of less than 25% for problem sizes between 1 0 − 1 and 1 0 3 MB. Finally, we find that the product of latency and bandwidth gives a minimum array size per compute node to achieve a scaling efficiency above 50% (both strong and weak). [ABSTRACT FROM AUTHOR]

Abstract The ab initio GW approach is a rigorous Green's-function-based framework that can be employed to compute electronic excitation properties of a wide variety of materials such as extended systems, molecules, as well as confined and nanostructured materials with a very satisfactory accuracy. However, GW calculations on complex systems are often hindered by the high computational cost associated with the method. Here, we demonstrate how to significantly speedup GW calculations with a novel algorithm for the computationally intense kernel based on a non-blocking chunked cyclic communication scheme that minimizes latency in MPI messages, allows for overlapping of communication and computation, and improves cache usage in matrix multiplication operations. The optimized version of the code, implemented in the BerkeleyGW software package, is capable of scaling well to the full Cori computer at NERSC (Cray XC40) and achieves over 11 Peta FLOP/s of sustained performance. We showcase our work by performing large-scale GW calculations of defect structures on silicon, which require simulation cells containing over 1700 atoms, and which can now be efficiently executed in just a few minutes on large pre-exascale high-performance computing systems. [ABSTRACT FROM AUTHOR]

Strong-field quantum electrodynamics (SFQED) processes are central in determining the dynamics of particles and plasmas in extreme electromagnetic fields such as those present in the vicinity of compact astrophysical objects or generated with ultraintense lasers. SFQEDtoolkit is an open source library designed to allow users for a straightforward implementation of SFQED processes in existing particle-in-cell (PIC) and Monte Carlo codes. Through advanced function approximation techniques, high-energy photon emission and electron-positron pair creation probability rates and energy distributions are calculated within the locally-constant-field approximation (LCFA) as well as with more advanced models [Phys. Rev. A 99 , 022125 (2019)]. SFQEDtoolkit is designed to provide users with high-performance and high-accuracy, and neat examples showing its usage are provided. In the near future, SFQEDtoolkit will be enriched to model the angular distribution of the generated particles, i.e., beyond the commonly employed collinear emission approximation, as well as to model spin and polarization dependent SFQED processes. Notably, the generality and flexibility of the presented function approximation approach makes it suitable to be employed in other areas of physics, chemistry and computer science. Program Title: SFQEDtoolkit (version 1.0) CPC Library link to program files: https://doi.org/10.17632/j7mhj3cg3m.1 Developer's repository link: https://github.com/QuantumPlasma/SFQEDtoolkit Licensing provisions: GPLv3 Programming language: C++, Fortran Nature of problem: The accurate simulation of strong-field QED processes in the interaction of particles or plasmas with strong electromagnetic fields is critical for present and future experiments with ultraintense lasers as well as to unveil the dynamics around compact astrophysical objects. State-of-the-art PIC and Monte Carlo codes implement strong-field QED processes by means of lookup tables, where the approximated function is evaluated on a fixed set of points. This typically results in numerical artifacts such as stairlike structures in the approximated function, and in a reduced efficiency when large size lookup tables are necessary. An easily implementable open-source accurate and efficient simulation tool is required to address problems at the high-intensity frontier of plasma physics and astrophysics. Solution method: Strong-field QED process rates and particle distributions are calculated by leveraging advanced approximation methods based on variable and function transformation, asymptotic expansions as well as Chebyshev polynomial expansions. This provides a numerically-stable and efficient approach that allows us to overcome all limitations of existing methods based on standard lookup tables. • Computationally efficient and numerically stable function approximation technique. • Ideally suited for the inverse transform sampling method. • Variable and function transformation for function smoothing. • Strong-field QED processes in extreme electromagnetic fields and plasmas. • Enables accurate QED cascade simulations and relativistic laboratory astrophysics. [ABSTRACT FROM AUTHOR]

Highlights • A High-Order Compact Scheme solver for stratified rotating flows is presented. • A parallelization strategy is proposed for a HOCS discretisation of flow equations. • The code is able to capture strato-rotational and baroclinic instabilities. Abstract To take advantage of modern generation computing hardware, a scalable numerical method, based on higher-order compact scheme, is described to solve rotating stratified flows in cylindrical annular domains. An original approach combining 2d-pencil decomposition and reduced Parallel Diagonal Dominant is proposed to improve the parallelization performance during the computation of Poisson/Helmholtz solvers and time explicit terms. The developed technique is validated with respect to analytical solutions, using the method of manufactured solutions, and available data for two specific configurations. The purpose is to demonstrate its ability to correctly capture the flow characteristics in strato-rotational instability and in baroclinic instability with associated small-scale features. Moreover, this code is found to drastically reduce the huge execution times often preventing detailed numerical investigations of these complex phenomena. Strong scaling test is carried out to assess the performance for up to 1024 cores using grid up to 128 × 568 × 568 in radial, axial and azimuthal directions. [ABSTRACT FROM AUTHOR]

Smilei is a collaborative, open-source, object-oriented (C++) particle-in-cell code. To benefit from the latest advances in high-performance computing (HPC), Smilei is co-developed by both physicists and HPC experts. The code’s structures, capabilities, parallelization strategy and performances are discussed. Additional modules (e.g. to treat ionization or collisions), benchmarks and physics highlights are also presented. Multi-purpose and evolutive, Smilei is applied today to a wide range of physics studies, from relativistic laser–plasma interaction to astrophysical plasmas. Program summary Program title: Smilei (version 3.2) Program Files doi: http://dx.doi.org/10.17632/gsn4x6mbrg.1 Licensing provisions: This version of the code is distributed under the GNU General Public License v3 Programming language: C++11, Python 2.7 Nature of the problem: The kinetic simulation of plasmas is at the center of various physics studies, from laser–plasma interaction to astrophysics. To address today’s challenges, a versatile simulation tool requires high-performance computing on massively parallel super-computers. Solution method: The Vlasov–Maxwell system describing the self-consistent evolution of a collisionless plasma is solved using the Particle-In-Cell (PIC) method. Additional physics modules allow to account for additional effects such as collisions and/or ionization. A hybrid MPI-OpenMP strategy, based on a patch-based super-decomposition, allows for efficient cache-use, dynamic load balancing and high-performance on massively parallel super-computers. Additional comments: Repository https://github.com/SmileiPIC/Smilei References: http://www.maisondelasimulation.fr/smilei [ABSTRACT FROM AUTHOR]

Particle-laden flows occur in a wide range of disciplines, from atmospheric flows to renewable energy to turbomachinery. They generally pose a challenging environment for the numerical prediction of particle-induced phenomena due to their often complex geometry and highly instationary flow field which covers a wide range of spatial and temporal scales. At the same time, confidence in the evolution of the particulate phase is crucial for the reliable prediction of non-linear effects such as erosion and fouling. As a result, the multiscale nature requires the time-accurate integration of the flow field and the dispersed phase, especially in the presence of transition and separation. In this work, we present the extension of the open-source high-order accurate CFD framework FLEXI towards particle-laden flows. FLEXI is a massively parallel solver for the compressible Navier-Stokes-Fourier equations which operates on (un-)structured grids including curved elements and hanging nodes. An efficient particle tracking approach in physical space based on methods from ray-tracing is employed to handle intersections with curved boundaries. We describe the models for a one- and two-way coupled dispersed phase and their numerical treatment, where particular emphasis is placed on discussing the background and motivation leading to specific implementation choices. Special care is taken to retain the excellent scaling properties of FLEXI on high performance computing infrastructures during the complete tool chain including high-order accurate post-processing. Finally, we demonstrate the applicability of the extended framework to large-scale problems. [ABSTRACT FROM AUTHOR]

The Tersoff potential is one of the empirical many-body potentials that has been widely used in simulation studies at atomic scales. Unlike pair-wise potentials, the Tersoff potential involves three-body terms, which require much more arithmetic operations and data dependency. In this contribution, we have implemented the GPU-accelerated version of several variants of the Tersoff potential for LAMMPS, an open-source massively parallel Molecular Dynamics code. Compared to the existing MPI implementation in LAMMPS, the GPU implementation exhibits a better scalability and offers a speedup of 2.2X when run on 1000 compute nodes on the Titan supercomputer. On a single node, the speedup ranges from 2.0 to 8.0 times, depending on the number of atoms per GPU and hardware configurations. The most notable features of our GPU-accelerated version include its design for MPI/accelerator heterogeneous parallelism, its compatibility with other functionalities in LAMMPS, its ability to give deterministic results and to support both NVIDIA CUDA- and OpenCL-enabled accelerators. Our implementation is now part of the GPU package in LAMMPS and accessible for public use. [ABSTRACT FROM AUTHOR]

One of the greatest efforts of computational scientists is to translate the mathematical model describing a class of physical phenomena into large and complex codes. Many of these codes face the difficulty of implementing the mathematical operations in the model in terms of low level optimized kernels offering both performance and portability. Legacy codes suffer from the additional curse of rigid design choices based on outdated performance metrics (e.g. minimization of memory footprint). Using a representative code from the Materials Science community, we propose a methodology to restructure the most expensive operations in terms of an optimized combination of dense linear algebra (BLAS3) kernels. The resulting algorithm guarantees an increased performance and an extended life span of this code, enabling larger scale simulations. [ABSTRACT FROM AUTHOR]

Material irradiation effect is one of the most important keys to use nuclear power. However, the lack of high-throughput irradiation facility and knowledge of evolution process, lead to little understanding of the addressed issues. With the help of high-performance computing, we could make a further understanding of micro-level-material. In this paper, a new data structure is proposed for the massively parallel simulation of the evolution of metal materials under irradiation environment. Based on the proposed data structure, we developed the new molecular dynamics software named Crystal MD. The simulation with Crystal MD achieved over 90% parallel efficiency in test cases, and it takes more than 25% less memory on multi-core clusters than LAMMPS and IMD, which are two popular molecular dynamics simulation software. Using Crystal MD, a two trillion particles simulation has been performed on Tianhe-2 cluster. [ABSTRACT FROM AUTHOR]

In an effort to determine which Computational Fluid Dynamics method offers the best trade-off between accuracy and computational cost for aerodynamic and aeroacoustic applications, the lattice Boltzmann and finite-volume Navier–Stokes methods are compared. Unlike previous studies, the present framework enables a fair and unbiased comparison of the core capabilities of each numerical approach and focuses on schemes of practical relevance. With the aim of providing a comprehensive comparison of the methods, an extended von Neumann analysis is performed and the High–Performance Computing capacities of both methods are thoroughly discussed. In addition, it is also shown through computations on canonical test cases that a "time to solution" metric has to be considered in order to objectively assess the suitability of one particular numerical method over the other. Three main conclusions are drawn: (1) both the lattice Boltzmann and Navier–Stokes schemes exhibit an anisotropic dissipative behaviour, (2) a cell update using the lattice Boltzmann method is 2 to 3 times faster than with the Navier–Stokes method dedicated to cartesian grids, and (3) the use of the "time to solution" metric demonstrates that the relevance of one method over the other is closely linked to the underlying physics and the intended error target. In light of these results, decision aids are provided to assist in selecting the most efficient method for a given application. • Theoretical and numerical one-to-one comparison of the LB and FV-NS methods. • A linear stability analysis is performed to study the methods' dissipative behaviour. • The core HPC capabilities of each method are thoroughly investigated. • A "time to solution" metric is introduced to fairly compare the LB and NS methods. • Decision aids are given to select the most efficient method for a given application. [ABSTRACT FROM AUTHOR]

High-order methods are being recognized as powerful tools for handling scale-resolving simulations over complex geometry. However, several obstacles still block their complete applications to practical engineering problems: a compromise between accuracy and efficiency on mixed-curved meshes, inherent vulnerability to numerical oscillations, and lack of open-source high-performance solvers for researchers. To address these issues, we present Deneb, an open-source high-order accurate numerical solver that enables high-performance scale-resolving simulations on PDE-based flow systems. Deneb uses the physical domain-based modal discontinuous Galerkin (DG) method; thus, it can provide an arbitrary high-order accurate solution on mixed-curved meshes and has the potential for handling polyhedral meshes as well. The direct reconstruction method (DRM) efficiently executes the numerical integration of DG volume and surface integrals without accuracy loss on non-affine elements where mapping functions are high-degree. The resulting DRM-DG method eliminates the severe cost of a quadrature-based approach on mixed-curved meshes. Deneb offers explicit and implicit Runge–Kutta methods as well to achieve high-order accuracy in time. In addition, Krylov subspace methods and preconditioners are available for high-performance linear system solving in parallel. Deneb possesses a strong capability to resolve multi-physical shocks without numerical instabilities with the aid of multi-dimensional limiting and artificial viscosity methods. In particular, the hierarchical multi-dimensional limiting process enables efficient computations of supersonic flows without time-step restriction. The current release of Deneb covers the simulations of hypersonic equilibrium and magneto-hydrodynamic flows as well as compressible Navier–Stokes equations, but it has the potential to solve any PDE-based multi-physical flow systems. Several benchmark problems are presented to highlight Deneb's capability to perform scale-resolving and multi-physical flow simulations. A scalability test is also presented to verify the scaling characteristics of Deneb for high-performance computing. Program title: Deneb CPC library link to program files: https://doi.org/10.17632/723n5r797n.1 Developer's repository link: https://github.com/HojunYouKr/Deneb Licensing provisions: BSD-3-Clause Programming language: C++17 Nature of problem: The physical domain-based modal DG method can achieve the expected order of accuracy with the optimal number of polynomial bases even on non-affine elements. However, the DG method becomes significantly expensive on high-order curved elements when using quadrature rules, blocking its applicability to practical engineering problems. The numerical integration should be much more efficient without compromising accuracy. In addition, the less diffusive nature of high-order methods makes them susceptible to producing spurious numerical oscillations near flow discontinuities, potentially leading to numerical instabilities. Thus, an accurate and robust shock-capturing method is essential to simulate multi-physical flows under compressible regimes. Finally, the solver needs high scalability to perform large-scale computations in parallel. Solution method: DRM is applied to the DG volume and surface integrals to perform efficient numerical integration on non-affine elements without accuracy loss. The resulting method, DRM-DG, provides arbitrary high-order accurate solutions to various PDE-based flow problems on mixed-curved meshes. The solution is also high-order accurate in time due to high-order explicit and implicit Runge–Kutta methods implemented. The external library enables high-performance linear system solving with various preconditioners in parallel. Both multi-dimensional limiting and artificial viscosity methods suppress unwanted subcell oscillations across physical discontinuities. In particular, the limiting methods simulate complex supersonic flows efficiently without time-step restriction. The solver is highly scalable on parallel computing with the aid of non-blocking communications and latency hiding. [ABSTRACT FROM AUTHOR]

We present the Fluid Transport Accelerated Solver, FluTAS, a scalable GPU code for multiphase flows with thermal effects. The code solves the incompressible Navier-Stokes equation for two-fluid systems, with a direct FFT-based Poisson solver for the pressure equation. The interface between the two fluids is represented with the Volume of Fluid (VoF) method, which is mass conserving and well suited for complex flows thanks to its capacity of handling topological changes. The energy equation is explicitly solved and coupled with the momentum equation through the Boussinesq approximation. The code is conceived in a modular fashion so that different numerical methods can be used independently, the existing routines can be modified, and new ones can be included in a straightforward and sustainable manner. FluTAS is written in modern Fortran and parallelized using hybrid MPI/OpenMP in the CPU-only version and accelerated with OpenACC directives in the GPU implementation. We present different benchmarks to validate the code, and two large-scale simulations of fundamental interest in turbulent multiphase flows: isothermal emulsions in HIT and two-layer Rayleigh-Bénard convection. FluTAS is distributed through a MIT license and arises from a collaborative effort of several scientists, aiming to become a flexible tool to study complex multiphase flows. Program Title: : Fluid Transport Accelerated Solver, FluTAS. CPC Library link to program files: https://doi.org/10.17632/tp6k8wky8m.1 Developer's repository link: https://github.com/Multiphysics-Flow-Solvers/FluTAS.git. Licensing provisions: MIT License. Programming language: Fortran 90, parallelized using MPI and slab/pencil decomposition, GPU accelerated using OpenACC directives. External libraries/routines: FFTW, cuFFT. Nature of problem: FluTAS is a GPU-accelerated numerical code tailored to perform interface resolved simulations of incompressible multiphase flows, optionally with heat transfer. The code combines a standard pressure correction algorithm with an algebraic volume of fluid method, MTHINC [1]. Solution method: the code employs a second-order-finite difference discretization and solves the two-fluid Navier-Stokes equation using a projection method. It can be run both on CPU-architectures and GPU-architectures. [ABSTRACT FROM AUTHOR]

We proposed a simple one-step index (OSI) algorithm for solving the lattice Boltzmann equation, particularly the streaming of particle distribution functions (PDFs) on a single grid system. The OSI algorithm is derived from the conventional A-B pattern. The memory addresses of the PDFs are fixed in this algorithm and consistent with collision principles. The streaming process is implicitly computed by reassigning their indexes corresponding to the time steps, spatial coordinates, and directions of the PDFs. The algorithm is simple to program because it reads and writes the PDFs only once per time step and does not require the synchronization of odd and even time steps. In this implementation, the data layout of the PDFs is the structure of arrays (SoA), suitable for the memory access pattern of graphics processing units (GPUs). The accuracy and single-precision performance of the proposed algorithm for the three-dimensional lid-driven cavity flow simulation with the D3Q19 model were validated and tested on an NVIDIA A100 having a 40 GB PCIe using CUDA and OpenACC. Performances of 8.4 and 8.1 giga lattice updates per second were obtained for CUDA and OpenACC, respectively. OpenACC can outperform CUDA by up to 95% with significantly less programming work. The bandwidth usage rates on a single GPU were 96% and 94% for CUDA and OpenACC, respectively, close to the theoretical values. Lattice Boltzmann method parallelism is implemented using CUDA and MPI for multi-GPU usage. Finally, computation and communication overlaps were implemented to optimize the parallel efficiency, where the weak scaling parallel efficiency exceeded 0.98 on up to 512 GPUs. • The algorithm integrated LB collision and streaming in a single step on one grid. • The PDFs are only read and written once every time step. • The performance is 8.4 and 8.1 GLUPS on A100 using CUDA and OpenACC, respectively. • The weak scaling parallel efficiency is higher than 0.98 on up to 512 GPUs. [ABSTRACT FROM AUTHOR]

A common feature of computational modelling and simulation research is the need to perform many tasks in complex sequences to achieve a usable result. This will typically involve tasks such as preparing input data, pre-processing, running simulations on a local or remote machine, post-processing, and performing coupling communications, validations and/or optimisations. Tasks like these can involve manual steps which are time and effort intensive, especially when it involves the management of large ensemble runs. Additionally, human errors become more likely and numerous as the research work becomes more complex, increasing the risk of damaging the credibility of simulation results. Automation tools can help ensure the credibility of simulation results by reducing the manual time and effort required to perform these research tasks, by making more rigorous procedures tractable, and by reducing the probability of human error due to a reduced number of manual actions. In addition, efficiency gained through automation can help researchers to perform more research within the budget and effort constraints imposed by their projects. This paper presents the main software release of FabSim3, and explains how our automation toolkit can improve and simplify a range of tasks for researchers and application developers. FabSim3 helps to prepare, submit, execute, retrieve, and analyze simulation workflows. By providing a suitable level of abstraction, FabSim3 reduces the complexity of setting up and managing a large-scale simulation scenario, while still providing transparent access to the underlying layers for effective debugging. The tool also facilitates job submission and management (including staging and curation of files and environments) for a range of different supercomputing environments. Although FabSim3 itself is application-agnostic, it supports a provably extensible plugin system where users automate simulation and analysis workflows for their own application domains. To highlight this, we briefly describe a selection of these plugins and we demonstrate the efficiency of the toolkit in handling large ensemble workflows. Program Title: FabSim3 CPC Library link to program files: https://doi.org/10.17632/6nfrwy7ptj.1 Licensing provisions: BSD 3-clause Programming language: Python 3 Nature of problem: Many aspects are crucial for obtaining reproducible and robust simulation results. For instance, we need to curate all the inputs and outputs for later scrutiny, scrutinize the model behaviour under slightly perturbed circumstances, quantify the propagation of key uncertainties from input data and known parameters and analyze the sensitivity for any parameters for which the exact specification eludes us. Solution method: FabSim3 uses a range of methods to provide automation. These primarily include: (i) SSH + Fabric2 to enable remote execution of SSH commands, (ii) an internal parameter state space using primarily Python dict objects that can be customized with machine- plugin- and user-specific modifications, (iii) Python templating to quickly enable the insertion of state space variables into supercomputing scripts, (iv) multiprocessing and/or QCG-PilotJob to enable efficient submission and execution of job arrays and (v) a system of flexibly installable and modifiable Python3 plugins which allows users to create and customize application-specific functionalities without modifying the core code base. In addition to the written code, FabSim3 also relies on a set of user conventions to maintain a separation of concerns (particularly between machine-, user- and application-specific settings). Additional comments including restrictions and unusual features: This paper serves as the definitive reference for FabSim3. [ABSTRACT FROM AUTHOR]

The exascale era of high performance computing promises to bring the field of computational plasma physics ever closer to the goal of accurate multiscale modeling. Such computers will rely on hardware acceleration to offload work to dedicated components, notably general-purpose graphics processing units (GPUs). However, devices from different manufacturers require software to be written with different parallel programming models, greatly increasing the code maintenance burden of applications designed to perform on more than one such device. hPIC2 is a hybrid plasma simulation code developed with the Kokkos performance portability framework to target the architectures that will drive exascale computing for the foreseeable future. As a hybrid simulation code, hPIC2 investigates the simultaneous use of various plasma models on the same domain, at the same time. hPIC2 also optionally couples to RustBCA, which accurately models ion-material interactions using the binary collision approximation (BCA) method [1]. hPIC2 therefore achieves scalable performance on a variety of computing architectures when simulating complex and diverse plasmas, particularly near plasma-material interfaces. [ABSTRACT FROM AUTHOR]

This paper presents a load balancing strategy for reaction rate evaluation and chemistry integration in reacting flow simulations. The large disparity in scales during combustion introduces stiffness in the numerical integration of the PDEs and generates load imbalance during the parallel execution. The strategy is based on the use of the DLB library to redistribute the computing resources at node level, lending additional CPU-cores to higher loaded MPI processes. This approach does not require explicit data transfer and is activated automatically at runtime. Two chemistry descriptions, detailed and reduced, are evaluated on two different configurations: laminar counterflow flame and a turbulent swirl-stabilized flame. For single-node calculations, speedups of 2.3x and 7x are obtained for the detailed and reduced chemistry, respectively. Results on multi-node runs also show that DLB improves the performance of the pure-MPI code similar to single node runs. It is shown DLB can get performance improvements in both detailed and reduced chemistry calculations. • A load balancing strategy for reaction rate and chemistry integration is presented. • It uses the DLB library to redistribute the computational resources at node level. • Code hybridization improves the performance over MPI-pure implementations. • Single-node and multi-node tests show speedups in reacting flow calculations. [ABSTRACT FROM AUTHOR]

This paper addresses non-linear gyrokinetic simulations of ion temperature gradient (ITG) turbulence in tokamak plasmas. The electrostatic Gysela code is one of the few international 5D gyrokinetic codes able to perform global, full- f and flux-driven simulations. Its has also the numerical originality of being based on a semi-Lagrangian (SL) method. This reference paper for the Gysela code presents a complete description of its multi-ion species version including: (i) numerical scheme, (ii) high level of parallelism up to 500k cores and (iii) conservation law properties. [ABSTRACT FROM AUTHOR]

We present an eigenspectrum partitioning scheme without inversion for the recently described real-space electronic transport code, TRANSEC. The primary advantage of TRANSEC is its highly parallel algorithm, which enables studying conductance in large systems. The present scheme adds a new source of parallelization, significantly enhancing TRANSEC’s parallel scalability, especially for systems with many electrons. In principle, partitioning could enable super-linear parallel speedup, as we demonstrate in calculations within TRANSEC. In practical cases, we report better than five-fold improvement in CPU time and similar improvements in wall time, compared to previously-published large calculations. Importantly, the suggested scheme is relatively simple to implement. It can be useful for general large Hermitian or weakly non-Hermitian eigenvalue problems, whenever relatively accurate inversion via direct or iterative linear solvers is impractical. [ABSTRACT FROM AUTHOR]

The recently developed semi-Lagrangian discontinuous Galerkin approach is used to discretize hyperbolic partial differential equations (usually first order equations). Since these methods are conservative, local in space, and able to limit numerical diffusion, they are considered a promising alternative to more traditional semi-Lagrangian schemes (which are usually based on polynomial or spline interpolation). In this paper, we consider a parallel implementation of a semi-Lagrangian discontinuous Galerkin method for distributed memory systems (so-called clusters). Both strong and weak scaling studies are performed on the Vienna Scientific Cluster 2 (VSC-2). In the case of weak scaling we observe a parallel efficiency above 0.8 for both two and four dimensional problems and up to 8192 cores. Strong scaling results show good scalability to at least 512 cores (we consider problems that can be run on a single processor in reasonable time). In addition, we study the scaling of a two dimensional Vlasov–Poisson solver that is implemented using the framework provided. All of the simulations are conducted in the context of worst case communication overhead; i.e., in a setting where the CFL (Courant–Friedrichs–Lewy) number increases linearly with the problem size. The framework introduced in this paper facilitates a dimension independent implementation of scientific codes (based on C++ templates) using both an MPI and a hybrid approach to parallelization. We describe the essential ingredients of our implementation. Program summary Program title: sldg Catalogue identifier: AEZO_v1_0 Program summary URL: http://cpc.cs.qub.ac.uk/summaries/AEZO_v1_0.html Program obtainable from: CPC Program Library, Queen’s University, Belfast, N. Ireland Licensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.html No. of lines in distributed program, including test data, etc.: 320111 No. of bytes in distributed program, including test data, etc.: 5725614 Distribution format: tar.gz Programming language: C++03. Computer: PC, HPC systems. Operating system: POSIX compatible (extensively tested on various Linux systems). Only the timing class requires POSIX routines; all other parts of the program can be run on any system where a C++ compiler, Boost, and FFTW is available. RAM: Megabytes to Terabytes (depending on the problem size) Classification: 6.5, 4.3, 19.8. External routines: Boost, FFTW Nature of problem: An approximate solution of the advection equation is computed using the semi-Lagrangian discontinuous Galerkin method. This procedure is used as the fundamental building block in a splitting based Vlasov solver. Solution method: The semi-Lagrangian discontinuous Galerkin approach is employed. For the Vlasov–Poisson problem we, in addition, use a splitting approach in time and the FFT to solve Poisson’s equation. Restrictions: We assume that the hyperbolic part of the problem under consideration can be (for example, using a splitting approach) decomposed into a sequence of one-dimensional advections. Unusual features: The framework is independent of the dimension of the problem under consideration. This is accomplished using template techniques. Running time: From minutes to weeks (depending on the problem size). [ABSTRACT FROM AUTHOR]

A formulation of elliptic boundary value problems is used to develop the first discrete exterior calculus (DEC) library for massively parallel computations with 3D domains. This can be used for steady-state analysis of any physical process driven by the gradient of a scalar quantity, e.g. temperature, concentration, pressure or electric potential, and is easily extendable to transient analysis. In addition to offering this library to the community, we demonstrate one important benefit from the DEC formulation: effortless introduction of strong heterogeneities and discontinuities. These are typical for real materials, but challenging for widely used domain discretization schemes, such as finite elements. Specifically, we demonstrate the efficiency of the method for calculating the evolution of thermal conductivity of a solid with a growing crack population. Future development of the library will deal with transient problems, and more importantly with processes driven by gradients of vector quantities. Program Title: ParaGEMS CPC Library link to program files: https://doi.org/10.17632/t8rpkc7xnx.1 Developer's repository link: https://bitbucket.org/pieterboom/paragems/ Licensing provisions: BSD 2-clause Programming language: Fortran 90 External routines/libraries: BLAS/LAPACK, MPI, PETSc [1], hypre ParaSails [2,3] Nature of problem: Large scale simulation of 3D elliptic boundary value problems in discrete media with heterogeneous or discontinuous properties. Solution method: ParaGEMS is a library implementing discrete exterior calculus (DEC) for distributed memory architectures. It is distributed with mini-applications for solving 3D elliptic boundary value problems, which describe a number of physical laws (Fourier's law, Fick's law, Darcy's law and Ohm's law), in discrete media with heterogeneous or discontinuous properties. Additional comments including restrictions and unusual features: ParaGEMS requires meshes generated in the default format used by Triangle [4] and TetGen [5]. [1] S. Balay et al., PETSc Web page, https://www.mcs.anl.gov/petsc , 2019. [2] R.D. Falgout, U.M. Yang, in: P.M.A. Sloot, A.G. Hoekstra, C.J.K. Tan, J.J. Dongarra (Eds.), Computational Science — ICCS 2002, Springer Berlin Heidelberg, Berlin, Heidelberg, 2002, pp. 632–641. [3] E. Chow, SIAM J. Sci. Comput. 21 (5) (2000) 1804–1822. [4] J.R. Shewchuk, in: M.C. Lin, D. Manocha (Eds.), Applied Computational Geometry: Towards Geometric Engineering, in: Lecture Notes in Computer Science, vol. 1148, Springer-Verlag, 1996, pp. 203–222, from the First ACM Workshop on Applied Computational Geometry. [5] H. Si, ACM Trans. Math. Softw. 41 (2) (2015) 11, https://doi.org/10.1145/2629697. [ABSTRACT FROM AUTHOR]

We introduce a virtual machine (VM) written in a numerically fast language like Fortran or C for evaluating very large expressions. We discuss the general concept of how to perform computations in terms of a VM and present specifically a VM that is able to compute tree-level cross sections for any number of external legs, given the corresponding byte-code from the optimal matrix element generator, O’Mega . Furthermore, this approach allows to formulate the parallel computation of a single phase space point in a simple and obvious way. We analyze hereby the scaling behavior with multiple threads as well as the benefits and drawbacks that are introduced with this method. Our implementation of a VM can run faster than the corresponding native, compiled code for certain processes and compilers, especially for very high multiplicities, and has in general runtimes in the same order of magnitude. By avoiding the tedious compile and link steps, which may fail for source code files of gigabyte sizes, new processes or complex higher order corrections that are currently out of reach could be evaluated with a VM given enough computing power. [ABSTRACT FROM AUTHOR]

Several species of specialized bees possess special structures to store and transport floral oils. By using closely spaced hairs at their back legs, the so called scopa, these bees can absorb and release oil droplets without loss. The high efficiency of this process is a matter of ongoing research. Based on recent X-ray microtomography scans from a bee’s scopa, we build a three-dimensional geometric computer model. Then, using NaSt3DGPF , a two-phase flow solver developed at the Institute for Numerical Simulation, we compute the micro flow in the scopa model. Our calculations reveal the laminar to turbulent air flow in the scopa during flight. Furthermore, we simulate the deformation of an oil droplet in the scopa due to surface tension effects on a microscopic length scale. Our results are in good agreement with measurements for an oil-wetted scopa at steady state which are obtained from X-ray scans. Both simulations are relevant for the understanding of the process of oil absorption and transportation in the real scopa of a bee. Due to the large computational complexity of the problem, massively parallel computations are essential for our simulations. [ABSTRACT FROM AUTHOR]

High fidelity simulations of the flow phenomena around complex geometries for turbomachinery applications require fluid solvers to run on ever increasing processor counts. For fully unsteady predictions in rotor–stator systems most of CFD codes employ the sliding interface technique. However, the scalability and efficiency of current sliding grid parallel implementations are significantly constrained by the computation and communication imbalances. They are associated with data transfer across discrete non-matching interfaces. To prepare for the challenges at extreme scales in this paper we attempt to redesign the algorithm in such a way that it maintains the scalability of the original CFD code on static grids. In the proposed parallel implementation the cell containment search and interpolation workloads are balanced by employing a deterministic geometric decomposition on an intermediate “rendezvous” set of processes. Rapidly changing dynamic communication patterns induced by the grids relative motion are handled with a sparse communication protocol. The scaling behavior and performance of the developed technique are analyzed using realistic test cases on two different computing systems. [ABSTRACT FROM AUTHOR]

The article presents research carried out by Lawrence Livermore Laboratory and Texas A&M University which analyzes the advanced materials development and computational fluid dynamics (CFD) to improve treatment for intracranial aneurysms. It reveals the complex blood flow patterns that emerge within and around the closed design for shape-memory-polymer (SMP) foam. It also unveils that closed SMP design is tested to be effective at increasing blood residence time within an aneurysm.

Highlights: [•] We develop a generalised modelling framework for shallow water simulations. [•] The framework allows expedited simulations with graphics processing units (GPUs). [•] We consider differences in device architecture to optimise performance. [•] The new software simulates a real-life event with scalable performance on GPUs. [Copyright &y& Elsevier]

Abstract: The UNEDF project was a large-scale collaborative effort that applied high-performance computing to the nuclear quantum many-body problem. The primary focus of the project was on constructing, validating, and applying an optimized nuclear energy density functional, which entailed a wide range of pioneering developments in microscopic nuclear structure and reactions, algorithms, high-performance computing, and uncertainty quantification. UNEDF demonstrated that close associations among nuclear physicists, mathematicians, and computer scientists can lead to novel physics outcomes built on algorithmic innovations and computational developments. This review showcases a wide range of UNEDF science results to illustrate this interplay. [Copyright &y& Elsevier]

We describe the new version (v2.49t) of the code hfodd which solves the nuclear Skyrme–Hartree–Fock (HF) or Skyrme–Hartree–Fock–Bogolyubov (HFB) problem by using the Cartesian deformed harmonic-oscillator basis. In the new version, we have implemented the following physics features: (i) the isospin mixing and projection, (ii) the finite-temperature formalism for the HFB and HF+BCS methods, (iii) the Lipkin translational energy correction method, (iv) the calculation of the shell correction. A number of specific numerical methods have also been implemented in order to deal with large-scale multi-constraint calculations and hardware limitations: (i) the two-basis method for the HFB method, (ii) the Augmented Lagrangian Method (ALM) for multi-constraint calculations, (iii) the linear constraint method based on the approximation of the RPA matrix for multi-constraint calculations, (iv) an interface with the axial and parity-conserving Skyrme–HFB code hfbtho, (v) the mixing of the HF or HFB matrix elements instead of the HF fields. Special care has been paid to using the code on massively parallel leadership class computers. For this purpose, the following features are now available with this version: (i) the Message Passing Interface (MPI) framework, (ii) scalable input data routines, (iii) multi-threading via OpenMP pragmas, (iv) parallel diagonalization of the HFB matrix in the simplex-breaking case using the ScaLAPACK library. Finally, several little significant errors of the previous published version were corrected. New version program summary: Program title: hfodd (v2.49t) Catalogue identifier: ADFL_v3_0 Program summary URL: http://cpc.cs.qub.ac.uk/summaries/ADFL_v3_0.html Program obtainable from: CPC Program Library, Queenʼs University, Belfast, N. Ireland Licensing provisions: GNU General Public Licence v3 No. of lines in distributed program, including test data, etc.: 190 614 No. of bytes in distributed program, including test data, etc.: 985 898 Distribution format: tar.gz Programming language: FORTRAN-90 Computer: Intel Pentium-III, Intel Xeon, AMD-Athlon, AMD-Opteron, Cray XT4, Cray XT5 Operating system: UNIX, LINUX, Windows XP Has the code been vectorized or parallelized?: Yes, parallelized using MPI RAM: 10 Mwords Word size: The code is written in single-precision for the use on a 64-bit processor. The compiler option -r8 or +autodblpad (or equivalent) has to be used to promote all real and complex single-precision floating-point items to double precision when the code is used on a 32-bit machine. Classification: 17.22 Catalogue identifier of previous version: ADFL_v2_2 Journal reference of previous version: Comput. Phys. Comm. 180 (2009) 2361 External routines: The user must have access to [1.] the NAGLIB subroutine f02axe, or LAPACK subroutines zhpev, zhpevx, zheevr, or zheevd, which diagonalize complex hermitian matrices, [2.] the LAPACK subroutines dgetri and dgetrf which invert arbitrary real matrices, [3.] the LAPACK subroutines dsyevd, dsytrf and dsytri which compute eigenvalues and eigenfunctions of real symmetric matrices, [4.] the LINPACK subroutines zgedi and zgeco, which invert arbitrary complex matrices and calculate determinants, [5.] the BLAS routines dcopy, dscal, dgeem and dgemv for double-precision linear algebra and zcopy, zdscal, zgeem and zgemv for complex linear algebra, or provide another set of subroutines that can perform such tasks. The BLAS and LAPACK subroutines can be obtained from the Netlib Repository at the University of Tennessee, Knoxville: http://netlib2.cs.utk.edu/. Does the new version supersede the previous version?: Yes Nature of problem: The nuclear mean field and an analysis of its symmetries in realistic cases are the main ingredients of a description of nuclear states. Within the Local Density Approximation, or for a zero-range velocity-dependent Skyrme interaction, the nuclear mean field is local and velocity dependent. The locality allows for an effective and fast solution of the self-consistent Hartree–Fock equations, even for heavy nuclei, and for various nucleonic (n-particle–n-hole) configurations, deformations, excitation energies, or angular momenta. Similarly, Local Density Approximation in the particle–particle channel, which is equivalent to using a zero-range interaction, allows for a simple implementation of pairing effects within the Hartree–Fock–Bogolyubov method. Solution method: The program uses the Cartesian harmonic oscillator basis to expand single-particle or single-quasiparticle wave functions of neutrons and protons interacting by means of the Skyrme effective interaction and zero-range pairing interaction. The expansion coefficients are determined by the iterative diagonalization of the mean-field Hamiltonians or Routhians which depend non-linearly on the local neutron and proton densities. Suitable constraints are used to obtain states corresponding to a given configuration, deformation or angular momentum. The method of solution has been presented in: [J. Dobaczewski, J. Dudek, Comput. Phys. Commun. 102 (1997) 166]. Reasons for new version: Version 2.49s of HFODD provides a number of new options such as the isospin mixing and projection of the Skyrme functional, the finite-temperature HF and HFB formalism and optimized methods to perform multi-constrained calculations. It is also the first version of HFODD to contain threading and parallel capabilities. Summary of revisions: [1.] Isospin mixing and projection of the HF states has been implemented. [2.] The finite-temperature formalism for the HFB equations has been implemented. [3.] The Lipkin translational energy correction method has been implemented. [4.] Calculation of the shell correction has been implemented. [5.] The two-basis method for the solution to the HFB equations has been implemented. [6.] The Augmented Lagrangian Method (ALM) for calculations with multiple constraints has been implemented. [7.] The linear constraint method based on the cranking approximation of the RPA matrix has been implemented. [8.] An interface between HFODD and the axially-symmetric and parity-conserving code HFBTHO has been implemented. [9.] The mixing of the matrix elements of the HF or HFB matrix has been implemented. [10.] A parallel interface using the MPI library has been implemented. [11.] A scalable model for reading input data has been implemented. [12.] OpenMP pragmas have been implemented in three subroutines. [13.] The diagonalization of the HFB matrix in the simplex-breaking case has been parallelized using the ScaLAPACK library. [14.] Several little significant errors of the previous published version were corrected. Running time: In serial mode, running 6 HFB iterations for 152Dy for conserved parity and signature symmetries in a full spherical basis of shells takes approximately 8 min on an AMD Opteron processor at 2.6 GHz, assuming standard BLAS and LAPACK libraries. As a rule of thumb, runtime for HFB calculations for parity and signature conserved symmetries roughly increases as , where N is the number of full HO shells. Using custom-built optimized BLAS and LAPACK libraries (such as in the ATLAS implementation) can bring down the execution time by 60%. Using the threaded version of the code with 12 threads and threaded BLAS libraries can bring an additional factor 2 speed-up, so that the same 6 HFB iterations now take of the order of 2 min 30 s. [ABSTRACT FROM AUTHOR]

Abstract: In this article we present a high-performance computing framework for advanced flow simulation and its application to wind energy based on the residual-based variational multiscale (RBVMS) method and isogeometric analysis. The RBVMS formulation and its suitability and accuracy for turbulent flow in a moving domain are presented. Particular emphasis is placed on the parallel implementation of the methodology and its scalability. Two challenging flow cases were considered: the turbulent Taylor–Couette flow and the NREL 5MW offshore baseline wind turbine rotor at full scale. In both cases, flow quantities of interest from the simulation results compare favorably with the reference data and near-perfect linear parallel scaling is achieved. [Copyright &y& Elsevier]

Abstract: We present a certified reduced basis method for high-fidelity real-time solution of parametrized partial differential equations on deployed platforms. Applications include in situ parameter estimation, adaptive design and control, interactive synthesis and visualization, and individuated product specification. We emphasize a new hierarchical architecture particularly well suited to the reduced basis computational paradigm: the expensive Offline stage is conducted pre-deployment on a parallel supercomputer (in our examples, the TeraGrid machine Ranger); the inexpensive Online stage is conducted “in the field” on ubiquitous thin/inexpensive platforms such as laptops, tablets, smartphones (in our examples, the Nexus One Android-based phone), or embedded chips. We illustrate our approach with three examples: a two-dimensional Helmholtz acoustics “horn” problem; a three-dimensional transient heat conduction “Swiss Cheese” problem; and a three-dimensional unsteady incompressible Navier–Stokes low-Reynolds-number “eddy-promoter” problem. [Copyright &y& Elsevier]

We present a multi-block finite-difference solver for massively parallel Direct Numerical Simulations (DNS) of incompressible flows. The algorithm combines the versatility of a multi-block solver with the method of eigenfunctions expansions, to speedup the solution of the pressure Poisson equation. This is achieved by employing FFT-based transforms along one homogeneous direction, which effectively reduce the problem complexity at a low cost. These FFT-based expansions are implemented in a framework that unifies all valid combinations of boundary conditions for this type of method. Subsequently, a geometric multigrid solver is employed to solve the reduced Poisson equation in a multi-block geometry. Particular care was taken here, to guarantee the parallel performance of the multigrid solver when solving the reduced linear systems equations. We have validated the overall numerical algorithm and assessed its performance in a massively parallel setting. The results show that 2- to 8-fold reductions in computational cost may be easily achieved when exploiting FFT-accelerated for the solution of the Poisson equation. The solver, SNaC , has been made freely available and open-source under the terms of an MIT license. [ABSTRACT FROM AUTHOR]

The last ten years have witnessed fast spreading of massively parallel computing clusters, from leading supercomputing facilities down to the average university computing center. Many companies in the private sector have undergone a similar evolution. In this scenario, the seamless integration of software and middleware libraries is a key ingredient to ensure portability of scientific codes and guarantees them an extended lifetime. In this work, we describe the integration of the ChASE library, a modern parallel eigensolver, into an existing legacy code for the first-principles computation of optical properties of materials via solution of the Bethe-Salpeter equation for the optical polarization function. Our numerical tests show that, as a result of integrating ChASE and parallelizing the reading routine, the code experiences a remarkable speedup and greatly improved scaling behavior on both multi- and many-core architectures. We demonstrate that such a modernized BSE code will, by fully exploiting parallel computing architectures and file systems, enable domain scientists to accurately study complex material systems that were not accessible before. [ABSTRACT FROM AUTHOR]

Coupled-cluster Green's function (GFCC) calculation has drawn much attention in the recent years for targeting the molecular and material electronic structure problems from a many-body perspective in a systematically improvable way. However, GFCC calculations on scientific computing clusters usually suffer from expensive higher dimensional tensor contractions in the complex space, expensive inter-process communication, and severe load imbalance, which limits it's use for tackling electronic structure problems. Here we present a numerical library prototype that is specifically designed for large-scale GFCC calculations. The design of the library is focused on a systematically optimal computing strategy to improve its scalability and efficiency. The performance of the library is demonstrated by the relevant profiling analysis of running GFCC calculations on remote giant computing clusters. The capability of the library is highlighted by computing a wide near valence band of a fullerene C60 molecule for the first time at the GFCCSD level that shows excellent agreement with the experimental spectrum. Program Title: GFCCLib CPC Library link to program files: https://doi.org/10.17632/j594wydctd.1 Code Ocean capsule: https://doi.org/10.24433/CO.5131827.v1 Licensing provisions: MIT License Programming language: C++ Nature of problem: The applications of coupled cluster Green's function on large scale molecular electronic structure problems suffer from expensive higher dimensional tensor contractions in the complex space, expensive inter-process communication, and severe load imbalance. Tackling these issues are a key step in building high-performance coupled cluster Green's function library for its routine use in large scale molecular science. Solution method: We have developed a C++ library for large scale molecular GFCC calculations on high-performance computing clusters. We provide implementations for high dimensional tensor algebra for many-body methods (TAMM), Cholesky decomposition of high dimensional electron repulsion integral tensors, process group technique for mitigating load imbalance. The library is written in C++. The source code, tutorials and documentation are provided online. A continuous integration mechanism is set up to automatically run a series of regression tests and check code coverage when the codebase is updated. [ABSTRACT FROM AUTHOR]

The solution of eigenproblems is often a key computational bottleneck that limits the tractable system size of numerical algorithms, among them electronic structure theory in chemistry and in condensed matter physics. Large eigenproblems can easily exceed the capacity of a single compute node, thus must be solved on distributed-memory parallel computers. We here present GPU-oriented optimizations of the ELPA two-stage tridiagonalization eigensolver (ELPA2). On top of cuBLAS-based GPU offloading, we add a CUDA kernel to speed up the back-transformation of eigenvectors, which can be the computationally most expensive part of the two-stage tridiagonalization algorithm. We benchmark the performance of this GPU-accelerated eigensolver on two hybrid CPU–GPU architectures, namely a compute cluster based on Intel Xeon Gold CPUs and NVIDIA Volta GPUs, and the Summit supercomputer based on IBM POWER9 CPUs and NVIDIA Volta GPUs. Consistent with previous benchmarks on CPU-only architectures, the GPU-accelerated two-stage solver exhibits a parallel performance superior to the one-stage counterpart. Finally, we demonstrate the performance of the GPU-accelerated eigensolver developed in this work for routine semi-local KS-DFT calculations comprising thousands of atoms. [ABSTRACT FROM AUTHOR]

• An efficient parallelization strategy for a high-order accurate sliding mesh method. • Investigation of the method's scaling behavior on high performance computing systems. • A wall-resolved large eddy simulation of a 1-1/2 stage turbine. Sliding meshes are a powerful method to treat deformed domains in computational fluid dynamics, where different parts of the domain are in relative motion. In this paper, we present an efficient implementation of a sliding mesh method into a discontinuous Galerkin compressible Navier-Stokes solver and its application to a large eddy simulation of a 1-1/2 stage turbine. The method is based on the mortar method and is high-order accurate. It can handle three-dimensional sliding mesh interfaces with various interface shapes. For plane interfaces, which are the most common case, conservativity and free-stream preservation are ensured. We put an emphasis on efficient parallel implementation. Our implementation generates little computational and storage overhead. Inter-node communication via MPI in a dynamically changing mesh topology is reduced to a bare minimum by ensuring a priori information about communication partners and data sorting. We provide performance and scaling results showing the capability of the implementation strategy. Apart from analytical validation computations and convergence results, we present a wall-resolved implicit LES of the 1-1/2 stage Aachen turbine test case as a large scale practical application example. [ABSTRACT FROM AUTHOR]

The aim of this study is to devise an efficient and scalable computational procedure to solve the many tridiagonal systems in multi-dimensional partial differential equations. The modified Thomas algorithm and a newly designed communication scheme were used to reduce the communication overhead encountered while solving the many tridiagonal systems. Benchmark test results reveal an advantage of the proposed procedures compared to global all-to-all communication methods — a significantly reduced communication time that becomes more prominent for larger problem sizes and greater number of cores. The proposed computational procedures are fully implemented in an open-source library called Parallel and Scalable Library for TDMA (PaScaL_TDMA). Considering a three-dimensional heat conduction problem as a practical example, we obtain good strong and weak scalability results up to 262,144 computing cores on the KISTI Nurion cluster system, which, to the best of our knowledge, is the largest parallel simulation for solving tridiagonal systems. The potential of this library for large-scale substantive problems in physics is also demonstrated through direct numerical simulations of the Rayleigh–Bénard convection problem, which yielded excellent scalability and accurate results. Program Title: PaScaL_TDMA CPC Library link to program files: http://dx.doi.org/10.17632/49z6fh94z3.1 Developer's repository link: https://github.com/MPMC-Lab Licensing provisions: MIT Programming language: Fortran90 Nature of problem: Tridiagonal systems for solving multi-dimensional partial differential equations. Solution method: The divide-and-conquer method is used to solve partitioned tridiagonal systems of equations in the distributed memory system. The partitioned tridiagonal systems of equations are transformed into modified sub-matrices using the modified Thomas algorithm [1]. Reduced tridiagonal systems are constructed by collecting the first and last rows of the modified submatrices from each computing core. The newly designed communication scheme based on MPI_Alltoallw accelerates to collect rows and construct the reduced tridiagonal systems. The solutions of the reduced tridiagonal systems are obtained via the sequential Thomas algorithm. Thereafter, the remaining unknowns in the modified sub-matrices are solved using solutions of the reduced tridiagonal systems. [ABSTRACT FROM AUTHOR]

Gaussian process (GP) models are widely used to analyze spatially referenced data and to predict values at locations without observations. They are based on a statistical framework, which enables uncertainty quantification of the model structure and predictions. Both the evaluation of the likelihood and the prediction involve solving linear systems. Hence, the computational costs are large and limit the amount of data that can be handled. While there are many approximation strategies that lower the computational cost of GP models, they often provide sub-optimal support for the parallel computing capabilities of (high-performance) computing environments. To bridge this gap a parallelizable parameter estimation and prediction method is presented. The key idea is to divide the spatial domain into overlapping subsets and to use cross-validation (CV) to estimate the covariance parameters in parallel. Although simulations show that CV is less effective for parameter estimation than the maximum likelihood method, it is amenable to parallel computing and enables the handling of large datasets. Exploiting the screen effect for spatial prediction helps to arrive at a spatial analysis that is close to a global computation despite performing parallel computations on local regions. Simulation studies assess the accuracy of the parameter estimates and predictions. The implementation shows good weak and strong parallel scaling properties. For illustration, an exponential covariance model is fitted to a scientifically relevant canopy height dataset with 5 million observations. Using 512 processor cores in parallel brings the evaluation time of one covariance parameter configuration to 1.5 minutes. [ABSTRACT FROM AUTHOR]

Furthering our understanding of many of today's interesting problems in plasma physics – including plasma based acceleration and magnetic reconnection with pair production due to quantum electrodynamic effects – requires large-scale kinetic simulations using particle-in-cell (PIC) codes. However, these simulations are extremely demanding, requiring that contemporary PIC codes be designed to efficiently use a new fleet of exascale computing architectures. To this end, the key issue of parallel load balance across computational nodes must be addressed. We discuss the implementation of dynamic load balancing by dividing the simulation space into many small, self-contained regions or "tiles," along with shared-memory (e.g., OpenMP) parallelism both over many tiles and within single tiles. The load balancing algorithm can be used with three different topologies, including two space-filling curves. We tested this implementation in the code Osiris and show low overhead and improved scalability with OpenMP thread number on simulations with both uniform load and severe load imbalance. Compared to other load-balancing techniques, our algorithm gives order-of-magnitude improvement in parallel scalability for simulations with severe load imbalance issues. • Particle-in-cell simulations of plasmas can suffer from severe load imbalance. • We divide the simulation space into many small, self-contained regions, or "tiles". • Dynamic load balancing is performed by shuffling tiles between MPI processes. • Performance is enhanced by processing tiles via shared-memory parallelism. • We show low overhead and order-of-magnitude improvement in parallel scalability. [ABSTRACT FROM AUTHOR]

Information about several papers discussed at the ShipIt day event held at the Livermore Valley Open Campus in Livermore, California on August 24-25, 2012. Topics include applications which navigates and analyze the massive amounts of data developed by the Software engineers in Global Security, application which displays EleCent data on Android device, and the prototyple of MyLC portlet. The event featured software developers from Global Security & Communications.

eWEEK PROCESSOR ANALYSIS: We are seeing a major change in how we think of PC graphics, and it shouldn't be a surprise that NVIDIA is leading the way. [ABSTRACT FROM AUTHOR]

The transport and manipulation of particles and cells in microfluidic devices has become a core methodology in domains ranging from molecular biology to manufacturing and drug design. The design and operation of such devices can benefit from simulations that resolve flow-structure interactions at sub-micron resolution. We present a computational tool for large scale, efficient and high throughput mesoscale simulations of fluids and deformable objects at complex microscale geometries. The code employs dissipative particle dynamics for the description of the flow coupled with visco-elastic membrane model for red blood cells and can also handle rigid bodies and complex geometries. The software (Mirheo) is deployed on hybrid GPU/CPU architectures exhibiting unprecedented time-to-solution performance and excellent weak and strong scaling for a number of benchmark problems. Mirheo exploits the capabilities of GPU clusters, leading to speedup of up to 10X in terms of time to solution as compared to state-of-the-art software packages and reaches 90%–99% weak scaling efficiency on 512 nodes of the Piz Daint supercomputer. The software Mirheo relies on a Python interface to facilitate the solution and analysis of complex problems. Mirheo is an open source, potent computational tool that can greatly assist studies of microfluidics. Program Title: Mirheo Program Files doi: http://dx.doi.org/10.17632/n2dvz7htvn.1 Licensing provisions: MIT Programming language: C++, CUDA, Python Nature of problem: 3D simulations of microfluidic flows in complex geometries with suspended rigid bodies and deformable membranes such as cells, bacteria and microparticles. Solution method: Dissipative particle dynamics are used to represent the fluid. Cell membrane dynamics are described through potentials for shear and bending energies that are discretized on a triangular mesh and by additional constraints on cell volume and membrane area. The model incorporates membrane viscosity and interactions between membranes and the surrounding fluid. Rigid objects and boundaries are represented by groups of particles with prescribed center of mass and rotation quaternion. Time integration is performed using the Velocity-Verlet algorithm. Additional comments including restrictions and unusual features: The code runs on Nvidia GPU accelerators starting with the Kepler generation. [ABSTRACT FROM AUTHOR]