Search

Maintaining computational load balance is important to the performantbehavior of codes which operate under a distributed computing model. This isespecially true for GPU architectures, which can suffer from memoryoversubscription if improperly load balanced. We present enhancements totraditional load balancing approaches and explicitly target GPU architectures,exploring the resulting performance. A key component of our enhancements is theintroduction of several GPU-amenable strategies for assessing compute work.These strategies are implemented and benchmarked to find the most optimal datacollection methodology for in-situ assessment of GPU compute work. For thefully kinetic particle-in-cell code WarpX, which supports MPI+CUDA parallelism,we investigate the performance of the improved dynamic load balancing via astrong scaling-based performance model and show that, for a laser-ionacceleration test problem run with up to 6144 GPUs on Summit, the enhanceddynamic load balancing achieves from 62%--74% (88% when running on 6 GPUs) ofthe theoretically predicted maximum speedup; for the 96-GPU case, we find thatdynamic load balancing improves performance relative to baselines without loadbalancing (3.8x speedup) and with static load balancing (1.2x speedup). Ourresults provide important insights into dynamic load balancing and performanceassessment, and are particularly relevant in the context of distributed memoryapplications ran on GPUs.

Maintaining computational load balance is important to the performantbehavior of codes which operate under a distributed computing model. This isespecially true for GPU architectures, which can suffer from memoryoversubscription if improperly load balanced. We present enhancements totraditional load balancing approaches and explicitly target GPU architectures,exploring the resulting performance. A key component of our enhancements is theintroduction of several GPU-amenable strategies for assessing compute work.These strategies are implemented and benchmarked to find the most optimal datacollection methodology for in-situ assessment of GPU compute work. For thefully kinetic particle-in-cell code WarpX, which supports MPI+CUDA parallelism,we investigate the performance of the improved dynamic load balancing via astrong scaling-based performance model and show that, for a laser-ionacceleration test problem run with up to 6144 GPUs on Summit, the enhanceddynamic load balancing achieves from 62%--74% (88% when running on 6 GPUs) ofthe theoretically predicted maximum speedup; for the 96-GPU case, we find thatdynamic load balancing improves performance relative to baselines without loadbalancing (3.8x speedup) and with static load balancing (1.2x speedup). Ourresults provide important insights into dynamic load balancing and performanceassessment, and are particularly relevant in the context of distributed memoryapplications ran on GPUs.

Particle-in-cell (PIC) simulations of relativistic flowing plasmas are of key interest to several fields of physics (including, e.g., laser-wakefield acceleration, when viewed in a Lorentz-boosted frame) but remain sometimes infeasible due to the well-known numerical Cherenkov instability (NCI). In this article, we show that, for a plasma drifting at a uniform relativistic velocity, the NCI can be eliminated by simply integrating the PIC equations in Galilean coordinates that follow the plasma (also sometimes known as comoving coordinates) within a spectral analytical framework. The elimination of the NCI is verified empirically and confirmed by a theoretical analysis of the instability. Moreover, it is shown that this method is applicable both to Cartesian geometry and to cylindrical geometry with azimuthal Fourier decomposition.

Particle-in-cell (PIC) simulations of relativistic flowing plasmas are of key interest to several fields of physics (including, e.g., laser-wakefield acceleration, when viewed in a Lorentz-boosted frame) but remain sometimes infeasible due to the well-known numerical Cherenkov instability (NCI). In this article, we show that, for a plasma drifting at a uniform relativistic velocity, the NCI can be eliminated by simply integrating the PIC equations in Galilean coordinates that follow the plasma (also sometimes known as comoving coordinates) within a spectral analytical framework. The elimination of the NCI is verified empirically and confirmed by a theoretical analysis of the instability. Moreover, it is shown that this method is applicable both to Cartesian geometry and to cylindrical geometry with azimuthal Fourier decomposition.

A new spectral particle-in-cell (PIC) method for plasma modeling is presented and discussed. In the proposed scheme, the Fourier-Bessel transform is used to translate the Maxwell equations to the quasi-cylindrical spectral domain. In this domain, the equations are solved analytically in time, and the spatial derivatives are approximated with high accuracy. In contrast to the finite-difference time domain (FDTD) methods, that are used commonly in PIC, the developed method does not produce numerical dispersion and does not involve grid staggering for the electric and magnetic fields. These features are especially valuable in modeling the wakefield acceleration of particles in plasmas. The proposed algorithm is implemented in the code PLARES-PIC, and the test simulations of laser plasma interactions are compared to the ones done with the quasi-cylindrical FDTD PIC code CALDER-CIRC.