Search

A new adaptive clipping-and-redistribution method is presented which provides bounds-preservation for multidimensional interpolation in the context of high-order finite-volume discretizations with adaptive mesh refinement (AMR). The underlying finite-volume method (FVM) for the computational fluid dynamics applications is fourth-order accurate for smooth solutions and utilizes AMR for computational efficiency in solving multiscale problems involving turbulence and combustion. High-order interpolation between different AMR levels is required. However, this operation often leads to numerical issues because combustion species must have physical bounds preserved. The present study overcomes two major challenges in the development of the high-order interpolation method. First, the method needs to be bound-preserving near extrema or discontinuities to prevent the emergence of unphysical oscillations while maintaining fourth-order accuracy in smooth flows. Second, the method needs to satisfy the conservation requirement in multiple dimensions, particularly in the context of curvilinear coordinate transformations. Additionally, the method is designed to be localized and computationally inexpensive. The new interpolation scheme is demonstrated by solving reacting flows, which are extremely sensitive to unphysical overshoots in conserved quantities. The test problems are shock-induced (Formula presented.) - (Formula presented.) combustion and a (Formula presented.) -air flame in a practical bluff-body combustor. Results show the method prevents new extrema near discontinuities while maintaining high-order accuracy in smooth regions. In particular, the method is extremely beneficial for combustion with stiff chemistry. With the proposed new method, even if flame fronts cross AMR interfaces or new grids are created in the vicinity of the flame, solution stability is retained.

Reacting flow simulations for combustion applications require extensive computing capabilities. Leveraging the AMReX library, the Pele suite of combustion simulation tools targets the largest supercomputers available and future exascale machines. We introduce PeleC, the compressible solver in the Pele suite, and detail its capabilities, including complex geometry representation, chemistry integration, and discretization. We present a comparison of development efforts using both OpenACC and AMReX’s C++ performance portability framework for execution on multiple GPU architectures. We discuss relevant details that have allowed PeleC to achieve high performance and scalability. PeleC’s performance characteristics are measured through relevant simulations on multiple supercomputers. The success of PeleC’s design for exascale is exhibited through demonstration of a 160 billion cell simulation and weak scaling onto 100% of Summit, an NVIDIA-based GPU supercomputer at Oak Ridge National Laboratory. Our results provide confidence that PeleC will enable future combustion science simulations with unprecedented fidelity.

Structured-grid adaptive mesh refinement (SAMR) is an approach to mesh generation that supports structured access to data and adaptive mesh refinement for discretized partial differential equations (PDEs). Solution algorithms often require that an inverse of an operator be applied, a system of algebraic equations must be solved, and this process is often the primary computational cost in an application. SAMR is well suited to geometric multigrid solvers, which can be effective, but often do not adapt well to complex geometry including material coefficients. Algebraic multigrid (AMG) is more robust in the face of complex geometry, in both boundary conditions and internal material interfaces. AMG requires a stored matrix linearization of the operator. We discuss an approach, and an implementation in the Chombo block-structured AMR framework, for constructing composite grid matrices from a SAMR hierarchy of grids for use in linear solvers in the PETSc numerical library. We consider a case study with the Chombo-based BISICLES ice sheet modeling application.

Structured-grid adaptive mesh refinement (SAMR) is an approach to mesh generation that supports structured access to data and adaptive mesh refinement for discretized partial differential equations (PDEs). Solution algorithms often require that an inverse of an operator be applied, a system of algebraic equations must be solved, and this process is often the primary computational cost in an application. SAMR is well suited to geometric multigrid solvers, which can be effective, but often do not adapt well to complex geometry including material coefficients. Algebraic multigrid (AMG) is more robust in the face of complex geometry, in both boundary conditions and internal material interfaces. AMG requires a stored matrix linearization of the operator. We discuss an approach, and an implementation in the Chombo block-structured AMR framework, for constructing composite grid matrices from a SAMR hierarchy of grids for use in linear solvers in the PETSc numerical library. We consider a case study with the Chombo-based BISICLES ice sheet modeling application.

Reactive transport in fractured media is conceptualized as a multi-scale problem that couples a pore-scale component, which comprises Navier–Stokes flow, multi-component transport and aqueous equilibrium in the fracture, and a Darcy-scale component, which comprises multi-component diffusive transport, aqueous equilibrium and mineral reactions in the porous matrix. The model that implements this multi-scale approach builds on an existing pore-scale model and is able to capture complex fracture geometries with the embedded-boundary method. The embedded boundary acts as the interface between pore- and Darcy-scale domains. Adaptive mesh refinement is used to match resolutions at the interface while using coarser resolution away from the interface when not needed in the Darcy-scale domain. The new model is validated and then compared to results from a pore-scale model. Multi-scale model results are shown to be equivalent to pore-scale results under diffusion-controlled reactions in the pore scale and very fast dissolution in the Darcy scale. The multi-scale model provides a more accurate solution for a given resolution as it effectively sets the equilibrium concentrations as boundary conditions. The multi-scale model is capable to capture flow channelization observed in an experimental fractured core and, at the same time, limitations in the dissolution of calcite by diffusive transport through an altered porous layer. Discrepancies in effluent calcium concentrations between the multi-scale results and results from a reduced-dimension Darcy-scale model for this fractured core experiment are attributed to the solution of the flow field and the gradients that develop inside the fracture. Discrepancies in effluent magnesium concentrations exemplify the limitations of the approach because the multi-scale model requires calibration of reactive surface areas as Darcy-scale continuum models.

Reactive transport in fractured media is conceptualized as a multi-scale problem that couples a pore-scale component, which comprises Navier–Stokes flow, multi-component transport and aqueous equilibrium in the fracture, and a Darcy-scale component, which comprises multi-component diffusive transport, aqueous equilibrium and mineral reactions in the porous matrix. The model that implements this multi-scale approach builds on an existing pore-scale model and is able to capture complex fracture geometries with the embedded-boundary method. The embedded boundary acts as the interface between pore- and Darcy-scale domains. Adaptive mesh refinement is used to match resolutions at the interface while using coarser resolution away from the interface when not needed in the Darcy-scale domain. The new model is validated and then compared to results from a pore-scale model. Multi-scale model results are shown to be equivalent to pore-scale results under diffusion-controlled reactions in the pore scale and very fast dissolution in the Darcy scale. The multi-scale model provides a more accurate solution for a given resolution as it effectively sets the equilibrium concentrations as boundary conditions. The multi-scale model is capable to capture flow channelization observed in an experimental fractured core and, at the same time, limitations in the dissolution of calcite by diffusive transport through an altered porous layer. Discrepancies in effluent calcium concentrations between the multi-scale results and results from a reduced-dimension Darcy-scale model for this fractured core experiment are attributed to the solution of the flow field and the gradients that develop inside the fracture. Discrepancies in effluent magnesium concentrations exemplify the limitations of the approach because the multi-scale model requires calibration of reactive surface areas as Darcy-scale continuum models.

Flows in which the primary features of interest do not rely on high-frequency acoustic effects, but in which long-wavelength acoustics play a nontrivial role, present a computational challenge. Integrating the entire domain with low-Mach-number methods would remove all acoustic wave propagation, while integrating the entire domain with the fully compressible equations can in some cases be prohibitively expensive due to the CFL time step constraint. For example, simulation of thermoacoustic instabilities might require fine resolution of the fluid/chemistry interaction but not require fine resolution of acoustic effects, yet one does not want to neglect the long-wavelength wave propagation and its interaction with the larger domain. The present paper introduces a new multi-level hybrid algorithm to address these types of phenomena. In this new approach, the fully compressible Euler equations are solved on the entire domain, potentially with local refinement, while their low-Mach-number counterparts are solved on subregions of the domain with higher spatial resolution. The finest of the compressible levels communicates inhomogeneous divergence constraints to the coarsest of the low-Mach-number levels, allowing the low-Mach-number levels to retain the long-wavelength acoustics. The performance of the hybrid method is shown for a series of test cases, including results from a simulation of the aeroacoustic propagation generated from a Kelvin–Helmholtz instability in low-Mach-number mixing layers. It is demonstrated that compared to a purely compressible approach, the hybrid method allows time-steps two orders of magnitude larger at the finest level, leading to an overall reduction of the computational time by a factor of 8.

Flows in which the primary features of interest do not rely on high-frequency acoustic effects, but in which long-wavelength acoustics play a nontrivial role, present a computational challenge. Integrating the entire domain with low-Mach-number methods would remove all acoustic wave propagation, while integrating the entire domain with the fully compressible equations can in some cases be prohibitively expensive due to the CFL time step constraint. For example, simulation of thermoacoustic instabilities might require fine resolution of the fluid/chemistry interaction but not require fine resolution of acoustic effects, yet one does not want to neglect the long-wavelength wave propagation and its interaction with the larger domain. The present paper introduces a new multi-level hybrid algorithm to address these types of phenomena. In this new approach, the fully compressible Euler equations are solved on the entire domain, potentially with local refinement, while their low-Mach-number counterparts are solved on subregions of the domain with higher spatial resolution. The finest of the compressible levels communicates inhomogeneous divergence constraints to the coarsest of the low-Mach-number levels, allowing the low-Mach-number levels to retain the long-wavelength acoustics. The performance of the hybrid method is shown for a series of test cases, including results from a simulation of the aeroacoustic propagation generated from a Kelvin–Helmholtz instability in low-Mach-number mixing layers. It is demonstrated that compared to a purely compressible approach, the hybrid method allows time-steps two orders of magnitude larger at the finest level, leading to an overall reduction of the computational time by a factor of 8.

We present a moving control volume (CV) approach to computing hydrodynamic forces and torques on complex geometries. The method requires surface and volumetric integrals over a simple and regular Cartesian box that moves with an arbitrary velocity to enclose the body at all times. The moving box is aligned with Cartesian grid faces, which makes the integral evaluation straightforward in an immersed boundary (IB) framework. Discontinuous and noisy derivatives of velocity and pressure at the fluid–structure interface are avoided and far-field (smooth) velocity and pressure information is used. We re-visit the approach to compute hydrodynamic forces and torques through force/torque balance equations in a Lagrangian frame that some of us took in a prior work (Bhalla et al., 2013 [13]). We prove the equivalence of the two approaches for IB methods, thanks to the use of Peskin's delta functions. Both approaches are able to suppress spurious force oscillations and are in excellent agreement, as expected theoretically. Test cases ranging from Stokes to high Reynolds number regimes are considered. We discuss regridding issues for the moving CV method in an adaptive mesh refinement (AMR) context. The proposed moving CV method is not limited to a specific IB method and can also be used, for example, with embedded boundary methods.

We present a moving control volume (CV) approach to computing hydrodynamic forces and torques on complex geometries. The method requires surface and volumetric integrals over a simple and regular Cartesian box that moves with an arbitrary velocity to enclose the body at all times. The moving box is aligned with Cartesian grid faces, which makes the integral evaluation straightforward in an immersed boundary (IB) framework. Discontinuous and noisy derivatives of velocity and pressure at the fluid–structure interface are avoided and far-field (smooth) velocity and pressure information is used. We re-visit the approach to compute hydrodynamic forces and torques through force/torque balance equations in a Lagrangian frame that some of us took in a prior work (Bhalla et al., 2013 [13]). We prove the equivalence of the two approaches for IB methods, thanks to the use of Peskin's delta functions. Both approaches are able to suppress spurious force oscillations and are in excellent agreement, as expected theoretically. Test cases ranging from Stokes to high Reynolds number regimes are considered. We discuss regridding issues for the moving CV method in an adaptive mesh refinement (AMR) context. The proposed moving CV method is not limited to a specific IB method and can also be used, for example, with embedded boundary methods.

We present an approach to designing arbitrarily high-order finitevolume spatial discretizations on locally-rectangular grids. It is based on the use of a simple class of high-order quadratures for computing the average of fluxes over faces. This approach has the advantage of being a variation on widely-used second-order methods, so that the prior experience in engineering those methods carries over in the higher-order case. Among the issues discussed are the basic design principles for uniform grids, the extension to locally-refined nest grid hierarchies, and the treatment of complex geometries using mapped grids, multiblock grids, and cut-cell representations.

The interaction of maintained homogeneous isotropic turbulence with lean premixed methane flames is investigated using direct numerical simulation with detailed chemistry. The conditions are chosen to be close to those found in atmospheric laboratory experiments. As the Karlovitz number is increased from 1 to 36, the preheat zone becomes thickened, while the reaction zone remains largely unaffected. A negative correlation of fuel consumption with mean flame surface curvature is observed. With increasing turbulence intensity, the chemical composition in the preheat zone tends towards that of an idealised unity Lewis number flame, which we argue is the onset of the transition to distributed burning, and the response of the various chemical species is shown to fall into broad classes. Smaller-scale simulations are used to isolate the specific role of species diffusion at high turbulent intensities. Diffusion of atomic hydrogen is shown to be related to the observed curvature correlations, but does not have significant consequential impact on the thickening of the preheat zone. It is also shown that susceptibility of the preheat zone to thickening by turbulence is related to the 'global' Lewis number (the Lewis number of the deficient reactant); higher global Lewis number flames tend to be more prone to thickening.

The interaction of maintained homogeneous isotropic turbulence with lean premixed methane flames is investigated using direct numerical simulation with detailed chemistry. The conditions are chosen to be close to those found in atmospheric laboratory experiments. As the Karlovitz number is increased from 1 to 36, the preheat zone becomes thickened, while the reaction zone remains largely unaffected. A negative correlation of fuel consumption with mean flame surface curvature is observed. With increasing turbulence intensity, the chemical composition in the preheat zone tends towards that of an idealised unity Lewis number flame, which we argue is the onset of the transition to distributed burning, and the response of the various chemical species is shown to fall into broad classes. Smaller-scale simulations are used to isolate the specific role of species diffusion at high turbulent intensities. Diffusion of atomic hydrogen is shown to be related to the observed curvature correlations, but does not have significant consequential impact on the thickening of the preheat zone. It is also shown that susceptibility of the preheat zone to thickening by turbulence is related to the 'global' Lewis number (the Lewis number of the deficient reactant); higher global Lewis number flames tend to be more prone to thickening.

We present an approach to designing arbitrarily high-order finitevolume spatial discretizations on locally-rectangular grids. It is based on the use of a simple class of high-order quadratures for computing the average of fluxes over faces. This approach has the advantage of being a variation on widely-used second-order methods, so that the prior experience in engineering those methods carries over in the higher-order case. Among the issues discussed are the basic design principles for uniform grids, the extension to locally-refined nest grid hierarchies, and the treatment of complex geometries using mapped grids, multiblock grids, and cut-cell representations.

We have developed a new 3D multi-physics multi-material code, ALE-AMR, which combines Arbitrary Lagrangian Eulerian (ALE) hydrodynamics with Adaptive Mesh Refinement (AMR) to connect the continuum to the microstructural regimes. The code is unique in its ability to model hot radiating plasmas and cold fragmenting solids. New numerical techniques were developed for many of the physics packages to work efficiently on a dynamically moving and adapting mesh. We use interface reconstruction based on volume fractions of the material components within mixed zones and reconstruct interfaces as needed. This interface reconstruction model is also used for void coalescence and fragmentation. A flexible strength/failure framework allows for pluggable material models, which may require material history arrays to determine the level of accumulated damage or the evolving yield stress in J2 plasticity models. For some applications laser rays are propagating through a virtual composite mesh consisting of the finest resolution representation of the modeled space. A new 2nd order accurate diffusion solver has been implemented for the thermal conduction and radiation transport packages. One application area is the modeling of laser/target effects including debris/shrapnel generation. Other application areas include warm dense matter, EUV lithography, and material wall interactions for fusion devices.

We have developed a new 3D multi-physics multi-material code, ALE-AMR, which combines Arbitrary Lagrangian Eulerian (ALE) hydrodynamics with Adaptive Mesh Refinement (AMR) to connect the continuum to the microstructural regimes. The code is unique in its ability to model hot radiating plasmas and cold fragmenting solids. New numerical techniques were developed for many of the physics packages to work efficiently on a dynamically moving and adapting mesh. We use interface reconstruction based on volume fractions of the material components within mixed zones and reconstruct interfaces as needed. This interface reconstruction model is also used for void coalescence and fragmentation. A flexible strength/failure framework allows for pluggable material models, which may require material history arrays to determine the level of accumulated damage or the evolving yield stress in J2 plasticity models. For some applications laser rays are propagating through a virtual composite mesh consisting of the finest resolution representation of the modeled space. A new 2nd order accurate diffusion solver has been implemented for the thermal conduction and radiation transport packages. One application area is the modeling of laser/target effects including debris/shrapnel generation. Other application areas include warm dense matter, EUV lithography, and material wall interactions for fusion devices.

We present a high-order finite-volume approach for solving the shallow-water equations on the sphere, using multiblock grids on the cubed sphere. This approach combines a Runge-Kutta time discretization with a fourth-order-accurate spatial discretization and includes adaptive mesh refinement and refinement in time. Results of tests show fourth-order convergence for the shallow-water equations as well as for advection in a highly deformational flow. Hierarchical adaptive mesh refinement allows solution error to be achieved that is comparable to that obtained with uniform resolution of the most refined level of the hierarchy but with many fewer operations.

We present a high-order finite-volume approach for solving the shallow-water equations on the sphere, using multiblock grids on the cubed sphere. This approach combines a Runge-Kutta time discretization with a fourth-order-accurate spatial discretization and includes adaptive mesh refinement and refinement in time. Results of tests show fourth-order convergence for the shallow-water equations as well as for advection in a highly deformational flow. Hierarchical adaptive mesh refinement allows solution error to be achieved that is comparable to that obtained with uniform resolution of the most refined level of the hierarchy but with many fewer operations.

We present a fourth-order accurate finite-volume method for solving time-dependent hyperbolic systems of conservation laws on Cartesian grids with multiple levels of refinement. The underlying method is a generalization of that in [5] to nonlinear systems, and is based on using fourth-order accurate quadratures for computing fluxes on faces, combined with fourth-order accurate Runge?Kutta discretization in time. To interpolate boundary conditions at refinement boundaries, we interpolate in time in a manner consistent with the individual stages of the Runge-Kutta method, and interpolate in space by solving a least-squares problem over a neighborhood of each target cell for the coefficients of a cubic polynomial. The method also uses a variation on the extremum-preserving limiter in [8], as well as slope flattening and a fourth-order accurate artificial viscosity for strong shocks. We show that the resulting method is fourth-order accurate for smooth solutions, and is robust in the presence of complex combinations of shocks and smooth flows.

In this paper we present a second-order accurate adaptive algorithm for solving multiphase, incompressible flows in porous media. We assume a multiphase form of Darcy's law with relative permeabilities given as a function of the phase saturation. The remaining equations express conservation of mass for the fluid constituents. In this setting the total velocity, defined to be the sum of the phase velocities, is divergence-free. The basic integration method is based on a total-velocity splitting approach in which we solve a second-order elliptic pressure equation to obtain a total velocity. This total velocity is then used to recast component conservation equations as nonlinear hyperbolic equations. Our approach to adaptive refinement uses a nested hierarchy of logically rectangular grids with simultaneous refinement of the grids in both space and time. The integration algorithm on the grid hierarchy is a recursive procedure in which coarse grids are advanced in time, fine grids are advanced multiple steps to reach the same time as the coarse grids and the data at different levels are then synchronized. The single grid algorithm is described briefly, but the emphasis here is on the time-stepping procedure for the adaptive hierarchy. Numerical examples are presented to demonstrate the algorithm's accuracy and convergence properties and to illustrate the behavior of the method.

We discuss a web-based portal for the exploration, encapsulation, and dissemination of visualization results over the Grid. This portal integrates three components: an interface client for structured visualization exploration, a visualization web application to manage the generation and capture of the visualization results, and a centralized portal application server to access and manage grid resources. Our approach uses standard web technologies to make the system accessible with minimal user setup. We demonstrate the usefulness of the developed system using an example for Adaptive Mesh Refinement (AMR) data visualization.

Adaptive Mesh Refinement (AMR) is a technique used in numerical simulations to automatically refine (or re-define) certain regions of the physical domain in an infinite difference calculation. AMR data consists of nested hierarchies of data grids. As AMR visualization is still a relatively unexplored topic, our work is motivated by the need to perform effcient visualization of large AMR data sets. We present a software algorithm for parallel direct volume rendering of AMR data using a cell-projection technique on several different parallel platforms. Our algorithm can use one of several different distribution methods, and we present performance results for each of these alternative approaches. By partitioning an AMR data set into blocks of constant resolution and estimating rendering costs of individual blocks using an application specific benchmark, it is possible to achieve even load balancing.

Adaptive mesh refinement (AMR) is a technique used in numerical simulations to automatically refine (or de-refine) certain regions of the physical domain in a finite difference calculation. AMR data consists of nested hierarchies of data grids. As AMR visualization is still a relatively unexplored topic, our work is motivated by the need to perform efficient visualization of large AMR data sets. We present a software algorithm for parallel direct volume rendering of AMR data using a cell-projection technique on several different parallel platforms. Our algorithm can use one of several different distribution methods, and we present performance results for each of these alternative approaches. By partitioning an AMR data set into blocks of constant resolution and estimating rendering costs of individual blocks using an application specific benchmark, it is possible to achieve even load balancing.

Adaptive mesh refinement (AMR) is a technique used in numerical simulations to automatically refine (or de-refine) certain regions of the physical domain in a finite difference calculation. AMR data consists of nested hierarchies of data grids. As AMR visualization is still a relatively unexplored topic, our work is motivated by the need to perform efficient visualization of large AMR data sets. We present a software algorithm for parallel direct volume rendering of AMR data using a cell-projection technique on several different parallel platforms. Our algorithm can use one of several different distribution methods, and we present performance results for each of these alternative approaches. By partitioning an AMR data set into blocks of constant resolution and estimating rendering costs of individual blocks using an application specific benchmark, it is possible to achieve even load balancing.

The complexity of physical phenomena often varies substantially over space and time. There can be regions where a physical phenomenon/quantity varies very little over a large extent. At the same time, there can be small regions where the same quantity exhibits highly complex variations. Adaptive mesh refinement (AMR) is a technique used in computational fluid dynamics (CFD) to simulate phenomena with drastically varying scales concerning the complexity of the simulated variables. Using multiple nested grids of different resolutions, AMR combines the topological simplicity of structured-rectilinear grids, permitting efficient computation and storage, with the possibility to adapt grid resolutions in regions of complex behavior. We present methods for direct volume rendering of AMR data. Our methods utilize AMR grids directly for efficiency of the visualization process. We apply a hardware-accelerated rendering method to AMR data supporting interactive manipulation of color-transfer functions and viewing parameters. We also present a cell-projection-based rendering technique for AMR data.