Search

We discuss pioneering heat and fluid flow simulations of fusion and fission energy systems with NekRS on exascale computing facilities, including Frontier and Aurora. The Argonne-based code, NekRS, is a highly-performant open-source code for the simulation of incompressible and low-Mach fluid flow, heat transfer, and combustion with a particular focus on turbulent flows in complex domains. It is based on rapidly convergent high-order spectral element discretizations that feature minimal numerical dissipation and dispersion. State-of-the-art multilevel preconditioners, efficient high-order time-splitting methods, and runtime-adaptive communication strategies are built on a fast OCCA-based kernel library, libParanumal, to provide scalability and portability across the spectrum of current and future high-performance computing platforms. On Frontier, Nek5000/RS has achieved an unprecedented milestone in breaching over 1 trillion degrees of freedom with the spectral element methods for the simulation of the CHIMERA fusion technology testing platform. We also demonstrate for the first time the use of high-order overset grids at scale., Comment: 10 pages, 3 figures, 3 tables

Heat pipes can efficiently and passively remove heat in nuclear microreactors. Nevertheless, the flow dynamics within heat pipes present a significant challenge in designing and optimizing them for nuclear energy applications. This work aims to explore the vapor core of heat pipes through comprehensive two- and three-dimensional simulations, with a primary focus on modeling the pressure recovery observed in the condenser section. The goal is to establish improved correlations for one-dimensional heat pipe codes. The simulations are validated against experimental data from a vapor pipe documented in the literature. The turbulence model is employed in the two-dimensional simulations through the open-source spectral-element code Nek5000. This model provides insights into pressure recovery within heat pipes with low computational cost. In addition, Large Eddy Simulations (LES) are used to capture turbulent flow features in a three-dimensional vapor pipe model, utilizing the code NekRS. Using LES is crucial for comprehending the impact of laminar-to-turbulent transition on pressure recovery. A simulation framework is created to model the heat pipe's vapor core, laying the foundation for an enhanced understanding of heat pipe behavior. The ultimate goal is to improve and optimize heat pipe designs, provide data to validate lower-fidelity approaches and enhance their performance in nuclear reactors., Comment: 14th International Topical Meeting on Nuclear Reactor Thermal-Hydraulics, Operation and Safety (NUTHOS-14)

Understanding the coolant thermal hydraulics in rod bundles is essential to the design of nuclear reactors. However, flows with low Reynolds numbers present serious modeling challenges, especially in heat transfer and natural convection. They are difficult to analyze through standard Computational Fluid Dynamics (CFD) tools. High-fidelity simulations, such as Direct Numerical Simulations (DNS), can provide invaluable insight into flow physics, supporting experiments in developing a deeper understanding and eventually enabling the accurate simulation of this class of flows. Data generated from these high-fidelity methods can then be used to benchmark available turbulence models and deliver cheap, faster running methods. In the present work, the convective heat transfer in hexagonal subchannels was studied through a DNS approach, using the high-order spectral element method code Nek5000, developed at Argonne National Laboratory. First, the geometric model composed of two hexagonal arrayed rod bundle subchannels with a pitch-to-diameter ratio of 1.5 is built, and then, the mesh is generated. These unusually high P/D and low Reynolds numbers represent conditions of interest for gas-fast reactors (GFRs). To our knowledge, there is no available dataset in these conditions. In this work, we detailed the development of the numerical benchmark and a series of preliminary LES simulations. Periodic boundary conditions are applied in the streamwise and spanwise directions and non-slip boundary conditions at the wall. Four cases are studied, with Reynolds of Re=2500, 5000, 7500, and 10000. All calculations have been performed with the Prandtl number of 0.61, corresponding to helium conditions of interest for Gas Fast Reactor applications. The results are analyzed with a polynomial-order convergence study, and the Reynolds stresses, and the turbulent kinetic budgets are presented and discussed., Comment: NURETH-19

The study of coolant flow behavior in rod bundles is of relevance to the design of nuclear reactors. Although laminar and turbulent flows have been researched extensively, there are still gaps in understanding the process of laminar-turbulent transition. Such a process may involve the formation of a gap vortex street as the consequence of a related linear instability. In the present work, a parametric study was performed to analyze the spatially developing turbulence in a simplified geometry setting. The geometry includes two square arrayed rod bundle subchannels with periodic boundary conditions in the cross-section. The pitch-to-diameter ratios range from 1.05 to 1.20, and the length of the domain was selected to be 100 diameters. No-slip condition at the wall, and inlet-outlet configuration were employed. Then, to investigate the stability of the flow, the Reynolds number was varied from 250 to 3000. The simulations were carried out using the spectral-element code Nek5000, with a Direct Numerical Simulation (DNS) approach. Data were analyzed to examine this Spatio-temporal developing instability. In particular, we evaluate the location of onset and spatial growth of the instability., Comment: ICONE-28

Nuclear power is a significant source of electricity in the United States, but the average U.S. nuclear power plant is around 40 years old. Safe management of spent nuclear fuel (SNF) is a crucial aspect of the back end of the nuclear fuel cycle. SNF dry storage systems are increasingly popular as they represent an effective solution in this area, given the absence of a final disposal system. In particular, the spent fuel cask system (dry cask method) provides a feasible solution for maintaining SNF ($\sim$60 years) prior to the final disposal. The HI-STORM overpack and MPC-32 canister are the primary components of the HI-STORM 100 dry cask storage system. They remove heat from the system via natural circulation with no human intervention required. This characteristic provides passive heat removal while requiring little maintenance in dry cask storage systems. This project aims to validate and compare the capabilities of a thermal model of HI-STORM overpack developed using the Multiphysics Object-Oriented Simulation Environment (MOOSE) based on the author's previous study. MOOSE is an open-source framework developed by Idaho National Laboratory for multiscale, multiphysics simulations. This study will improve the capabilities of the thermal-hydraulic model of the HI-STORM dry cask storage system by producing high-fidelity results for the air circulation in the overpack. Large Eddy Simulations (LES) are performed using the open-source spectral element code NekRS, developed by Argonne National Laboratory (ANL), for simulating transitional and turbulent flows in complex geometries. NekRS will produce high-fidelity results for the HI-STORM overpack to assess the validity of the current thermal-hydraulic model., Comment: 12 pages, 9 figures, Proceedings of NURETH-20 Conference

Wall channeling is a phenomena of interest for Pebble Bed Reactors (PBRs) where flow is diverted into high-porosity regions near the wall. This diversion of flow can have a significant impact on maximum fuel temperatures and core bypass flow. Porous media models that are currently used to model PBRs for design scoping and transient simulation are lacking in their capabilities to model the wall channel effect. Recent efforts at Penn State have produced an improved porous media pressure drop equation that is more capable of modeling the velocity variations caused by the wall channel effect in a porous media model. Several pebble beds were divided into concentric rings of $0.05D_{peb}$, and average flow quantities and porosities were extracted for the ring. A correlation between the form loss coefficient and the local ring porosity was found, allowing for the addition of a correction factor to the form loss term of the KTA equation. The developed correlation was purely empirical, and thus a more thorough understanding of the underlying flow phenomena is desired. This study investigates geometric and flow features that can explain the observed correlation between the form coefficient and the local porosity that was used to generate the improved pressure drop equation. The solid surface area to volume ratio $S_v$ along with the production of Turbulent Kinetic Energy (TKE) is analyzed. A relationship between $S_v$ and the local porosity and an inverse relationship between the negative TKE production and the local porosity were found, pointing to the idea that inertial effects caused by different pore geometry in each ring contribute to the variation of the form constant with the local porosity., Comment: 12 pages, 10 figures, part of the NURETH-20 conference

The design of advanced nuclear reactors (Gen IV) involves an array of challenging fluid-flow issues that affect its safety and performance. The calculated DNS database will be instrumental in understanding the flow behavior in the downcomer. Ultimately, we aim to evaluate existing heat transfer correlations and some modifications are proposed., Comment: arXiv admin note: substantial text overlap with arXiv:2203.14157

Single-phase natural circulation thermosiphon loops have been attracting increased interest as they represent the prototype of passive safety systems. However, the stability properties of thermosiphon loops, which can affect and compromise their functionality, are still actively investigated. Traditionally, the stability analysis of thermosiphon loops has been simplified to one-dimensional (1D) calculations, on the argument that the flow would be mono-dimensional when the diameter of the pipe D is orders of magnitude smaller than the length of the loop Lt. However, at lower Lt/D ratios, rectangular thermosiphon loops show that the flow presents 3D effect, which also has been confirmed by stability analyses in toroidal loops. In this paper, we performed a series of high-fidelity simulations using the spectral element code Nek5000 to investigate the stability behavior of the flow in rectangular thermosiphon loops. A wide range of Lt/D ratio from 10 to 200 has been considered and the results show many different outcomes compared to previous 1D analytical calculations or stability theory. Moreover, we analyzed the flow in rectangular thermosiphon loops using Proper Orthogonal Decomposition (POD) and we observed that the cases without flow reversal are characterized by swirl modes typical of bent pipes and high-frequency oscillation of the related time coefficients obtained by Galerkin projection. However, the swirl mode was not observed in cases with flow reversals, these cases are characterized by symmetric flow field at 2nd POD mode and the similarity of low-frequency oscillation in the projection of POD modes.

In this paper, we propose a novel reduced order model (ROM) lengthscale that is constructed by using energy distribution arguments. The new energy-based ROM lengthscale is fundamentally different from the current ROM lengthscales, which are built by using dimensional arguments. To assess the novel, energy-based ROM lengthscale, we compare it with a standard, dimensionality-based ROM lengthscale in two fundamentally different types of models: (i) the mixing-length ROM (ML-ROM), which is a ROM closure model; and (ii) the evolve-filter-relax ROM (EFR-ROM), which is a regularized ROM. We test the four combinations (i.e., ML-ROM and EFR-ROM equipped with the energy-based and dimensionality-based lengthscales) in the numerical simulation of the turbulent channel flow at $Re_{\tau} = 395$. The numerical investigation yields the following conclusions: (i) The new energy-based ROM lengthscale is significantly (almost two orders of magnitude) larger than the standard dimensionality-based ROM lengthscale. As a result, the energy-based lengthscale yields more stable ML-ROMs and EFR-ROMs than the dimensionality-based lengthscale. (ii) The energy-based lengthscale displays the correct asymptotic behavior with respect to the ROM dimension, whereas the dimensionality-based lengthscale does not. (iii) The energy-based lengthscale yields ML-ROMs and (when significant filtering is effected) EFR-ROMs whose parameters are less sensitive (i.e., more robust) than the parameters of the ML-ROMs and EFR-ROMs based on the dimensionality-based lengthscale. The novel energy-based lengthscale could enable the development of better scale-aware ROM strategies for flow-specific applications and is expected to have long term applications in nuclear reactor thermal-hydraulics., Comment: arXiv admin note: substantial text overlap with arXiv:2108.02254

Buoyancy effect on low-flow condition convective heat transfer of non-conventional coolants, such as liquid metal and molten salts, is a crucial safety factor to advanced reactors under transient or accidental scenarios. The distinct heat transfer characteristics of non-unitary Prandtl fluids and the inherent complexity of the low-flow mixed convection phenomena requires the development of novel turbulent and heat transfer models that are adaptive to different spatiotemporal scales involved in the mixed convection heat transfer. In this work, direct numerical simulation of low-flow mixed convection is carried out at low-to-unitary Prandtl numbers that are of industrial interest. Time-averaged statistics, turbulent Prandtl number, as well as time signals are analyzed to investigate mixed convection phenomenon. From the time-averaged statistics, buoyant plume altered velocity boundary layer as well as the intensity of the fluctuation near both walls and channel centerline. Buoyancy effect also rendered different degree of convective heat transfer enhancement and impairment depends on Prandtl and Richardson number. Analysis of time series was conducted on the sodium mixed convection case to emphasize on the low-Pr mixed convection behavior at transition region. Resulting power spectra density and wavelet spectrogram suggests possible large convective structure in transition region. Future work will focus on providing broader data coverage on Pr-Re-Ri parameter space to facilitate more comprehensive analysis of mixed convection.

The passive safety is a crucial feature of advanced nuclear reactor (Gen IV) design. During loss of power scenarios, the downcomer plays a crucial role. The fluid-flow behavior in the downcomer can involve forced to mixed to natural convection and characterizing the heat transfer for these changing regimes is a daunting challenge. High-resolution heat transfer numerical database can potentially support the development of precise and affordable reduced resolution heat transfer models. These models can be designed based on a multiscale hierarchy developed as part of the recently DOE-funded center of excellence for thermal-fluids applications in nuclear energy. In this paper, the downcomer is simplified to heated parallel plates and High Prandtl number fluid (FLiBe) is considered for all simulations. The calculations are performed for a wide range of Richardson number from 0 to 400 at 2 different FLiBe Prandtl numbers (12 and 24) which result in 40 simulated Cases in total. Time averaged and time series statistics as well as Nusselt number correlations are investigated to illuminate the mixed convection behaviors. The calculated database will be instrumental in understanding the flow behaviors in the downcomer. Ultimately, we aimed to evaluate existing heat transfer correlations and some modifications will be proposed for Cases where no satisfactory choice is available., Comment: arXiv admin note: text overlap with arXiv:2203.06670

The design of advanced nuclear reactors (Gen IV) involves an array of challenging fluid-flow issues that affect safety and performance. Currently, these problems are addressed in an ad-hoc manner at varying scales which are time-consuming and expensive. The creation of a high-resolution heat transfer numerical database has the potential to help develop to accurate and inexpensively reduced resolution heat transfer models. Such models can help address industrial-driven issues associated with the heat transfer behavior of advanced reactors. The models can be developed using the multiscale hierarchy developed as part of the recently DOE-funded center of excellence for thermal-fluids applications in nuclear energy. Ultimately this can lead to fast-running reliable models, thus accelerating the deployment of advanced reactors. In this paper, we performed a series of Direct Numerical Simulation using the spectral element codes Nek5000 and NekRS to investigate heat transfer in mixed convection conditions. First, we investigate the heat transfer of the flow in heated parallel plates for high Prandtl number fluids. The calculated database will eventually be used to evaluate existing heat transfer correlations and some modifications will be proposed for cases where no satisfactory choice is available. We have also investigated the heated transfer alteration phenomena in a straight heated tube for supercritical carbon dioxide (sCO2). The low-Mach-number approximation is used to decouple thermal and dynamic pressure, as pressure drop is negligible in this problem. The properties of sCO2 are calculated using multi-region polynomials. We observed that the heat transfer deterioration occurred in combination with the property changes of sCO2 and the depreciation of turbulence kinetic energy (TKE) for upward flow. Whereas, in downward flow, the heat transfer is enhanced thanks to the increase of TKE., Comment: NURETH-19 conference paper

The development of nuclear reactors that utilize pebble fuel has drastically increased the demand for improving the capabilities to simulate the packed beds found in these reactors. The complex flow fields found in a pebble bed make computational fluid dynamics (CFD) simulations time consuming and costly. Intermediate fidelity porous media models, however, are capable of approximating these flow fields in a much more computationally efficient manner. These models require the use of closures to capture the effects of complex flow phenomena without modeling them explicitly. This research employs data obtained from high-fidelity CFD simulations of a pebble bed to improve the drag closures used in porous media models in the near-wall region of the bed. Specifically, NekRS, a GPU-enabled spectral element CFD code, was used to simulate a bed of 1,568 pebbles at multiple Reynolds numbers. The case was divided into five concentric subdomains to extract radial profiles of the average porosity, velocity, and wall shear in each subdomain. A model consistent with the high-fidelity model was created in Idaho National Laboratory's Pronghorn porous media code and the KTA correlation was chosen as the drag closure of comparison. It was found that the KTA correlation overestimates the velocity in the near-wall region. An investigation of the drag coefficients between the two codes revealed that the KTA correlation underestimated the form factor in the outermost region while overestimating it in the inner four regions. This analysis in this work has revealed the underlying inaccuracy in the near-wall region of the KTA correlation and has set up the process for using high-fidelity simulation to predict more accurate drag coefficients a priori, rather than with a manual velocity-matching approach. This process will allow for the development of an improved drag closure for use in porous media models., Comment: 12 pages, 11 figures, part of the NURETH19 (2022) conference

Traditional 1D system thermal hydraulic analysis has been widely applied in nuclear industry for licensing purposes due to its numerical efficiency. However, such codes are inherently deficient for modeling of multiscale multidimensional flows. For such scenarios coarse-grid 3D simulations are useful due to the balance between the cost and the amount of information a modeler can extract from the results. At the same time, coarse grids do not allow to accurately resolve and capture turbulent mixing in reactor enclosures, while existing turbulence models (closures for the Reynolds stresses or turbulent viscosity) have large model form uncertainties. Thus, there is an interest in the improvement of such solvers. In this work two data-driven high-to-low methodologies to reduce mesh and model-induced errors are explored using a case study based on the Texas A&M upper plenum of high-temperature gas-cooled reactor facility. The first approach relies on the usage of turbulence closure for eddy viscosity with higher resolution/fidelity in a coarse grid solver. The second methodology employs artificial neural networks to map low-fidelity input features with errors in quantities of interest (velocity fields). Both methods have shown potential to improve the coarse grid simulation results by using data with higher fidelity (Reynolds-averaged Navier-Stokes and large eddy simulations), though, influence of the mesh-induced (discretization) error is quite complex and requires further investigations.

Recent interest for the development of high-temperature gas reactors has increased the need for more advanced understanding of flow characteristics in randomly packed pebble beds. A proper understanding of these flow characteristics can provide a better idea of the cooling capabilities of the system in both normal operation and accident scenarios. In order to enhance the accuracy of computationally efficient, intermediate fidelity modeling, high-fidelity simulation may be used to generate correlative data. For this research, NekRS, a GPU-enabled spectral-element computational fluid dynamics code, was used in order to produce the high-fidelity flow data for beds of 1,568 and 45,000 pebbles. Idaho National Lab's Pronghorn porous media code was used as the intermediate fidelity code. The results of the high-fidelity model were separated into multiple concentric regions in order to extract porosity and velocity averages in each region. The porosity values were input into the Pronghorn model and the resulting velocity profile was compared with that from NekRS. Both cases were run with a Reynolds number of 20,000 based on pebble diameter. The Pronghorn results were found to significantly overestimate the velocity in the outermost region indicating that changes in the porosity alone do not cause the difference in fluid velocity. We conclude that further work is necessary to develop a more effective drag coefficient correlation for the near-wall region and improve predictive capabilities of intermediate fidelity models., Comment: 10 pages, 9 figures, from the proceedings of the ICONE-28 (2021) conference

In the present study, the capabilities of a new Convolutional Neural Network (CNN) model are explored with the paramount objective of reconstructing the temperature field of wall-bounded flows based on a limited set of measurement points taken at the boundaries of the fluid domain. For that, we employ an algorithm that leverages the CNN capabilities provided with additional information about the governing equations of the physical problem. Great progress has been made in recent years towards reconstructing and characterizing the spatial distribution of physical variables of interest using CNNs. In principle, CNNs can represent any continuous mathematical function with a relatively reduced number of parameters. However, depending on the complexity imposed by the physical problem, this technique becomes unfeasible. The present study employs a Physics Informed Neuron Network technique featuring a data-efficient function approximator. As a proof of concept, the CNN is trained to retrieve the temperature of a heated channel based on a limited number of sensors placed at the boundaries of the domain. In this context, the training data are the temperature fields solutions considering various flows conditions at steady state, e.g varying the Reynolds and the Prandtl numbers. Additionally, a demonstration case considering the more complex geometry of a MSR is also provided. Assessment on the performance of the CNN is done by the mean L2 and the maximum Linf Euclidean norms stemmed from the difference between the actual solutions and the predictions made by the CNN. Finally, a sensitivity analysis is carried out such that the robustness of the CNN is tested considering a potential real application scenario where noise is inevitable. For that, the original test inputs are overlaid with a normal distribution of random numbers targeting to mimic different levels of noise in the measurement points., Comment: This paper has been submitted and accepted to be published in the 19th International Conference on Nuclear Reactor Thermal Hydraulics (NURETH-19) The paper number in that conference is 35774

Nek5000/RS is a highly-performant open-source spectral element code for simulation of incompressible and low-Mach fluid flow, heat transfer, and combustion with a particular focus on turbulent flows in complex domains. It is based on high-order discretizations that realize the same (or lower) cost per gridpoint as traditional low-order methods. State-of-the-art multilevel preconditioners, efficient high-order time-splitting methods, and runtime-adaptive communication strategies are built on a fast OCCA-based kernel library, libParanumal, to provide scalability and portability across the spectrum of current and future high-performance computing platforms. On Summit, Nek5000/RS has recently achieved an milestone in the simulation of nuclear reactors: the first full-core computational fluid dynamics simulations of reactor cores, including pebble beds with > 350,000 pebbles and 98M elements advanced in less than 0.25 seconds per Navier-Stokes timestep. With carefully tuned algorithms, it is possible to simulate a single flow-through time for a full reactor core in less than six hours on all of Summit., Comment: 9 pages, 3 figures, 6 tables

In this paper we describe the research and development activities in the Center for Efficient Exascale Discretization within the US Exascale Computing Project, targeting state-of-the-art high-order finite-element algorithms for high-order applications on GPU-accelerated platforms. We discuss the GPU developments in several components of the CEED software stack, including the libCEED, MAGMA, MFEM, libParanumal, and Nek projects. We report performance and capability improvements in several CEED-enabled applications on both NVIDIA and AMD GPU systems.

Efficient exploitation of exascale architectures requires rethinking of the numerical algorithms used in many large-scale applications. These architectures favor algorithms that expose ultra fine-grain parallelism and maximize the ratio of floating point operations to energy intensive data movement. One of the few viable approaches to achieve high efficiency in the area of PDE discretizations on unstructured grids is to use matrix-free/partially-assembled high-order finite element methods, since these methods can increase the accuracy and/or lower the computational time due to reduced data motion. In this paper we provide an overview of the research and development activities in the Center for Efficient Exascale Discretizations (CEED), a co-design center in the Exascale Computing Project that is focused on the development of next-generation discretization software and algorithms to enable a wide range of finite element applications to run efficiently on future hardware. CEED is a research partnership involving more than 30 computational scientists from two US national labs and five universities, including members of the Nek5000, MFEM, MAGMA and PETSc projects. We discuss the CEED co-design activities based on targeted benchmarks, miniapps and discretization libraries and our work on performance optimizations for large-scale GPU architectures. We also provide a broad overview of research and development activities in areas such as unstructured adaptive mesh refinement algorithms, matrix-free linear solvers, high-order data visualization, and list examples of collaborations with several ECP and external applications., Comment: 22 pages, 18 figures

In this paper, we propose a novel reduced order model (ROM) lengthscale definition that is based on energy distribution arguments. This novel ROM lengthscale is fundamentally different from the current ROM lengthscales, which are generally based on dimensional arguments. As a first step in the assessment of the new, energy based ROM lengthscale, we compare it with a standard, dimensional based ROM lengthscale in the mixing-length ROM (ML-ROM) simulation of the turbulent channel flow at $Re_{\tau} = 395$. The numerical investigation shows that the energy based ROM lengthscale yields a significantly more stable ML-ROM than the dimensional based ROM lengthscale. The new energy based lengthscale definition could allow the development of scale-aware reduced order modeling strategies that are better suited for flow-specific applications.

It has previously been shown that by increasing the Reynolds number across a channel by spatially varying the viscosity does not cause an immediate change in the size of turbulent structures and a delay is in fact observed in both wall shear and friction Reynolds number (Coppo Leite, V. & Merzari, E., Proceedings of the ASME 2020 FEDSM, p. V003T05A019). Furthermore, it is also shown that depending on the length in which the flow condition changes, turbulence bursts are observed in the turbulence field. For the present work we propose a new version of the standard Reynolds Averaged Navier Stokes (RANS) k-tau model that includes some modifications in the production term in order to account for these effects. The new proposed model may be useful for many engineering applications as turbulent flows featuring temperature gradients and high heat transfer rates are often seen in heat exchangers, combustion chambers and nuclear reactors. In these applications, thermal and viscous properties f the working fluid are important design parameters that depende on temperature; hence it is likely to observe strong gradients on these scalars' fields. To accomplish our goal, the modifications for the k-tau model are implemented and tested for a channel flow with spatial varying viscosity in the streamwise direction. The numerical simulations are performed using Nek5000, a spectral-element code developed at Argonne National Laboratory (ANL). Finally, the results considering a turbulence channel using the proposed model are compared against data obtained using Direct Numerical Simulations from the earlier work.

We develop an all-hex meshing strategy for the interstitial space in beds of densely packed spheres that is tailored to turbulent flow simulations based on the spectral element method (SEM). The SEM achieves resolution through elevated polynomial order N and requires two to three orders of magnitude fewer elements than standard finite element approaches do. These reduced element counts place stringent requirements on mesh quality and conformity. Our meshing algorithm is based on a Voronoi decomposition of the sphere centers. Facets of the Voronoi cells are tessellated into quads that are swept to the sphere surface to generate a high-quality base mesh. Refinements to the algorithm include edge collapse to remove slivers, node insertion to balance resolution, localized refinement in the radial direction about each sphere, and mesh optimization. We demonstrate geometries with 10^2-10^5 spheres using approximately 300 elements per sphere (for three radial layers), along with mesh quality metrics, timings, flow simulations, and solver performance., Comment: 13 pages, 10 figures

The development of NekRS, a GPU-oriented thermal-fluids simulation code based on the spectral element method (SEM) is described. For performance portability, the code is based on the open concurrent compute abstraction and leverages scalable developments in the SEM code Nek5000 and in libParanumal, which is a library of high-performance kernels for high-order discretizations and PDE-based miniapps. Critical performance sections of the Navier-Stokes time advancement are addressed. Performance results on several platforms are presented, including scaling to 27,648 V100s on OLCF Summit, for calculations of up to 60B gridpoints., Comment: 14 pages, 8 figures

Rod bundle flows are commonplace in nuclear engineering, and are present in light water reactors (LWRs) as well as other more advanced concepts. Inhomogeneities in the bundle cross section can lead to complex flow phenomena, including varying local conditions of turbulence. Despite the decades of numerical and experimental investigations regarding this topic, and the importance of elucidating the physics of the flow field, to date there are few publicly available direct numerical simulations (DNS) of the flow in multiple-pin rod bundles. Thus a multiple-pin DNS study can provide significant value toward reaching a deeper understanding of the flow physics, as well as a reference simulation for development of various reduced-resolution analysis techniques. To this end, DNS of the flow in a square 5x5 rod bundle at Reynolds number of 19,000 has been performed using the highly-parallel spectral element code Nek5000. The geometrical dimensions were representative of typical LWR fuel designs. The DNS was designed using microscales estimated from an advanced Reynolds-Averaged Navier-Stokes (RANS) model. Characteristics of the velocity field, Reynolds stresses, and anisotropy are presented in detail for various regions of the bundle. The turbulent kinetic energy budget is also presented and discussed

Rod bundle flows are prevalent in nuclear engineering for both light water reactors (LWR) and advanced reactor concepts. Unlike canonical channel flow, the flow in rod bundles presents some unique characteristics, notably due to the inhomogeneous cross section which can present different local conditions of turbulence as well as localized effects characteristic of external flows. Despite the ubiquity of rod bundle flows and the decades of experimental and numerical knowledge acquired in this field, there are no publicly available direct numerical simulations (DNS) of the flow in multiple pin rod bundles with heat transfer. A multiple pin DNS study is of great value as it would allow for assessment of the reliability of various turbulence models in the presence of heat transfer, as well as allow for a deeper understanding of the flow physics. We present work towards DNS of the flow in a square 5x5 rod bundle representative of LWR fuel. We consider standard configurations as well as configurations where the central pin is replaced with a guide thimble. We perform simulations in STAR-CCM+ to design the numerical DNS, which is to be conducted using the open source spectral element code Nek5000. Large Eddy Simulations are also performed in Nek5000 to confirm that the resolution requirements are adequate. We compare results from STAR-CCM+ and Nek5000, which show very good agreement in the wide gaps with larger discrepancies in the narrow gaps. In particular, evidence of a gap vortex street is seen in the edge subchannels in LES but is not predicted by STAR-CCM+.

In this article we examine channel flow subject to spatially varying viscosity in the streamwise direction. The Reynolds number is imposed locally with three different ramps. The setup is reminiscent of transient channel flow, but with a space dependent viscosity rather than a time dependent viscosity. It is also relevant to various applications in nuclear engineering and in particular in test reactors, where the viscosity changes significantly in the streamwise direction, and there is a severe lack of Direct Numerical Simulation (DNS) data to benchmark turbulence models in these conditions. As part of this work we set up a novel benchmark case: the channel is extended in the stream-wise direction up to 20p. The viscosity is kept constant in the first 4p region. This inlet region is used as a cyclic region to obtain a fully developed flow profile at the beginning of the ramping region. In the ramping region the Reynolds number is linearly increased along the channel. The flow is homogenous in the spanwise direction, while it is nonhomogenous in the stream-wise and wall-normal direction. We perform here Direct Numerical Simulation (DNS) with Nek5000, a spectral-element computational fluid dynamics (CFD) code developed at Argonne National Laboratory. In this study, specific focus is given to the investigation of turbulence properties and structures in the near-wall region along the flow direction. Turbulent statistics are collected and investigated. Similarly to transient channel flow, the results show that a variation in the Reynolds across a channel does not cause an immediate change in the size of turbulent structures in the ramp region and a delay is in fact observed in both wall shear and friction Reynolds number. The results from the present study are compared with a correlation available in the literature for the friction velocity and as a function of the Reynolds number.

Funded by the U.S. Department of Energy, the Nuclear Energy Advanced Modeling and Simulation (NEAMS) program aims to develop an integrated multiphysics simulation capability for the design and analysis of future generations of nuclear power plants. NEAMS embraces a multiresolution hierarchy designing the code suite structure to ultimately span the full range of length and time scales present in relevant reactor design and safety analyses. Advanced reactors, such as liquid metal reactors, rely on innovative component designs to meet cost and safety targets. In order to span a wider design range, advanced modeling and simulation capabilities that rely on minimal assumptions play an important role in optimizing the design. Over the past several years the NEAMS program has developed the integrated multiphysics code suite (thermal-hydraulics, structural analysis and neutronics) SHARP aimed at streamlining the prototyping of such components. For the simulation of fluid flow and heat transfer, SHARP focuses on the high-fidelity end, aiming primarily at turbulence-resolving techniques such large eddy simulation (LES) and direct numerical simulation (DNS). The computational fluid dynamics code (CFD) selected for SHARP is Nek5000, a state-of-the-art highly scalable tool employing the spectral element method (SEM). In this manuscript, to be published in a Von karman institute lecture series monograph on liquid metal reactors, we review the method and its implementation in Nek5000. We also examine several applications. We note that Nek5000 is also regularly employed for intermediate-fidelity approaches such as Reynolds-averaged Navier-Stokes (RANS) and for reduced-order models employing momentum sources or porous media, especially when coupled to neutronics modeling., Comment: 23 pages and 15 figures

Swirl-switching is a low-frequency oscillatory phenomenon which affects the Dean vortices in bent pipes and may cause fatigue in piping systems. Despite thirty years worth of research, the mechanism that causes these oscillations and the frequencies that characterise them remain unclear. Here we show that a three-dimensional wave-like structure is responsible for the low-frequency switching of the dominant Dean vortex. The present study, performed via direct numerical simulation, focuses on the turbulent flow through a 90 \degree pipe bend preceded and followed by straight pipe segments. A pipe with curvature 0.3 (defined as ratio between pipe radius and bend radius) is studied for a bulk Reynolds number Re = 11 700, corresponding to a friction Reynolds number Re_\tau \approx 360. Synthetic turbulence is generated at the inflow section and used instead of the classical recycling method in order to avoid the interference between recycling and swirl-switching frequencies. The flow field is analysed by three-dimensional proper orthogonal decomposition (POD) which for the first time allows the identification of the source of swirl-switching: a wave-like structure that originates in the pipe bend. Contrary to some previous studies, the flow in the upstream pipe does not show any direct influence on the swirl-switching modes. Our analysis further shows that a three- dimensional characterisation of the modes is crucial to understand the mechanism, and that reconstructions based on 2D POD modes are incomplete.

The present work is targeted at performing a strong scaling study of the high-order spectral element fluid dynamics solver Nek5000. Prior studies indicated a recommendable metric for strong scalability from a theoretical viewpoint, which we test here extensively on three parallel machines with different performance characteristics and interconnect networks, namely Mira (IBM Blue Gene/Q), Beskow (Cray XC40) and Titan (Cray XK7). The test cases considered for the simulations correspond to a turbulent flow in a straight pipe at four different friction Reynolds numbers $Re_{\tau}$ = 180, 360, 550 and 1000. Considering the linear model for parallel communication we quantify the machine characteristics in order to better assess the scaling behaviors of the code. Subsequently sampling and profiling tools are used to measure the computation and communication times over a large range of compute cores. We also study the effect of the two coarse grid solvers XXT and AMG on the computational time. Super-linear scaling due to a reduction in cache misses is observed on each computer. The strong scaling limit is attained for roughly 5000 - 10,000 degrees of freedom per core on Mira, 30,000 - 50,0000 on Beskow, with only a small impact of the problem size for both machines, and ranges between 10,000 and 220,000 depending on the problem size on Titan. This work aims at being a reference for Nek5000 users and also serves as a basis for potential issues to address as the community heads towards exascale supercomputers., Comment: 10 pages, 9 figures, Proceedings of the Exascale Applications and Software Conference 2016 (EASC '16, Stockholm)