Search

We exploit a suite of large \emph{N}-body simulations (up to N=$4096^3$) performed with \Abacus, of scale-free models with a range of spectral indices $n$, to better understand and quantify convergence of the matter power spectrum. Using self-similarity to identify converged regions, we show that the maximal wavenumber resolved at a given level of accuracy increases monotonically as a function of time. At the 1\% level it starts at early times from a fraction of $k_\Lambda$, the Nyquist wavenumber of the initial grid, and reaches at most, if the force softening is sufficiently small, $\sim 2-3 k_\Lambda$ at the very latest times we evolve to. At the $5\%$ level, accuracy extends up to wavenumbers of order $5k_\Lambda$ at late times. Expressed as a suitable function of the scale-factor, accuracy shows a very simple $n$-dependence, allowing a extrapolation to place conservative bounds on the accuracy of \emph{N}-body simulations of non-scale free models like LCDM. We note that deviations due to discretization in the converged range are not well modelled by shot noise, and subtracting it in fact degrades accuracy. Quantitatively our findings are broadly in line with the conservative assumptions about resolution adopted by recent studies using large cosmological simulations (e.g. Euclid Flagship) aiming to constrain the mildly non-linear regime. On the other hand, we remark that conclusions about small scale clustering (e.g. concerning the validity of stable clustering) obtained using PS data at wavenumbers larger than a few $k_\Lambda$ may need revision in light of our convergence analysis., Comment: 14 pages, 10 figures, version to match published version. Additional subsection in Shot noise and discreteness effects

We present $\texttt{Abacus}$, a fast and accurate cosmological $N$-body code based on a new method for calculating the gravitational potential from a static multipole mesh. The method analytically separates the near- and far-field forces, reducing the former to direct $1/r^2$ summation and the latter to a discrete convolution over multipoles. The method achieves 70 million particle updates per second per node of the Summit supercomputer, while maintaining a median fractional force error of $10^{-5}$. We express the simulation time step as an event-driven "pipeline", incorporating asynchronous events such as completion of co-processor work, Input/Output, and network communication. $\texttt{Abacus}$ has been used to produce the largest suite of $N$-body simulations to date, the $\texttt{AbacusSummit}$ suite of 60 trillion particles (Maksimova et al., 2021), incorporating on-the-fly halo finding. $\texttt{Abacus}$ enables the production of mock catalogs of the volume and resolution required by the coming generation of cosmological surveys.

We demonstrate that testing for self-similarity in scale-free simulations provides an excellent tool to quantify the resolution at small scales of cosmological N-body simulations. Analysing two-point correlation functions measured in simulations using ABACUS, we show how observed deviations from self-similarity reveal the range of time and distance scales in which convergence is obtained. While the well-converged scales show accuracy below 1 percent, our results show that, with a small force softening length, the spatial resolution is essentially determined by the mass resolution. At later times the lower cut-off scale on convergence evolves in comoving units as $a^{-1/2}$ ($a$ being the scale factor), consistent with a hypothesis that it is set by two-body collisionality. A corollary of our results is that N-body simulations, particularly at high red-shift, contain a significant spatial range in which clustering appears converged with respect to the time-stepping and force softening but has not actually converged to the physical continuum result. The method developed can be applied to determine the resolution of any clustering statistic and extended to infer resolution limits for non-scale-free simulations., 15 pages, 9 figures, version published in MNRAS, small changes (more discussion of time-stepping)

Tracking the formation and evolution of dark matter haloes is a critical aspect of any analysis of cosmological $N$-body simulations. In particular, the mass assembly of a halo and its progenitors, encapsulated in the form of its merger tree, serves as a fundamental input for constructing semi-analytic models of galaxy formation and, more generally, for building mock catalogues that emulate galaxy surveys. We present an algorithm for constructing halo merger trees from AbacusSummit, the largest suite of cosmological $N$-body simulations performed to date consisting of nearly 60 trillion particles, and which has been designed to meet the Cosmological Simulation Requirements of the Dark Energy Spectroscopic Instrument (DESI) survey. Our method tracks the cores of haloes to determine associations between objects across multiple timeslices, yielding lists of halo progenitors and descendants for the several tens of billions of haloes identified across the entire suite. We present an application of these merger trees as a means to enhance the fidelity of AbacusSummit halo catalogues by flagging and "merging" haloes deemed to exhibit non-monotonic past merger histories. We show that this cleaning technique identifies portions of the halo population that have been deblended due to choices made by the halo finder, but which could have feasibly been part of larger aggregate systems. We demonstrate that by cleaning halo catalogues in this post-processing step, we remove potentially unphysical features in the default halo catalogues, leaving behind a more robust halo population that can be used to create highly-accurate mock galaxy realisations from AbacusSummit., Comment: 19 pages, 13 figures, 2 table. Accepted for publication in MNRAS; updated one citation

We present a novel simulation-based hybrid emulator approach that maximally derives cosmological and Halo Occupation Distribution (HOD) information from non-linear galaxy clustering, with sufficient precision for DESI Year 1 (Y1) analysis. Our hybrid approach first samples the HOD space on a fixed cosmological simulation grid to constrain the high-likelihood region of cosmology+HOD parameter space, and then constructs the emulator within this constrained region. This approach significantly reduces the parameter volume emulated over, thus achieving much smaller emulator errors with fixed number of training points. We demonstrate that this combined with state-of-the-art simulations result in tight emulator errors comparable to expected DESI Y1 LRG sample variance. We leverage the new AbacusSummit simulations and apply our hybrid approach to CMASS non-linear galaxy clustering data. We infer constraints on $\sigma_8 = 0.762\pm0.024$ and $f\sigma_8 (z_{eff} = 0.52) = 0.444\pm0.016$, the tightest among contemporary galaxy clustering studies. We also demonstrate that our $f\sigma_8$ constraint is robust against secondary biases and other HOD model choices, a critical first step towards showcasing the robust cosmology information accessible in non-linear scales. We speculate that the additional statistical power of DESI Y1 should tighten the growth rate constraints by at least another 50-60%, significantly elucidating any potential tension with Planck. We also address the "lensing is low" tension, where we find that the combined effect of a lower $f\sigma_8$ and environment-based bias lowers the predicted lensing signal by 15%, accounting for approximately 50% of the discrepancy between the lensing measurement and clustering-based predictions., Comment: Accepted to MNRAS, comments welcome

We describe a method for generating halo catalogues on the light cone using the \Abacus{AbacusSummit} suite of $N$-body simulations. The main application of these catalogues is the construction of realistic mock galaxy catalogues and weak lensing maps on the sky. Our algorithm associates the haloes from a set of coarsely-spaced snapshots with their positions at the time of light-cone crossing by matching halo particles to on-the-fly light cone particles. It then records the halo and particle information into an easily accessible product, which we call the \Abacus{AbacusSummit} halo light cone catalogues. Our recommended use of this product is in the halo mass regime of $M_{\rm halo} > 2.1 \times 10^{11} \ M_\odot/h$ for the \texttt{base} resolution simulations, i.e. haloes containing at least 100 particles, where the interpolated halo properties are most reliable. To test the validity of the obtained catalogues, we perform various visual inspections and consistency checks. In particular, we construct galaxy mock catalogues of emission-line galaxies (ELGs) at $z \sim 1$ by adopting a modified version of the \Abacus{AbacusHOD} script, which builds on the standard halo occupation distribution (HOD) method by including various extensions. We find that the multipoles of the auto-correlation function are consistent with the predictions from the full-box snapshot, implicitly validating our algorithm. In addition, we compute and output CMB convergence maps and find that the auto- and cross-power spectrum agrees with the theoretical prediction at the subpercent level. Halo light cone catalogues for 25 \texttt{base} and 2 \texttt{huge} simulations at the fiducial cosmology is available at DOI:\href{https://www.doi.org/10.13139/OLCF/1825069}{10.13139/OLCF/1825069}

The two-point correlation function (2PCF) is the most widely used tool for quantifying the spatial distribution of galaxies. Since the distribution of galaxies is determined by galaxy formation physics as well as the underlying cosmology, fitting an observed correlation function yields valuable insights into both. The calculation for a 2PCF involves computing pair-wise separations and consequently, the computing time scales quadratically with the number of galaxies. The next-generation galaxy surveys are slated to observe many millions of galaxies, and computing the 2PCF for such surveys would be prohibitively time-consuming. Additionally, modern modelling techniques require the 2PCF to be calculated thousands of times on simulated galaxy catalogues of {\em at least} equal size to the data and would be completely unfeasible for the next generation surveys. Thus, calculating the 2PCF forms a substantial bottleneck in improving our understanding of the fundamental physics of the universe, and we need high-performance software to compute the correlation function. In this paper, we present Corrfunc --- a suite of highly optimised, OpenMP parallel clustering codes. The improved performance of Corrfunc arises from both efficient algorithms as well as software design that suits the underlying hardware of modern CPUs. Corrfunc can compute a wide range of 2-D and 3-D correlation functions in either simulation (Cartesian) space or on-sky coordinates. Corrfunc runs efficiently in both single- and multi-threaded modes and can compute a typical 2-point projected correlation function ($w_p(r_p)$) for ~1 million galaxies within a few seconds on a single thread. Corrfunc is designed to be both user-friendly and fast and is publicly available at https://github.com/manodeep/Corrfunc., Comment: Accepted for publication to MNRAS

We present a high-fidelity realization of the cosmological $N$-body simulation from the Schneider et al. (2016) code comparison project. The simulation was performed with our Abacus $N$-body code, which offers high force accuracy, high performance, and minimal particle integration errors. The simulation consists of $2048^3$ particles in a $500\ h^{-1}\mathrm{Mpc}$ box, for a particle mass of $1.2\times 10^9\ h^{-1}\mathrm{M}_\odot$ with $10\ h^{-1}\mathrm{kpc}$ spline softening. Abacus executed 1052 global time steps to $z=0$ in 107 hours on one dual-Xeon, dual-GPU node, for a mean rate of 23 million particles per second per step. We find Abacus is in good agreement with Ramses and Pkdgrav3 and less so with Gadget3. We validate our choice of time step by halving the step size and find sub-percent differences in the power spectrum and 2PCF at nearly all measured scales, with $, 13 pages, 8 figures. Minor changes to match MNRAS accepted version

Analysis of self-similarity in scale-free $N$-body simulations reveals the spatial and temporal scales for which statistics measured in cosmological simulations are converged to the physical continuum limit. We examine how the range of scales in which the two-point correlation function is converged depends on the force softening length and whether it is held constant in comoving or proper coordinates. We find that a proper softening that reaches roughly 1/30th of the inter-particle spacing by the end of the simulation resolves the same spatial and temporal scales as a comoving softening of the same length while using a third fewer time steps, for a range of scale factors typical to $\Lambda$CDM simulations. We additionally infer an inherent resolution limit, set by the particle mass and scaling as $a^{-1/2}$, beyond which reducing the softening does not improve the resolution. We postulate a mapping of these results with spectral index $n=-2$ to $\Lambda$CDM simulations., Comment: 14 pages, 12 figures. Minor changes to match accepted version. Comments welcome!

We use the $k$-nearest neighbor probability distribution function ($k$NN-PDF, Banerjee & Abel 2021) to assess convergence in a scale-free $N$-body simulation. Compared to our previous two-point analysis, the $k$NN-PDF allows us to quantify our results in the language of halos and numbers of particles, while also incorporating non-Gaussian information. We find good convergence for 32 particles and greater at densities typical of halos, while 16 particles and fewer appears unconverged. Halving the softening length extends convergence to higher densities, but not to fewer particles. Our analysis is less sensitive to voids, but we analyze a limited range of underdensities and find evidence for convergence at 16 particles and greater even in sparse voids., 9+1 pages, 9+1 figures, submitted to journal. Comments welcome!

We introduce the AbacusHOD model and present two applications of AbacusHOD and the AbacusSummit simulations to observations. AbacusHOD is an HOD framework written in Python that is particle-based, multi-tracer, highly generalized, and highly efficient. It is designed specifically with multi-tracer/cosmology analyses for next generation large-scale structure surveys in mind, and takes advantage of the volume and precision offered by the new state-of-the-art AbacusSummit cosmological simulations. The model is also highly customizable and should be broadly applicable to any upcoming surveys and a diverse range of cosmological analyses. In this paper, we demonstrate the capabilities of the AbacusHOD framework through two example applications. The first example demonstrates the high efficiency and the large HOD extension feature set through an analysis full-shape redshift-space clustering of BOSS galaxies at intermediate to small scales (, 20 pages, 16 figures, accepted to MNRAS, comments welcome

We present the public data release of the AbacusSummit cosmological $N$-body simulation suite, produced with the $\texttt{Abacus}$ $N$-body code on the Summit supercomputer of the Oak Ridge Leadership Computing Facility. $\texttt{Abacus}$ achieves $\mathcal{O}\left(10^{-5}\right)$ median fractional force error at superlative speeds, calculating 70M particle updates per second per node at early times, and 45M particle updates per second per node at late times. The simulation suite totals roughly 60 trillion particles, the core of which is a set of 139 simulations with particle mass $2\times10^{9}\,h^{-1}\mathrm{M}_\odot$ in box size $2\,h^{-1}\mathrm{Gpc}$. The suite spans 97 cosmological models, including Planck 2018, previous flagship simulation cosmologies, and a linear derivative and cosmic emulator grid. A sub-suite of 1883 boxes of size $500\,h^{-1}\mathrm{Mpc}$ is available for covariance estimation. AbacusSummit data products span 33 epochs from $z=8$ to $0.1$ and include lightcones, full particle snapshots, halo catalogs, and particle subsets sampled consistently across redshift. AbacusSummit is the largest high-accuracy cosmological $N$-body data set produced to date., Comment: 30 pages, 10 figures, 6 tables. Published in MNRAS. Data available at https://abacusnbody.org and https://abacussummit.readthedocs.io/en/latest/data-access.html (DOI: 10.13139/OLCF/1811689)

We use self-similarity in N-body simulations of scale-free models to test for resolution dependence in the mass function and two-point correlation functions of dark matter halos. We use 1024$^3$ particle simulations performed with ABACUS, and compare results obtained with two halo finders: friends-of-friends (FOF) and ROCKSTAR. The FOF mass functions show a systematic deviation from self-similarity which is explained by resolution dependence of the FOF mass assignment previously reported in the literature. Weak evidence for convergence is observed only starting from halos of several thousand particles, and mass functions are overestimated by at least as much as 20-25 percent for halos of 50 particles. The mass function of the default ROCKSTAR halo catalog (with bound virial spherical overdensity mass), on the other hand, shows good convergence from of order 50 to 100 particles per halo, with no detectable evidence at the few percent level of any systematic dependence for larger particle number. Tests show that the mass unbinding procedure in ROCKSTAR is the key factor in obtaining this much improved resolution. Applying the same analysis to the halo-halo two point correlation function, we find again strong evidence for convergence only for ROCKSTAR halos, at separations sufficiently large so that halos do not overlap. At these separations we can exclude dependence on resolution at the 5-10 percent level once halos have of order 50 to 100 particles. At smaller separations results are not converged even at significantly larger particle number, and bigger simulations would be required to establish the resolution required for convergence., 10 pages, 8 figures, version to match published version, small changes (note slightly more conservative bounds on convergence)

We present the GeneRalized ANd Differentiable Halo Occupation Distribution (GRAND-HOD) routine that generalizes the standard 5 parameter halo occupation distribution model (HOD) with various halo-scale physics and assembly bias. We describe the methodology of 4 different generalizations: satellite distribution generalization, velocity bias, closest approach distance generalization, and assembly bias. We showcase the signatures of these generalizations in the 2-point correlation function (2PCF) and the squeezed 3-point correlation function (squeezed 3PCF). We identify generalized HOD prescriptions that are nearly degenerate in the projected 2PCF and demonstrate that these degeneracies are broken in the redshift-space anisotropic 2PCF and the squeezed 3PCF. We also discuss the possibility of identifying degeneracies in the anisotropic 2PCF and further demonstrate the extra constraining power of the squeezed 3PCF on galaxy-halo connection models. We find that within our current HOD framework, the anisotropic 2PCF can predict the squeezed 3PCF better than its statistical error. This implies that a discordant squeezed 3PCF measurement could falsify the particular HOD model space. Alternatively, it is possible that further generalizations of the HOD model would open opportunities for the squeezed 3PCF to provide novel parameter measurements. The GRAND-HOD Python package is publicly available at https://github.com/SandyYuan/GRAND-HOD ., Comment: 16 pages, 9 figures, published on MNRAS

We present a method for generating suites of dark-matter halo catalogs with only a few $N$-body simulations, focusing on making small changes to the underlying cosmology of a simulation with high precision. In the context of blind challenges, this allows us to reuse a simulation by giving it a new cosmology after the original cosmology is revealed. Starting with full $N$-body realizations of an original cosmology and a target cosmology, we fit a transfer function that displaces halos in the original so that the galaxy/HOD power spectrum matches that of the target cosmology. This measured transfer function can then be applied to a new realization of the original cosmology to create a new realization of the target cosmology. For a 1% change in $\sigma_8$, we achieve 0.1% accuracy to $k = 1h\,\mathrm{Mpc}^{-1}$ in the real-space power spectrum; this degrades to 0.3% when the transfer function is applied to a new realization. We achieve similar accuracy in the redshift-space monopole and quadrupole. In all cases, the result is better than the sample variance of our $1.1h^{-1}\,\mathrm{Gpc}$ simulation boxes., Comment: 15 pages, 6 figures. Comments welcome!

Correlation functions are widely used in extra-galactic astrophysics to extract insights into how galaxies occupy dark matter halos and in cosmology to place stringent constraints on cosmological parameters. A correlation function fundamentally requires computing pair-wise separations between two sets of points and then computing a histogram of the separations. Corrfunc is an existing open-source, high-performance software package for efficiently computing a multitude of correlation functions. In this paper, we will discuss the SIMD AVX512F kernels within Corrfunc, capable of processing 16 floats or 8 doubles at a time. The latest manually implemented Corrfunc AVX512F kernels show a speedup of up to \(\sim \)4\(\times \) relative to compiler-generated code for double-precision calculations. The AVX512F kernels show \(\sim \)1.6\(\times \) speedup relative to the AVX kernels and compares favorably to a theoretical maximum of \(2\times \). In addition, by pruning pairs with too large of a minimum possible separation, we achieve a \(\sim \)5–10% speedup across all the SIMD kernels. Such speedups highlight the importance of programming explicitly with SIMD vector intrinsics for complex calculations that can not be efficiently vectorized by compilers. Corrfunc is publicly available at https://github.com/manodeep/Corrfunc/.

The combination of galaxy-galaxy lensing (GGL) with galaxy clustering is one of the most promising routes to determining the amplitude of matter clustering at low redshifts. We show that extending clustering+GGL analyses from the linear regime down to $\sim 0.5 \, h^{-1}$ Mpc scales increases their constraining power considerably, even after marginalizing over a flexible model of non-linear galaxy bias. Using a grid of cosmological N-body simulations, we construct a Taylor-expansion emulator that predicts the galaxy autocorrelation $\xi_{\text{gg}}(r)$ and galaxy-matter cross-correlation $\xi_{\text{gm}}(r)$ as a function of $\sigma_8$, $\Omega_m$, and halo occupation distribution (HOD) parameters, which are allowed to vary with large scale environment to represent possible effects of galaxy assembly bias. We present forecasts for a fiducial case that corresponds to BOSS LOWZ galaxy clustering and SDSS-depth weak lensing (effective source density $\sim 0.3$ arcmin$^{-2}$). Using tangential shear and projected correlation function measurements over $0.5 \leq r_p \leq 30 \, h^{-1}$ Mpc yields a 1.8% constraint on the parameter combination $\sigma_8\Omega_m^{0.58}$, a factor of two better than a constraint that excludes non-linear scales ($r_p > 2 \, h^{-1}$ Mpc, $4 \, h^{-1}$ Mpc for $\gamma_t,w_p$). Much of this improvement comes from the non-linear clustering information, which breaks degeneracies among HOD parameters that would otherwise degrade the inference of matter clustering from GGL. Increasing the effective source density to $3$ arcmin$^{-2}$ sharpens the constraint on $\sigma_8\Omega_m^{0.58}$ by a further factor of two. With robust modeling into the non-linear regime, low-redshift measurements of matter clustering at the 1-percent level with clustering+GGL alone are well within reach of current data sets such as those provided by the Dark Energy Survey., Comment: 23 pages, 7 figures

In preparation for deep extragalactic imaging with the James Webb Space Telescope, we explore the clustering of massive halos at $z=8$ and $10$ using a large N-body simulation. We find that halos with masses $10^9$ to $10^{11}$ $h^{-1}\;M_\odot$, which are those expected to host galaxies detectable with JWST, are highly clustered with bias factors ranging from 5 and 30 depending strongly on mass, as well as on redshift and scale. This results in correlation lengths of 5--10$h^{-1}\;{\rm Mpc}$, similar to that of today's galaxies. Our results are based on a simulation of 130 billion particles in a box of $250h^{-1}\;{\rm Mpc}$ size using our new high-accuracy ABACUS simulation code, the corrections to cosmological initial conditions of (Garrison et al. 2016, 2016MNRAS.461.4125G), and the Planck 2015 cosmology. We use variations between sub-volumes to estimate the detectability of the clustering. Because of the very strong inter-halo clustering, we find that surveys of order 25$h^{-1}\;{\rm Mpc}$ comoving transverse size may be able to detect the clustering of $z=8$--$10$ galaxies with only 500-1000 survey objects if the galaxies indeed occupy the most massive dark matter halos.

In cosmological $N$-body simulations, the representation of dark matter as discrete "macroparticles" suppresses the growth of structure, such that simulations no longer reproduce linear theory on small scales near $k_{\rm Nyquist}$. Marcos et al. demonstrate that this is due to sparse sampling of modes near $k_{\rm Nyquist}$ and that the often-assumed continuum growing modes are not proper growing modes of the particle system. We develop initial conditions that respect the particle linear theory growing modes and then rescale the mode amplitudes to account for growth suppression. These ICs also allow us to take advantage of our very accurate $N$-body code Abacus to implement 2LPT in configuration space. The combination of 2LPT and rescaling improves the accuracy of the late-time power spectra, halo mass functions, and halo clustering. In particular, we achieve 1% accuracy in the power spectrum down to $k_{\rm Nyquist}$, versus $k_{\rm Nyquist}/4$ without rescaling or $k_{\rm Nyquist}/13$ without 2LPT, relative to an oversampled reference simulation. We anticipate that our 2LPT will be useful for large simulations where FFTs are expensive and that rescaling will be useful for suites of medium-resolution simulations used in cosmic emulators and galaxy survey mock catalogs. Code to generate initial conditions is available at https://github.com/lgarrison/zeldovich-PLT, 22 pages, 14 figures. Minor changes to match MNRAS published version. Code available at https://github.com/lgarrison/zeldovich-PLT

Dark Energy Spectroscopic Instrument (DESI) will construct a large and precise three-dimensional map of our Universe. The survey effective volume reaches ∼20h−3Gpc3⁠. It is a great challenge to prepare high-resolution simulations with a much larger volume for validating the DESI analysis pipelines. ABACUSSUMMIT is a suite of high-resolution dark-matter-only simulations designed for this purpose, with 200h−3Gpc3 (10 times DESI volume) for the base cosmology. However, further efforts need to be done to provide a more precise analysis of the data and to cover also other cosmologies. Recently, the CARPool method was proposed to use paired accurate and approximate simulations to achieve high statistical precision with a limited number of high-resolution simulations. Relying on this technique, we propose to use fast quasi-N-body solvers combined with accurate simulations to produce accurate summary statistics. This enables us to obtain 100 times smaller variance than the expected DESI statistical variance at the scales we are interested in, e.g. k, ZD and YY were supported by the National Key Basic Research and Development Program of China (No. 2018YFA0404504) and the National Science Foundation of China (grant nos. 11621303, 11890691, 11773048). LHG is supported by the Center for Computational Astrophysics at the Flatiron Institute, which is supported by the Simons Foundation. ABACUS development has been supported by NSF AST-1313285 and DOE-SC0013718, as well as by Harvard University start-up funds. This research is supported by the Director, Office of Science, Office of High Energy Physics of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231, and by the National Energy Research Scientific Computing Center, a DOE Office of Science User Facility under the same contract; additional support for DESI is provided by the U.S. National Science Foundation, Division of Astronomical Sciences under Contract No. AST-0950945 to the NSF’s National Optical-Infrared Astronomy Research Laboratory; the Science and Technologies Facilities Council of the United Kingdom; the Gordon and Betty Moore Foundation; the Heising-Simons Foundation; the French Alternative Energies and Atomic Energy Commission (CEA); the National Council of Science and Technology of Mexico; the Ministry of Economy of Spain, and by the DESI Member Institutions.