Search

Interferometric experiments designed to detect the highly redshifted 21-cm signal from neutral hydrogen are producing increasingly stringent constraints on the 21-cm power spectrum, but some k-modes remain systematics-dominated. Mutual coupling is a major systematic that must be overcome in order to detect the 21-cm signal, and simulations that reproduce effects seen in the data can guide strategies for mitigating mutual coupling. In this paper, we analyse 12 nights of data from the Hydrogen Epoch of Reionization Array and compare the data against simulations that include a computationally efficient and physically motivated semi-analytic treatment of mutual coupling. We find that simulated coupling features qualitatively agree with coupling features in the data; however, coupling features in the data are brighter than the simulated features, indicating the presence of additional coupling mechanisms not captured by our model. We explore the use of fringe-rate filters as mutual coupling mitigation tools and use our simulations to investigate the effects of mutual coupling on a simulated cosmological 21-cm power spectrum in a "worst case" scenario where the foregrounds are particularly bright. We find that mutual coupling contaminates a large portion of the "EoR Window", and the contamination is several orders-of-magnitude larger than our simulated cosmic signal across a wide range of cosmological Fourier modes. While our fiducial fringe-rate filtering strategy reduces mutual coupling by roughly a factor of 100 in power, a non-negligible amount of coupling cannot be excised with fringe-rate filters, so more sophisticated mitigation strategies are required., Comment: 19 pages, 12 figures, submitted to MNRAS

Radio interferometers targeting the 21cm brightness temperature fluctuations at high redshift are subject to systematic effects that operate over a range of different timescales. These can be isolated by designing appropriate Fourier filters that operate in fringe-rate (FR) space, the Fourier pair of local sidereal time (LST). Applications of FR filtering include separating effects that are correlated with the rotating sky vs. those relative to the ground, down-weighting emission in the primary beam sidelobes, and suppressing noise. FR filtering causes the noise contributions to the visibility data to become correlated in time however, making interpretation of subsequent averaging and error estimation steps more subtle. In this paper, we describe fringe rate filters that are implemented using discrete prolate spheroidal sequences, and designed for two different purposes -- beam sidelobe/horizon suppression (the `mainlobe' filter), and ground-locked systematics removal (the `notch' filter). We apply these to simulated data, and study how their properties affect visibilities and power spectra generated from the simulations. Included is an introduction to fringe-rate filtering and a demonstration of fringe-rate filters applied to simple situations to aid understanding., Comment: 21 pages, 18 figures, submitted to Monthly Notices of the Royal Astronomical Society

This paper presents the design and deployment of the Hydrogen Epoch of Reionization Array (HERA) phase II system. HERA is designed as a staged experiment targeting 21 cm emission measurements of the Epoch of Reionization. First results from the phase I array are published as of early 2022, and deployment of the phase II system is nearing completion. We describe the design of the phase II system and discuss progress on commissioning and future upgrades. As HERA is a designated Square Kilometer Array (SKA) pathfinder instrument, we also show a number of "case studies" that investigate systematics seen while commissioning the phase II system, which may be of use in the design and operation of future arrays. Common pathologies are likely to manifest in similar ways across instruments, and many of these sources of contamination can be mitigated once the source is identified.

Detection of the faint 21 cm line emission from the Cosmic Dawn and Epoch of Reionisation will require not only exquisite control over instrumental calibration and systematics to achieve the necessary dynamic range of observations but also validation of analysis techniques to demonstrate their statistical properties and signal loss characteristics. A key ingredient in achieving this is the ability to perform high-fidelity simulations of the kinds of data that are produced by the large, many-element, radio interferometric arrays that have been purpose-built for these studies. The large scale of these arrays presents a computational challenge, as one must simulate a detailed sky and instrumental model across many hundreds of frequency channels, thousands of time samples, and tens of thousands of baselines for arrays with hundreds of antennas. In this paper, we present a fast matrix-based method for simulating radio interferometric measurements (visibilities) at the necessary scale. We achieve this through judicious use of primary beam interpolation, fast approximations for coordinate transforms, and a vectorised outer product to expand per-antenna quantities to per-baseline visibilities, coupled with standard parallelisation techniques. We validate the results of this method, implemented in the publicly-available matvis code, against a high-precision reference simulator, and explore its computational scaling on a variety of problems., Comment: 25 pages, 20 figures, submitted to RAS Techniques and Instruments, matvis is publicly available at https://github.com/HERA-Team/matvis

Observations with radio arrays that target the 21-cm signal originating from the early Universe suffer from a variety of systematic effects. An important class of these are reflections and spurious couplings between antennas. We apply a Hamiltonian Monte Carlo sampler to the modelling and mitigation of these systematics in simulated Hydrogen Epoch of Reionisation Array (HERA) data. This method allows us to form statistical uncertainty estimates for both our models and the recovered visibilities, which is an important ingredient in establishing robust upper limits on the Epoch of Reionisation (EoR) power spectrum. In cases where the noise is large compared to the EoR signal, this approach can constrain the systematics well enough to mitigate them down to the noise level for both systematics studied. Where the noise is smaller than the EoR, our modelling can mitigate the majority of the reflections with there being only a minor level of residual systematics, while cross-coupling sees essentially complete mitigation. Our approach performs similarly to existing filtering/fitting techniques used in the HERA pipeline, but with the added benefit of rigorously propagating uncertainties. In all cases it does not significantly attenuate the underlying signal., Comment: 19 pages, 14 figures, submitted to MNRAS

The key to detecting neutral hydrogen during the epoch of reionization (EoR) is to separate the cosmological signal from the dominating foreground radiation. We developed direct optimal mapping (DOM) to map interferometric visibilities; it contains only linear operations, with full knowledge of point spread functions from visibilities to images. Here, we demonstrate a fast Fourier transform-based image power spectrum and its window functions computed from the DOM images. We use noiseless simulation, based on the Hydrogen Epoch of Reionization Array Phase I configuration, to study the image power spectrum properties. The window functions show $<10^{-11}$ of the integrated power leaks from the foreground-dominated region into the EoR window; the 2D and 1D power spectra also verify the separation between the foregrounds and the EoR., Comment: Published in ApJ

Radio interferometers aiming to measure the power spectrum of the redshifted 21 cm line during the Epoch of Reionisation (EoR) need to achieve an unprecedented dynamic range to separate the weak signal from overwhelming foreground emissions. Calibration inaccuracies can compromise the sensitivity of these measurements to the effect that a detection of the EoR is precluded. An alternative to standard analysis techniques makes use of the closure phase, which allows one to bypass antenna-based direction-independent calibration. Similarly to standard approaches, we use a delay spectrum technique to search for the EoR signal. Using 94 nights of data observed with Phase I of the Hydrogen Epoch of Reionization Array (HERA), we place approximate constraints on the 21 cm power spectrum at $z=7.7$. We find at 95% confidence that the 21 cm EoR brightness temperature is $\le$(372)$^2$ "pseudo" mK$^2$ at 1.14 "pseudo" $h$ Mpc$^{-1}$, where the "pseudo" emphasises that these limits are to be interpreted as approximations to the actual distance scales and brightness temperatures. Using a fiducial EoR model, we demonstrate the feasibility of detecting the EoR with the full array. Compared to standard methods, the closure phase processing is relatively simple, thereby providing an important independent check on results derived using visibility intensities, or related., Comment: 16 pages, 14 figures, accepted for publication by MNRAS

Radio Frequency Interference (RFI) is one of the systematic challenges preventing 21cm interferometric instruments from detecting the Epoch of Reionization. To mitigate the effects of RFI on data analysis pipelines, numerous inpaint techniques have been developed to restore RFI corrupted data. We examine the qualitative and quantitative errors introduced into the visibilities and power spectrum due to inpainting. We perform our analysis on simulated data as well as real data from the Hydrogen Epoch of Reionization Array (HERA) Phase 1 upper limits. We also introduce a convolutional neural network that capable of inpainting RFI corrupted data in interferometric instruments. We train our network on simulated data and show that our network is capable at inpainting real data without requiring to be retrained. We find that techniques that incorporate high wavenumbers in delay space in their modeling are best suited for inpainting over narrowband RFI. We also show that with our fiducial parameters Discrete Prolate Spheroidal Sequences (DPSS) and CLEAN provide the best performance for intermittent ``narrowband'' RFI while Gaussian Progress Regression (GPR) and Least Squares Spectral Analysis (LSSA) provide the best performance for larger RFI gaps. However we caution that these qualitative conclusions are sensitive to the chosen hyperparameters of each inpainting technique. We find these results to be consistent in both simulated and real visibilities. We show that all inpainting techniques reliably reproduce foreground dominated modes in the power spectrum. Since the inpainting techniques should not be capable of reproducing noise realizations, we find that the largest errors occur in the noise dominated delay modes. We show that in the future, as the noise level of the data comes down, CLEAN and DPSS are most capable of reproducing the fine frequency structure in the visibilities of HERA data., Comment: 21 pages, 13 figures

We report the most sensitive upper limits to date on the 21 cm epoch of reionization power spectrum using 94 nights of observing with Phase I of the Hydrogen Epoch of Reionization Array (HERA). Using similar analysis techniques as in previously reported limits (HERA Collaboration 2022a), we find at 95% confidence that $\Delta^2(k = 0.34$ $h$ Mpc$^{-1}$) $\leq 457$ mK$^2$ at $z = 7.9$ and that $\Delta^2 (k = 0.36$ $h$ Mpc$^{-1}) \leq 3,496$ mK$^2$ at $z = 10.4$, an improvement by a factor of 2.1 and 2.6 respectively. These limits are mostly consistent with thermal noise over a wide range of $k$ after our data quality cuts, despite performing a relatively conservative analysis designed to minimize signal loss. Our results are validated with both statistical tests on the data and end-to-end pipeline simulations. We also report updated constraints on the astrophysics of reionization and the cosmic dawn. Using multiple independent modeling and inference techniques previously employed by HERA Collaboration (2022b), we find that the intergalactic medium must have been heated above the adiabatic cooling limit at least as early as $z = 10.4$, ruling out a broad set of so-called "cold reionization" scenarios. If this heating is due to high-mass X-ray binaries during the cosmic dawn, as is generally believed, our result's 99% credible interval excludes the local relationship between soft X-ray luminosity and star formation and thus requires heating driven by evolved low-metallicity stars., Comment: 57 pages, 37 figures. Updated to match the accepted ApJ version. Corresponding author: Joshua S. Dillon

Combining the visibilities measured by an interferometer to form a cosmological power spectrum is a complicated process. In a delay-based analysis, the mapping between instrumental and cosmological space is not a one-to-one relation. Instead, neighbouring modes contribute to the power measured at one point, with their respective contributions encoded in the window functions. To better understand the power measured by an interferometer, we assess the impact of instrument characteristics and analysis choices on these window functions. Focusing on the Hydrogen Epoch of Reionization Array (HERA) as a case study, we find that long-baseline observations correspond to enhanced low-k tails of the window functions, which facilitate foreground leakage, whilst an informed choice of bandwidth and frequency taper can reduce said tails. With simple test cases and realistic simulations, we show that, apart from tracing mode mixing, the window functions help accurately reconstruct the power spectrum estimator of simulated visibilities. The window functions depend strongly on the beam chromaticity, and less on its spatial structure - a Gaussian approximation, ignoring side lobes, is sufficient. Finally, we investigate the potential of asymmetric window functions, down-weighting the contribution of low-k power to avoid foreground leakage. The window functions presented here correspond to the latest HERA upper limits for the full Phase I data. They allow an accurate reconstruction of the power spectrum measured by the instrument and will be used in future analyses to confront theoretical models and data directly in cylindrical space., Comment: 18 pages, 19 figures, accepted for publication in MNRAS

Reionization of helium is expected to occur at redshifts $z\sim3$ and have important consequences for quasar populations, galaxy formation, and the morphology of the intergalactic medium, but there is little known empirically about the process. Here we show that kinetic Sunyaev-Zeldovich (kSZ) tomography, based on the combination of CMB measurements and galaxy surveys, can be used to infer the primordial helium abundance as well as the time and duration of helium reionization. We find a high-significance detection at ${\sim10\sigma}$ can be expected from Vera Rubin Observatory and CMB-S4 in the near future. A more robust characterization of helium reionization will require next-generation experiments like MegaMapper (a proposed successor to DESI) and CMB-HD., Comment: 4+2 pages, 2 figures, comments welcome

The reionization of the intergalactic medium at redshifts $z\gtrsim 6$ is expected to have a lasting impact on galaxies residing in low-mass dark matter halos. Unable to accrete or retain gas photo-heated to temperatures $T \gtrsim 10^4$ K, the star formation histories of faint galaxies in the early Universe are expected to decline as they exhaust their gas supply, resulting in a turn-over in the galaxy luminosity function (LF) and a potential solution to the missing satellites problem in the local group. Unfortunately, there are several challenges to constraining `reionization feedback' empirically, most notably that galaxies in low-mass halos are intrinsically faint, and that there are other physical mechanisms capable of inducing a turn-over in the LF. In this work, we investigate a new signature of reionization feedback that is in principle distinct from other processes: as faint galaxies are quenched by reionization, their stellar populations passively age and grow redder while the brighter galaxies nearby continue to form stars at an increasing rate and so remain relatively blue. We find that this contrast, between quenched and un-quenched galaxies induces a scale and colour-dependent signature in the present-day near-infrared background comparable to the expected sensitivity of NASA's upcoming SPHEREx mission. Whereas models with pure mass suppression largely affect the signal at wavelengths $\lesssim 2 \mu\rm{m}$, $\sim 5$%-level differences in the background persist out to $\sim 5 \mu\rm{m}$ for reionization feedback models. Finally, the power spectra of intensity ratio maps exhibit larger variations, and may thus be a promising target for future analyses., Comment: 13 pages, 7 figures, submitted to MNRAS

The cross-correlation between the 21 cm field and the galaxy distribution is a potential probe of the Epoch of Reionization (EoR). The 21 cm signal traces neutral gas in the intergalactic medium and, on large spatial scales, this should be anti-correlated with the high-redshift galaxy distribution which partly sources and tracks the ionized gas. In the near future, interferometers such as the Hydrogen Epoch of Reionization Array (HERA) are projected to provide extremely sensitive measurements of the 21 cm power spectrum. At the same time, the Nancy Grace Roman Space Telescope (Roman) will produce the most extensive catalog to date of bright galaxies from the EoR. Using semi-numeric simulations of reionization, we explore the prospects for measuring the cross-power spectrum between the 21 cm and galaxy fields during the EoR. We forecast a 14$\sigma$ detection between HERA and Roman, assuming an overlapping survey area of 500 deg$^2$, redshift uncertainties of $\sigma_z = 0.01$ (as expected for the high-latitude spectroscopic survey of Ly$\alpha$-emitting galaxies), and an effective Ly$\alpha$ emitter duty cycle of $f_\mathrm{LAE} = 0.1$. Thus the HERA-Roman cross-power spectrum may be used to help verify 21 cm detections from HERA. We find that the shot-noise in the galaxy distribution is a limiting factor for detection, and so supplemental observations using Roman should prioritize deeper observations, rather than covering a wider field of view., Comment: 26 pages, 15 figures, accepted at ApJ

Motivated by the desire for wide-field images with well-defined statistical properties for 21cm cosmology, we implement an optimal mapping pipeline that computes a maximum likelihood estimator for the sky using the interferometric measurement equation. We demonstrate this direct optimal mapping with data from the Hydrogen Epoch of Reionization (HERA) Phase I observations. After validating the pipeline with simulated data, we develop a maximum likelihood figure-of-merit for comparing four sky models at 166MHz with a bandwidth of 100kHz. The HERA data agree with the GLEAM catalogs to <10%. After subtracting the GLEAM point sources, the HERA data discriminate between the different continuum sky models, providing most support for the model of Byrne et al. 2021. We report the computation cost for mapping the HERA Phase I data and project the computation for the HERA 320-antenna data; both are feasible with a modern server. The algorithm is broadly applicable to other interferometers and is valid for wide-field and non-coplanar arrays., Comment: 16 pages, 10 figures, 2 tables, published on ApJ

Interpreting 21cm measurements from current and upcoming experiments like HERA and the SKA will provide new scientific insights and exciting implications for astrophysics and cosmology regarding the Epoch of Reionization (EoR). Several recent works have proposed using machine learning methods, such as convolutions neural networks (CNNs), to analyze images of reionization generated by these experiments since they could take full advantage of information contained in the image. Generally, these studies have used only a single semi-numeric method to generate the input 21cm data. In this work, we investigate the extent to which training CNNs for reionization applications depends on the underlying semi-numeric models. Working in the context of predicting CMB optical depth from 21cm images, we compare networks trained on similar datasets from 21cmfast and zreion, two widely used semi-numeric reionization methods. We show that neural networks trained on input data from only one model produce poor predictions on data from the other model. Satisfactory results are only achieved when both models are included in the training data. This finding has important implications for future analyses on observation data, and encourages the use of multiple models to produce images that capture the full complexity of the EoR., Comment: 33 pages, 18 figures

We explore a new approach for extracting reionization-era contributions to the kinetic Sunyaev-Zel'dovich (kSZ) effect. Our method utilizes the cross-power spectrum between filtered and squared maps of the cosmic microwave background (CMB) and photometric galaxy surveys during the Epoch of Reionization (EoR). This kSZ$^2$-galaxy cross-power spectrum statistic has been successfully detected at lower redshifts ($z \lesssim 1.5$). Here we extend this method to $z \gtrsim 6$ as a potential means to extract signatures of patchy reionization. We model the expected signal across multiple photometric redshift bins using semi-numeric simulations of the reionization process. In principle, the cross-correlation statistic robustly extracts reionization-era contributions to the kSZ signal, while its redshift evolution yields valuable information regarding the timing of reionization. Specifically, the model cross-correlation signal near $\ell \sim 1,000$ peaks during the early stages of the EoR, when about 20% of the volume of the universe is ionized. Detectible $\ell$ modes mainly reflect squeezed triangle configurations of the related bispectrum, quantifying correlations between the galaxy overdensity field on large scales and the smaller-scale kSZ power. We forecast the prospects for detecting this signal using future wide-field samples of Lyman-break galaxies from the Roman Space Telescope and next-generation CMB surveys including the Simons Observatory, CMB-S4, and CMB-HD. We find that a roughly 13$\sigma$ detection is possible for CMB-HD and Roman after summing over all $\ell$ modes. We discuss the possibilities for improving this approach and related statistics, with the aim of moving beyond simple detections to measure the scale and redshift dependence of the cross-correlation signals., Comment: 19 pages, 12 figures, published in ApJ

We present a framework for identifying and flagging malfunctioning antennas in large radio interferometers. We outline two distinct categories of metrics designed to detect outliers along known failure modes of large arrays: cross-correlation metrics, based on all antenna pairs, and auto-correlation metrics, based solely on individual antennas. We define and motivate the statistical framework for all metrics used, and present tailored visualizations that aid us in clearly identifying new and existing systematics. We implement these techniques using data from 105 antennas in the Hydrogen Epoch of Reionization Array (HERA) as a case study. Finally, we provide a detailed algorithm for implementing these metrics as flagging tools on real data sets., Comment: 31 pages, 17 figures

Recently, the Hydrogen Epoch of Reionization Array (HERA) collaboration has produced the experiment's first upper limits on the power spectrum of 21-cm fluctuations at z~8 and 10. Here, we use several independent theoretical models to infer constraints on the intergalactic medium (IGM) and galaxies during the epoch of reionization (EoR) from these limits. We find that the IGM must have been heated above the adiabatic cooling threshold by z~8, independent of uncertainties about the IGM ionization state and the nature of the radio background. Combining HERA limits with galaxy and EoR observations constrains the spin temperature of the z~8 neutral IGM to 27 K < T_S < 630 K (2.3 K < T_S < 640 K) at 68% (95%) confidence. They therefore also place a lower bound on X-ray heating, a previously unconstrained aspects of early galaxies. For example, if the CMB dominates the z~8 radio background, the new HERA limits imply that the first galaxies produced X-rays more efficiently than local ones (with soft band X-ray luminosities per star formation rate constrained to L_X/SFR = { 10^40.2, 10^41.9 } erg/s/(M_sun/yr) at 68% confidence), consistent with expectations of X-ray binaries in low-metallicity environments. The z~10 limits require even earlier heating if dark-matter interactions (e.g., through millicharges) cool down the hydrogen gas. Using a model in which an extra radio background is produced by galaxies, we rule out (at 95% confidence) the combination of high radio and low X-ray luminosities of L_{r,\nu}/SFR > 3.9 x 10^24 W/Hz/(M_sun/yr) and L_X/SFR<10^40 erg/s/(M_sun/yr). The new HERA upper limits neither support nor disfavor a cosmological interpretation of the recent EDGES detection. The analysis framework described here provides a foundation for the interpretation of future HERA results., Comment: 40 pages, 19 figures, accepted to ApJ

We report upper-limits on the Epoch of Reionization (EoR) 21 cm power spectrum at redshifts 7.9 and 10.4 with 18 nights of data ($\sim36$ hours of integration) from Phase I of the Hydrogen Epoch of Reionization Array (HERA). The Phase I data show evidence for systematics that can be largely suppressed with systematic models down to a dynamic range of $\sim10^9$ with respect to the peak foreground power. This yields a 95% confidence upper limit on the 21 cm power spectrum of $\Delta^2_{21} \le (30.76)^2\ {\rm mK}^2$ at $k=0.192\ h\ {\rm Mpc}^{-1}$ at $z=7.9$, and also $\Delta^2_{21} \le (95.74)^2\ {\rm mK}^2$ at $k=0.256\ h\ {\rm Mpc}^{-1}$ at $z=10.4$. At $z=7.9$, these limits are the most sensitive to-date by over an order of magnitude. While we find evidence for residual systematics at low line-of-sight Fourier $k_\parallel$ modes, at high $k_\parallel$ modes we find our data to be largely consistent with thermal noise, an indicator that the system could benefit from deeper integrations. The observed systematics could be due to radio frequency interference, cable sub-reflections, or residual instrumental cross-coupling, and warrant further study. This analysis emphasizes algorithms that have minimal inherent signal loss, although we do perform a careful accounting in a companion paper of the small forms of loss or bias associated with the pipeline. Overall, these results are a promising first step in the development of a tuned, instrument-specific analysis pipeline for HERA, particularly as Phase II construction is completed en route to reaching the full sensitivity of the experiment., Comment: Accepted to ApJ. https://reionization.org/science/public-data-release-1/

Precision calibration poses challenges to experiments probing the redshifted 21-cm signal of neutral hydrogen from the Cosmic Dawn and Epoch of Reionization (z~30-6). In both interferometric and global signal experiments, systematic calibration is the leading source of error. Though many aspects of calibration have been studied, the overlap between the two types of instruments has received less attention. We investigate the sky based calibration of total power measurements with a HERA dish and an EDGES style antenna to understand the role of auto-correlations in the calibration of an interferometer and the role of sky in calibrating a total power instrument. Using simulations we study various scenarios such as time variable gain, incomplete sky calibration model, and primary beam model. We find that temporal gain drifts, sky model incompleteness, and beam inaccuracies cause biases in the receiver gain amplitude and the receiver temperature estimates. In some cases, these biases mix spectral structure between beam and sky resulting in spectrally variable gain errors. Applying the calibration method to the HERA and EDGES data, we find good agreement with calibration via the more standard methods. Although instrumental gains are consistent with beam and sky errors similar in scale to those simulated, the receiver temperatures show significant deviations from expected values. While we show that it is possible to partially mitigate biases due to model inaccuracies by incorporating a time-dependent gain model in calibration, the resulting errors on calibration products are larger and more correlated. Completely addressing these biases will require more accurate sky and primary beam models., Comment: 16 pages, 13 figures, 1 table; accepted for publication in MNRAS main journal

We describe the validation of the HERA Phase I software pipeline by a series of modular tests, building up to an end-to-end simulation. The philosophy of this approach is to validate the software and algorithms used in the Phase I upper limit analysis on wholly synthetic data satisfying the assumptions of that analysis, not addressing whether the actual data meet these assumptions. We discuss the organization of this validation approach, the specific modular tests performed, and the construction of the end-to-end simulations. We explicitly discuss the limitations in scope of the current simulation effort. With mock visibility data generated from a known analytic power spectrum and a wide range of realistic instrumental effects and foregrounds, we demonstrate that the current pipeline produces power spectrum estimates that are consistent with known analytic inputs to within thermal noise levels (at the 2 sigma level) for k > 0.2 h/Mpc for both bands and fields considered. Our input spectrum is intentionally amplified to enable a strong `detection' at k ~0.2 h/Mpc -- at the level of ~25 sigma -- with foregrounds dominating on larger scales, and thermal noise dominating at smaller scales. Our pipeline is able to detect this amplified input signal after suppressing foregrounds with a dynamic range (foreground to noise ratio) of > 10^7. Our validation test suite uncovered several sources of scale-independent signal loss throughout the pipeline, whose amplitude is well-characterized and accounted for in the final estimates. We conclude with a discussion of the steps required for the next round of data analysis., Comment: 32 pages, 20 figures. Submitted to the Astrophysical Journal

As current- and next-generation astronomical instruments come online, they will generate an unprecedented deluge of data. Analyzing these data in real time presents unique conceptual and computational challenges, and their long-term storage and archiving is scientifically essential for generating reliable, reproducible results. We present here the real-time processing (RTP) system for the Hydrogen Epoch of Reionization Array (HERA), a radio interferometer endeavoring to provide the first detection of the highly redshifted 21 cm signal from Cosmic Dawn and the Epoch of Reionization by an interferometer. The RTP system consists of analysis routines run on raw data shortly after they are acquired, such as calibration and detection of radio-frequency interference (RFI) events. RTP works closely with the Librarian, the HERA data storage and transfer manager which automatically ingests data and transfers copies to other clusters for post-processing analysis. Both the RTP system and the Librarian are public and open source software, which allows for them to be modified for use in other scientific collaborations. When fully constructed, HERA is projected to generate over 50 terabytes (TB) of data each night, and the RTP system enables the successful scientific analysis of these data., Comment: 15 pages, 3 figures, published in Astronomy and Computing

Upcoming measurements of the high-redshift 21 cm signal from the Epoch of Reionization (EoR) are a promising probe of the astrophysics of the first galaxies and of cosmological parameters. In particular, the optical depth $\tau$ to the last scattering surface of the cosmic microwave background (CMB) should be tightly constrained by direct measurements of the neutral hydrogen state at high redshift. A robust measurement of $\tau$ from 21 cm data would help eliminate it as a nuisance parameter from CMB estimates of cosmological parameters. Previous proposals for extracting $\tau$ from future 21 cm datasets have typically used the 21 cm power spectra generated by semi-numerical models to reconstruct the reionization history. We present here a different approach which uses convolution neural networks (CNNs) trained on mock images of the 21 cm EoR signal to extract $\tau$. We construct a CNN that improves upon on previously proposed architectures, and perform an automated hyperparameter optimization. We show that well-trained CNNs are able to accurately predict $\tau$, even when removing Fourier modes that are expected to be corrupted by bright foreground contamination of the 21 cm signal. Typical random errors for an optimized network are less than $3.06\%$, with biases factors of several smaller. While preliminary, this approach could yield constraints on $\tau$ that improve upon sample-variance limited measurements of the low-$\ell$ EE observations of the CMB, making this approach a valuable complement to more traditional methods of inferring $\tau$., Comment: 28 pages, 9 figures, accepted in PASP

Precise measurements of the 21 cm power spectrum are crucial for understanding the physical processes of hydrogen reionization. Currently, this probe is being pursued by low-frequency radio interferometer arrays. As these experiments come closer to making a first detection of the signal, error estimation will play an increasingly important role in setting robust measurements. Using the delay power spectrum approach, we have produced a critical examination of different ways that one can estimate error bars on the power spectrum. We do this through a synthesis of analytic work, simulations of toy models, and tests on small amounts of real data. We find that, although computed independently, the different error bar methodologies are in good agreement with each other in the noise-dominated regime of the power spectrum. For our preferred methodology, the predicted probability distribution function is consistent with the empirical noise power distributions from both simulated and real data. This diagnosis is mainly in support of the forthcoming HERA upper limit, and also is expected to be more generally applicable., Comment: 35 Pages, 9 Figures, 4 Tables. Replaced with accepted ApJ version; some clarifying text added in response to referee comments with no changes to results

Upcoming galaxy surveys and 21-cm experiments targeting high redshifts $z\gtrsim 6$ are highly complementary probes of galaxy formation and reionization. However, in order to expedite the large volume simulations relevant for 21-cm observations, many models of galaxies within reionization codes are entirely subgrid and/or rely on halo abundances only. In this work, we explore the extent to which resolving and modeling individual galaxy formation histories affects predictions both for the galaxy populations detectable by upcoming surveys and the signatures of reionization accessible to upcoming 21-cm experiments. We find that a common approach, in which galaxy luminosity is assumed to be a function of halo mass only, is biased with respect to models in which galaxy properties are evolved through time via semi-analytic modeling and thus reflective of the diversity of assembly histories that naturally arise in $N$-body simulations. The diversity of galaxy formation histories also results in scenarios in which the brightest galaxies do \textit{not} always reside in the centers of large ionized regions, as there are often relatively low-mass halos undergoing dramatic, but short-term, growth. This has clear implications for attempts to detect or validate the 21-cm background via cross correlation. Finally, we show that a hybrid approach -- in which only halos hosting galaxies bright enough to be detected in surveys are modeled in detail, with the rest modeled as an unresolved field of halos with abundance related to large-scale overdensity -- is a viable way to generate large-volume `simulations` well-suited to wide-area surveys and current-generation 21-cm experiments targeting relatively large $k \lesssim 1 \ h \ \text{Mpc}^{-1}$ scales., Comment: 17 pages, 14 figures, accepted for publication in MNRAS

The Hydrogen Epoch of Reionization Array (HERA) is a radio telescope in the Karoo desert of South Africa endeavoring to observe Cosmic Dawn and the Epoch of Reionization. When fully constructed, it will consist of 350 antennas and generate over 60 terabytes (TB) of data each night. In order to keep pace with the relatively large rate of data, we have developed the real-time processing (RTP) system for HERA. The RTP is responsible for inspecting data for acceptable levels of quality and flagging unusable data, as well as providing an initial calibration solution. The RTP system consists of the pipeline management system, called the hera_opm package, and the data management system, called the librarian. Though the systems have been developed in the context of analyzing HERA data, they feature public and open source software, and have been designed to be adapted and used in other contexts as necessary., Comment: 4 pages, 1 figure, presented at URSI GASS 2020, submitted to Radio Science Letters

The central challenge in 21~cm cosmology is isolating the cosmological signal from bright foregrounds. Many separation techniques rely on the accurate knowledge of the sky and the instrumental response, including the antenna primary beam. For drift-scan telescopes such as the Hydrogen Epoch of Reionization Array \citep[HERA, ][]{DeBoer2017} that do not move, primary beam characterization is particularly challenging because standard beam-calibration routines do not apply \citep{Cornwell2005} and current techniques require accurate source catalogs at the telescope resolution. We present an extension of the method from \citet{Pober2012} where they use beam symmetries to create a network of overlapping source tracks that break the degeneracy between source flux density and beam response and allow their simultaneous estimation. We fit the beam response of our instrument using early HERA observations and find that our results agree well with electromagnetic simulations down to a -20~dB level in power relative to peak gain for sources with high signal-to-noise ratio. In addition, we construct a source catalog with 90 sources down to a flux density of 1.4~Jy at 151~MHz., Comment: 22 pages, 22 figures, accepted for publication in ApJ

Characterizing the epoch of reionization (EoR) at $z\gtrsim 6$ via the redshifted 21 cm line of neutral Hydrogen (HI) is critical to modern astrophysics and cosmology, and thus a key science goal of many current and planned low-frequency radio telescopes. The primary challenge to detecting this signal is the overwhelmingly bright foreground emission at these frequencies, placing stringent requirements on the knowledge of the instruments and inaccuracies in analyses. Results from these experiments have largely been limited not by thermal sensitivity but by systematics, particularly caused by the inability to calibrate the instrument to high accuracy. The interferometric bispectrum phase is immune to antenna-based calibration and errors therein, and presents an independent alternative to detect the EoR HI fluctuations while largely avoiding calibration systematics. Here, we provide a demonstration of this technique on a subset of data from the Hydrogen Epoch of Reionization Array (HERA) to place approximate constraints on the brightness temperature of the intergalactic medium (IGM). From this limited data, at $z=7.7$ we infer "$1\sigma$" upper limits on the IGM brightness temperature to be $\le 316$ "pseudo" mK at $\kappa_\parallel=0.33$ "pseudo" $h$ Mpc$^{-1}$ (data-limited) and $\le 1000$ "pseudo" mK at $\kappa_\parallel=0.875$ "pseudo" $h$ Mpc$^{-1}$ (noise-limited). The "pseudo" units denote only an approximate and not an exact correspondence to the actual distance scales and brightness temperatures. By propagating models in parallel to the data analysis, we confirm that the dynamic range required to separate the cosmic HI signal from the foregrounds is similar to that in standard approaches, and the power spectrum of the bispectrum phase is still data-limited (at $\gtrsim 10^6$ dynamic range) indicating scope for further improvement in sensitivity as the array build-out continues., Comment: 22 pages, 12 figures (including sub-figures). Published in PhRvD. Abstract may be slightly abridged compared to the actual manuscript due to length limitations on arXiv

The high-redshift 21 cm signal from the Epoch of Reionization (EoR) is a promising observational probe of the early universe. Current- and next-generation radio interferometers such as the Hydrogen Epoch of Reionization Array (HERA) and Square Kilometre Array (SKA) are projected to measure the 21 cm auto power spectrum from the EoR. Another observational signal of this era is the kinetic Sunyaev-Zel'dovich (kSZ) signal in the cosmic microwave background (CMB), which will be observed by the upcoming Simons Observatory (SO) and CMB-S4 experiments. The 21 cm signal and the contribution to the kSZ from the EoR are expected to be anti-correlated, the former coming from regions of neutral gas in the intergalactic medium and the latter coming from ionized regions. However, the naive cross-correlation between the kSZ and 21 cm maps suffers from a cancellation that occurs because ionized regions are equally likely to be moving toward or away from the observer and so there is no net correlation with the 21 cm signal. We present here an investigation of the 21 cm-kSZ-kSZ bispectrum, which should not suffer the same cancellation as the simple two-point cross-correlation. We show that there is a significant and non-vanishing signal that is sensitive to the reionization history, suggesting the statistic may be used to confirm or infer the ionization fraction as a function of redshift. In the absence of foreground contamination, we forecast that this signal is detectable at high statistical significance with HERA and SO. The bispectrum we study suffers from the fact that the kSZ signal is sensitive only to Fourier modes with long-wavelength line-of-sight components, which are generally lost in the 21 cm data sets owing to foreground contamination. We discuss possible strategies for alleviating this contamination, including an alternative four-point statistic that may help circumvent this issue., Comment: 24 pages, 13 figures

We introduce DAYENU, a linear, spectral filter for HI intensity mapping that achieves the desirable foreground mitigation and error minimization properties of inverse co-variance weighting with minimal modeling of the underlying data. Beyond 21 cm power-spectrum estimation, our filter is suitable for any analysis where high dynamic-range removal of spectrally smooth foregrounds in irregularly (or regularly) sampled data is required, something required by many other intensity mapping techniques. Our filtering matrix is diagonalized by Discrete Prolate Spheroidal Sequences which are an optimal basis to model band-limited foregrounds in 21 cm intensity mapping experiments in the sense that they maximally concentrate power within a finite region of Fourier space. We show that DAYENU enables the access of large-scale line-of-sight modes that are inaccessible to tapered DFT estimators. Since these modes have the largest SNRs, DAYENU significantly increases the sensitivity of 21 cm analyses over tapered Fourier transforms. Slight modifications allow us to use DAYENU as a linear replacement for iterative delay CLEANing (DAYENUREST). We refer readers to the Code section at the end of this paper for links to examples and code., Comment: Accepted to MNRAS, 19 pages, 17 figures. Replacement matches accepted version

In 21 cm cosmology, precision calibration is key to the separation of the neutral hydrogen signal from very bright but spectrally-smooth astrophysical foregrounds. The Hydrogen Epoch of Reionization Array (HERA), an interferometer specialized for 21 cm cosmology and now under construction in South Africa, was designed to be largely calibrated using the self-consistency of repeated measurements of the same interferometric modes. This technique, known as "redundant-baseline calibration" resolves most of the internal degrees of freedom in the calibration problem. It assumes, however, on antenna elements with identical primary beams placed precisely on a redundant grid. In this work, we review the detailed implementation of the algorithms enabling redundant-baseline calibration and report results with HERA data. We quantify the effects of real-world non-redundancy and how they compare to the idealized scenario in which redundant measurements differ only in their noise realizations. Finally, we study how non-redundancy can produce spurious temporal structure in our calibration solutions--both in data and in simulations--and present strategies for mitigating that structure., Comment: 24 Pages, 19 Figures. Updated to match the accepted MNRAS version

We discuss absolute calibration strategies for Phase I of the Hydrogen Epoch of Reionization Array (HERA), which aims to measure the cosmological 21 cm signal from the Epoch of Reionization (EoR). HERA is a drift-scan array with a 10 degree wide field of view, meaning bright, well-characterized point source transits are scarce. This, combined with HERA's redundant sampling of the uv plane and the modest angular resolution of the Phase I instrument, make traditional sky-based and self-calibration techniques difficult to implement with high dynamic range. Nonetheless, in this work we demonstrate calibration for HERA using point source catalogues and electromagnetic simulations of its primary beam. We show that unmodeled diffuse flux and instrumental contaminants can corrupt the gain solutions, and present a gain smoothing approach for mitigating their impact on the 21 cm power spectrum. We also demonstrate a hybrid sky and redundant calibration scheme and compare it to pure sky-based calibration, showing only a marginal improvement to the gain solutions at intermediate delay scales. Our work suggests that the HERA Phase I system can be well-calibrated for a foreground-avoidance power spectrum estimator by applying direction-independent gains with a small set of degrees of freedom across the frequency and time axes., Comment: Accepted to ApJ

We present a study of internal reflection and cross coupling systematics in Phase I of the Hydrogen Epoch of Reionization Array (HERA). In a companion paper, we outlined the mathematical formalism for such systematics and presented algorithms for modeling and removing them from the data. In this work, we apply these techniques to data from HERA's first observing season as a method demonstration. The data show evidence for systematics that, without removal, would hinder a detection of the 21 cm power spectrum for the targeted EoR line-of-sight modes in the range 0.2 < k_parallel < 0.5\ h^-1 Mpc. After systematic removal, we find we can recover these modes in the power spectrum down to the integrated noise-floor of a nightly observation, achieving a dynamic range in the EoR window of 10^-6 in power (mK^2 units) with respect to the bright galactic foreground signal. In the absence of other systematics and assuming the systematic suppression demonstrated here continues to lower noise levels, our results suggest that fully-integrated HERA Phase I may have the capacity to set competitive upper limits on the 21 cm power spectrum. For future observing seasons, HERA will have upgraded analog and digital hardware to better control these systematics in the field., Comment: Submitted to ApJ

The detection of the Epoch of Reionization (EoR) delay power spectrum using a "foreground avoidance method" highly depends on the instrument chromaticity. The systematic effects induced by the radio-telescope spread the foreground signal in the delay domain, which contaminates the EoR window theoretically observable. Applied to the Hydrogen Epoch of Reionization Array (HERA), this paper combines detailed electromagnetic and electrical simulations in order to model the chromatic effects of the instrument, and quantify its frequency and time responses. In particular, the effects of the analogue receiver, transmission cables, and mutual coupling are included. These simulations are able to accurately predict the intensity of the reflections occurring in the 150-m cable which links the antenna to the back-end. They also show that electromagnetic waves can propagate from one dish to another one through large sections of the array due to mutual coupling. The simulated system time response is attenuated by a factor $10^{4}$ after a characteristic delay which depends on the size of the array and on the antenna position. Ultimately, the system response is attenuated by a factor $10^{5}$ after 1400 ns because of the reflections in the cable, which corresponds to characterizable ${k_\parallel}$-modes above 0.7 $h\;\rm{Mpc}^{-1}$ at 150 MHz. Thus, this new study shows that the detection of the EoR signal with HERA Phase I will be more challenging than expected. On the other hand, it improves our understanding of the telescope, which is essential to mitigate the instrument chromaticity., Comment: 13 pages, 17 figures - Submitted to MNRAS - 2nd revision

In this white paper, we lay out a US roadmap for high-redshift 21 cm cosmology (30 < z < 6) in the 2020s. Beginning with the currently-funded HERA and MWA Phase II projects and advancing through the decade with a coordinated program of small-scale instrumentation, software, and analysis projects targeting technology development, this roadmap incorporates our current best understanding of the systematics confronting 21 cm cosmology into a plan for overcoming them, enabling next-generation, mid-scale 21 cm arrays to be proposed late in the decade. Submitted for consideration by the Astro2020 Decadal Survey Program Panel for Radio, Millimeter, and Submillimeter Observations from the Ground as a Medium-Sized Project., Comment: 10 pages (plus a cover page and references), 6 figures. Submitted as a APC White Paper for Astro2020

In response to the need for the Astro2020 Decadal Survey to explicitly engage early career astronomers, the National Academies of Sciences, Engineering, and Medicine hosted the Early Career Astronomer and Astrophysicist Focus Session (ECFS) on October 8-9, 2018 under the auspices of Committee of Astronomy and Astrophysics. The meeting was attended by fifty six pre-tenure faculty, research scientists, postdoctoral scholars, and senior graduate students, as well as eight former decadal survey committee members, who acted as facilitators. The event was designed to educate early career astronomers about the decadal survey process, to solicit their feedback on the role that early career astronomers should play in Astro2020, and to provide a forum for the discussion of a wide range of topics regarding the astrophysics career path. This white paper presents highlights and themes that emerged during two days of discussion. In Section 1, we discuss concerns that emerged regarding the coming decade and the astrophysics career path, as well as specific recommendations from participants regarding how to address them. We have organized these concerns and suggestions into five broad themes. These include (sequentially): (1) adequately training astronomers in the statistical and computational techniques necessary in an era of "big data", (2) responses to the growth of collaborations and telescopes, (3) concerns about the adequacy of graduate and postdoctoral training, (4) the need for improvements in equity and inclusion in astronomy, and (5) smoothing and facilitating transitions between early career stages. Section 2 is focused on ideas regarding the decadal survey itself, including: incorporating early career voices, ensuring diverse input from a variety of stakeholders, and successfully and broadly disseminating the results of the survey., Comment: 9 pages; Astro2020 APC White Paper: State of the Profession Consideration

In addition to being a probe of Cosmic Dawn and Epoch of Reionization astrophysics, the 21cm line at $z>6$ is also a powerful way to constrain cosmology. Its power derives from several unique capabilities. First, the 21cm line is sensitive to energy injections into the intergalactic medium at high redshifts. It also increases the number of measurable modes compared to existing cosmological probes by orders of magnitude. Many of these modes are on smaller scales than are accessible via the CMB, and moreover have the advantage of being firmly in the linear regime (making them easy to model theoretically). Finally, the 21cm line provides access to redshifts prior to the formation of luminous objects. Together, these features of 21cm cosmology at $z>6$ provide multiple pathways toward precise cosmological constraints. These include the "marginalizing out" of astrophysical effects, the utilization of redshift space distortions, the breaking of CMB degeneracies, the identification of signatures of relative velocities between baryons and dark matter, and the discovery of unexpected signs of physics beyond the $\Lambda$CDM paradigm at high redshifts., Comment: Science white paper submitted to Decadal 2020 survey

The "cosmic dawn" refers to the period of the Universe's history when stars and black holes first formed and began heating and ionizing hydrogen in the intergalactic medium (IGM). Though exceedingly difficult to detect directly, the first stars and black holes can be constrained indirectly through measurements of the cosmic 21-cm background, which traces the ionization state and temperature of intergalactic hydrogen gas. In this white paper, we focus on the science case for such observations, in particular those targeting redshifts z $\gtrsim$ 10 when the IGM is expected to be mostly neutral. 21-cm observations provide a unique window into this epoch and are thus critical to advancing first star and black hole science in the next decade., Comment: White paper submitted to the Astro2020 Decadal Survey

The Dark Ages are the period between the last scattering of the cosmic microwave background and the appearance of the first luminous sources, spanning approximately 1100 < z < 30. The only known way to measure fluctuations in this era is through the 21-cm line of neutral hydrogen. Such observations have enormous potential for cosmology, because they span a large volume while the fluctuations remain linear even on small scales. Observations of 21-cm fluctuations during this era can therefore constrain fundamental aspects of our Universe, including inflation and any exotic physics of dark matter. While the observational challenges to these low-frequency 21-cm observations are enormous, especially from the terrestrial environment, they represent an important goal for cosmology., Comment: 10 pages, 4 figures

The epoch of reionization, when photons from early galaxies ionized the intergalactic medium about a billion years after the Big Bang, is the last major phase transition in the Universe's history. Measuring the characteristics of the transition is important for understanding early galaxies and the cosmic web and for modeling dwarf galaxies in the later Universe. But such measurements require probes of the intergalactic medium itself. Here we describe how the 21-cm line of neutral hydrogen provides a powerful probe of the reionization process and therefore important constraints on both the galaxies and intergalactic absorbers at that time. While existing experiments will make precise statistical measurements over the next decade, we argue that improved 21-cm analysis techniques - allowing imaging of the neutral gas itself - as well as improved theoretical models, are crucial for testing our understanding of this important era., Comment: 8 pages, 4 figures

The early phases of galaxy formation constitute one of the most exciting frontiers in astrophysics. It is during this era that the first luminous sources reionize the intergalactic medium - the moment when structure formation affects every baryon in the Universe. Here we argue that we will obtain a complete picture of this era by combining observations of galaxies with direct measurements of the reionization process: the former will provide a detailed understanding of bright sources, while the latter will constrain the (substantial) faint source population. We further describe how optimizing the comparison of these two measurements requires near-infrared galaxy surveys covering large volumes and retaining redshift information and also improvements in 21-cm analysis, moving those experiments into the imaging regime., Comment: 9 pages, 3 figures

Cosmic dawn and the Epoch of Reionization (EoR) are among the least explored observational eras in cosmology: a time at which the first galaxies and supermassive black holes formed and reionized the cold, neutral Universe of the post-recombination era. With current instruments, only a handful of the brightest galaxies and quasars from that time are detectable as individual objects, due to their extreme distances. Fortunately, a multitude of multi-wavelength intensity mapping measurements, ranging from the redshifted 21 cm background in the radio to the unresolved X-ray background, contain a plethora of synergistic information about this elusive era. The coming decade will likely see direct detections of inhomogenous reionization with CMB and 21 cm observations, and a slew of other probes covering overlapping areas and complementary physical processes will provide crucial additional information and cross-validation. To maximize scientific discovery and return on investment, coordinated survey planning and joint data analysis should be a high priority, closely coupled to computational models and theoretical predictions., Comment: 5 pages, 1 figure, submitted to the Astro2020 Decadal Survey Science White Paper call

In recent years, machine learning (ML) methods have remarkably improved how cosmologists can interpret data. The next decade will bring new opportunities for data-driven cosmological discovery, but will also present new challenges for adopting ML methodologies and understanding the results. ML could transform our field, but this transformation will require the astronomy community to both foster and promote interdisciplinary research endeavors., Comment: Submitted to the Astro2020 call for science white papers

Radio Frequency Interference (RFI) is an ever-present limiting factor among radio telescopes even in the most remote observing locations. When looking to retain the maximum amount of sensitivity and reduce contamination for Epoch of Reionization studies, the identification and removal of RFI is especially important. In addition to improved RFI identification, we must also take into account computational efficiency of the RFI-Identification algorithm as radio interferometer arrays such as the Hydrogen Epoch of Reionization Array grow larger in number of receivers. To address this, we present a Deep Fully Convolutional Neural Network (DFCN) that is comprehensive in its use of interferometric data, where both amplitude and phase information are used jointly for identifying RFI. We train the network using simulated HERA visibilities containing mock RFI, yielding a known "ground truth" dataset for evaluating the accuracy of various RFI algorithms. Evaluation of the DFCN model is performed on observations from the 67 dish build-out, HERA-67, and achieves a data throughput of 1.6$\times 10^{5}$ HERA time-ordered 1024 channeled visibilities per hour per GPU. We determine that relative to an amplitude only network including visibility phase adds important adjacent time-frequency context which increases discrimination between RFI and Non-RFI. The inclusion of phase when predicting achieves a Recall of 0.81, Precision of 0.58, and $F_{2}$ score of 0.75 as applied to our HERA-67 observations., Comment: 11 pages, 7 figures