The non-linear universe : the role of simulations, theory & machine learning in weak lensing cosmology

The coherent distortions in the observed shapes of distant galaxies, a consequence of the spacetime curvature induced by the intervening large-scale structure of the Universe, is an abundant reservoir of cosmological information. Via this phenomenon of weak gravitational lensing, and a number of other independent cosmological probes, the parameters of the standard model, ΛCDM, have been inferred, now with uncertainties approaching the per cent level. In this era of precision cosmology, however, we face new challenges. Elements of tension have emerged between the measurements of the cosmological parameters from lowand high-redshift probes, seemingly implying either a failure to account for all relevant systematics, or perhaps even an incompleteness in the ΛCDM paradigm. In this thesis, I develop novel methodologies in weak lensing, to enhance the cosmological information extracted from current and future data sets. In this pursuit, I adopt a three-pronged approach, combining new advances in theoretical modelling, cutting-edge numerical simulations and recent developments in machine learning. Applying this trinity of techniques to three distinct bodies of research, described below, I construct new routes to improving the constraining power of this cosmological probe. A notable shortfall of the standard two-point statistics conventionally used in weak lensing, is their inability to capture all of the information contained in the non-linear cosmological fields of the real Universe. In answer to this problem, I develop the use of "clipping" transformations, which suppress the signal from the highest density regions observed. I present the first "clipped" cosmic shear measurement using data from the Kilo-Degree Survey (KiDS-450), and employ a suite of numerical simulations to calibrate and explore the cosmological dependence of this novel statistic. I show that these transformations improve constraints on S8 = σ8(Ωm/0.3)0.5, where Ωm is the mass energy density and σ8 is the amplitude of matter density fluctuations, when used in combination with conventional, "unclipped" two-point statistics, by 17% in the case of the KiDS-450 data. Clipping is but one member of the non-Gaussian statistics family, which have great potential for improving cosmological constraints, but are reliant both on numerical simulations, and a robust means to interpolate the statistics measured in the simulations to arbitrary cosmologies for comparison to the data. In this thesis, I develop a general framework to facilitate this, by designing and training a Gaussian process emulator, employing Bayesian supervised machine learning, on the state-of-the-art cosmo-SLICS suite, consisting of 26 different wCDM cosmologies. I demonstrate that this emulator achieves per cent level interpolation accuracy, in turn yielding unprecedented precision in the estimation of non-Gaussian statistics. I subsequently show how the cosmo-SLICS emulator might be employed within a likelihood analysis to constrain the cosmology of next-generation lensing data using these non-standard statistical probes. Taking clipped shear correlation functions as an example, I find that the low levels of noise present in the cosmo-SLICS emulator's predictions facilitate improved constraints on cosmological parameters when the clipped and unclipped two-point probes are combined, not only for S8, but also for Ωm, and the Hubble and dark energy equation of state parameters, by 18%-26%. Finally, I combine the emulator approach with recent progress in theoretical modelling, to create a comprehensive framework for accurately predicting the non-linear matter power spectrum in arbitrary models of cosmology. This requires only a suite of vanilla ΛCDM N-body simulations with their initial conditions suitably tailored, such that the late-time non-linear power spectrum deviates from the standard model within a range permitted by observational constraints. These "pseudo" power spectra serve as the training set for the emulator, the predictions from which can be rescaled by reaction functions, analytically computed from the halo model, to obtain per cent level accurate non-linear predictions in a broadclass of beyond-ΛCDM cosmologies. In this proof-of-concept analysis, with a halofit training set substituting the simulation suite, I find that the emulator recovers the power spectra corresponding to f(R) gravity, massive neutrino cosmologies, combinations thereof, and even artificially generated departures from the ΛCDM prediction, with errors ≲ 1% deep in the highly non-linear regime. This work thus demonstrates a flexible and powerful method to not only test the validity of the standard model in the non-linear regime with next-generation cosmological data, but to also limit our reliance on costly numerical simulations in the future.