Time delay lens modelling challenge

In recent years, breakthroughs in methods and data have enabled gravitational time delays to emerge as a very powerful tool to measure the Hubble constant $H_0$. However, published state-of-the-art analyses require of order 1 year of expert investigator time and up to a million hours of computing time per system. Furthermore, as precision improves, it is crucial to identify and mitigate systematic uncertainties. With this time delay lens modelling challenge we aim to assess the level of precision and accuracy of the modelling techniques that are currently fast enough to handle of order 50 lenses, via the blind analysis of simulated datasets. The results in Rung 1 and Rung 2 show that methods that use only the point source positions tend to have lower precision ($10 - 20\%$) while remaining accurate. In Rung 2, the methods that exploit the full information of the imaging and kinematic datasets can recover $H_0$ within the target accuracy ($ |A| < 2\%$) and precision ($< 6\%$ per system), even in the presence of poorly known point spread function and complex source morphology. A post-unblinding analysis of Rung 3 showed the numerical precision of the ray-traced cosmological simulations to be insufficient to test lens modelling methodology at the percent level, making the results difficult to interpret. A new challenge with improved simulations is needed to make further progress in the investigation of systematic uncertainties. For completeness, we present the Rung 3 results in an appendix, and use them to discuss various approaches to mitigating against similar subtle data generation effects in future blind challenges.
23 pages, 12 figures, 6 tables, MNRAS accepted