A 400 Trillion-Grid Vlasov Simulation on Fugaku Supercomputer: Large-Scale Distribution of Cosmic Relic Neutrinos in a Six-dimensional Phase Space

We report a Vlasov simulation of cosmic relic neutrinos combined with N-body simulation of cold dark matter in the context of large-scale structure formation in the Universe performed on Fugaku supercomputer. Gravitational dynamics of the neutrinos is followed, for the first time, by directly integrating the Vlasov equation in a six-dimensional phase space. Our largest simulation combines the Vlasov simulation on 400 trillion grids and 330 billion-body calculations in a self-consistent manner, and reproduces accurately the nonlinear dynamics of neutrinos in the Universe. The novel high-order Vlasov solver is optimized by combining an array of state-of-the-art numerical schemes and fully utilizing the SIMD instructions on the A64FX processors. Time-To-Solution of our simulation is an order of magnitude shorter than the largest N-body simulations. The performance scales excellently with up to 147,456 nodes (7 million CPU cores) on Fugaku; the weak and strong scaling efficiencies are 82% - 96% and 82% - 93%, respectively.
Comment: 11 pages, 8 figures. 2021 ACM Gordon-Bell Prize Finalist. A simulation movie is available at https://youtu.be/p2fHt8f7chQ