1. On learning higher-order cumulants in diffusion models
- Author
-
Aarts, Gert, Habibi, Diaa E., Wang, Lingxiao, and Zhou, Kai
- Subjects
High Energy Physics - Lattice ,Condensed Matter - Disordered Systems and Neural Networks ,Computer Science - Machine Learning - Abstract
To analyse how diffusion models learn correlations beyond Gaussian ones, we study the behaviour of higher-order cumulants, or connected n-point functions, under both the forward and backward process. We derive explicit expressions for the moment- and cumulant-generating functionals, in terms of the distribution of the initial data and properties of forward process. It is shown analytically that during the forward process higher-order cumulants are conserved in models without a drift, such as the variance-expanding scheme, and that therefore the endpoint of the forward process maintains nontrivial correlations. We demonstrate that since these correlations are encoded in the score function, higher-order cumulants are learnt in the backward process, also when starting from a normal prior. We confirm our analytical results in an exactly solvable toy model with nonzero cumulants and in scalar lattice field theory., Comment: 21 pages, many figures. Extended version of contribution accepted in the NeurIPS 2024 workshop "Machine Learning and the Physical Sciences"
- Published
- 2024