Machine learning that predicts well may not learn the correct physical descriptions of glassy systems

The complexity of glasses makes it challenging to explain their dynamics. Machine learning (ML) has emerged as a promising pathway for understanding glassy dynamics by linking their structural features to rearrangement dynamics. Support vector machine (SVM) was one of the first methods used to detect such correlations. Specifically, a certain output of SVMs trained to predict dynamics from structure, the distance from the separating hyperplane, was interpreted as being linearly related to the activation energy for the rearrangement. By numerical analysis of toy models, we explore under which conditions it is possible to infer the energy barrier to rearrangements from the distance to the separating hyperplane. We observe that such successful inference is possible only under very restricted conditions. Typical tests, such as the apparent Arrhenius dependence of the probability of rearrangement on the inferred energy and the temperature, or high cross-validation accuracy do not guarantee success. Since even in such relatively simple toy models, prediction success of ML models does not necessarily translate into success of learning the underlying physics, we suggest that more careful investigations are needed when such claims are made. For this, we propose practical approaches for measuring the quality of the energy inference and for modifying the inferred model to improve the inference, which should be usable in the context of realistic datasets.