Search

In simplified models of glasses we clarify the existence of two different kinds of coexisting activated dynamics, with one of the two dominating over the other. One is the energy barrier hopping that is typically used to understand activation, and the other, which we call entropic activation, is driven by the scarcity of convenient directions in phase space. When entropic activation dominates, the height of the energy barriers is no longer the primary factor governing the system's slowdown. In our analysis, dominance of one mechanism over the other depends on temperature and the shape of the density of states. We also find that at low temperatures a phase transition between the two kinds of activation can occur. Our observations are used to provide a scenario that can harmonize the facilitation and thermodynamic pictures of the slowdown of glasses into a single description.

We use a simple model to extend network models for activated dynamics to a continuous landscape with a well-defined notion of distance and a direct connection to many-body systems. The model consists of a tracer in a high-dimensional funnel landscape with no disorder. We find a nonequilibrium low-temperature phase with aging dynamics that is effectively equivalent to that of models with built-in disorder, such as the trap model, step model and random energy model. Finally, we compare entropy with energy-driven activation, and we remark that the former is more robust to the choice of the dynamics since it does not depend on whether one uses local or global updates.

Deep learning has been immensely successful at a variety of tasks, ranging from classification to artificial intelligence. Learning corresponds to fitting training data, which is implemented by descending a very high-dimensional loss function. Understanding under which conditions neural networks do not get stuck in poor minima of the loss, and how the landscape of that loss evolves as depth is increased, remains a challenge. Here we predict, and test empirically, an analogy between this landscape and the energy landscape of repulsive ellipses. We argue that in fully connected deep networks a phase transition delimits the over- and underparametrized regimes where fitting can or cannot be achieved. In the vicinity of this transition, properties of the curvature of the minima of the loss (the spectrum of the Hessian) are critical. This transition shares direct similarities with the jamming transition by which particles form a disordered solid as the density is increased, which also occurs in certain classes of computational optimization and learning problems such as the perceptron. Our analysis gives a simple explanation as to why poor minima of the loss cannot be encountered in the overparametrized regime. Interestingly, we observe that the ability of fully connected networks to fit random data is independent of their depth, an independence that appears to also hold for real data. We also study a quantity Δ which characterizes how well (Δ<0) or badly (Δ>0) a datum is learned. At the critical point it is power-law distributed on several decades, P&#95;{+}(Δ)∼Δ^{θ} for Δ>0 and P&#95;{-}(Δ)∼(-Δ)^{-γ} for Δ<0, with exponents that depend on the choice of activation function. This observation suggests that near the transition the loss landscape has a hierarchical structure and that the learning dynamics is prone to avalanche-like dynamics, with abrupt changes in the set of patterns that are learned.

To study the activated dynamics of mean-field glasses, which takes place on times of order exp(N), where N is the system size, we introduce a new model, the correlated random energy model (CREM), that allows for a smooth interpolation between the REM and the p-spin models. We study numerically and analytically the CREM in the intermediate regime between REM and p-spin. We fully characterize its energy landscape, which is like a golf course but, at variance with the REM, has metabasins (or holes) containing several configurations. We find that an effective description for the dynamics, in terms of traps, emerges, provided that one identifies metabasins in the CREM with configurations in the trap model.