Description length of canonical and microcanonical models

Non-equivalence between the canonical and the microcanonical ensemble has been shown to arise for models defined by an extensive (i.e. scaling with the size of the system) number of constraints (e.g. the Configuration Model). Here, we compare the description length of binary canonical and microcanonical models in light of ensemble non-equivalence. Specifically, we consider the description length induced by the Normalized Maximum Likelihood (NML), which consists of two terms, i.e. a model log-likelihood and its complexity. While the effects of ensemble non-equivalence on the log-likelihood term are well understood, its effects on the complexity term have not yet been systematically studied. Here, we find that i) microcanonical models are always more complex than their canonical counterparts, ii) the best-scoring model in terms of description length highly depends on the numerical values of the constraints, and iii) the difference between the canonical and the microcanonical description length is strongly influenced by the degree of non-equivalence, a result suggesting that non-equivalence should be taken into account when selecting models. Finally, we compare the NML-based approach to model selection with the Bayesian one in light of our results, showing that the Bayesian description length becomes much more sensitive to the choice of the prior when an extensive number of constraints is involved