In this thesis, a new mathematical formalism for analyzing evolutionary dynamics is developed. This formalism combines ideas and methods from statistical mechanics, mathematical population genetics, and dynamical systems theory to describe the dynamics of evolving populations. In particular, the work shows how the maximum entropy formalism of statistical mechanics can be extended to apply to simple evolutionary systems, such that "macroscopic" equations of motion can be constructed from an underlying "microscopic" evolutionary dynamics. More specifically, the thesis studies "epochal evolution"; a dynamical phenomenon which is frequently observed in evolutionary dynamics. In epochal evolution, some macroscopic state variables that describe the evolving population exhibit an alternation of periods of stasis (epochs) and sudden transitions (innovations). For populations evolving in a constant selective environment there are two main mechanisms that may bring about epochal evolution. In the first mechanism the population is imagined to evolve on a "fitness landscape" that assigns a fitness to each point in a space of genotypes. Metastability then occurs when the population gets stuck around a "local optimum" in the fitness landscape. An innovation takes place when a rare sequence of mutations creates a lineage of individuals that crosses a valley of low fitness toward a local optimum of higher fitness. In the second mechanism, the genotype space is thought to decompose into a relatively small number of "neutral subbasins": large connected sets of equal fitness genotypes. In this view, an epoch corresponds to a time period in which the fittest members of the population diffuse through a neutral subbasin under mutation until one of them discovers a (rare) connection to a neutral subbasin of higher fitness. In somewhat different methodology, in the first mechanism the metastability is caused by a "fitness barrier", whereas in the second mechanism it is caused by an "entropy barrier". In this thesis, the evolutionary dynamics in the presence of both fitness and entropy barriers is studied, although the focus is on fitness functions with entropy barriers. For a large class of simple fitness functions we derive, using the maximum entropy methodology, equations of motion on the level of fitness distributions from the underlying microscopic dynamics of selection and mutation on genotypes. In the "thermodynamic limit" of infinite population sizes the population follows these equations of motion precisely while for finite populations the population only follows these equations of motion on average at each time step. From this formulation of the finite population dynamics we determine explicitly the locations of epochs in the space of fitness distributions, the stochastic dynamics within and between the epochs, the average durations of epochs, and the stability of epochs. The results also bear directly on the dynamics of evolutionary search algorithms such as genetic algorithms. In two chapters of this thesis, our mathematical model is used to derive optimal parameter settings for evolutionary search on a wide class of fitness functions. The analysis suggests that optimal evolutionary search occurs in a parameter regime where the highest fitness strings in the population are only marginally stable. We also studied in detail the way in which an evolving population will spread through a neutral subbasin or neutral network. As the theory shows, the limit distribution of the population over a neutral network is independent of almost all evolutionary parameters and is determined solely by the topology of the neutral network. Additionally, this distribution is concentrated at those areas of the neutral network where the "neutrality" is largest. This implies that under neutral evolution the population evolves to increase its "mutational robustness" and that the mutational robustness that it attains is only dependent on the topology of the neutral network on which the population evolves. Finally, we studied and compared the barrier crossing times for fitness and entropy barriers. The results show that entropy barrier crossing in evolving population takes place on much shorter time scales than fitness barrier crossing. This suggests that in evolution, the escape from a metastable state is most likely to occur along neutral paths in genotype space.