Search

Long-term memory behavior is one of the most important phenomena that has appeared in the time series analysis. Different from most definitions of second-order properties, an excess entropy approach is developed for stationary time series to classify long-term and short-term memory. A stationary sequence with finite block entropy is long-term memory if its excess entropy is infinite. The simulation results are graphically demonstrated after some theoretical results are simply presented by various stochastic sequences. Such an approach has advantages over the traditional ways that the excess entropy of stationary sequence with finite block entropy is invariant under instantaneous one-to-one transformation, and that it only requires very weak moment conditions rather than second-order moment conditions and thus can be applied to distinguish the LTM behavior of stationary sequences with unbounded second moment (e.g., heavy tail distribution). Finally, several applications on real data are exhibited.

Quantifying behaviors of robots which were generated autonomously from task-independent objective functions is an important prerequisite for objective comparisons of algorithms and movements of animals. The temporal sequence of such a behavior can be considered as a time series and hence complexity measures developed for time series are natural candidates for its quantification. The predictive information and the excess entropy are such complexity measures. They measure the amount of information the past contains about the future and thus quantify the nonrandom structure in the temporal sequence. However, when using these measures for systems with continuous states one has to deal with the fact that their values will depend on the resolution with which the systems states are observed. For deterministic systems both measures will diverge with increasing resolution. We therefore propose a new decomposition of the excess entropy in resolution dependent and resolution independent parts and discuss how they depend on the dimensionality of the dynamics, correlations and the noise level. For the practical estimation we propose to use estimates based on the correlation integral instead of the direct estimation of the mutual information based on next neighbor statistics because the latter allows less control of the scale dependencies. Using our algorithm we are able to show how autonomous learning generates behavior of increasing complexity with increasing learning duration.

We present two examples of finite-alphabet, infinite excess entropy processes generated by stationary hidden Markov models (HMMs) with countable state sets. The first, simpler example is not ergodic, but the second is. These are the first explicit constructions of processes of this type.

Recently, the duality between values (words) and orderings (permutations) has been proposed by the authors as a basis to discuss the relationship between information theoretic measures for finite-alphabet stationary stochastic processes and their permutatio nanalogues. It has been used to give a simple proof of the equality between the entropy rate and the permutation entropy rate for any finite-alphabet stationary stochastic process and to show some results on the excess entropy and the transfer entropy for finite-alphabet stationary ergodic Markov processes. In this paper, we extend our previous results to hidden Markov models and show the equalities between various information theoretic complexity and coupling measures and their permutation analogues. In particular, we show the following two results within the realm of hidden Markov models with ergodic internal processes: the two permutation analogues of the transfer entropy, the symbolic transfer entropy and the transfer entropy on rank vectors, are both equivalent to the transfer entropy if they are considered as the rates, and the directed information theory can be captured by the permutation entropy approach.

Causal states are minimal sufficient statistics of prediction of a stochastic process, their coding cost is called statistical complexity, and the implied causal structure yields a sense of the process’ “intrinsic computation”. We discuss how statistical complexity changes with slight changes to the underlying model– in this case, a biologically-motivated dynamical model, that of a Monod-Wyman-Changeux molecule. Perturbations to kinetic rates cause statistical complexity to jump from finite to infinite. The same is not true for excess entropy, the mutual information between past and future, or for the molecule’s transfer function. We discuss the implications of this for the relationship between intrinsic and functional computation of biological sensory systems.

High-entropy alloys (HEAs) are a new class of solid-solution alloys that have attracted worldwide attention for their outstanding properties. Owing to the demand from transportation and defense industries, light-weight HEAs have also garnered widespread interest from scientists for use as potential structural materials. Great efforts have been made to study the phase-formation rules of HEAs to accelerate and refine the discovery process. In this paper, many proposed solid-solution phase-formation rules are assessed, based on a series of known and newly-designed light-weight HEAs. The results indicate that these empirical rules work for most compositions but also fail for several alloys. Light-weight HEAs often involve the additions of Al and/or Ti in great amounts, resulting in large negative enthalpies for forming solid-solution phases and/or intermetallic compounds. Accordingly, these empirical rules need to be modified with the new experimental data. In contrast, CALPHAD (acronym of the calculation of phase diagrams) method is demonstrated to be an effective approach to predict the phase formation in HEAs as a function of composition and temperature. Future perspectives on the design of light-weight HEAs are discussed in light of CALPHAD modeling and physical metallurgy principles.