Search

The Kullback-Leibler (KL) divergence is a fundamental measure of information geometry that is used in a variety of contexts in artificial intelligence. We show that, when system dynamics are given by distributed nonlinear systems, this measure can be decomposed as a function of two information-theoretic measures, transfer entropy and stochastic interaction. More specifically, these measures are applicable when selecting a candidate model for a distributed system, where individual subsystems are coupled via latent variables and observed through a filter. We represent this model as a directed acyclic graph (DAG) that characterises the unidirectional coupling between subsystems. Standard approaches to structure learning are not applicable in this framework due to the hidden variables; however, we can exploit the properties of certain dynamical systems to formulate exact methods based on differential topology. We approach the problem by using reconstruction theorems to derive an analytical expression for the KL divergence of a candidate DAG from the observed dataset. Using this result, we present a scoring function based on transfer entropy to be used as a subroutine in a structure learning algorithm. We then demonstrate its use in recovering the structure of coupled Lorenz and Rössler systems. [ABSTRACT FROM AUTHOR]

We introduce a novel measure, Fisher transfer entropy (FTE), which quantifies a gain in sensitivity to a control parameter of a state transition, in the context of another observable source. The new measure captures both transient and contextual qualities of transfer entropy and the sensitivity characteristics of Fisher information. FTE is exemplified for a ferromagnetic two-dimensional lattice Ising model with Glauber dynamics and is shown to diverge at the critical point. [ABSTRACT FROM AUTHOR]

We propose a thermodynamic interpretation of transfer entropy near equilibrium, using a specialised Boltzmann's principle. The approach relates conditional probabilities to the probabilities of the corresponding state transitions. This in turn characterises transfer entropy as a difference of two entropy rates: the rate for a resultant transition and another rate for a possibly irreversible transition within the system affected by an additional source. We then show that this difference, the local transfer entropy, is proportional to the external entropy production, possibly due to irreversibility. Near equilibrium, transfer entropy is also interpreted as the difference in equilibrium stabilities with respect to two scenarios: a default case and the case with an additional source. Finally, we demonstrated that such a thermodynamic treatment is not applicable to information flow, a measure of causal effect. [ABSTRACT FROM AUTHOR]

We have recently presented a framework for the information dynamics of distributed computation that locally identifies the component operations of information storage, transfer, and modification. We have observed that while these component operations exist to some extent in all types of computation, complex computation is distinguished in having coherent structure in its local information dynamics profiles. In this article, we conjecture that coherent information structure is a defining feature of complex computation, particularly in biological systems or artificially evolved computation that solves human-understandable tasks. We present a methodology for studying coherent information structure, consisting of state-space diagrams of the local information dynamics and a measure of structure in these diagrams. The methodology identifies both clear and 'hidden' coherent structure in complex computation, most notably reconciling conflicting interpretations of the complexity of the Elementary Cellular Automata rule 22. [ABSTRACT FROM AUTHOR]

Animal groups are often composed of individuals that vary according to behavioral, morphological, and internal state parameters. Understanding the importance of such individual-level heterogeneity to the establishment and maintenance of coherent group responses is of fundamental interest in collective behavior. We examined the influence of hunger on the individual and collective behavior of groups of shoaling fish, x-ray tetras (Pristella maxillaris). Fish were assigned to one of two nutritional states, satiated or hungry, and then allocated to 5 treatments that represented different ratios of satiated to hungry individuals (8 hungry, 8 satiated, 4:4 hungry:satiated, 2:6 hungry:satiated, 6:2 hungry:satiated). Our data show that groups with a greater proportion of hungry fish swam faster and exhibited greater nearest neighbor distances. Within groups, however, there was no difference in the swimming speeds of hungry versus well-fed fish, suggesting that group members conform and adapt their swimming speed according to the overall composition of the group. We also found significant differences in mean group transfer entropy, suggesting stronger patterns of information flow in groups comprising all, or a majority of, hungry individuals. In contrast, we did not observe differences in polarization, a measure of group alignment, within groups across treatments. Taken together these results demonstrate that the nutritional state of animals within social groups impacts both individual and group behavior, and that members of heterogenous groups can adapt their behavior to facilitate coherent collective motion. [ABSTRACT FROM AUTHOR]