Search

How the human brain processes information during different cognitive tasks is one of the greatest questions in contemporary neuroscience. Understanding the statistical properties of brain signals during specific activities is one promising way to address this question. Here we analyze freely available data from implanted electrocorticography (ECoG) in five human subjects during two different cognitive tasks in the light of information theory quantifiers ideas. We employ a symbolic information approach to determine the probability distribution function associated with the time series from different cortical areas. Then we utilize these probabilities to calculate the associated Shannon entropy and a statistical complexity measure based on the disequilibrium between the actual time series and one with a uniform probability distribution function. We show that an Euclidian distance in the complexity-entropy plane and an asymmetry index for complexity are useful for comparing the two conditions. We show that our method can distinguish visual search epochs from blank screen intervals in different electrodes and patients. By using a multi-scale approach and embedding time delays to downsample the data, we find important time scales in which the relevant information is being processed. We also determine cortical regions and time intervals along the 2-second-long trials that present more pronounced differences between the two cognitive tasks. Finally, we show that the method is useful to distinguish cognitive processes using brain activity on a trial-by-trial basis.

In 2002, in a seminal article, Christoph Bandt and Bernd Pompe proposed a new methodology for the analysis of complex time series, now known as Ordinal Analysis. The ordinal methodology is based on the computation of symbols (known as ordinal patterns) which are defined in terms of the temporal ordering of data points in a time series, and whose probabilities are known as ordinal probabilities. With the ordinal probabilities, the Shannon entropy can be calculated, which is the permutation entropy. Since it was proposed, the ordinal method has found applications in fields as diverse as biomedicine and climatology. However, some properties of ordinal probabilities are still not fully understood, and how to combine the ordinal approach of feature extraction with machine learning techniques for model identification, time series classification or forecasting remains a challenge. The objective of this perspective article is to present some recent advances and to discuss some open problems., Comment: Accepted in EPL

The main motivation of this paper is to introduce the permutation Jensen-Shannon distance, a symbolic tool able to quantify the degree of similarity between two arbitrary time series. This quantifier results from the fusion of two concepts, the Jensen-Shannon divergence and the encoding scheme based on the sequential ordering of the elements in the data series. The versatility and robustness of this ordinal symbolic distance for characterizing and discriminating different dynamics are illustrated through several numerical and experimental applications. Results obtained allow us to be optimistic about its usefulness in the field of complex time series analysis. Moreover, thanks to its simplicity, low computational cost, wide applicability and less susceptibility to outliers and artifacts, this ordinal measure can efficiently handle large amounts of data and help to tackle the current big data challenges., Comment: 23 two-column pages, 30 figures, supplementary information; accepted for publication in Physical Review E

How the brain processes information from external stimuli in order to perceive the world and act on it is one of the greatest questions in neuroscience. To address this question different time series analyzes techniques have been employed to characterize the statistical properties of brain signals during cognitive tasks. Typically response-specific processes are addressed by comparing the time course of average event-related potentials in different trials type. Here we analyze monkey Local Field Potentials data during visual pattern discrimination called Go/No-Go task in the light of information theory quantifiers. We show that the Bandt-Pompe symbolization methodology to calculate entropy and complexity of data is a useful tool to distinguish response-related differences between Go and No-Go trials. We propose to use an asymmetry index to statistically validate trial type differences. Moreover, by using the multi-scale approach and embedding time delays to downsample the data we can estimate the important time scales in which the relevant information is been processed.

Complex systems are typically characterized as an intermediate situation between a complete regular structure and a random system. Brain signals can be studied as a striking example of such systems: cortical states can range from highly synchronous and ordered neuronal activity (with higher spiking variability) to desynchronized and disordered regimes (with lower spiking variability). It has been recently shown, by testing independent signatures of criticality, that a phase transition occurs in a cortical state of intermediate spiking variability. Here, we use a symbolic information approach to show that, despite the monotonical increase of the Shannon entropy between ordered and disordered regimes, we can determine an intermediate state of maximum complexity based on the Jensen disequilibrium measure. More specifically, we show that statistical complexity is maximized close to criticality for cortical spiking data of urethane-anesthetized rats, as well as for a network model of excitable elements that presents a critical point of a non-equilibrium phase transition., Comment: 8 pages, 6 figures

Two identical autonomous dynamical systems unidirectionally coupled in a sender-receiver configuration can exhibit anticipated synchronization (AS) if the Receiver neuron (R) also receives a delayed negative self-feedback. Recently, AS was shown to occur in a three-neuron motif with standard chemical synapses where the delayed inhibition was provided by an interneuron. Here we show that a two-neuron model in the presence of an inhibitory autapse, which is a massive self-innervation present in the cortical architecture, may present AS. The GABAergic autapse regulates the internal dynamics of the Receiver neuron and acts as the negative delayed self-feedback required by dynamical systems in order to exhibit AS. In this biologically plausible scenario, a smooth transition from the usual delayed synchronization (DS) to AS typically occurs when the inhibitory conductance is increased. The phenomenon is shown to be robust when model parameters are varied within a physiological range. For extremely large values of the inhibitory autapse the system undergoes to a phase-drift regime in which the Receiver is faster than the Sender. Furthermore, we show that the inhibitory autapse promotes a faster internal dynamics of the free-running Receiver when the two neurons are uncoupled, which could be the mechanism underlying anticipated synchronization and the DS-AS transition., Comment: 6 pages, 6 figures

This paper discusses the dynamics of intraday prices of twelve cryptocurrencies during last months' boom and bust. The importance of this study lies on the extended coverage of the cryptoworld, accounting for more than 90\% of the total daily turnover. By using the complexity-entropy causality plane, we could discriminate three different dynamics in the data set. Whereas most of the cryptocurrencies follow a similar pattern, there are two currencies (ETC and ETH) that exhibit a more persistent stochastic dynamics, and two other currencies (DASH and XEM) whose behavior is closer to a random walk. Consequently, similar financial assets, using blockchain technology, are differentiated by market participants., Comment: 17 pages, 5 figures. arXiv admin note: text overlap with arXiv:1508.04748

Bandt and Pompe introduced Permutation Entropy in 2002 for Time Series where equal values, xt1 = xt2, t1 = t2, were neglected and only inequalities between the xt were considered. Since then, this measure has been modified and extended, in particular in cases when the amount of equal values in the series can not be neglected, (i.e. heart rate variability (HRV) time series). We review the different existing methodologies that treats this subject by classifying them according to their different strategies. In addition, a novel Bayesian Missing Data Imputation is presented that proves to outperform the existing methodologies that deals with type of time series. All this facts are illustrated by simulations and also by distinguishing patients suffering from Congestive Heart Failure from a (healthy) control group using HRV time series

Financial global crisis has devastating impacts to economies since early XX century and continues to impose increasing collateral damages for governments, enterprises, and society in general. Up to now, all efforts to obtain efficient methods to predict these events have been disappointing. However, the quest for a robust estimator of the degree of the market efficiency, or even, a crisis predictor, is still one of the most studied subjects in the field. We present here an original contribution that combines Information Theory with graph concepts, to study the return rate series of 32 global trade markets. Specifically, we propose a very simple quantifier that shows to be highly correlated with global financial instability periods, being also a good estimator of the market crisis risk and market resilience. We show that this estimator displays striking results when applied to countries that played central roles during the last major global market crisis. The simplicity and effectiveness of our quantifier allow us to anticipate its use in a wide range of disciplines., Comment: 5 pages, 3 figures

This paper analyzes the informational efficiency of oil market during the last three decades, and examines changes in informational efficiency with major geopolitical events, such as terrorist attacks, financial crisis and other important events. The series under study is the daily prices of West Texas Intermediate (WTI) in USD/BBL, commonly used as a benchmark in oil pricing. The analysis is performed using information-theory-derived quantifiers, namely permutation entropy and permutation statistical complexity. These metrics allow capturing the hidden structure in the market dynamics, and allow discriminating different degrees of informational efficiency. We find that some geopolitical events impact on the underlying dynamical structure of the market., Comment: arXiv admin note: text overlap with arXiv:1603.02874

We have experimentally quantified the temporal structural diversity from the coordinate fluctuations of a laser beam propagating through isotropic optical turbulence. The main focus here is on the characterization of the long-range correlations in the wandering of a thin Gaussian laser beam over a screen after propagating through a turbulent medium. To fulfill this goal, a laboratory-controlled experiment was conducted in which coordinate fluctuations of the laser beam were recorded at a sufficiently high sampling rate for a wide range of turbulent conditions. Horizontal and vertical displacements of the laser beam centroid were subsequently analyzed by implementing the symbolic technique based on ordinal patterns to estimate the well-known permutation entropy. We show that the permutation entropy estimations at multiple time scales evidence an interplay between different dynamical behaviors. More specifically, a crossover between two different scaling regimes is observed. We confirm a transition from an integrated stochastic process contaminated with electronic noise to a fractional Brownian motion with a Hurst exponent H = 5/6 as the sampling time increases. Besides, we are able to quantify, from the estimated entropy, the amount of electronic noise as a function of the turbulence strength. We have also demonstrated that these experimental observations are in very good agreement with numerical simulations of noisy fractional Brownian motions with a well-defined crossover between two different scaling regimes., Comment: 8 pages, 6 figures

This paper studies the 28 time series of Libor rates, classified in seven maturities and four currencies), during the last 14 years. The analysis was performed using a novel technique in financial economics: the Complexity-Entropy Causality Plane. This planar representation allows the discrimination of different stochastic and chaotic regimes. Using a temporal analysis based on moving windows, this paper unveals an abnormal movement of Libor time series arround the period of the 2007 financial crisis. This alteration in the stochastic dynamics of Libor is contemporary of what press called "Libor scandal", i.e. the manipulation of interest rates carried out by several prime banks. We argue that our methodology is suitable as a market watch mechanism, as it makes visible the temporal redution in informational efficiency of the market., Comment: 17 pages, 9 figures. arXiv admin note: text overlap with arXiv:1508.04748, arXiv:1509.00217

We present a new approach for online handwritten signature classification and verification based on descriptors stemming from Information Theory. The proposal uses the Shannon Entropy, the Statistical Complexity, and the Fisher Information evaluated over the Bandt and Pompe symbolization of the horizontal and vertical coordinates of signatures. These six features are easy and fast to compute, and they are the input to an One-Class Support Vector Machine classifier. The results produced surpass state-of-the-art techniques that employ higher-dimensional feature spaces which often require specialized software and hardware. We assess the consistency of our proposal with respect to the size of the training sample, and we also use it to classify the signatures into meaningful groups., Comment: Submitted to PLOS One

This work proposes the use of Information Theory for the characterization of vehicles behavior through their velocities. Three public data sets were used: i.Mobile Century data set collected on Highway I-880, near Union City, California; ii.Borl\"ange GPS data set collected in the Swedish city of Borl\"ange; and iii.Beijing taxicabs data set collected in Beijing, China, where each vehicle speed is stored as a time series. The Bandt-Pompe methodology combined with the Complexity-Entropy plane were used to identify different regimes and behaviors. The global velocity is compatible with a correlated noise with f^{-k} Power Spectrum with k >= 0. With this we identify traffic behaviors as, for instance, random velocities (k aprox. 0) when there is congestion, and more correlated velocities (k aprox. 3) in the presence of free traffic flow.

We revisit the Fisher-Shannon representation plane ${\mathcal H} \times {\mathcal F}$, evaluated using the Bandt and Pompe recipe to assign a probability distribution to a time series. Several stochastic dynamical (noises with $f^{-k}$, $k \geq 0$, power spectrum) and chaotic processes (27 chaotic maps) are analyzed so as to illustrate the approach. Our main achievement is uncovering the informational properties of the planar location., Comment: 6 pages, 1 figure. arXiv admin note: text overlap with arXiv:1401.2139

The role of credit rating agencies has been under severe scrutiny after the subprime crisis. In this paper we explore the relationship between credit ratings and informational efficiency of a sample of thirty nine corporate bonds of US oil and energy companies from April 2008 to November 2012. For that purpose, we use a powerful statistical tool relatively new in the financial literature: the complexity-entropy causality plane. This representation space allows to graphically classify the different bonds according to their degree of informational efficiency. We find that this classification agrees with the credit ratings assigned by Moody's. Particularly, we detect the formation of two clusters, that correspond to the global categories of investment and speculative grades. Regarding to the latter cluster, two subgroups reflect distinct levels of efficiency. Additionally, we also find an intriguing absence of correlation between informational efficiency and firm characteristics. This allows us to conclude that the proposed permutation-information-theory approach provides an alternative practical way to justify bond classification.

This paper analyzes Libor interest rates for seven different maturities and referred to operations in British Pounds, Euro, Swiss Francs and Japanese Yen, during the period years 2001 to 2015. The analysis is performed by means of two quantifiers derived from Information Theory: the permutation Shannon entropy and the permutation Fisher information measure. An anomalous behavior in the Libor is detected in all currencies except Euro during the years 2006--2012. The stochastic switch is more severe in 1, 2 and 3 months maturities. Given the special mechanism of Libor setting, we conjecture that the behavior could have been produced by the manipulation that was uncovered by financial authorities. We argue that our methodology is pertinent as a market overseeing instrument., Comment: arXiv admin note: text overlap with arXiv:1304.0399

This paper analyzes several interest rates time series from the United Kingdom during the period 1999 to 2014. The analysis is carried out using a pioneering statistical tool in the financial literature: the complexity-entropy causality plane. This representation is able to classify different stochastic and chaotic regimes in time series. We use sliding temporal windows to assess changes in the intrinsic stochastic dynamics of the time series. Anomalous behavior in the Libor is detected, especially around the time of the last financial crisis, that could be consistent with data manipulation., Comment: PACS 89.65.Gh Econophysics; 74.40.De noise and chaos

A crucial challenge in network theory is the study of the robustness of a network after facing a sequence of failures. In this work, we propose a dynamical definition of network's robustness based on Information Theory, that considers measurements of the structural changes caused by failures of the network's components. Failures are defined here, as a temporal process defined in a sequence. The robustness of the network is then evaluated by measuring dissimilarities between topologies after each time step of the sequence, providing a dynamical information about the topological damage. We thoroughly analyze the efficiency of the method in capturing small perturbations by considering both, the degree and distance distributions. We found the network's distance distribution more consistent in capturing network structural deviations, as better reflects the consequences of the failures. Theoretical examples and real networks are used to study the performance of this methodology., Comment: 5 pages, 2 figures, submitted

This paper presents and discusses the use of a new feature for PolSAR imagery: the Generalized Statistical Complexity. This measure is able to capture the disorder of the data by means of the entropy, as well as its departure from a reference distribution. The latter component is obtained by measuring a stochastic distance between two models: the $\mathcal G^0$ and the Gamma laws. Preliminary results on the intensity components of AIRSAR image of San Francisco are encouraging., Comment: Proceedings of The 4th Asia-Pacific Conference on Synthetic Aperture Radar (APSAR), 2013, 100-103

A recently proposed methodology called the Horizontal Visibility Graph (HVG) [Luque {\it et al.}, Phys. Rev. E., 80, 046103 (2009)] that constitutes a geometrical simplification of the well known Visibility Graph algorithm [Lacasa {\it et al.\/}, Proc. Natl. Sci. U.S.A. 105, 4972 (2008)], has been used to study the distinction between deterministic and stochastic components in time series [L. Lacasa and R. Toral, Phys. Rev. E., 82, 036120 (2010)]. Specifically, the authors propose that the node degree distribution of these processes follows an exponential functional of the form $P(\kappa)\sim \exp(-\lambda~\kappa)$, in which $\kappa$ is the node degree and $\lambda$ is a positive parameter able to distinguish between deterministic (chaotic) and stochastic (uncorrelated and correlated) dynamics. In this work, we investigate the characteristics of the node degree distributions constructed by using HVG, for time series corresponding to $28$ chaotic maps and $3$ different stochastic processes. We thoroughly study the methodology proposed by Lacasa and Toral finding several cases for which their hypothesis is not valid. We propose a methodology that uses the HVG together with Information Theory quantifiers. An extensive and careful analysis of the node degree distributions obtained by applying HVG allow us to conclude that the Fisher-Shannon information plane is a remarkable tool able to graphically represent the different nature, deterministic or stochastic, of the systems under study., Comment: Submitted to PLOS One

Wireless sensor networks are an important technology for making distributed autonomous measures in hostile or inaccessible environments. Among the challenges they pose, the way data travel among them is a relevant issue since their structure is quite dynamic. The operational topology of such devices can often be described by complex networks. In this work, we assess the variation of measures commonly employed in the complex networks literature applied to wireless sensor networks. Four data communication strategies were considered: geometric, random, small-world, and scale-free models, along with the shortest path length measure. The sensitivity of this measure was analyzed with respect to the following perturbations: insertion and removal of nodes in the geometric strategy; and insertion, removal and rewiring of links in the other models. The assessment was performed using the normalized Kullback-Leibler divergence and Hellinger distance quantifiers, both deriving from the Information Theory framework. The results reveal that the shortest path length is sensitive to perturbations., Comment: 12 pages, 4 figures, Central European Journal of Physics

When inhibitory neurons constitute about 40% of neurons they could have an important antinociceptive role, as they would easily regulate the level of activity of other neurons. We consider a simple network of cortical spiking neurons with axonal conduction delays and spike timing dependent plasticity, representative of a cortical column or hypercolumn with large proportion of inhibitory neurons. Each neuron fires following a Hodgkin-Huxley like dynamics and it is interconnected randomly to other neurons. The network dynamics is investigated estimating Bandt and Pompe probability distribution function associated to the interspike intervals and taking different degrees of inter-connectivity across neurons. More specifically we take into account the fine temporal ``structures'' of the complex neuronal signals not just by using the probability distributions associated to the inter spike intervals, but instead considering much more subtle measures accounting for their causal information: the Shannon permutation entropy, Fisher permutation information and permutation statistical complexity. This allows us to investigate how the information of the system might saturate to a finite value as the degree of inter-connectivity across neurons grows, inferring the emergent dynamical properties of the system., Comment: 26 pages, 3 Figures; Physica A, in press

A new generalized Statistical Complexity Measure (SCM) was proposed by Rosso et al in 2010. It is a functional that captures the notions of order/disorder and of distance to an equilibrium distribution. The former is computed by a measure of entropy, while the latter depends on the definition of a stochastic divergence. When the scene is illuminated by coherent radiation, image data is corrupted by speckle noise, as is the case of ultrasound-B, sonar, laser and Synthetic Aperture Radar (SAR) sensors. In the amplitude and intensity formats, this noise is multiplicative and non-Gaussian requiring, thus, specialized techniques for image processing and understanding. One of the most successful family of models for describing these images is the Multiplicative Model which leads, among other probability distributions, to the G0 law. This distribution has been validated in the literature as an expressive and tractable model, deserving the "universal" denomination for its ability to describe most types of targets. In order to compute the statistical complexity of a site in an image corrupted by speckle noise, we assume that the equilibrium distribution is that of fully developed speckle, namely the Gamma law in intensity format, which appears in areas with little or no texture. We use the Shannon entropy along with the Hellinger distance to measure the statistical complexity of intensity SAR images, and we show that it is an expressive feature capable of identifying many types of targets., Comment: Article accepted for publication in the proceedings of the 17 Iberoamerican Conference on Pattern Recognition (CIARP), to be published in the Lecture Notes in Computer Science

We deal here with the issue of determinism versus randomness in time series. One wishes to identify their relative importance in a given time series. To this end we extend i) the use of ordinal patterns-based probability distribution functions associated with a time series [Bandt and Pompe, Phys. Rev. Lett. 88 (2002) 174102] and ii) the so-called Amig\'o paradigm of forbidden/missing patterns [Amig\'o, Zambrano, Sanju\'an, Europhys. Lett. 79 (2007) 50001], to analyze deterministic finite time series contaminated with strong additive noises of different correlation-degree. Insights pertaining to the deterministic component of the original time series are obtained with the help of the causal entropy-complexity plane [Rosso et al. Phys. Rev. Lett. 99 (2007) 154102]., Comment: submitted to Physica A (october 2011). arXiv admin note: text overlap with arXiv:1105.4550

We deal here with the issue of determinism versus randomness in time series. One wishes to identify their relative weights in a given time series. Two different tools have been advanced in the literature to such effect, namely, i) the "causal" entropy-complexity plane [Rosso et al. Phys. Rev. Lett. 99 (2007) 154102] and ii) the estimation of the decay rate of missing ordinal patterns [Amig\'o et al. Europhys. Lett. 79 (2007) 50001, and Carpi et al. Physica A 389 (2010) 2020-2029]. In this work we extend the use of these techniques to address the analysis of deterministic finite time series contaminated with additive noises of different degree of correlation. The chaotic series studied here was via the logistic map (r = 4) to which we added correlated noise (colored noise with f-k Power Spectrum, 0 {\leq} k {\leq} 2) of varying amplitudes. In such a fashion important insights pertaining to the deterministic component of the original time series can be gained. We find that in the entropy-complexity plane this goal can be achieved without additional computations., Comment: submitted to Physica A

We analyze the statistical complexity vs. entropy plane-representation of sampled chaotic attractors as a function of the sampling period {\tau}. It is shown that if the Bandt and Pompe procedure is used to assign a probability distribution function (PDF) to the pertinent time series, the statistical complexity measure (SCM) attains a definite maximum for a specific sampling period tM. If the usual histogram approach is used instead in order to assign the PDF to the time series, the SCM remains almost constant at any sampling period {\tau}. The significance of tM is further investigated by comparing it with typical times given in the literature for the two main reconstruction processes: the Takens' one in a delay-time embedding, and the exact Nyquist-Shannon reconstruction. It is shown that tM is compatible with those times recommended as adequate delay ones in Takens' reconstruction. The reported results correspond to three representative chaotic systems having correlation dimension 2 < D2 < 3. One recent experiment confirms the analysis presented here., Comment: Submitted for publication to Physica D

We study the functional link between the Hurst parameter and the Normalized Total Wavelet Entropy when analyzing fractional Brownian motion (fBm) time series--these series are synthetically generated. Both quantifiers are mainly used to identify fractional Brownian motion processes (Fractals 12 (2004) 223). The aim of this work is understand the differences in the information obtained from them, if any., Comment: 10 pages, 2 figures, submitted to Physica A for considering its publication

We analyze, within the wavelet theory framework, the wandering over a screen of the centroid of a laser beam after it has propagated through a time-changing laboratory-generated turbulence. Following a previous work (Fractals 12 (2004) 223) two quantifiers are used, the Hurst parameter, $H$, and the Normalized Total Wavelet Entropy, $\text{NTWS}$. The temporal evolution of both quantifiers, obtained from the laser spot data stream is studied and compared. This allows us to extract information of the stochastic process associated to the turbulence dynamics., Comment: 11 pages, 3 figures, accepted to be published in Physica A

The propagation of a laser beam through turbulent media is modeled as a fractional Brownian motion (fBm). Time series corresponding to the center position of the laser spot (coordinates x and y) after traveling across air in turbulent motion, with different strength, are analyzed by the wavelet theory. Two quantifiers are calculated, the Hurst exponent and the mean Normalized Total Wavelet Entropy. It is verified that both quantifiers gives complementary information about the turbulence state., Comment: 20 Pages, 7 Figures, LaTeX, published in Fractals