Search

We have developed a neural network model that imitates the central nervous system's control of motor sensors (Sánchez-Torres and Rodríguez-Romo in Neurocomputing 581:127511, 2024). Our research explored various levels of connectivity in our neural network related to neuroplasticity in the central nervous system. We have conducted a study comparing healthy individuals to those with motor impairments by utilizing reinforcement learning and transfer entropy. In our previous research (Sánchez-Torres and Rodríguez-Romo in Neurocomputing 581:127511, 2024), we have simulated human walking while encountering obstacles as an instance of gross motor activities. Now, we have used the same model to simulate fine motor activities. Our goal is to identify differences in information transmission between gross and fine motor activities among healthy individuals and those with motor impairments by evaluating the effective connectivity of our network. To regulate learning accuracy in our model, we introduced a variable called numClusterToFire. However, we discovered that the value for this variable requires careful calibration. If the value is too small, agent exploration is insufficient, and network learning is inefficient. Conversely, learning times increase exponentially, often unnecessarily if the value is too large. We conducted simulations for gross and fine motor skills using three different numClusterToFire values and found that as we increased numClusterToFire, the time required for the network to memorize the outputs for each of the objects in the test set also increased. Our findings indicate that in gross motor skills, which do not require precision, changes in the numClusterToFire variable do not affect information transfer behavior. Conversely, in fine motor skills, information transfer decreases as numClusterToFire increases. On the other hand, our model revealed that for healthy and disabled individuals, the transfer of information between the input layer and the first hidden layer is higher for fine motor skills; this important biological fact suggests the influence of external cues in performing this activity successfully. Additionally, our neural network model showed that movements that do not require precision do not necessarily require a high level of neuroplasticity. Increasing neuroplasticity may cause some neurons to transmit more information than others. Whereas, increasing neuroplasticity through practice is essential for precise movements like fine motor skills. We also found that information transfer in the network's hidden layers is similar for fine and gross motor activities, as we observed identical patterns. However, the distribution and proportion of these patterns differ, concluding that more neurons are involved in fine motor activities, and more information is transferred compared to gross motor activities. Finally, a pattern was observed in the transfer of information in the last hidden layer, which is only present in fine motor skills. This pattern is associated with the precision of the movements. [ABSTRACT FROM AUTHOR]

Multivariate time series modeling has been an important topic of interest for researchers in various fields. However, most of the existing methods focus on univariate forecasts and their historical values with little consideration of potential spatial dependencies among multiple variables. Multivariate time series forecasting can be naturally viewed from the perspective of a graph, where each variable can be considered as a node in the graph and they are interrelated through hidden dependencies. To this end, this paper proposes a multivariate time series forecasting graph neural network model based on multi-scale temporal feature extraction and attention mechanisms. Specifically, extremely symmetric modal decomposition is used to extract the time-domain features of multivariate time series at different time scales to form the node features of the graph; meanwhile, transfer entropy is computed to represent the adjacency matrix between nodes as a priori information, so as to identify the causal relationship between variables. In addition, a graph convolutional neural network is used to generate node embeddings containing rich spatial relationships. Finally, the temporal relationships of the node embeddings are established by FEDformer to enable multivariate time series forecasting. The effectiveness of the model is validated with real data from the power, climate and financial sectors. [ABSTRACT FROM AUTHOR]

In this work, we present a method that determines optimal multistep Dynamic Mode Decomposition (DMD) models via Entropic Regression (ER), which is a nonlinear information flow detection algorithm. Motivated by the Higher-Order DMD (HODMD) method of [Le Clainche & Vega, 2017], and the ER technique for network detection and model construction found in [Sun et al., 2015; AlMomani et al., 2020], we develop a method that we call ERDMD, which produces high fidelity time-delay DMD models that allow for nonuniformity in the delays. This optimal choice of delays is discovered by maximizing informativity as measured through ER. These models are shown to be highly efficient and robust. We test our method over several data sets generated by chaotic attractors and show that we are able to build excellent reconstructions using relatively minimal models. We likewise are able to better identify multiscale features via our models which enhances the utility of DMD. [ABSTRACT FROM AUTHOR]

Drug-resistant epilepsy is frequent, persistent, and brings a heavy economic burden to patients and their families. Traditional epilepsy detection methods ignore the causal relationship of seizures and focus on a single time or spatial dimension, and the effect varies greatly in different patients. Therefore, it is necessary to research accurate automatic detection technology of epilepsy in different patients. We propose a causal-spatio-temporal graph attention network (CSTGAT), which uses transfer entropy (TE) to construct a causal graph between multiple channels, combining graph attention network (GAT) and bi-directional long short-term memory (BiLSTM) to capture temporal dynamic correlation and spatial topological structure information. The accuracy, specificity, and sensitivity of the SWEZ dataset were 97.24%, 97.92%, and 98.11%. The accuracy of the private dataset reached 98.55%. The effectiveness of each module was proven through ablation experiments and the impact of different network construction methods was compared. The experimental results indicate that the causal relationship network constructed by TE could accurately capture the information flow of epileptic seizures, and GAT and BiLSTM could capture spatiotemporal dynamic correlations. This model accurately captures causal relationships and spatiotemporal correlations on two datasets, and it overcomes the variability of epileptic seizures in different patients, which may contribute to clinical surgical planning. [ABSTRACT FROM AUTHOR]

This study evaluates the links between representative indices of companies with high environmental performance and the propensity of such indices to economic and financial shocks. Five indices, representing environmental segments and four global macroeconomic and financial variables, were analyzed over a thirteen-year period, which included various crisis moments, such as the sovereign debt crisis, the COVID-19 pandemic and the onset of the Russia/Ukraine conflict. Using dynamic and nonlinear models, our research reveals statistically significant and consistent relationships between the variables under investigation, particularly during periods of global financial and pandemic crises. The analysis revealed that the VIXCLS is the most influential global risk factor, with certain risk factors being influenced by environmental segments, particularly Alternative Energy. This influence can create conditions conducive to contagion risk and diminish the benefits of portfolio diversification. This study contributes to a deeper understanding of the connection between environmental investments and their vulnerability to significant global events and risks. [ABSTRACT FROM AUTHOR]

Membrane fouling caused by many direct and indirect triggering factors has become an obstacle to the application of membrane bioreactors (MBRs). The nonlinear relationship between those factors is subject to complex causality or affiliation, which is difficult to clarify for the diagnosis of membrane fouling. To solve this problem, this paper proposes a compressible diagnosis model (CDM) based on transfer entropy to facilitate the fault diagnosis of the root cause for membrane fouling. The novelty of this model includes the following points: Firstly, a framework of a CDM between membrane fouling and causal variables is built based on a feature extraction algorithm and mechanism analysis. The framework can identify fault transfer scenarios following the changes in operating conditions. Secondly, the fault transfer topology of a CDM based on transfer entropy is constructed to describe the causal relationship between variables dynamically. Thirdly, an information compressible strategy is designed to simplify the fault transfer topology. This strategy can eliminate the repetitious affiliation relationship, which contributes to diagnosing the root causal variables speedily and accurately. Finally, the effectiveness of the proposed CDM is verified by the measured data from an actual MBR. The results of experiments demonstrate that the proposed CDM fulfills the diagnosis of membrane fouling. [ABSTRACT FROM AUTHOR]

Abstract In the process of penicillin fermentation, there is a strong nonlinear relationship between the input eigenvector and multiple output vectors, which makes the prediction accuracy of the existing model difficult to meet the requirements of chemical production. Therefore, a local selective ensemble learning multi-objective soft sensing modeling strategy is proposed in this study. Firstly, a localization method based on transfer entropy and k-means is proposed to reconstruct the sample set. Then, based on the reconstructed local samples, the local soft sensing model is established by the multi-objective support vector regression method, and the selective ensemble of sub-models and the adaptive calculation of prediction weights are realized. At the same time, to reduce the adverse effects caused by improper selection of model parameters, the sparrow search algorithm is used to realize the tuning of the mentioned model parameters. Finally, the proposed modeling strategy is simulated. The results show that, compared with other methods, the proposed local selective ensemble learning multi-objective soft sensing modeling strategy has better prediction performance.

The hydraulic and structural vibration sources of pumping station are investigated in current research. Based on a vertical shaft tubular pumping station project in an eastern province of China, a three-dimensional fluid-solid coupling model of the pumping station is established, including the fluid in the flow channel, concrete structure, and hydraulic machinery. Two-way fluid-structure interaction (FSI) method is used to calculate the coupled vibration responses of the pumping station. Based on the FSI simulation results, the hydraulic and structural vibration sources of the pumping station are explored from the frequency perspective based on the Fourier transform theory and the vibration energy transmission perspective based on the time-delayed transfer entropy method. Research results show that the fluid pulsation generated by the rotation of the impeller is the main hydraulic vibration source, and the rotating impeller is the main structural vibration source that causes the vibration of the unit and the concrete structure. This research could provide a scientific basis for structural vibration control, vibration reduction, and isolation design of the pumping station. [ABSTRACT FROM AUTHOR]

Diabetic peripheral neuropathy (DPN) is a prevalent complication of chronic diabetes mellitus and has a significant impact on quality of life. DPN typically manifests itself as a symmetrical, length-dependent sensorimotor polyneuropathy with severe effects on gait. Surface electromyography (sEMG) is a valuable low-cost tool for assessing muscle activation patterns and precise identification of abnormalities. For the present study, we used information theory methods, such as cross-correlation (CC), normalized mutual information (NMI), conditional granger causality (CG-Causality), and transfer entropy (TE), to evaluate muscle network connectivity in three population groups: 33 controls (healthy volunteers, CT), 10 diabetic patients with a low risk of DPN (LW), and 17 moderate/high risk patients (MH). The results obtained indicated significant alterations in the intermuscular coupling mechanisms due to diabetes and DPN, with the TE group showing the best performance in detecting differences. The data revealed a significant increase in information transfer and muscle connectivity in the LW group over the CT group, while the MH group obtained significantly lower values for these metrics than the other two groups. These findings highlight the sEMG coupling metrics' potential to reveal neuromuscular mechanisms that could aid the development of targeted rehabilitation strategies and help monitor DPN patients. [ABSTRACT FROM AUTHOR]

Accurate runoff prediction is crucial for watershed water resource management, flood prevention, and hydropower station scheduling. Data-driven models have been increasingly applied to runoff prediction tasks and have achieved impressive results. However, existing data-driven methods may produce unreasonable predictions due to the lack of prior knowledge guidance. This study proposes a multivariate runoff prediction model that couples knowledge embedding with data-driven approaches, integrating information contained in runoff probability distributions as constraints into the data-driven model and optimizing the existing loss function with prior probability density functions (PDFs). Using the main stream in the Yellow River Basin with nine hydrological stations as an example, we selected runoff feature factors using the transfer entropy method, chose a temporal convolutional network (TCN) as the data-driven model, and optimized model parameters with the IPSO algorithm, studying univariate input models (TCN-UID), multivariable input models (TCN-MID), and the coupling model. The results indicate the following: (1) Among numerous influencing factors, precipitation, sunshine duration, and relative humidity are the key feature factors driving runoff occurrence; (2) the coupling model can effectively fit the extremes of runoff sequences, improving prediction accuracy in the training set by 6.9% and 4.7% compared to TCN-UID and TCN-MID, respectively, and by 5.7% and 2.8% in the test set. The coupling model established through knowledge embedding not only retains the advantages of data-driven models but also effectively addresses the poor prediction performance of data-driven models at extremes, thereby enhancing the accuracy of runoff predictions. [ABSTRACT FROM AUTHOR]

Background: Inferring directional connectivity of brain regions from functional magnetic resonance imaging (fMRI) data has been shown to provide additional insights into predicting mental disorders such as schizophrenia. However, existing research has focused on the magnitude data from complex-valued fMRI data without considering the informative phase data, thus ignoring potentially important information. Methods: We propose a new complex-valued transfer entropy (CTE) method to measure causal links among brain regions in complex-valued fMRI data. We use the transfer entropy to model a general non-linear magnitude-magnitude and phase-phase directed connectivity and utilize partial transfer entropy to measure the complementary phase and magnitude effects on magnitude-phase and phase-magnitude causality. We also define the significance of the causality based on a statistical test and the shuffling strategy of the two complex-valued signals. Results: Simulated results verified higher accuracy of CTE than four causal analysis methods, including a simplified complex-valued approach and three real-valued approaches. Using experimental fMRI data from schizophrenia and controls, CTE yields results consistent with previous findings but with more significant group differences. The proposed method detects new directed connectivity related to the right frontal parietal regions and achieves 10.2-20.9% higher SVM classification accuracy when inferring directed connectivity using anatomical automatic labeling (AAL) regions as features. Conclusion: The proposed CTE provides a new general method for fully detecting highly predictive directed connectivity from complex-valued fMRI data, with magnitude-only fMRI data as a specific case. [ABSTRACT FROM AUTHOR]

The brain-computer interface (BCI) is a direct pathway of communication between the electrical activity of the brain and an external device. The present paper was aimed to investigate directed connectivity between different areas of the brain during motor imagery (MI)-based BCI. For this purpose, two methods were implemented including, Limited Penetrable Horizontal Visibility Graph (LPHVG) and Direct Lingam. The visibility graph (VG) is a robust algorithm for analyzing complex systems such as the brain. Direct Lingam uses a non-Gaussian model to extract causal links which is appropriate for analyzing large-scale connectivity. First, LPHVG map MI-EEG (electroencephalogram) signals into networks. After extracting the topological features of the networks, a support vector machine classifier was applied to categorize multi-classes MI. The network of all classes was found to be different from one another, and the kappa value of classification was 0.68. The degree sequence of LPHVG was calculated for each channel in order to obtain the direction of brain information flow. Transfer entropy (TE) is used to compute the relations of the channel degree sequence. Therefore, the directed graph between channels was formed. This method is called LPHVG_TE directed graph. The Bayesian network, also known as the Direct LiNGAM model, was implemented for the second method. Finally, images of the LPHVG and Direct Lingam were classified by convolutional neural network (CNN). In this study, Data sets 2a of BCI competition IV was used. The outcomes reveal that the brain network developed by LPHVG (92.7%) might be more effective to distinguish 4 classes of MI than the Direct Lingam (90.6%) and it was shown that graph theory has the potential to get better efficiency of BCI. [ABSTRACT FROM AUTHOR]

This study aims to employ a causal network model based on transfer entropy for the early warning of systemic risk in commodity markets. We analyzed the dynamic causal relationships of prices for 25 commodities related to China (including futures and spot prices of energy, industrial metals, precious metals, and agricultural products), validating the effect of the causal network structure among commodity markets on systemic risk. Our research results identified commodities and categories playing significant roles, revealing that industry and precious metal markets possess stronger market information transmission capabilities, with price fluctuations impacting a broader range and with greater force on other commodity markets. Under the influence of different types of crisis events, such as economic crises and the Russia–Ukraine conflict, the causal network structure among commodity markets exhibited distinct characteristics. The results of the effect of external shocks to the causal network structure of commodity markets on the entropy of systemic risk suggest that network structure indicators can warn of systemic risk. This article can assist investors and policymakers in managing systemic risk to avoid unexpected losses. [ABSTRACT FROM AUTHOR]

Monitoring groundwater level provides sufficient information on groundwater quantity and quality and is vital in effective management of water resources. This study applies transfer entropy coupled with directed-weighted complex network for the analysis of groundwater levels in the Sina river-basin, Maharashtra, India. All observation wells present in study area have been classified into five clusters using canopy clustering method. The direction and weight of the links of this complex network have been obtained by employing transfer entropy. Seasonal groundwater level data for pre-monsoon (May) and post-monsoon (October) were obtained from centre for groundwater board, Pune, Maharashtra for the period 1990–2009. Data analysis show that both pre-monsoon and post-monsoon groundwater level show significant decreasing trend. The proposed methodology determines the directional relationships between the selected observation wells of different clusters. It recognizes the most influenced well by using node strength and directed clustering coefficients. For each cluster, clustering coefficients and in-strength and out-strength have been calculated. Clustering coefficients for the selected wells of cluster 0 are 1, 1, 1, 0.6844 and 0.6342 which indicates Cluster 0 emerges as the strongest cluster. Similarly, clustering coefficients for cluster 4 are 0.6604, 0.6540, 0.3095, 0.2616 and 0 which means cluster 4 is the weakest among all the clusters formed. Clustering coefficients obtained for all clusters indicate that all wells within a cluster are forming clusters among themselves, however, some are strong whereas others are weak clusters. Therefore, transfer entropy can be effectively applied to groundwater and results obtained from it can be used for forecasting and water resources management. [ABSTRACT FROM AUTHOR]

Internal flow has a significant impact on the performance and stability of a mixed-flow pump during deceleration. This article aims to explore the complex internal flow in mixed-flow pumps during deceleration. A multi-channel testing system was constructed to clarify the impact of different decelerations on the pressure pulsation inside the pump. External characteristic parameters and internal pressure pulsation of mixed-flow pumps were collected under different deceleration modes. Non-stationary signal processing was introduced to determine the effects of speed reduction on the pressure pulsation at the impeller outlet and inside the guide vanes. Three domains were divided into time, frequency, and time-frequency, as well as the correlation between them. The results show that the decline gradient of head and flow under different deceleration modes was positively correlated with the decline speed. The energy entropy ratio of the pressure pulsation in the pump showed a downward trend during deceleration. It indicated that the pressure pulsation was gradually stabilized with the decrease of the speed in the pump. The peak value of pressure pulsation and the changing trend of sample entropy were positively correlated with the exponential of the deceleration modes. The changing trend of impeller outlet pressure pulsation was consistent with the speed transformation during the whole deceleration. The forward transfer entropy under different deceleration modes was greater than the reverse transfer entropy. Therefore, the pressure pulsation at the outlet of the impeller was dominated that in the guide vane. The high energy region of the pressure pulsation was mainly concentrated in the blade passing frequency 4fn and the high-order harmonic 8fn, whereas, the amplitude was positively correlated with the index of the speed transformation. The strong correlation region was concentrated in the two frequency bands of rotation frequency fn and blade passing frequency 4fn in the process of deceleration. The high amplitude region of pressure pulsation at the outlet of the impeller was concentrated in the blade passing frequency of 4fn, while the frequency of the pressure fluctuation component with the highest amplitude inside the guide vane was 3fn. The partial coherence frequency between the pressure pulsation at the impeller outlet and inside the guide vane was in the range of 80 and 120 Hz. There was no change in the frequency range of the strong correlation region between the pressure pulsations at the impeller outlet and inside the guide vane during different decelerations. But there was a variation in the proportion of the strong correlation region in the frequency range of the entire deceleration. In the deceleration with the fixed initial speed and target speed, the deceleration mode shared no effect on the coherent frequency of pressure fluctuation in the impeller outlet and guide vane. The finding can provide a strong reference to reveal the transient operation for the high stability of mixed-flow pump in down-speed operation. [ABSTRACT FROM AUTHOR]

This paper proposes a new hybrid model for classifying stress states using EEG signals, combining multi-domain transfer entropy (TrEn) with a two-dimensional PCANet (2D-PCANet) approach. The aim is to create an automated system for identifying stress levels, which is crucial for early intervention and mental health management. A major challenge in this field lies in extracting meaningful emotional information from the complex patterns observed in EEG. Our model addresses this by initially applying independent component analysis (ICA) to purify the EEG signals, enhancing the clarity for further analysis. We then leverage the adaptability of the fractional Fourier transform (FrFT) to represent the EEG data in time, frequency, and time–frequency domains. This multi-domain representation allows for a more nuanced understanding of the brain's activity in response to stress. The subsequent stage involves the deployment of a two-layer 2D-PCANet network designed to autonomously distill EEG features associated with stress. These features are then classified by a support vector machine (SVM) to determine the stress state. Moreover, stress induction and data acquisition experiments are designed. We employed two distinct tasks known to trigger stress responses. Other stress-inducing elements that enhance the stress response were included in the experimental design, such as time limits and performance feedback. The EEG data collected from 15 participants were retained. The proposed algorithm achieves an average accuracy of over 92% on this self-collected dataset, enabling stress state detection under different task-induced conditions. [ABSTRACT FROM AUTHOR]

The interaction between new energy vehicle (NEV) stock prices and the crude oil market is crucial for resource allocation and risk management. This study employs Multifractal detrended cross-correlation analysis (MF-DCCA) to investigate the multifractal characteristics of the cross-correlation between Tesla stock price (TSLA) and crude oil price (Brent), as well as between TSLA and other NEV stocks (excluding TSLA). The experimental results reveal long-term persistence and multiple fractal characteristics in the cross-correlations. Additionally, multifractal asymmetric detrended cross-correlation analysis (MF-ADCCA) demonstrates the asymmetry of the cross-correlation during upward or downward trends between TSLA and Brent, as well as between TSLA and other NEV stocks (excluding TSLA). Furthermore, utilizing the transfer entropy (TE) method, we assess the strength and direction of information flows between TSLA and Brent, and between TSLA and other NEV stocks (excluding TSLA). Interestingly, we observe bidirectional information transmission between TSLA and other NEV stocks, while only unidirectional information transmission from NIO to TSLA is evident. These findings provide valuable insights for resource allocation, supply chain management and sustainable development strategies for decision-makers in the NEV market. [ABSTRACT FROM AUTHOR]

This study tackles the complex task of measuring investor sentiment, a latent variable often measured through various proxies. The focus here is on textual sentiment extracted from online sources, specifically news media and social media sentiment. The central inquiry is whether these proxies are equivalent indicators of investor sentiment. Employing firm-level daily sentiment scores for DJIA stocks and leveraging Granger causality and transfer entropy, the research investigates the dynamics of information flow between these proxies. The findings show a prevailing pattern: information predominantly flows from social media to news for the majority of stocks while a reverse relationship is established for some stocks. The variations across stocks suggest that these proxies do not uniformly capture the same underlying phenomena. The study shows the significant role of social media in shaping news media sentiment and prompts considerations about regulating social media platforms in the context of their impact on financial markets.

Abstract In a rapidly urbanizing world, heavy air pollution and increasing surface temperature pose significant threats to human health and lives, especially in densely populated cities. In this study, we took an information theory perspective to investigate the causal relationship between diel land surface temperature (LST) and transboundary air pollution (TAP) from 2003 to 2020 in the Bangkok Metropolitan Region (BMR), which includes Bangkok Metropolis and its five adjacent provinces. We found an overall increasing trend of LST over the study region, with the mean daytime LST rising faster than nighttime LST. Evident seasonal variations showed high aerosol optical depth (AOD) loadings during the dry period and low loadings at the beginning of the rainy season. Our study revealed that TAP affected diel surface temperature in Bangkok Metropolis significantly. Causality tests show that air pollutants of two adjacent provinces west of Bangkok, i.e., Nakhon Pathom and Samut Sakhon, have a greater influence on the LST of Bangkok than other provinces. Also, the bidirectional relationship indicates that air pollution has a greater impact on daytime LST than nighttime LST. While LST has an insignificant influence on AOD during the daytime, it influences AOD significantly at night. Our study offers a new approach to understanding the causal impact of TAP and can help policymakers to identify the most relevant locations that cause pollution, leading to appropriate planning and management.

In a rapidly urbanizing world, heavy air pollution and increasing surface temperature pose significant threats to human health and lives, especially in densely populated cities. In this study, we took an information theory perspective to investigate the causal relationship between diel land surface temperature (LST) and transboundary air pollution (TAP) from 2003 to 2020 in the Bangkok Metropolitan Region (BMR), which includes Bangkok Metropolis and its five adjacent provinces. We found an overall increasing trend of LST over the study region, with the mean daytime LST rising faster than nighttime LST. Evident seasonal variations showed high aerosol optical depth (AOD) loadings during the dry period and low loadings at the beginning of the rainy season. Our study revealed that TAP affected diel surface temperature in Bangkok Metropolis significantly. Causality tests show that air pollutants of two adjacent provinces west of Bangkok, i.e., Nakhon Pathom and Samut Sakhon, have a greater influence on the LST of Bangkok than other provinces. Also, the bidirectional relationship indicates that air pollution has a greater impact on daytime LST than nighttime LST. While LST has an insignificant influence on AOD during the daytime, it influences AOD significantly at night. Our study offers a new approach to understanding the causal impact of TAP and can help policymakers to identify the most relevant locations that cause pollution, leading to appropriate planning and management. [ABSTRACT FROM AUTHOR]

Information theory explains how systems encode and transmit information. This article examines the neuronal system, which processes information via neurons that react to stimuli and transmit electrical signals. Specifically, we focus on transfer entropy to measure the flow of information between sequences and explore its use in determining effective neuronal connectivity. We analyze the causal relationships between two discrete time series, X : = X t : t ∈ Z and Y : = Y t : t ∈ Z , which take values in binary alphabets. When the bivariate process (X , Y) is a jointly stationary ergodic variable-length Markov chain with memory no larger than k, we demonstrate that the null hypothesis of the test—no causal influence—requires a zero transfer entropy rate. The plug-in estimator for this function is identified with the test statistic of the log-likelihood ratios. Since under the null hypothesis, this estimator follows an asymptotic chi-squared distribution, it facilitates the calculation of p-values when applied to empirical data. The efficacy of the hypothesis test is illustrated with data simulated from a neuronal network model, characterized by stochastic neurons with variable-length memory. The test results identify biologically relevant information, validating the underlying theory and highlighting the applicability of the method in understanding effective connectivity between neurons. [ABSTRACT FROM AUTHOR]

The causal connectivity of a network is often inferred to understand network function. It is arguably acknowledged that the inferred causal connectivity relies on the causality measure one applies, and it may differ from the network's underlying structural connectivity. However, the interpretation of causal connectivity remains to be fully clarified, in particular, how causal connectivity depends on causality measures and how causal connectivity relates to structural connectivity. Here, we focus on nonlinear networks with pulse signals as measured output, e.g., neural networks with spike output, and address the above issues based on four commonly utilized causality measures, i.e., time-delayed correlation coefficient, time-delayed mutual information, Granger causality, and transfer entropy. We theoretically show how these causality measures are related to one another when applied to pulse signals. Taking a simulated Hodgkin-Huxley network and a real mouse brain network as two illustrative examples, we further verify the quantitative relations among the four causality measures and demonstrate that the causal connectivity inferred by any of the four well coincides with the underlying network structural connectivity, therefore illustrating a direct link between the causal and structural connectivity.We stress that the structural connectivity of pulse-output networks can be reconstructed pairwise without conditioning on the global information of all other nodes in a network, thus circumventing the curse of dimensionality. Our framework provides a practical and effective approach for pulseoutput network reconstruction. [ABSTRACT FROM AUTHOR]

Analysis of cell–cell communication (CCC) in the tumor micro-environment helps decipher the underlying mechanism of cancer progression and drug tolerance. Currently, single-cell RNA-Seq data are available on a large scale, providing an unprecedented opportunity to predict cellular communications. There have been many achievements and applications in inferring cell–cell communication based on the known interactions between molecules, such as ligands, receptors and extracellular matrix. However, the prior information is not quite adequate and only involves a fraction of cellular communications, producing many false-positive or false-negative results. To this end, we propose an improved hierarchical variational autoencoder (HiVAE) based model to fully use single-cell RNA-seq data for automatically estimating CCC. Specifically, the HiVAE model is used to learn the potential representation of cells on known ligand–receptor genes and all genes in single-cell RNA-seq data, respectively, which are then utilized for cascade integration. Subsequently, transfer entropy is employed to measure the transmission of information flow between two cells based on the learned representations, which are regarded as directed communication relationships. Experiments are conducted on single-cell RNA-seq data of the human skin disease dataset and the melanoma dataset, respectively. Results show that the HiVAE model is effective in learning cell representations, and transfer entropy could be used to estimate the communication scores between cell types. [ABSTRACT FROM AUTHOR]

We propose a method for characterizing the local structure of weighted multivariate time series networks. We draw intensity and coherence of network motifs, i.e. statistically recurrent subgraphs, to characterize the system behavior via higher-order structures derived upon effective transfer entropy networks. The latter consists of a model-free methodology enabling to correct for small sample biases affecting Shannon transfer entropy, other than conducting inference on the estimated directional time series information flows. We demonstrate the usefulness of our proposed method with an application to a set of global commodity prices. Our main result shows that, despite simple triadic structures are the most intense, coherent and statistically recurrent over time, their intensity suddenly decreases after the Global Financial Crisis, in favor of most complex triadic structures, while all types of subgraphs tend to become more coherent thereafter. [ABSTRACT FROM AUTHOR]

The variability in the magnetic activity of the Sun is the main source of the observed changes in the plasma and electromagnetic environments within the heliosphere. The primary way in which solar activity affects the Earth's environment is via the solar wind and its transients. However, the relationship between solar activity and solar wind is not the same at the Space Weather and Space Climate time scales. In this work, we investigate this relationship exploiting five solar cycles data of Ca II K index and solar wind parameters, by taking advantage of the Hilbert–Huang Transform, which allows to separate the contribution at the different time scales. By filtering out the high-frequency components and looking at decennial time scales, we confirm the presence of a delayed response of solar wind to Ca II K index variations, with a time lag of ∼ 3.1-year for the speed and ∼ 3.4-year for the dynamic pressure. To assess the results in a stronger framework, we make use of a Transfer Entropy approach to investigate the information flow between the quantities and to test the causality of the relation. The time lag results from the latter are consistent with the cross-correlation ones, pointing out the presence of a statistical significant information flow from Ca II K index to solar wind dynamic pressure that peaks at time lag of 3.6-year. Such a result could be of relevance to build up a predictive model in a Space Climate context. [ABSTRACT FROM AUTHOR]

BACKGROUND AND HYPOTHESIS: Prior research has shown that patients with schizophrenia (SZ) show disruption in brain network connectivity that is thought to underlie their cognitive and psychotic symptoms. However, most studies examining functional network disruption in schizophrenia have focused on the temporally correlated coupling of the strength of network connections. Here, we move beyond correlative metrics to assay causal computations of connectivity changes in directed neural information flow, assayed from a neural source to a target in SZ. STUDY DESIGN: This study describes a whole-brain magnetoencephalography-imaging approach to examine causal computations of connectivity changes in directed neural information flow between brain regions during resting states, quantified by phase-transfer entropy (PTE) metrics, assayed from a neural source to an endpoint, in 21 SZ compared with 21 healthy controls (HC), and associations with cognitive and clinical psychotic symptoms in SZ. STUDY RESULTS: We found that SZ showed significant disruption in information flow in alpha (8-12 Hz) and beta (12-30 Hz) frequencies, compared to HC. Reduced information flow in alpha frequencies from the precuneus to the medio-ventral occipital cortex was associated with more severe clinical psychopathology (ie, positive psychotic symptoms), while reduced information flow between insula and middle temporal gyrus was associated with worsening cognitive symptoms. CONCLUSIONS: The present findings highlight the importance of delineating dysfunction in neural information flow in specific oscillatory frequencies between distinct regions that underlie the cognitive and psychotic symptoms in SZ, and provide potential neural biomarkers that could lead to innovations in future neuromodulation treatment development.

Precise control of strength is of significant importance in upper limb functional rehabilitation. Understanding the neuro-muscular response in strength regulation can help optimize the rehabilitation prescriptions and facilitate the relative training process for recovery control. This study aimed to investigate the inherent characteristics of neural-muscular activity during dynamic hand strength adjustment. Four dynamic grip force tracking modes were set by manipulating different magnitude and speed of force variations, and thirteen healthy young individuals took participation in the experiment. Electroencephalography were recorded in the contralateral sensorimotor cortex area, as well as the electromyography from the first dorsal interosseous muscle were collected synchronously. The metrics of the Event-related desynchronization, the electromyography stability index, and the force variation, were used to represent the corresponding cortical neural responses, muscle contraction activities, and the level of strength regulation, respectively; and further neuro-muscular coupling between the sensorimotor cortex and the first dorsal interosseous muscle was investigated by transfer entropy analysis. The results indicated a strong relationship that the increase of force regulation demand would result in a force variation increase as well as a stability reduction in muscle motor unit output. Meanwhile, the intensity of neural response increased in both the $\alpha $ and $\beta $ frequency bands. As the force regulation demand increased, the strength of bidirectional transfer entropy showed a clear shift from $\beta $ to the $\gamma $ frequency band, which facilitate rapid integration of dynamic strength compensation to adapt to motor task changes.

A significant issue for traffic safety has been drowsy driving for decades. A number of studies have investigated the effects of acute fatigue on spectral power; and recent research has revealed that drowsy driving is associated with a variety of brain connections in a specific cortico-cortical pathway. In spite of this, it is still unclear how different brain regions are connected in drowsy driving at different levels of daily fatigue. This study identified the brain connectivity-behavior relationship among three different daily fatigue levels (low-, median- and high-fatigue) with the EEG data transfer entropy. According to the results, only low- and medium-fatigue groups demonstrated an inverted U-shaped change in connectivity from high performance to poor behavioral performance. In addition, from low- to high-fatigue groups, connectivity magnitude decreased in the frontal region and increased in the occipital region. These study results suggest that brain connectivity and driving behavior would be affected by different levels of daily fatigue.

Drug-resistant epilepsy is frequent, persistent, and brings a heavy economic burden to patients and their families. Traditional epilepsy detection methods ignore the causal relationship of seizures and focus on a single time or spatial dimension, and the effect varies greatly in different patients. Therefore, it is necessary to research accurate automatic detection technology of epilepsy in different patients. We propose a causal-spatio-temporal graph attention network (CSTGAT), which uses transfer entropy (TE) to construct a causal graph between multiple channels, combining graph attention network (GAT) and bi-directional long short-term memory (BiLSTM) to capture temporal dynamic correlation and spatial topological structure information. The accuracy, specificity, and sensitivity of the SWEZ dataset were 97.24%, 97.92%, and 98.11%. The accuracy of the private dataset reached 98.55%. The effectiveness of each module was proven through ablation experiments and the impact of different network construction methods was compared. The experimental results indicate that the causal relationship network constructed by TE could accurately capture the information flow of epileptic seizures, and GAT and BiLSTM could capture spatiotemporal dynamic correlations. This model accurately captures causal relationships and spatiotemporal correlations on two datasets, and it overcomes the variability of epileptic seizures in different patients, which may contribute to clinical surgical planning.

Membrane fouling caused by many direct and indirect triggering factors has become an obstacle to the application of membrane bioreactors (MBRs). The nonlinear relationship between those factors is subject to complex causality or affiliation, which is difficult to clarify for the diagnosis of membrane fouling. To solve this problem, this paper proposes a compressible diagnosis model (CDM) based on transfer entropy to facilitate the fault diagnosis of the root cause for membrane fouling. The novelty of this model includes the following points: Firstly, a framework of a CDM between membrane fouling and causal variables is built based on a feature extraction algorithm and mechanism analysis. The framework can identify fault transfer scenarios following the changes in operating conditions. Secondly, the fault transfer topology of a CDM based on transfer entropy is constructed to describe the causal relationship between variables dynamically. Thirdly, an information compressible strategy is designed to simplify the fault transfer topology. This strategy can eliminate the repetitious affiliation relationship, which contributes to diagnosing the root causal variables speedily and accurately. Finally, the effectiveness of the proposed CDM is verified by the measured data from an actual MBR. The results of experiments demonstrate that the proposed CDM fulfills the diagnosis of membrane fouling.

The objectives of this study are to: (1) develop an approach for extracting representative information from dam behaviour data and (2) diagnose deformation and crack opening behaviour by integrating techniques in transfer entropy (TE), chaos theory and wavelet transform (WT). First, TE is employed to map the information flow between measuring points to select the representative measuring point. The phase space reconstruction of observation data on dam behaviour is implemented, where lag time τ and embedding dimension m are identified by the statistical C–C method. The non-linear structural dynamical behaviour of the dam is then quantified in the form of the largest Lyapunov exponent representing system chaoticity. In addition, a moving window formula is employed to scan along the observation data to ensure an on-line continuous diagnosis. WT may be employed to assist in detecting the time where significant changes occur. A simple 4 degrees of freedom spring-mass-damper (K-M-C) system is employed to illustrate the performance of the proposed approach. The approach is then applied to an actual dam using monitoring data. From the deformation and crack opening behaviour, it is concluded that the studied dam is performing with no significant changes in behaviour over time. Transfer entropy is applied to map the information flow between measuring points. The representative information on structural behaviour of an actual dam is identified. TE and chaos theory are combined to diagnose the dam behaviour. A moving window is adopted to facilitate online statistical diagnosis. [ABSTRACT FROM AUTHOR]

The study applies the effective transfer entropy (ETE) and Renyi transfer entropy (RTE) to quantify the information flow between government bonds and equity market of G7 countries. The magnitude and direction of information flow between these two markets are dynamic across the state of the markets and time thus confirming their adaptiveness to the evolution of cross-market information flow under different market conditions over time. Although information flow from the equity dominates the flow from the bond market, it is dynamic over the time and state of the markets. However, during the period of market turbulence, the bond dominates the equity market. This study is the first of its kind to validate the adaptive market hypothesis using the novel transfer entropy framework in the bond and equity market. [ABSTRACT FROM AUTHOR]

In collective systems, influence of individuals can permeate an entire group through indirect interactionscomplicating any scheme to understand individual roles from observations. A typical approach to understand an individuals influence on another involves consideration of confounding factors, for example, by conditioning on other individuals outside of the pair. This becomes unfeasible in many cases as the number of individuals increases. In this article, we review some of the unforeseen problems that arise in understanding individual influence in a collective such as single cells, as well as some of the recent works which address these issues using tools from information theory. [ABSTRACT FROM AUTHOR]

Process abnormalities can result in serious accidents that may lead to unexpected loss of life and property. Early fault detection and diagnosis are essential to prevent these accidents. In the efficient operation of industrial systems, alarm systems play a crucial role. Developing new sensors and alarm adjustment networks has increased the possibility of generating multiple alarms. The alarm flood is the most important type of this problem. In this study, a new algorithm is proposed to diagnose the root cause of the fault through the classification of alarm floods. The main novelty of this algorithm is to use transfer entropy as a criterion to detect the similarity between the alarm flood sequences. The other innovations include calculating transfer entropy between the process variable and the alarm data, multi-sensor information fusion for large-scale plants and simultaneous alarms, and proposing an online version of the algorithm for the early prediction of the type of fault occurring. Finally, the Tennessee Eastman Process system, as a simulator, and the Saveh rotary cement kiln, as a real industrial system, are applied to evaluate the proposed method. • Diagnosing the root cause of the faults through the classification of the alarm floods. • Using transfer entropy to detect the similarity between the alarm flood sequences. • Calculation of the transfer entropy between the process variable and the alarm data. • Multi-sensor information fusion for large-scale plant and simultaneous alarms. • An online method for early predict of the root cause of the alarm floods occurring. [ABSTRACT FROM AUTHOR]

This study presents a novel approach to predicting price fluctuations for U.S. sector index ETFs. By leveraging information-theoretic measures like mutual information and transfer entropy, we constructed threshold networks highlighting nonlinear dependencies between log returns and trading volume rate changes. We derived centrality measures and node embeddings from these networks, offering unique insights into the ETFs' dynamics. By integrating these features into gradient-boosting algorithm-based models, we significantly enhanced the predictive accuracy. Our approach offers improved forecast performance for U.S. sector index futures and adds a layer of explainability to the existing literature. [ABSTRACT FROM AUTHOR]

Anticipatory dynamics (AD) is unusual in that responses from an information receiver can appear ahead of triggers from the source, and direction of information flow (DIF) is needed to establish causality. Although it is believed that anticipatory dynamics is important for animals' survival, natural examples are rare. Time series (trajectories) from a pair of interacting zebrafish are used to look for the existence of AD in natural systems. In order to obtain the DIF between the two trajectories, we have made use of a special experimental design to designate information source. However, we have also used common statistical tools such as Granger causality and transfer entropy to detect DIF. In our experiments, we found that a majority of the fish pairs do not show any anticipatory behaviors and only a few pairs displayed possible AD. Interestingly, for fish in this latter group, they do not display AD all the time. Our findings suggest that the formation of schooling of fish might not need the help of AD, and new tools are needed in the detection of causality in AD system. [ABSTRACT FROM AUTHOR]

Visual scanning is achieved via head motion and gaze movement for visual information acquisition and cognitive processing, which plays a critical role in undertaking common sensorimotor tasks such as driving. The coordination of the head and eyes is an important human behavior to make a key contribution to goal-directed visual scanning and sensorimotor driving. In this paper, we basically investigate the two most common patterns in eye–head coordination: "head motion earlier than eye movement" and "eye movement earlier than head motion". We utilize bidirectional transfer entropies between head motion and eye movements to determine the existence of these two eye–head coordination patterns. Furthermore, we propose a unidirectional information difference to assess which pattern predominates in head–eye coordination. Additionally, we have discovered a significant correlation between the normalized unidirectional information difference and driving performance. This result not only indicates the influence of eye–head coordination on driving behavior from a computational perspective but also validates the practical significance of our approach utilizing transfer entropy for quantifying eye–head coordination. [ABSTRACT FROM AUTHOR]

This paper provides a methodology to better understand the relationships between different aspects of vocal fold motion, which are used as features in machine learning-based approaches for detecting respiratory infections from voice recordings. The relationships are derived through a joint multivariate analysis of the vocal fold oscillations of speakers. Specifically, the multivariate setting explores the displacements and velocities of the left and right vocal folds derived from recordings of five extended vowel sounds for each speaker (/aa/, /iy/, /ey/, /uw/, and /ow/). In this multivariate setting, the differences between the bivariate and conditional interactions are analyzed by information-theoretic quantities based on transfer entropy. Incorporation of the conditional quantities reveals information regarding the confounding factors that can influence the statistical interactions among other pairs of variables. This is demonstrated on a vector autoregressive process where the analytical derivations can be carried out. As a proof of concept, the methodology is applied on a clinically curated dataset of COVID-19. The findings suggest that the interaction between the vocal fold oscillations can change according to individuals and presence of any respiratory infection, such as COVID-19. The results are important in the sense that the proposed approach can be utilized to determine the selection of appropriate features as a supplementary or early detection tool in voice-based diagnostics in future studies. [ABSTRACT FROM AUTHOR]

Despite the growing literature in cryptocurrency forecasting and their price drivers, the relationship between their price and other financial time series is an ongoing matter of debate. This study proposes a three‐step methodology to cover these arguments. First, we conduct an ad hoc analysis using transfer entropy (TE) to study the causal relationship between Bitcoin (BTC) returns and a vast array of financial time series. Then, we utilize variables with a significant amount of information flow toward BTC returns to forecast multi‐step‐ahead BTC returns. Finally, we use explainable artificial intelligence post hoc analysis methods to discover the contribution of each input feature to the overall forecasting. The results indicate a significant change in the information flow pattern in the first days of the COVID‐19 pandemic outbreak. Additionally, our proposed TE‐based feature‐selection method outperforms both benchmarks, a nonfeature‐selection model, and backward stepwise regression. [ABSTRACT FROM AUTHOR]

In collective systems, influence of individuals can permeate an entire group through indirect interactionscom-plicating any scheme to understand individual roles from observations. A typical approach to understand an individuals influence on another involves consideration of confounding factors, for example, by conditioning on other individuals outside of the pair. This becomes unfeasible in many cases as the number of individuals increases. In this article, we review some of the unforeseen problems that arise in understanding individual influence in a collective such as single cells, as well as some of the recent works which address these issues using tools from information theory.

Diabetic peripheral neuropathy (DPN) is a prevalent complication of chronic diabetes mellitus and has a significant impact on quality of life. DPN typically manifests itself as a symmetrical, length-dependent sensorimotor polyneuropathy with severe effects on gait. Surface electromyography (sEMG) is a valuable low-cost tool for assessing muscle activation patterns and precise identification of abnormalities. For the present study, we used information theory methods, such as cross-correlation (CC), normalized mutual information (NMI), conditional granger causality (CG-Causality), and transfer entropy (TE), to evaluate muscle network connectivity in three population groups: 33 controls (healthy volunteers, CT), 10 diabetic patients with a low risk of DPN (LW), and 17 moderate/high risk patients (MH). The results obtained indicated significant alterations in the intermuscular coupling mechanisms due to diabetes and DPN, with the TE group showing the best performance in detecting differences. The data revealed a significant increase in information transfer and muscle connectivity in the LW group over the CT group, while the MH group obtained significantly lower values for these metrics than the other two groups. These findings highlight the sEMG coupling metrics’ potential to reveal neuromuscular mechanisms that could aid the development of targeted rehabilitation strategies and help monitor DPN patients.

Accurate runoff prediction is crucial for watershed water resource management, flood prevention, and hydropower station scheduling. Data-driven models have been increasingly applied to runoff prediction tasks and have achieved impressive results. However, existing data-driven methods may produce unreasonable predictions due to the lack of prior knowledge guidance. This study proposes a multivariate runoff prediction model that couples knowledge embedding with data-driven approaches, integrating information contained in runoff probability distributions as constraints into the data-driven model and optimizing the existing loss function with prior probability density functions (PDFs). Using the main stream in the Yellow River Basin with nine hydrological stations as an example, we selected runoff feature factors using the transfer entropy method, chose a temporal convolutional network (TCN) as the data-driven model, and optimized model parameters with the IPSO algorithm, studying univariate input models (TCN-UID), multivariable input models (TCN-MID), and the coupling model. The results indicate the following: (1) Among numerous influencing factors, precipitation, sunshine duration, and relative humidity are the key feature factors driving runoff occurrence; (2) the coupling model can effectively fit the extremes of runoff sequences, improving prediction accuracy in the training set by 6.9% and 4.7% compared to TCN-UID and TCN-MID, respectively, and by 5.7% and 2.8% in the test set. The coupling model established through knowledge embedding not only retains the advantages of data-driven models but also effectively addresses the poor prediction performance of data-driven models at extremes, thereby enhancing the accuracy of runoff predictions.

This study aims to employ a causal network model based on transfer entropy for the early warning of systemic risk in commodity markets. We analyzed the dynamic causal relationships of prices for 25 commodities related to China (including futures and spot prices of energy, industrial metals, precious metals, and agricultural products), validating the effect of the causal network structure among commodity markets on systemic risk. Our research results identified commodities and categories playing significant roles, revealing that industry and precious metal markets possess stronger market information transmission capabilities, with price fluctuations impacting a broader range and with greater force on other commodity markets. Under the influence of different types of crisis events, such as economic crises and the Russia–Ukraine conflict, the causal network structure among commodity markets exhibited distinct characteristics. The results of the effect of external shocks to the causal network structure of commodity markets on the entropy of systemic risk suggest that network structure indicators can warn of systemic risk. This article can assist investors and policymakers in managing systemic risk to avoid unexpected losses.