Search

Single-cell transcriptomics offers a powerful way to reveal the heterogeneity of individual cells. To date, many information theoretical approaches have been proposed to assess diversity and similarity, and characterize the latent heterogeneity in transcriptome data. Diversity implies gene expression variations and can facilitate the identification of signature genes; while, similarity unravels co-expression patterns for cell type clustering. In this review, we summarized 16 measures of information theory used for evaluating diversity and similarity in single-cell transcriptomic data, provide references and shed light on selected theoretical properties when there is a need to select proper measurements in general cases. We further provide an R package assembling discussed approaches to improve the researchers own single-cell transcriptome study. At last, we prospected further applications of diversity and similarity measures in support of depicting heterogeneity in single-cell multi-omics data.

We propose herein a six-parameter McDonald modified Burr-III distribution. This distribution contains several special cases, including the modified Burr-III, Burr-III, Beta Burr-III, and Kum Burr-III distributions. Accordingly, we obtained its hazard function, survival function, moments, Renyi’s entropy, β-entropy, and mean deviation. We also used the method of maximum likelihood to estimate the parameters of our proposed distribution. Simulation studies were also performed for different values of sample sizes. The proposed model is applied on real-life datasets and shows that our proposed distribution yields better fits than other existing models.

The notion of statistical order derives from the disequilibrium concept introduced by López-Ruiz, Mancini, and Calbet thirty years ago. In this effort, it is shown that the disequilibrium is intimately linked to the celebrated Rényi entropy. One also explores this link in connection with the van der Waals gas description.

For perfect diagnosis of brain tumour, it is necessary to identify tumour affected regions in the brain in MRI (Magnetic Resonance Imaging) images effectively and compression of these images for transmission over a communication channel at high speed with better visual quality to the experts. An attempt has been made in this paper for identifying tumour regions with optimal thresholds which are optimized with the proposed Hybrid Bacteria Foraging Optimization Algorithm (BFOA) and Particle Swarm Optimization (PSO) named (HBFOA-PSO) by maximizing the Renyi’s entropy and Kapur’s entropy. BFOA may be trapped into local optimal problem and delay in execution time (convergence time) because of random chemo taxis steps in the procedure of algorithm and to get global solution, a theory of swarming is commenced in the structure of HBFOA-PSO. Effectiveness of this HBFOA-PSO is evaluated on six different MRI images of brain with tumours and proved to be better in Peak Signal to Noise Ratio (PSNR), Mean Square Error (MSE) and Fitness Function.

Cross-frequency interactions, a form of oscillatory neural activity, are thought to play an essential role in the integration of distributed information in the brain. Indeed, phase-amplitude interactions are believed to allow for the transfer of information from large-scale brain networks, oscillating at low frequencies, to local, rapidly oscillating neural assemblies. A promising approach to estimating such interactions is the use of transfer entropy (TE), a non-linear, information-theory-based effective connectivity measure. The conventional method involves feeding instantaneous phase and amplitude time series, extracted at the target frequencies, to a TE estimator. In this work, we propose that the problem of directed phase-amplitude interaction detection is recast as a phase TE estimation problem, under the hypothesis that estimating TE from data of the same nature, i.e., two phase time series, will improve the robustness to the common confounding factors that affect connectivity measures, such as the presence of high noise levels. We implement our proposal using a kernel-based TE estimator, defined in terms of Renyi’s α entropy, which has successfully been used to compute single-trial phase TE. We tested our approach on the synthetic data generated through a simulation model capable of producing a time series with directed phase-amplitude interactions at two given frequencies, and on EEG data from a cognitive task designed to activate working memory, a memory system whose underpinning mechanisms are thought to include phase–amplitude couplings. Our proposal detected statistically significant interactions between the simulated signals at the desired frequencies for the synthetic data, identifying the correct direction of the interaction. It also displayed higher robustness to noise than the alternative methods. The results attained for the working memory data showed that the proposed approach codes connectivity patterns based on directed phase–amplitude interactions, that allow for the different cognitive load levels of the working memory task to be differentiated.

Neural oscillations are present in the brain at different spatial and temporal scales, and they are linked to several cognitive functions. Furthermore, the information carried by their phases is fundamental for the coordination of anatomically distributed processing in the brain. The concept of phase transfer entropy refers to an information theory-based measure of directed connectivity among neural oscillations that allows studying such distributed processes. Phase TE is commonly obtained from probability estimations carried out over data from multiple trials, which bars its use as a characterization strategy in brain–computer interfaces. In this work, we propose a novel methodology to estimate TE between single pairs of instantaneous phase time series. Our approach combines a kernel-based TE estimator defined in terms of Renyi’s α entropy, which sidesteps the need for probability distribution computation with phase time series obtained by complex filtering the neural signals. Besides, a kernel-alignment-based relevance analysis is added to highlight relevant features from effective connectivity-based representation supporting further classification stages in EEG-based brain–computer interface systems. Our proposal is tested on simulated coupled data and two publicly available databases containing EEG signals recorded under motor imagery and visual working memory paradigms. Attained results demonstrate how the introduced effective connectivity succeeds in detecting the interactions present in the data for the former, with statistically significant results around the frequencies of interest. It also reflects differences in coupling strength, is robust to realistic noise and signal mixing levels, and captures bidirectional interactions of localized frequency content. Obtained results for the motor imagery and working memory databases show that our approach, combined with the relevance analysis strategy, codes discriminant spatial and frequency-dependent patterns for the different conditions in each experimental paradigm, with classification performances that do well in comparison with those of alternative methods of similar nature.

Transfer entropy (TE) is a model-free effective connectivity measure based on information theory. It has been increasingly used in neuroscience because of its ability to detect unknown non-linear interactions, which makes it well suited for exploratory brain effective connectivity analyses. Like all information theoretic quantities, TE is defined regarding the probability distributions of the system under study, which in practice are unknown and must be estimated from data. Commonly used methods for TE estimation rely on a local approximation of the probability distributions from nearest neighbor distances, or on symbolization schemes that then allow the probabilities to be estimated from the symbols' relative frequencies. However, probability estimation is a challenging problem, and avoiding this intermediate step in TE computation is desirable. In this work, we propose a novel TE estimator using functionals defined on positive definite and infinitely divisible kernels matrices that approximate Renyi's entropy measures of order α. Our data-driven approach estimates TE directly from data, sidestepping the need for probability distribution estimation. Also, the proposed estimator encompasses the well-known definition of TE as a sum of Shannon entropies in the limiting case when α → 1. We tested our proposal on a simulation framework consisting of two linear models, based on autoregressive approaches and a linear coupling function, respectively, and on the public electroencephalogram (EEG) database BCI Competition IV, obtained under a motor imagery paradigm. For the synthetic data, the proposed kernel-based TE estimation method satisfactorily identifies the causal interactions present in the data. Also, it displays robustness to varying noise levels and data sizes, and to the presence of multiple interaction delays in the same connected network. Obtained results for the motor imagery task show that our approach codes discriminant spatiotemporal patterns for the left and right-hand motor imagination tasks, with classification performances that compare favorably to the state-of-the-art.

Multi-level image thresholding is the most direct and effective method for image segmentation, which is a key step for image analysis and computer vision, however, as the number of threshold values increases, exhaustive search does not work efficiently and effectively and evolutionary algorithms often fall into a local optimal solution. In the paper, a meta-heuristics algorithm based on the breeding mechanism of Chinese hybrid rice is proposed to seek the optimal multi-level thresholds for image segmentation and Renyi’s entropy is utilized as the fitness function. Experiments have been run on four scanning electron microscope images of cement and four standard images, moreover, it is compared with other six classical and novel evolutionary algorithms: genetic algorithm, particle swarm optimization algorithm, differential evolution algorithm, ant lion optimization algorithm, whale optimization algorithm, and salp swarm algorithm. Meanwhile, some indicators, including the average fitness values, standard deviation, peak signal to noise ratio, and structural similarity index are used as evaluation criteria in the experiments. The experimental results show that the proposed method prevails over the other algorithms involved in the paper on most indicators and it can segment cement scanning electron microscope image effectively.

Permutation entropy (PE) has been widely exploited to measure the complexity of the electroencephalogram (EEG), especially when complexity is linked to diagnostic information embedded in the EEG. Recently, the authors proposed a spatial-temporal analysis of the EEG recordings of absence epilepsy patients based on PE. The goal here is to improve the ability of PE in discriminating interictal states from ictal states in absence seizure EEG. For this purpose, a parametrical definition of permutation entropy is introduced here in the field of epileptic EEG analysis: the permutation Rényi entropy (PEr). PEr has been extensively tested against PE by tuning the involved parameters (order, delay time and alpha). The achieved results demonstrate that PEr outperforms PE, as there is a statistically-significant, wider gap between the PEr levels during the interictal states and PEr levels observed in the ictal states compared to PE. PEr also outperformed PE as the input to a classifier aimed at discriminating interictal from ictal states.

Time Frequency Signal Analysis and Processing (TFSAP) have been proposed in order to analyse the signal in both the time and the frequency domains. Electroencephalography (EEG) as a time-varying frequency signal is an interesting field in which Time Frequency Distribution (TFD) could be used in order to visualize the simultaneous distributions of signal energy in different physiological and pathological brain states. Particularly, epileptic signals due to their great features of seizure activity are introduced as the most attractive research field among researchers. This study outlines an investigation on two main pathologic brain states including, pre-ictal activity and seizure activity compared to normal activity. Pseudo-Wigner -Ville and Choi-William distributions are used in order to visualize the energy content of signals in these states. Different segments of brain electrical activity are analyzed using these distributions. Finally, Renyi’s entropy as an important characteristic which offer insight towards the EEG signal processing has been extracted from TFDs. The results obtained indicate that Renyi’s entropy is a high-quality discriminative feature especially in alpha and delta sub-bands of the EEG signal.

Hierarchical clustering has been extensively used in practice, where clusters can be assigned and analyzed simultaneously, especially when estimating the number of clusters is challenging. However, due to the conventional proximity measures recruited in these algorithms, they are only capable of detecting mass-shape clusters and encounter problems in identifying complex data structures. Here, we introduce two bottom-up hierarchical approaches that exploit an information theoretic proximity measure to explore the nonlinear boundaries between clusters and extract data structures further than the second order statistics. Experimental results on both artificial and real datasets demonstrate the superiority of the proposed algorithm compared to conventional and information theoretic clustering algorithms reported in the literature, especially in detecting the true number of clusters.

Urban sprawl (US), propelled by rapid population growth leads to the shrinkage of productive agricultural lands and pristine forests in the suburban areas and, in turn, adversely affects the provision of ecosystem services. The quantification of US is thus crucial for effective urban planning and environmental management. Like many megacities in fast growing developing countries, Chennai, the capital of Tamilnadu and one of the business hubs in India, has experienced extensive US triggered by the doubling of total population over the past three decades. However, the extent and level of US has not yet been quantified and a prediction for future extent of US is lacking. We employed the Random Forest (RF) classification on Landsat imageries from 1991, 2003, and 2016, and computed six landscape metrics to delineate the extent of urban areas within a 10 km suburban buffer of Chennai. The level of US was then quantified using Renyi’s entropy. A land change model was subsequently used to project land cover for 2027. A 70.35% expansion in urban areas was observed mainly towards the suburban periphery of Chennai between 1991 and 2016. The Renyi’s entropy value for year 2016 was 0.9, exhibiting a two-fold level of US when compared to 1991. The spatial metrics values indicate that the existing urban areas became denser and the suburban agricultural, forests and particularly barren lands were transformed into fragmented urban settlements. The forecasted land cover for 2027 indicates a conversion of 13,670.33 ha (16.57% of the total landscape) of existing forests and agricultural lands into urban areas with an associated increase in the entropy value to 1.7, indicating a tremendous level of US. Our study provides useful metrics for urban planning authorities to address the social-ecological consequences of US and to protect ecosystem services.