Search

Uncertainty measures are important tools for analyzing various data. However, there are relatively few studies on the uncertainty measures for covering rough set models. In this paper, from the viewpoint of the lower and upper approximations, we propose new uncertainty measures, the lower rough entropy and the upper rough entropy, for covering rough set models. Then, we define the concepts of the joint entropy and the conditional entropy in the covering rough set models. Some important properties of these measures are obtained, and their relationships are investigated. Furthermore, we provide a certain characterization of reducible element of a covering by means of the proposed measures, and apply the proposed rough entropy to evaluate the significance of covering granules of a covering. Finally, we apply these rough entropies to measure a dual degree between covering lower and upper approximations. The theoretical analysis and examples show that the proposed uncertainty measures for covering rough set models are reasonable and useful.

The purpose of this study is to propose novel and efficient competitiveness indexes to measure the level of competition among container shipping operators based on a specific region. These indexes should require only basic data, which is full container throughput on the basis of terminal/ hinterland and ship operator. This study takes advantages of two methods to propose novel indexes as alternatives to Herfindahl Hirschman Index (HHI), which is very popular to measure level of competition. Originally named Competition-based Overall Similarity Measurement Index (COSMI) and Entropy Competitiveness Index (ECI) utilize overall similarity measure from clustering analysis and entropy methodologies, respectively. Both indexes have been proposed with two variants for each. COSMI200+ ignores the throughput of each SO having an amount less than 200 Twentyfoot Equivalent Units (TEUs), but COSMITOP5 takes into account only the top 5 SOs in terms of local throughput in a hinterland. ECI-JOINT includes a joint entropy coefficient which is constant for each hinterland, but ECI-VAR takes into account a variable entropy coefficient defined by the number of ship operators in each hinterland. Analyzing a dataset for the terminals located in Turkey, the Entropy Competitiveness Index (by means of ECI-JOINT variant) has been exhibited as a good alternative to HHI with a great correlation coefficient with it: 0.97. Theoretically, Competitionbased Overall Similarity Measurement Index (by means of COSMITOP5 variant) seems a promising method, but it is highly affected by outliers and inconstant numbers of ship operators per route, indicating a moderate correlation coefficient with HHI: 0.45., Bu çalışmanın amacı, belli bir bölge bazlı olarak konteyner gemi operatörleri arasındaki rekabet seviyesini ölçmeye yarayan yeni ve etkin rekabet indeksleri oluşturmaktır. Bu indeksler terminal/bölge ve gemi operatörü bazlı olarak, sadece başlıca verilerden olan dolu konteyner sevkiyat verilerine ihtiyaç duymalıdır. Bu çalışma, rekabet seviyesi ölçümünde popüler olan Herfindahl Hirschman İndeksine (HHI) alternatif yeni indeksler oluşturabilmek için iki farklı yöntemden yararlanmaktadır. Özgün olarak Rekabet Bazlı Toplam Benzerlik Ölçüsü İndeksi (COSMI) ve Entropi Rekabet İndeksi (ECI) olarak adlandırılan bu indeksler, sırasıyla kümeleme analizi toplam benzerlik ölçüsü ve entropi yöntemlerinden yararlanmaktadırlar. Çalışmada her iki indeksin ikişer uyarlaması incelenmiştir. COSMI200+, bir lokal bölgedeki 200 TEU’dan daha az yükleme-tahliye performansı gösteren gemi operatörlerini gözardı ederken, COSMITOP5 sadece en yüksek performansa sahip 5 gemi operatörünü dikkate almaktadır. ECI-JOINT uyarlamasında sabit olan ortak bir entropi katsayısı kullanılırken, ECI-VAR uyarlamasında her bir lokal bölgedeki gemi operatörü sayısına göre değişen entropi katsayısı kullanılmaktadır. Türkiye’de mukim terminallere ait verilerin analizi, Entropi Rekabet İndeksi’nin (ECI-JOINT uyarlaması ile) HHI ile 0,97 değerinde korelasyon katsayısına sahip olduğunu ve bu indekse iyi bir alternatif olduğunu göstermektedir. Teorik olarak Rekabet Bazlı Toplam Rekabet Ölçüsü İndeksi (COSMITOP5 uyarlaması ile) umut veren bir yöntemi barındırsa da, dışadüşenler ve rota bazlı değişken gemi operatörü sayısı nedeniyle HHI ile 0,45 değerinde orta dereceli bir korelasyon katsayısına sahiptir.

Kernel density estimation, which is a non-parametric method about estimating probability density distribution of random variables, has been used in feature selection. However, existing feature selection methods based on kernel density estimation seldom consider interval-valued data. Actually, interval-valued data exist widely. In this paper, a feature selection method based on kernel density estimation for interval-valued data is proposed. Firstly, the kernel function in kernel density estimation is defined for interval-valued data. Secondly, the interval-valued kernel density estimation probability structure is constructed by the defined kernel function, including kernel density estimation conditional probability, kernel density estimation joint probability and kernel density estimation posterior probability. Thirdly, kernel density estimation entropies for interval-valued data are proposed by the constructed probability structure, including information entropy, conditional entropy and joint entropy of kernel density estimation. Fourthly, we propose a feature selection approach based on kernel density estimation entropy. Moreover, we improve the proposed feature selection algorithm and propose a fast feature selection algorithm based on kernel density estimation entropy. Finally, comparative experiments are conducted from three perspectives of computing time, intuitive identifiability and classification performance to show the feasibility and the effectiveness of the proposed method.

In this paper, some new results and inequalities of dynamic failure entropy (DFE) defined by Abbasnejad (2011) will be obtained. Then weighted versions of failure and survival entropies of order β introduced and some properties and inequalities based on these uncertainty measures found. Then weighted DFE will be introduced and some properties investigated. Finally, empirical cumulative entropy is applied to estimate the new information measures.

Cross-frequency phase–amplitude coupling (PAC) plays an important role in neuronal oscillations network, reflecting the interaction between the phase of low-frequency oscillation (LFO) and amplitude of the high-frequency oscillations (HFO). Thus, we applied four methods based on permutation analysis to measure PAC, including multiscale permutation mutual information (MPMI), permutation conditional mutual information (PCMI), symbolic joint entropy (SJE), and weighted-permutation mutual information (WPMI). To verify the ability of these four algorithms, a performance test including the effects of coupling strength, signal-to-noise ratios (SNRs), and data length was evaluated by using simulation data. It was shown that the performance of SJE was similar to that of other approaches when measuring PAC strength, but the computational efficiency of SJE was the highest among all these four methods. Moreover, SJE can also accurately identify the PAC frequency range under the interference of spike noise. All in all, the results demonstrate that SJE is better for evaluating PAC between neural oscillations.

In this paper, we propose the selection of most suitable mother wavelet function for Turkish phonemes using discrete wavelet transform. The determination of most similar mother wavelet function to the signal has been a challenge in speech processing. The optimum mother wavelet function for Turkish phonemes have been determined by using quantitative measures which are energy and Shannon entropy, information theoretic measures which are joint entropy, conditional entropy, mutual information, and relative entropy from wavelet coefficients of the phonemes. In this study, 101 potential functions were investigated to determine the most appropriate mother wavelet. Experimental results show that the most appropriate wavelet functions for /ç/ and /ş/ phonemes which are unvoiced fricatives have been found as Bi-orthogonal 3.9 and Bi-orthogonal 5.5, respectively. By considering all the results, it is seen that the Bi-orthogonal 3.1 and Discrete Meyer wavelet functions are the most suitable mother wavelets for all other phonemes.

Inspired on the classical Bayesian method also called mutual information, by generalizing the classical case to quantum approach to study the multipartite correlation in quantum image by using the flexible representation of quantum images (FRQI) method. Some important entanglement measures are calculated including von Neumann entropy, conditional entropy, joint entropy, discord and mutual information. We take two typical images of size 2 × 2 pixels and 8 × 8 pixels and study their entanglement measures from different classical and quantum point of view. The quantum image of size 2 × 2 is studied under the change of color for one pixel, but the quantum image of 8 × 8 is treated under translation transform. We find that the classical joint entropy remains invariant under the change of color for one pixel but the quantum entropy is sensitive to such change. It is shown that the total correlation IT could arrive to the double amount of the classical joint entropy. The discord in the classical case is zero, but it varies much in the quantum case.

This article considers the problem of estimation of the parameter of DUS-exponential distribution based on upper records through classical as well as Bayesian procedures. Maximum likelihood estimator is calculated under the classical scheme and Bayes estimator is obtained under the squared error loss function by using MCMC technique. The performances of both the estimators are compared on the basis of their estimated mean squared errors. The asymptotic and HPD intervals for the unknown parameter is also calculated. In addition to that, the entropy of ith upper record and joint entropy based on m upper records are obtained for considered distribution. A real data set is considered to illustrate the suitability of the proposed methodology.

The estimation of relative entropy measures such as mutual information, conditional and joint entropy, or transfer entropy requires the estimation of conditional and joint densities. When the data are continuous, a multi-variate kernel density estimation or a discretization scheme is usually applied. The problem with this discretization approach is that for mutual information as well as transfer entropy the resulting measure does not converge monotonically to the true value when the number of discrete bins is increased. In the absence of a distribution theory, hypothesis testing is only possible by means of bootstrapping. We propose to estimate the necessary joint and conditional frequencies by means of quantile regression. This allows us to avoid arbitrary binning and all associated problems. Moreover, due to the semi-parametric nature of this approach, the computational burden is decisively reduced compared to multi-variate kernel density estimation. Instead, we show that we can flexibly use quantile regressions to estimate densities in order to calculate joint, conditional and transfer entropy as well as mutual information. Furthermore, by casting our into a Generalized Method of Moments framework, we develop the asymptotic theory to conduct inference on relative entropy measures for multiple variables.

In this research paper, it is proved that an approximation to the Gibbs-Shannon entropy measure naturally leads to the Tsallis entropy for the real parameter q = 2. Several interesting measures based on the input as well as the output of a discrete memoryless channel are provided and some of the properties of those measures are discussed. It is expected that these results will be of utility in Information Theoretic research.

First, information theory is reviewed for readers who are unfamiliar with basic concepts and ideas. Information of events and entropy of sample spaces and/or random variables are introduced and the properties are discussed. To measure the differences between distributions, the Kullback–Leibler (KL) information is explained through a theoretical consideration. In order to treat the association between random variables, the joint entropy and the conditional entropy are also introduced. Second, basic statistics are considered from a viewpoint of entropy. The log-likelihood ratio test statistic is interpreted in entropy. By introducing entropy of continuous variables, t and F statistics are also discussed through KL information.

We propose a mathematical model of the entropy estimator in the Linux random number generator. First, we construct a probability model for random event times in entropy sources, and then precisely derive probability distributions for the first, second, and third time differences. Second, we obtain the probability distribution for the minimum of absolute values of these differences, which is used for the estimated entropy in the Linux system. Moreover, we provide several simulations that display the accuracy of our results for various parameters.

Recently, Sati and Gupta (2015) have introduced a generalized cumulative residual entropy based on the non-additive Tsallis entropy. The cumulative residual entropy, introduced by Rao et al. (2004) is a generalized measure of uncertainty which is applied in reliability and image alignment and non-additive measures of entropy. This entropy finds justifications in many physical, biological and chemical phenomena. In this paper, we derive the weighted form of this measure and call it Weighted Cumulative Residual Tsallis Entropy (WCRTE). Being a “length-biased” shift-dependent information measure, WCRTE is related to the differential information in which higher weight is assigned to large values of observed random variables. Based on the dynamic version of this new information measure, we propose ageing classes and it is shown that it can uniquely determine the survival function and Rayleigh distribution. Several properties, including linear transformations, bounds and related results to stochastic orders are obtained for these measures. Also, we identify classes of distributions in which some well-known distributions are maximum dynamic version of WCRTE. The empirical WCRTE is finally proposed to estimate the new information measure.

The convergence and divergence of oscillatory behavior of different brain regions are very important for the procedure of information processing. Measurements of coupling or correlation are very useful to study the difference of brain activities. In this study, EEG signals were collected from ten subjects under two conditions, i.e. eyes closed state and idle with eyes open. We propose a nonlinear algorithm, symbolic joint entropy, to compare the coupling strength among the frontal, temporal, parietal and occipital lobes and between two different states. Instead of decomposing the EEG into different frequency bands (theta, alpha, beta, gamma etc.), the novel algorithm is to investigate the coupling from the entire spectrum of brain wave activities above 4Hz. The coupling coefficients in two states with different time delay steps are compared and the group statistics are presented as well. We find that the coupling coefficient of eyes open state with delay consistently lower than that of eyes close state across the group except for one subject, whereas the results without delay are not consistent. The differences between two brain states with non-zero delay can reveal the intrinsic inter-region coupling better. We also use the well-known Henon map data to validate the algorithm proposed in this paper. The result shows that the method is robust and has a great potential for other physiologic time series.

We propose a new definition of entropy of basic probability assignments (BPAs) in the Dempster–Shafer (DS) theory of belief functions, which is interpreted as a measure of total uncertainty in the BPA. Our definition is different from those proposed by Hohle, Smets, Yager, Nguyen, Dubois–Prade, Lamata–Moral, Klir–Ramer, Klir–Parviz, Pal et al., Maeda–Ichihashi, Harmanec–Klir, Abellan–Moral, Jousselme et al., Pouly et al., and Deng. We state a list of six desired properties of entropy for DS belief functions theory, four of which are motivated by Shannon's definition of entropy of probability functions, and the remaining two are requirements that adapt this measure to the philosophy of the DS theory. Three of our six desired properties are different from the five properties proposed by Klir and Wierman. We demonstrate that our definition satisfies all six properties in our list, whereas none of the existing definitions do. Our new definition has two components. The first component is Shannon's entropy of an equivalent probability mass function obtained using the plausibility transform, which constitutes the conflict measure of entropy. The second component is Dubois–Prade's definition of entropy of basic probability assignments in the DS theory, which constitutes the non-specificity measure of entropy. Our new definition is the sum of these two components. Our definition does not satisfy the subadditivity property. Whether there exists a definition that satisfies our six properties plus subadditivity remains an open question.

The cumulative entropy is an information measure which is alternative to the differential entropy. Indeed, the cumulative entropy of a random lifetime X can be expressed as the expectation of its mean inactivity time evaluated at X . In this paper we propose a new generalized cumulative entropy based on Tsallis entropy (CTE) and its dynamic version (DCTE). We study some properties and characterization results for this measure.

In this article, we propose high order, semi-discrete, entropy stable, finite difference schemes for Ten-Moment Gaussian Closure equations. The crucial components of these schemes are an entropy conservative flux and suitable high order entropy dissipative operators to ensure entropy stability. We design two numerical fluxes, one is approximately entropy conservative, and another is entropy conservative flux. For the construction of appropriate entropy dissipative operators, we also derive entropy scaled right eigenvectors. This is used for sign preserving reconstruction of scaled entropy variables, which results in second and third order entropy stable schemes. We also extend these schemes to a plasma flow model with source term. Several numerical results are presented for homogeneous and non-homogeneous cases to demonstrate stability and performance of these schemes.

In this paper, we perform a multiscale entropy analysis on the Dow Jones Industrial Average Index using the Shannon entropy. The stock index shows the characteristic of multi-scale entropy that caused by noise in the market. The entropy is demonstrated to have significant predictive ability for the stock index in both long-term and short-term, and empirical results verify that noise does exist in the market and can affect stock price. It has important implications on market participants such as noise traders.

This paper proposes some new distance measures between interval-valued neutrosophic sets (IvNSs) and their similarity measures. Then, some entropy measures of IvNS based on the distances are proposed as the extension of the entropy measures of interval-valued intuitionistic fuzzy sets (IvIFSs). Also, we investigate the relationship between the presented entropy measures and the similarity measures for IvNSs. Finally, the comparison of the new entropy measures with existing entropy measures for IvNSs is given by the numerical and decision-making examples to demonstrate that the proposed new entropy measures for IvNSs are effective and reasonable and more intelligible in representing the degree of fuzziness of IvNSs than the existing ones.

Background The Electroencephalogram (EEG) signal is a time series depictive signal that contains the useful knowledge about the state of the brain. It has high temporal resolution for detection of chronic brain disorders such as epilepsy/seizure, dementia, sleep apnea, schizophrenia, etc. In this work, EEG is a prime concern for seizure/epilepsy detection and localization. Methods Entropy estimator is a good solution to this problem. Here, the time series complexity analysis of brain signal is carried using five different entropy estimators: Shannon Entropy, Renyi Entropy, Approximate Entropy, Sample Entropy, and Fuzzy Entropy. The average entropy values of EEG signal is significantly found lower for epileptic data sets compared to non-epileptic EEG. Results Experimental results evaluated for discriminating ability of each entropy measure demonstrated that among all entropies, Fuzzy Entropy discriminates between epileptic EEG and non-epileptic EEG more efficiently with a minimum p-value (0.001) compared to other four entropy estimators. Fuzzy Entropy defines the similarity between two vectors fuzzily on the basis of exponential function. Unlike to Approximate and Sample Entropy, the Fuzzy Entropy is free from parameter limitations and offers efficient results even for the small tolerance (r Conclusion The Fuzzy entropy gives better stability and consistency with highest discriminating ability compared to other entropy estimators. Eventually, results for detection and localization of epilepsy for affected channel and region through the variance and FuzzyEn are cross-validated by expert Neuro-physician.

There is a certain relationship between the global financial markets, which creates an interactive network of global finance. Transfer entropy, a measurement for information transfer, offered a good way to analyse the relationship. In this paper, we analysed the relationship between 9 stock indices from the U.S., Europe and China (from 1995 to 2015) by using transfer entropy (TE), effective transfer entropy (ETE), Renyi transfer entropy (RTE) and effective Renyi transfer entropy (ERTE). We compared the four methods in the sense of the effectiveness for identification of the relationship between stock markets. In this paper, two kinds of information flows are given. One reveals that the U.S. took the leading position when in terms of lagged-current cases, but when it comes to the same date, China is the most influential. And ERTE could provide superior results.

here is a new theory of information based on logic. The definition of Shannon entropy as well as the notions on joint, conditional, and mutual entropy as defined by Shannon can all be derived by a uniform transformation from the corresponding formulas of logical information theory. Information is first defined in terms of sets of distinctions without using any probability measure. When a probability measure is introduced, the logical entropies are simply the values of the (product) probability measure on the sets of distinctions. The compound notions of joint, conditional, and mutual entropies are obtained as the values of the measure, respectively, on the union, difference, and intersection of the sets of distinctions. These compound notions of logical entropy satisfy the usual Venn diagram relationships (e.g., inclusion-exclusion formulas) since they are values of a measure (in the sense of measure theory). The uniform transformation into the formulas for Shannon entropy is linear so it explains the long-noted fact that the Shannon formulas satisfy the Venn diagram relations--as an analogy or mnemonic--since Shannon entropy is not a measure (in the sense of measure theory) on a given set. What is the logic that gives rise to logical information theory? Partitions are dual (in a category-theoretic sense) to subsets, and the logic of partitions was recently developed in a dual/parallel relationship to the Boolean logic of subsets (the latter being usually mis-specified as the special case of "propositional logic"). Boole developed logical probability theory as the normalized counting measure on subsets. Similarly the normalized counting measure on partitions is logical entropy--when the partitions are represented as the set of distinctions that is the complement to the equivalence relation for the partition. In this manner, logical information theory provides the set-theoretic and measure-theoretic foundations for information theory. The Shannon theory is then derived by the transformation that replaces the counting of distinctions with the counting of the number of binary partitions (bits) it takes, on average, to make the same distinctions by uniquely encoding the distinct elements--which is why the Shannon theory perfectly dovetails into coding and communications theory.

Entropy-based measures have been used to characterize the structure of complex networks. Various graph entropies based on different graph parameters have been proposed and studied. So, it has been intricate to decide which measure is the right one for solving a particular problem. In this paper, we introduce graph entropy measures based on independent sets and matchings of graphs. The values of entropies of some special graphs are calculated and we draw several conclusions regrading the usability of the measures. Based on our findings, we conclude that it should be demanding studying extremal values of the novel entropies.

In this paper, the exponential weighted entropy (EWE) and exponential weighted mutual information (EWMI) are proposed as the more generalized forms of Shannon entropy and mutual information (MI), respectively. They are position-related and causal systems that redefine the foundations of information-theoretic metrics. As the special forms of the weighted entropy and the weighted mutual information, EWE and EWMI have been proved that they preserve nonnegativity and concavity properties similar to Shannon frameworks. They can be adopted as the information measures in spatial interaction modeling. Paralleling with the normalized mutual information (NMI), the normalized exponential weighted mutual information (NEWMI) is also investigated. Image registration experiments demonstrate that EWMI and NEWMI algorithms can achieve higher aligned accuracy than MI and NMI algorithms.

In this paper, we introduce a quantity measure which is called (R, S)-norm entropy and discuss some of its major properties with Shannon’s and other entropies in the literature. Based on this (R, S)-norm entropy, we have proposed a new (R, S)-norm fuzzy information measure and discussed its validity and properties. Further, we have given its comparison with other fuzzy information measures to prove its effectiveness. Attribute weights play an important role in multiple-attribute decision-making problems. In the present communication, two methods of determining the attribute weights are introduced. First is the case when the information regarding attribute weights is incompletely known or completely unknown and second is when we have partial information about attribute weights. For the first case, the extension of ordinary entropy weight method is used to calculate attribute weights and minimum entropy principle method based on solving a linear programming model is used in the second case. Finally, two methods are explained through numerical examples.

Entropy is a measure of the uncertainty associated with a variable whose value cannot be exactly predicted. Based on the notion of chance measure, a concept of uncertain random entropy is introduced and used to provide a quantitative measurement of the uncertainty associated with uncertain random variables and its properties are studied in this paper. Relative entropy is a measure of the difference between two distribution functions. In order to deal with the divergence of uncertain random variables via chance distributions, this paper proposes also the relative entropy for uncertain random variables, as well as it investigates some mathematical properties of this concept. As an application, a model is presented to formulate a minimum spanning tree problem with uncertain random edge weights involving a relative entropy chance distribution. Finally, a numerical example of an uncertain random network is put forward to illustrate the effectiveness of the proposed model.

Characterising the capacity region for a network can be extremely difficult. Even with independent sources, determining the capacity region can be as hard as the open problem of characterising all information inequalities. The majority of computable outer bounds in the literature are relaxations of the Linear Programming bound which involves entropy functions of random variables related to the sources and link messages. When sources are not independent, the problem is even more complicated. Extension of Linear Programming bounds to networks with correlated sources is largely open. Source dependence is usually specified via a joint probability distribution, and one of the main challenges in extending linear program bounds is the difficulty (or impossibility) of characterising arbitrary dependencies via entropy functions. This paper tackles the problem by answering the question of how well entropy functions can characterise correlation among sources. We show that by using carefully chosen auxiliary random variables, the characterisation can be fairly "accurate" Using such auxiliary random variables we also give implicit and explicit outer bounds on the capacity of networks with correlated sources. The characterisation of correlation or joint distribution via Shannon entropy functions is also applicable to other information measures such as Renyi entropy and Tsallis entropy., Comment: 24 pager, 1 figure, submitted to IEEE Transactions on Information Theory. arXiv admin note: text overlap with arXiv:1309.1517

Evolutionary algorithms are used widely in optimization studies on water distribution networks. The optimization algorithms use simulation models that analyse the networks under various operating conditions. The solution process typically involves cost minimization along with reliability constraints that ensure reasonably satisfactory performance under abnormal operating conditions also. Flow entropy has been employed previously as a surrogate reliability measure. While a body of work exists for a single operating condition under steady state conditions, the effectiveness of flow entropy for systems with multiple operating conditions has received very little attention. This paper describes a multi-objective genetic algorithm that maximizes the flow entropy under multiple operating conditions for any given network. The new methodology proposed is consistent with the maximum entropy formalism that requires active consideration of all the relevant information. Furthermore, an alternative but equivalent flow entropy model that emphasizes the relative uniformity of the nodal demands is described. The flow entropy of water distribution networks under multiple operating conditions is discussed with reference to the joint entropy of multiple probability spaces, which provides the theoretical foundation for the optimization methodology proposed. Besides the rationale, results are included that show that the most robust or failure-tolerant solutions are achieved by maximizing the sum of the entropies.

We propose a statistical monitoring method to detect the increase of entropy in categorical data. First, we propose a distribution estimation method to approximate the probability distribution of t...

In this article, the concept of cumulative residual entropy (CRE) given by Rao et al. (2004) is extended to Tsallis entropy function and dynamic version, both residual and past of it. We study some properties and characterization results for these generalized measures. In addition, we provide some characterization results of the first order statistic based on the Tsallis survival entropy.

Uncertain variables, applied to modelling imprecise quantities, are fundamental concepts in uncertainty theory. In order to characterize the information deficiency of an uncertain variable, this paper proposes a definition of quadratic entropy. Compared with the traditional entropy, it is much easier to compute and has a wider range. Furthermore, the principle of maximum entropy is applied to quadratic entropy, and two theorems of maximum quadratic entropy with moment constraint are proved.

SYNOPTIC ABSTRACTAbbasnejad (2011) proposed new measures of information known as failure and dynamic failure entropy of order α based on generalization of the well-known cumulative entropy. In this article, we propose the weighted forms of failure entropy (WFE) and dynamic failure entropy (DWFE) of order α. Some relationships of these measures with well-known reliability measures and ageing classes are studied and some characterization results for reflected Weibull distribution are provided. We also propose an empirical estimator for the WDFE of order α. We demonstrate the performance of the proposed estimator with a real life data set and further carried out a comprehensive Monte-Carlo simulation study.

Recently, Di Crescenzo and Longobardi (2006) have studied “length-biased” shift-dependent information measure and its dynamic versions. On the other hand, Renyi's entropy plays a vital role in the ...

Considering the effects of environment disturbances and instrument systems, the actual detecting signals always are carrying different trends, which result in that it is difficult to accurately catch signals complexity. So choosing steady and effective analysis methods is very important. In this paper, we applied entropy measures—the base-scale entropy and approximate entropy to analyze signal complexity, and studied the effect of trends on the ideal signal and the heart rate variability (HRV) signals, that is, linear, periodic, and power-law trends which are likely to occur in actual signals. The results show that approximate entropy is unsteady when we embed different trends into the signals, so it is not suitable to analyze signal with trends. However, the base-scale entropy has preferable stability and accuracy for signal with different trends. So the base-scale entropy is an effective method to analyze the actual signals.

The key issue in image fusion is the process of defining evaluation indices for the output image and for multi-scale image data set. This paper attempted to develop a fusion model for plantar pressure distribution images, which is expected to contribute to feature points construction based on shoe-last surface generation and modification. First, the time series plantar pressure distribution image was preprocessed, including back removing and Laplacian of Gaussian (LoG) filter. Then, discrete wavelet transform and a multi-scale pixel conversion fusion operating using a parameter estimation optimized Gaussian mixture model (PEO-GMM) were performed. The output image was used in a fuzzy weighted evaluation system, that included the following evaluation indices: mean, standard deviation, entropy, average gradient, and spatial frequency; the difference with the reference image, including the root mean square error, signal to noise ratio (SNR), and the peak SNR; and the difference with source image including the cross entropy, joint entropy, mutual information, deviation index, correlation coefficient, and the degree of distortion. These parameters were used to evaluate the results of the comprehensive evaluation value for the synthesized image. The image reflected the fusion of plantar pressure distribution using the proposed method compared with other fusion methods, such as up-down, mean-mean, and max-min fusion. The experimental results showed that the proposed LoG filtering with PEO-GMM fusion operator outperformed other methods.

In this paper, a type of entropy of action of a semi-group on a measure space is introduced. Then we apply this concept to specific semi-groups to extract metric entropy of a dynamical system as a special case.

The purpose of this paper is to explore the concept of mixing entropy for systems of coupled harmonic oscillators. First, entropy is examined theoretically for a generalized oscillator configuration. A general formulation for the mixing entropy is obtained by comparing the actual entropy of a system to its hypothetical decoupled entropy. It is proven that the mixing entropy of any system is nonnegative, and that the entropy inequality property reduces to an equality with the introduction of a mixing entropy term. This result is validated using a series of numerical examples, which provide insight into the relationship between energy and entropy behavior. First, the mixing entropy is examined for an undamped system of two identical coupled oscillators. The same methodology is then applied for a system of nonidentical oscillators. Lastly, the mixing entropy is determined for a system composed of a main oscillator attached to a fuzzy structure.

In this paper, a new coefficient is proposed with the objective of quantifying the level of information flow between financial time series. This transfer entropy coefficient, which provides an assessment on the multiscale information flow between measurements, is defined in terms of the transfer entropy method and the multiscale method. The implementation of this transfer entropy coefficient is illustrated with simulated time series and financial time series. Examples taken from simulated and financial data demonstrate that the dynamic mechanism of a complex system cannot be detected solely on the basis of transfer entropy of single scale.

Shannon entropy is being increasingly used in biomedical research as an index of complexity and information content in sequences of symbols, e.g. languages, amino acid sequences, DNA methylation patterns and animal vocalizations. Yet, distributional properties of information entropy as a random variable have seldom been the object of study, leading to researchers mainly using linear models or simulation-based analytical approach to assess differences in information content, when entropy is measured repeatedly in different experimental conditions. Here a method to perform inference on entropy in such conditions is proposed. Building on results coming from studies in the field of Bayesian entropy estimation, a symmetric Dirichlet-multinomial regression model, able to deal efficiently with the issue of mean entropy estimation, is formulated. Through a simulation study the model is shown to outperform linear modeling in a vast range of scenarios and to have promising statistical properties. As a practical example, the method is applied to a data set coming from a real experiment on animal communication.

The theory of Evidence, or Shafer-Dempster theory (DST), has been widely used in applications. The DST is based on the concept of a basic probability assignment. An important part of this theory is the quantification of the information-based-uncertainty that this function represents. A recent measure of uncertainty (or information) in this theory, called the Deng entropy, has appeared as an interesting alternative to the measures presented so far. This measure quantifies the both types of uncertainty found in DST, then it is considered as a total uncertainty measure (TU). It is shown that this measure does not verify some of the essential properties for a TU in DST such as monotonicity, additivity and subadditivity. Also, the definition of this new measure produces other debatable situations. These shortcomings call in question the utility of this measure in applications.

In this paper, we consider a fuzzy c-means (FCM) clustering algorithm combined with the deterministic annealing method and the Tsallis entropy maximization. The Tsallis entropy is a q-parameter extension of the Shannon entropy. By maximizing the Tsallis entropy within the framework of FCM, membership functions similar to statistical mechanical distribution functions can be derived. One of the major considerations when using this method is how to determine appropriate q values and the highest annealing temperature, Thigh , for a given data set. Accordingly, in this paper, a method for determining these values simultaneously without introducing any additional parameters is presented. In our approach, the membership function is approximated by a series of expansion methods and the K-means clustering algorithm is utilized as a preprocessing step to estimate a radius of each data distribution. The results of experiments indicate that the proposed method is effective and both q and Thigh can be determined automatically and algebraically from a given data set.

Maximum entropy procedures for estimating coarse-grain parameters from molecular dynamics (MD) simulation data are considered within the specific context of the sequence-dependent cgDNA rigid-base model of DNA. We describe a quite general approach that exploits a maximum absolute entropy principle to fit an observed matrix of covariances subject to the constraint of only allowing a prescribed sparsity pattern of nearest-neighbor interactions in the free energy. We also allow indefinite local stiffness-matrix parameter blocks that nevertheless always generate a positive-definite model stiffness matrix. Beginning from a database of atomic-resolution MD simulations of a library of short DNA oligomers in explicit solvent, these procedures deliver a complete parameter set for the cgDNA model. Due to the intrinsic linear structure of DNA and the convergence characteristics of the MD time series data, the maximum absolute entropy parameter set yields significantly improved predictions of persistence lengths, whe...

Tsallis entropy is a generalized form of entropy and tends to be Shannon entropy when q → 1. Using Tsallis entropy, an alternative estimation methodology (generalized maximum Tsallis entropy) is introduced and used to estimate the parameters in a linear regression model when the basic data are ill-conditioned. We describe the generalized maximum Tsallis entropy and for q = 2 we call that GMET2 estimator. We apply the GMET2 estimator for estimating the linear regression model Y = Xβ + e where the design matrix X is subject to severe multicollinearity. We compared the GMET2, generalized maximum entropy (GME), ordinary least-square (OLS), and inequality restricted least-square (IRLS) estimators on the analyzed dataset on Portland cement.

Kolmogorov–Sinai entropy-based irregularity measures such as approximate entropy (ApEn), sample entropy and fuzzy entropy are widely used for short-term heart rate variability analysis. These entropy statistics are estimated for a specific value of the tolerance parameter (r) that is mostly chosen from a common recommended range. Entropy measurement on short-term signals is highly sensitive to the choice of r. An incorrect selection of r results in an inaccurate entropy value, thereby leading to unreliable information retrieval. By addressing this inaccuracy due to r selection, the quality and reliability of information retrieval can be improved. Thus, we hypothesize that generating a complete entropy profile using all potential r values will give a more complete and useful information about signal irregularity in contrast to the case of finding entropy at a single selected value of r. In order to do so, one must be able to accurately select all potential r candidates. In this paper, we use a data-driven algorithm based on cumulative histograms to automatically select potential r values for an individual signal based on its dynamics. An appropriate set of r values is designated by the algorithm for generating a series of ApEn values (ApEn profile) instead of a single value of ApEn. ApEn profile- based secondary measures such as TotalApEn and SDApEn have been used as features to classify sets of synthetic and physiologic data. Our study proves that secondary measures obtained from an ApEn profile are more efficient in indicating irregularity levels in comparison with the traditional measure of ApEn evaluated at a single r value, specially in the case of short length data.

Lattices provide useful structure for distributed coding of correlated sources. A common lattice encoder construction is to first round an observed sequence to a `fine&rsquo, lattice with dither, then produce the result&rsquo, s modulo to a `coarse&rsquo, lattice as the encoding. However, such encodings may be jointly-dependent. A class of upper bounds is established on the conditional entropy-rates of such encodings when sources are correlated and Gaussian and the lattices involved are a from an asymptotically-well-behaved sequence. These upper bounds guarantee existence of a joint&ndash, compression stage which can increase encoder efficiency. The bounds exploit the property that the amount of possible values for one encoding collapses when conditioned on other sufficiently informative encodings. The bounds are applied to the scenario of communicating through a many-help-one network in the presence of strong correlated Gaussian interferers, and such a joint&ndash, compression stage is seen to compensate for some of the inefficiency in certain simple encoder designs.

The matrix-based Renyi's $\alpha$ α -order entropy functional was recently introduced using the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS). However, the current theory in the matrix-based Renyi's $\alpha$ α -order entropy functional only defines the entropy of a single variable or mutual information between two random variables. In information theory and machine learning communities, one is also frequently interested in multivariate information quantities, such as the multivariate joint entropy and different interactive quantities among multiple variables. In this paper, we first define the matrix-based Renyi's $\alpha$ α -order joint entropy among multiple variables. We then show how this definition can ease the estimation of various information quantities that measure the interactions among multiple variables, such as interactive information and total correlation. We finally present an application to feature selection to show how our definition provides a simple yet powerful way to estimate a widely-acknowledged intractable quantity from data. A real example on hyperspectral image (HSI) band selection is also provided.

In this paper, we investigate the role of convexity in entropy power inequalities. We establish Renyi entropy power inequalities of order r ∈ (0, 1) for a large class of densities, the so-called s-concave densities. This extends recent works on Renyi entropy power inequalities.

We resume the investigation of the problem of independent local compression of correlated quantum sources, the classical case of which is covered by the celebrated Slepian-Wolf theorem. We focus specifically on classical-quantum (cq) sources, for which one edge of the rate region, corresponding to the compression of the classical part, using the quantum part as side information at the decoder, was previously determined by Devetak and Winter [Phys. Rev. A 68, 042301 (2003)]. Whereas the Devetak-Winter protocol attains a rate-sum equal to the von Neumann entropy of the joint source, here we show that the full rate region is much more complex, due to the partially quantum nature of the source. In particular, in the opposite case of compressing the quantum part of the source, using the classical part as side information at the decoder, typically the rate sum is strictly larger than the von Neumann entropy of the total source.We determine the full rate region in the generic case, showing that, apart from the Devetak-Winter point, all other points in the achievable region have a rate sum strictly larger than the joint entropy. We can interpret the difference as the price paid for the quantum encoder being ignorant of the classical side information. In the general case, we give an achievable rate region, via protocols that are built on the decoupling principle, and the principles of quantum state merging and quantum state redistribution. Our achievable region is matched almost by a single-letter converse, which however still involves asymptotic errors and an unbounded auxiliary system.