Search

In this research paper, it is proved that an approximation to the Gibbs-Shannon entropy measure naturally leads to the Tsallis entropy for the real parameter q = 2. Several interesting measures based on the input as well as the output of a discrete memoryless channel are provided and some of the properties of those measures are discussed. It is expected that these results will be of utility in Information Theoretic research.

This paper considers a fuzzyc-means (FCM) clustering algorithm in combination with deterministic annealing and the Tsallis entropy maximization. The Tsallis entropy is aq-parameter extension of the Shannon entropy. By maximizing the Tsallis entropy within the framework of FCM, statistical mechanical membership functions can be derived. One of the major considerations when using this method is how to determine appropriate values forqand the highest annealing temperature,Thigh, for a given data set. Accordingly, in this paper, a method for determining these values simultaneously without introducing any additional parameters is presented, where the membership function is approximated using a series expansion method. The results of experiments indicate that the proposed method is effective, and bothqandThighcan be determined automatically and algebraically from a given data set.

In this article, we propose high order, semi-discrete, entropy stable, finite difference schemes for Ten-Moment Gaussian Closure equations. The crucial components of these schemes are an entropy conservative flux and suitable high order entropy dissipative operators to ensure entropy stability. We design two numerical fluxes, one is approximately entropy conservative, and another is entropy conservative flux. For the construction of appropriate entropy dissipative operators, we also derive entropy scaled right eigenvectors. This is used for sign preserving reconstruction of scaled entropy variables, which results in second and third order entropy stable schemes. We also extend these schemes to a plasma flow model with source term. Several numerical results are presented for homogeneous and non-homogeneous cases to demonstrate stability and performance of these schemes.

There is a certain relationship between the global financial markets, which creates an interactive network of global finance. Transfer entropy, a measurement for information transfer, offered a good way to analyse the relationship. In this paper, we analysed the relationship between 9 stock indices from the U.S., Europe and China (from 1995 to 2015) by using transfer entropy (TE), effective transfer entropy (ETE), Renyi transfer entropy (RTE) and effective Renyi transfer entropy (ERTE). We compared the four methods in the sense of the effectiveness for identification of the relationship between stock markets. In this paper, two kinds of information flows are given. One reveals that the U.S. took the leading position when in terms of lagged-current cases, but when it comes to the same date, China is the most influential. And ERTE could provide superior results.

Partial entropy is a device to measure how much of entropy of an uncertain random variable belongs to uncertain variables. However, partial entropy may fail to measure the uncertainty of an uncertain random variable. In this paper, for refining this problem, a definition of partial triangular entropy is presented. In addition, several properties of partial triangular entropy are obtained. Moreover, partial triangular entropy of an uncertain random variable is applied to mean-variance portfolio selection.

Entropy-based measures have been used to characterize the structure of complex networks. Various graph entropies based on different graph parameters have been proposed and studied. So, it has been intricate to decide which measure is the right one for solving a particular problem. In this paper, we introduce graph entropy measures based on independent sets and matchings of graphs. The values of entropies of some special graphs are calculated and we draw several conclusions regrading the usability of the measures. Based on our findings, we conclude that it should be demanding studying extremal values of the novel entropies.

In this paper, we introduce a quantity measure which is called (R, S)-norm entropy and discuss some of its major properties with Shannon’s and other entropies in the literature. Based on this (R, S)-norm entropy, we have proposed a new (R, S)-norm fuzzy information measure and discussed its validity and properties. Further, we have given its comparison with other fuzzy information measures to prove its effectiveness. Attribute weights play an important role in multiple-attribute decision-making problems. In the present communication, two methods of determining the attribute weights are introduced. First is the case when the information regarding attribute weights is incompletely known or completely unknown and second is when we have partial information about attribute weights. For the first case, the extension of ordinary entropy weight method is used to calculate attribute weights and minimum entropy principle method based on solving a linear programming model is used in the second case. Finally, two methods are explained through numerical examples.

Characterising the capacity region for a network can be extremely difficult. Even with independent sources, determining the capacity region can be as hard as the open problem of characterising all information inequalities. The majority of computable outer bounds in the literature are relaxations of the Linear Programming bound which involves entropy functions of random variables related to the sources and link messages. When sources are not independent, the problem is even more complicated. Extension of Linear Programming bounds to networks with correlated sources is largely open. Source dependence is usually specified via a joint probability distribution, and one of the main challenges in extending linear program bounds is the difficulty (or impossibility) of characterising arbitrary dependencies via entropy functions. This paper tackles the problem by answering the question of how well entropy functions can characterise correlation among sources. We show that by using carefully chosen auxiliary random variables, the characterisation can be fairly "accurate" Using such auxiliary random variables we also give implicit and explicit outer bounds on the capacity of networks with correlated sources. The characterisation of correlation or joint distribution via Shannon entropy functions is also applicable to other information measures such as Renyi entropy and Tsallis entropy., Comment: 24 pager, 1 figure, submitted to IEEE Transactions on Information Theory. arXiv admin note: text overlap with arXiv:1309.1517

Uncertain variables, applied to modelling imprecise quantities, are fundamental concepts in uncertainty theory. In order to characterize the information deficiency of an uncertain variable, this paper proposes a definition of quadratic entropy. Compared with the traditional entropy, it is much easier to compute and has a wider range. Furthermore, the principle of maximum entropy is applied to quadratic entropy, and two theorems of maximum quadratic entropy with moment constraint are proved.

Recently, Di Crescenzo and Longobardi (2006) have studied “length-biased” shift-dependent information measure and its dynamic versions. On the other hand, Renyi's entropy plays a vital role in the ...

In this paper, a type of entropy of action of a semi-group on a measure space is introduced. Then we apply this concept to specific semi-groups to extract metric entropy of a dynamical system as a special case.

The purpose of this paper is to explore the concept of mixing entropy for systems of coupled harmonic oscillators. First, entropy is examined theoretically for a generalized oscillator configuration. A general formulation for the mixing entropy is obtained by comparing the actual entropy of a system to its hypothetical decoupled entropy. It is proven that the mixing entropy of any system is nonnegative, and that the entropy inequality property reduces to an equality with the introduction of a mixing entropy term. This result is validated using a series of numerical examples, which provide insight into the relationship between energy and entropy behavior. First, the mixing entropy is examined for an undamped system of two identical coupled oscillators. The same methodology is then applied for a system of nonidentical oscillators. Lastly, the mixing entropy is determined for a system composed of a main oscillator attached to a fuzzy structure.

In this paper, recently introduced permutation entropy and sample entropy are further developed to the fractional cases, weighted fractional permutation entropy (WFPE) and fractional sample entropy (FSE). The fractional order generalization of information entropy is utilized in the above two complexity approaches, to detect the statistical characteristics of fractional order information in complex systems. The effectiveness analysis of proposed methods on the synthetic data and the real-world data reveals that tuning the fractional order allows a high sensitivity and more accurate characterization to the signal evolution, which is useful in describing the dynamics of complex systems. Moreover, the numerical research on nonlinear complexity behaviors is compared between the returns series of Potts financial model and the actual stock markets. And the empirical results confirm the feasibility of the proposed model.

Maximum entropy procedures for estimating coarse-grain parameters from molecular dynamics (MD) simulation data are considered within the specific context of the sequence-dependent cgDNA rigid-base model of DNA. We describe a quite general approach that exploits a maximum absolute entropy principle to fit an observed matrix of covariances subject to the constraint of only allowing a prescribed sparsity pattern of nearest-neighbor interactions in the free energy. We also allow indefinite local stiffness-matrix parameter blocks that nevertheless always generate a positive-definite model stiffness matrix. Beginning from a database of atomic-resolution MD simulations of a library of short DNA oligomers in explicit solvent, these procedures deliver a complete parameter set for the cgDNA model. Due to the intrinsic linear structure of DNA and the convergence characteristics of the MD time series data, the maximum absolute entropy parameter set yields significantly improved predictions of persistence lengths, whe...

In this paper, we investigate the role of convexity in entropy power inequalities. We establish Renyi entropy power inequalities of order r ∈ (0, 1) for a large class of densities, the so-called s-concave densities. This extends recent works on Renyi entropy power inequalities.

We review the notion of weighted quantum entropy and consider the weighted quantum entropy for bipartite and noncomposite quantum systems. We extend the subadditivity condition, the inequality known for the weighted entropy information, to the case of indivisible qudit system, such as a qutrit. We discuss the new inequality for the qutrit density matrix for different weights and states, as well as the role of weighted entropy with respect to nonlinear quantum channels.

As a fundamental concept in describing complex system, entropy measure has been proposed to various forms, like Boltzmann–Gibbs (BG) entropy, one-index entropy, two-index entropy, sample entropy, permutation entropy etc. This paper proposes a new two-index entropy S q,δ and we find the new two-index entropy is applicable to measure the complexity of wide range of systems in the terms of randomness and fluctuation range. For more complex system, the value of two-index entropy is smaller and the correlation between parameter δ and entropy S q,δ is weaker. By combining the refined two-index entropy S q,δ with scaling exponent h ( δ ), this paper analyzes the complexities of simulation series and classifies several financial markets in various regions of the world effectively.

We examine the entropy of low-significance bits in analog-to-digital measurements of chaotic dynamical systems. We find the partition of measurement space corresponding to low-significance bits has a corrugated structure. Using simulated measurements of a map and experimental data from a circuit, we identify two consequences of this corrugated partition. First, entropy rates for sequences of low-significance bits more closely approach the metric entropy of the chaotic system, because the corrugated partition better approximates a generating partition. Second, accurate estimation of the entropy rate using low-significance bits requires long block lengths as the corrugated partition introduces more long-term correlation, and using only short block lengths overestimates the entropy rate. This second phenomenon may explain recent reports of experimental systems producing binary sequences that pass statistical tests of randomness at rates that may be significantly beyond the metric entropy rate of the physical source.

This study presents the Shannon and Renyi information entropy for both position and momentum space and the Fisher information for the position-dependent mass Schr¨odinger equation with the Frost-Musulin potential. The analysis of the quantum mechanical probability has been obtained via the Fisher information. The variance information of this potential is equally computed. This controls both the chemical properties and physical properties of some of the molecular systems. We have observed the behaviour of the Shannon entropy. Renyi entropy, Fisher information and variance with the quantum number n respectively.

In this expository article, we study optimization problems specified via linear and relative entropy inequalities. Such relative entropy programs (REPs) are convex optimization problems as the relative entropy function is jointly convex with respect to both its arguments. Prominent families of convex programs such as geometric programs (GPs), second-order cone programs, and entropy maximization problems are special cases of REPs, although REPs are more general than these classes of problems. We provide solutions based on REPs to a range of problems such as permanent maximization, robust optimization formulations of GPs, and hitting-time estimation in dynamical systems. We survey previous approaches to some of these problems and the limitations of those methods, and we highlight the more powerful generalizations afforded by REPs. We conclude with a discussion of quantum analogs of the relative entropy function, including a review of the similarities and distinctions with respect to the classical case. We also describe a stylized application of quantum relative entropy optimization that exploits the joint convexity of the quantum relative entropy function.

In this paper we establish a formula of conditional entropy and give two examples of applications of the formula.

We propose a new criterion for randomness of a word $x_{1}x_{2}\cdots x_{n}\in \mathbb{A}^{n}$ over a finite alphabet $\mathbb{A}$ defined by $$\begin{eqnarray}\unicode[STIX]{x1D6EF}^{n}(x_{1}x_{2}\cdots x_{n})=\mathop{\sum }_{\unicode[STIX]{x1D709}\in \mathbb{A}^{+}}\unicode[STIX]{x1D713}(|x_{1}x_{2}\cdots x_{n}|_{\unicode[STIX]{x1D709}}),\end{eqnarray}$$ where $\mathbb{A}^{+}=\bigcup _{k=1}^{\infty }\mathbb{A}^{k}$ is the set of non-empty finite words over $\mathbb{A}$, for $\unicode[STIX]{x1D709}\in \mathbb{A}^{k}$, $$\begin{eqnarray}|x_{1}x_{2}\cdots x_{n}|_{\unicode[STIX]{x1D709}}=\#\{i;~1\leq i\leq n-k+1,~x_{i}x_{i+1}\cdots x_{i+k-1}=\unicode[STIX]{x1D709}\},\end{eqnarray}$$ and for $t\geq 0$, $\unicode[STIX]{x1D713}(0)=0$ and $\unicode[STIX]{x1D713}(t)=t\log t~(t>0)$. This value represents how random the word $x_{1}x_{2}\cdots x_{n}$ is from the viewpoint of the block frequency. In fact, we define a randomness criterion as $$\begin{eqnarray}Q(x_{1}x_{2}\cdots x_{n})=(1/2)(n\log n)^{2}/\unicode[STIX]{x1D6EF}^{n}(x_{1}x_{2}\cdots x_{n}).\end{eqnarray}$$ Then, $$\begin{eqnarray}\lim _{n\rightarrow \infty }(1/n)Q(X_{1}X_{2}\cdots X_{n})=h(X)\end{eqnarray}$$ holds with probability 1 if $X_{1}X_{2}\cdots \,$ is an ergodic, stationary process over $\mathbb{A}$ either with a finite energy or $h(X)=0$, where $h(X)$ is the entropy of the process. Another criterion for randomness using $t^{2}$ instead of $t\log t$ has already been proposed in Kamae and Xue [An easy criterion for randomness. Sankhya A77(1) (2015), 126–152]. In comparison, our new criterion provides a better fit with the entropy. We also claim that our criterion not only represents the entropy asymptotically but also gives a good representation of the randomness of fixed finite words.

In the literature of information theory, the concept of generalized entropy has been proposed and the length-based shift dependent information measure has been studied. In this paper, the concept o...

This paper proposes the continuous controller design method for quantum Shannon entropy, which can continuously drive the entropy to track a desired trajectory. We also analyzed the controllability of Shannon entropy in very short time interval. Simulations are done on five dimensional quantum system, which can verify the validation of the method.

In this paper, at first, the amount of irregularity or the uncertainty of Brownian motion process is obtained. The results shows that this uncertainty will be high by the time. In the second step, the entropy of $(B(t_1),B(t_2),...,B(t_n))$ as a Gaussian process of Brownian motion is introduced. In special case, it is shown that the joint entropy of two Brownian motion increases with increasing s and t. In the third step, the conditional entropy of Brownian motion is computed and via it the entropy of Brownian Bridge is obtained. The mutual information between two processes is introduced as a measures of how much knowing one of these processes reduces uncertainty about the other. Finally, the Entropy of Wiener integral is introduced.

We first investigate on the asymptotics of the Kolmogorov metric entropy and nonlinear n -widths of approximation spaces on some function classes on manifolds and quasi-metric measure spaces. Secondly, we develop constructive algorithms to represent those functions within a prescribed accuracy. The constructions can be based on either spectral information or scattered samples of the target function. Our algorithmic scheme is asymptotically optimal in the sense of nonlinear n -widths and asymptotically optimal up to a logarithmic factor with respect to the metric entropy.

In the present study, several characterizations of order statistics are obtained on the basis of the generalized entropy. Under some conditions, it is shown that the parent distribution can be uniquely determined by equality of generalized entropy of order statistics.

In this paper, we advance the entropy theory of discrete non-autonomous dynamical systems that was initiated by Kolyada and Snoha in 1996. The first part of the paper is devoted to the measure-theoretic entropy theory of general topological systems. We derive several conditions guaranteeing that an initial probability measure, when pushed forward by the system, produces an invariant measure sequence whose entropy captures the dynamics on arbitrarily fine scales. In the second part of the paper, we apply the general theory to the non-stationary subshifts of finite type, introduced by Fisher and Arnoux. In particular, we give sufficient conditions for the variational principle, relating the topological and measure-theoretic entropy, to hold.

The final article in a series of works on creation of ion‐molecular models of memory is dedicated to the entropy factor. The authors examine the correlation between information and entropy. They introduce an entropy equivalent in the specificity of memory (by I. Prigozhin). The memory system is considered markedly nonlinear and non‐equilibrium. The article contains a general conclusion to the series of works. The entropy factor is, from the position of bio‐physic‐chemistry, an important aspect of memory, as a higher form of functioning of the organism, including human organism ‐ in the first place. It is possible to allocate to factor the actual memory several types of entropy, instead of one integrated, as for example in the case of the classical thermodynamic processes. Moreover, the thermodynamic entropy, as a defining static processes, virtually ignored in ion‐molecular memory model (Boltzmann entropy). Here more significant is the consideration of the role of information Shannonʹs entropy. And another important point: to evaluate the entropy of systems with non‐equilibrium processes, which undoubtedly applies to the memory, I. Prigozhin proposed to use the sum of the elements of the correlation matrix, the diagonal members which are set in accordance with probability, and extra diagonal ‐ correlations. The authors use it in the estimation of entropic characteristics of ion‐molecular memory model. I.e. on the basis of proposals Prigozhin, the authors introduce entropy equivalent, which corresponds to the provisions of non‐equilibrium thermodynamics and the requirement for orientation changes in two of its components, that allows to estimate the partial contributions of both types of entropy to the total entropy of memory.

Biological networks are open systems that can utilize nutrients and energy from their environment for use in their metabolic processes, and produce metabolic products. System entropy is defined as the difference between input and output signal entropy, i.e., the net signal entropy of the biological system. System entropy is an important indicator for living or non-living biological systems, as biological systems can maintain or decrease their system entropy. In this study, system entropy is determined for the first time for stochastic biological networks, and a computation method is proposed to measure the system entropy of nonlinear stochastic biological networks that are subject to intrinsic random fluctuations and environmental disturbances. We find that intrinsic random fluctuations could increase the system entropy, and that the system entropy is inversely proportional to the robustness and stability of the biological networks. It is also determined that adding feedback loops to shift all eigenvalues to the farther left-hand plane of the complex s-domain could decrease the system entropy of a biological network.

In this paper, we provide the R\'enyi entropy and complexity measure for a novel, flexible class of skew-gaussian distributions and their related families, as a characteristic form of the skew-gaussian Shannon entropy. We give closed expressions considering a more general class of closed skew-gaussian distributions and the weighted moments estimation method. In addition, closed expressions of R\'enyi entropy are presented for extended skew-gaussian and truncated skew-gaussian distributions. Finally, additional inequalities for skew-gaussian and extended skew-gaussian R\'enyi and Shannon entropies are reported., Comment: 13 pages

We present novel complementary relations in non-equilibrium stochastic processes. Specifically, by utilising path integral formulation, we derive statistical measures (entropy, information, and work) and investigate their dependence on variables (x, v), reference frames, and time. In particular, we show that the equilibrium state maximises the simultaneous information quantified by the product of the Fisher information based on x and v while minimising the simultaneous disorder/uncertainty quantified by the sum of the entropy based on x and v as well as by the product of the variances of the PDFs of x and v. We also elucidate the difference between Eulerian and Lagrangian entropy. Our theory naturally leads to Hamilton–Jacobi relation for forced-dissipative systems.

In the literature of information theory, there exist many well known measures of entropy suitable for entropy optimization principles towards applications in different disciplines of science and technology. The object of this article is to develop a new generalized measure of entropy and to establish the relation between entropy and queueing theory. To fulfill our aim, we have made use of maximum entropy principle which provides the most uncertain probability distribution subject to some constraints expressed by mean values.

After generalization of Shannon's entropy measure by Renyi in 1961, many generalized versions of Shannon measure were proposed by different authors. Shannon measure can be obtained from these generalized measures asymptotically. A natural question arises in the parametric generalization of Shannon's entropy measure. What is the role of the parameter(s) from application point of view? In the present communication, super additivity and fast scalability of generalized hyperbolic measure (Bhatia and Singh, 2013) of probabilistic entropy as compared to some classical measures of entropy has been shown. Application of a generalized hyperbolic measure of probabilistic entropy in certain situations has been discussed. Also, application of generalized hyperbolic measure of fuzzy entropy in multi attribute decision making have been presented where the parameter affects the preference order. i i i i i p p E Inf p P H       

The resultant measures of the entropy/information content in complex elec- tronic states are discussed in the canonical position and momentum representations of molecular quantum mechanics. The nonclassical (phase/current) supplements of the classical (probability) descriptors of the overall entropy/information content in electronic states are identified and the associated entropy deficiency (information dis- tance) quantities are introduced. The Shannon (global, logarithmic) and Fisher (local, gradient) information descriptors in both spaces are summarized, and the momentum continuity equation is used to establish the associated probability source. General relations between global and local information densities are examined and the etropic principles determining molecular phase equilibria are investigated.

Recently, Zografos and Nadarajah (2005) proposed two measures of uncertainty based on the survival function, called the survival exponential entropy and the generalized survival exponential entropy. In this article, we explore properties of the generalized survival entropy and the dynamic version of it. We study conditions under which the generalized survival entropy of first order statistic can uniquely determines the parent distribution. The exponential, Pareto, and finite range distributions, which are commonly used in reliability, have been characterized using this generalized measure. Another measure of entropy is also introduced in analogy with cumulative entropy which has been proposed by Di Crescenzo and Longobardi (2009) and some properties of it are given.

We consider the relative operator entropy constructed by the spectral geometric mean and see its properties analogous to those of the Tsallis relative operator entropy by the usual geometric mean. Furthermore, we define the quantum relative entropy constructed by the spectral geometric mean and derive its subadditivity under tensor product.

The classical entropy power inequality is extended to the Renyi entropy. We also discuss the question of the existence of the entropy for sums of independent random variables.

We consider a non-additive generalization of the Shannon entropy measure using order statistics and show that this entropy measure characterizes the distribution function uniquely. Further we propose a residual non-additive entropy measure for order statistics and prove a characterization result for that also.

Using a statistical approach, it is demonstrated that the complexity of a crystal structure measured as the Shannon information per atom [Krivovichev (2012).Acta Cryst.A68, 393–398] represents a negative contribution to the configurational entropy of a crystalline solid. This conclusion is in full accordance with the general agreement that information and entropy are reciprocal variables. It also agrees well with the understanding that complex structures possess lower entropies relative to their simpler counterparts. The obtained equation is consistent with the Landauer principle and points out that the information encoded in a crystal structure has a physical nature.

Permutation entropy is a relatively new promising concept for measuring complexity of time series and of the systems behind with applications in various fields. The idea of permutation entropy it is to quantify the amount of up and down in a time series on the base of considering the distribution of ordinal patterns in it. Although metric features are neglected to a great extend, this approch often preserves more information than expected on a first glance. The aim of the talk is to explain why permutation entropy is interesting for complex data analysis and to discuss perspectives, challenges and limits of its application. Our discussion emphasizes the viewpoint of symbolic dynamics.

The aim of the article is application of entropy defined in combinatorics on words for finite words; entropy function will be applied for solving classification and clusterisation problems.

International audience; This paper deals with the existence of global weak solutions for a wide class of (multiple species) cross-diffusions systems. The existence is based on two different ingredients: an entropy estimate giving some gradient control and a duality estimate that gives naturally L^2 control. The proof relies on a semi-implicit scheme tailored for cross-diffusion systems firstly defined by the two authors and collaborators. These results are applied to models having an entropy relying on the detailed balance condition recently exhibited by Chen et. al.

This paper explores some applications of a two-moment inequality for the integral of the r-th power of a function, where 0 < r < 1. The first contribution is an upper bound on the Renyi entropy of a random vector in terms of the two different moments. When one of the moments is the zeroth moment, these bounds recover previous results based on maximum entropy distributions under a single moment constraint. More generally, evaluation of the bound with two carefully chosen nonzero moments can lead to significant improvements with a modest increase in complexity. The second contribution is a method for upper bounding mutual information in terms of certain integrals with respect to the variance of the conditional density. The bounds have a number of useful properties arising from the connection with variance decompositions.

Information entropy and its extension, which are important generalizations of entropy, are currently applied to many research domains. In this paper, a novel generalized relative entropy is constructed to avoid some defects of traditional relative entropy. We present the structure of generalized relative entropy after the discussion of defects in relative entropy. Moreover, some properties of the provided generalized relative entropy are presented and proved. The provided generalized relative entropy is proved to have a finite range and is a finite distance metric. Finally, we predict nucleosome positioning of fly and yeast based on generalized relative entropy and relative entropy respectively. The experimental results show that the properties of generalized relative entropy are better than relative entropy.