Search

The main objective of this contribution is to develop information about how entropy measures of linguistic terms can be designed. Two different ideas have been put forward to explain this designation: (1) The idea that comes from the seminal definition of fuzziness measure; (2) The idea of transforming similarity measures to entropy ones. To demonstrate the utility and effectiveness of the proposed entropy measures, an entropy-based approach of determining objective weights of attributes is developed to solve multiple-attribute decision-making problems in the context of linguistic term sets.

In this paper we propose a new image randomness measure using Shannon entropy over local image blocks. The proposed local Shannon entropy measure overcomes several weaknesses of the conventional global Shannon entropy measure, including unfair randomness comparisons between images of different sizes, failure to discern image randomness before and after image shuffling, and possible inaccurate scores for synthesized images. Statistical tests pertinent to this new measure are also derived. This new measure is therefore both quantitative and qualitative. The parameters in the local Shannon entropy measure are further optimized for a better capture of local image randomness. The estimated statistics and observed distribution from 50,000 experiments match the theoretical ones. Finally, two examples are given, applying the proposed measure to image randomness among shuffled images and encrypted images. Both examples show that the proposed method is more effective and more accurate than the global Shannon entropy measure.

In this paper, we introduce an axiomatic definition of the similarity measure of intuitionistic fuzzy sets (IFS) that differs from the definition of Li [15]. The relationship between the similarity measure and the entropy of IFS is investigated in detail. Six theorems on how the similarity measure could be transformed into the entropy for IFS and vice versa are proposed based on their axiomatic definitions. Some formulas have been proposed to calculate the similarity measure and the entropy of IFS. Finally, sufficient conditions to transform the similarity measures to the entropy for IFS and vice versa are given.

This paper introduces the notion of an entropy measurement for populations of candidate solutions in evolutionary algorithms, developing both conditional and joint entropy-based algorithms. We describe the inherent characteristics of the entropy measurement and how these affect the search process. Following these discussions, we develop a recognition mechanism through which promising candidate solutions can be identified without the need of invoking costly evaluation functions. This on-demand evaluation strategy (ODES) is able to perform decision making tasks regardless of whether the actual fitness evaluation is necessary or not, making it an ideal efficiency enhancement technique for accelerating the computational process of evolutionary algorithms. Two different evolutionary algorithms, a traditional genetic algorithm and a multivariate estimation of distribution algorithm, are employed as example targets for the application of our on-demand evaluation strategy. Ultimately, experimental results confirm that our method is able to broadly improve the performance of various population-based global searchers.

Recently, the minimum error entropy criterion, an information theoretic alternative to the traditional mean square error criterion, has been successfully used in the contexts of machine learning and signal processing. For system identification, however, the MEE criterion will be no longer suitable if the training data are discrete-valued, since minimizing error's discrete entropy cannot constrain error's dispersion. In this paper, to make the MEE criterion suitable for the discrete-valued data cases, we give a new entropy definition for the discrete random variables, i.e. the @D-entropy, based on Riemann sums for finite size partitions. A probability weighted formula is established to calculate the average partition.This new entropy retains some important properties of the differential entropy and reduces to discrete entropy under certain conditions. Unlike discrete entropy, the @D-entropy is sensitive to the dynamic range of the data, and can be used as a superior optimality criterion in system identification problems. Also, we present a plug-in estimate of @D-entropy, analyze its asymptotic behavior and explore the links to the kernel based and m-spacing based estimates for differential entropy. Finally, the @D-entropy criterion is applied in system identification with coarsely quantized input-output data to search for the optimum parameter set. Monte Carlo simulations demonstrate the performance improvement that may be achieved with the @D-entropy criterion.

This survey seeks to describe methods for measuring the entropy of graphs and to demonstrate the wide applicability of entropy measures. Setting the scene with a review of classical measures for determining the structural information content of graphs, we discuss graph entropy measures which play an important role in a variety of problem areas, including biology, chemistry, and sociology. In addition, we examine relationships between selected entropy measures, illustrating differences quantitatively with concrete examples.

A new information entropy measure of interval-valued intuitionistic fuzzy set (IvIFS) is proposed by using membership interval and non-membership interval of IvIFS, which complies with the extended form of Deluca-Termini axioms for fuzzy entropy. Then the cross-entropy of IvIFSs is presented and the relationship between the proposed entropy measures and the existing information measures of IvIFSs is discussed. Additionally, some numerical examples are given to illustrate the applications of the proposed entropy and cross-entropy of IvIFSs to pattern recognition and decision-making.

This paper explores properties of the residual Renyi entropy of some ordered random variables. The residual Renyi entropy of the kth order statistic from a continuous distribution function is represented in terms of the residual Renyi entropy of the kth order statistic from uniform distribution. The monotone behavior of the residual Renyi entropy of order statistic under various conditions is discussed. Analogues results for the residual Renyi entropy of record values are also given.

In this paper, we define the conditional Renyi entropy and show that the so-called chain rule holds for the Renyi entropy. Then, we introduce a relation for the rate of Renyi entropy and use it to derive the rate of the Renyi entropy for an irreducible-aperiodic Markov chain. We also show that the bound for the Renyi entropy rate is simply the Shannon entropy rate.

Due to the deficiency of information, the membership function of a fuzzy variable cannot be obtained explicitly. It is a challenging work to find an appropriate membership function when certain partial information about a fuzzy variable is given, such as expected value or moments. This paper solves such problems for discrete fuzzy variables via maximum entropy principle and proves some maximum entropy theorems with certain constraints. A genetic algorithm is designed to solve the general maximum entropy model for discrete fuzzy variables, which is illustrated by some numerical experiments.

In this paper, we propose some new approaches for attribute reduction in covering decision systems from the viewpoint of information theory. Firstly, we introduce information entropy and conditional entropy of the covering and define attribute reduction by means of conditional entropy in consistent covering decision systems. Secondly, in inconsistent covering decision systems, the limitary conditional entropy of the covering is proposed and attribute reductions are defined. And finally, by the significance of the covering, some algorithms are designed to compute all the reducts of consistent and inconsistent covering decision systems. We prove that their computational complexity are polynomial. Numerical tests show that the proposed attribute reductions accomplish better classification performance than those of traditional rough sets. In addition, in traditional rough set theory, MIBARK-algorithm [G.Y. Wang, H. Hu, D. Yang, Decision table reduction based on conditional information entropy, Chinese J. Comput., 25 (2002) 1-8] cannot ensure the reduct is the minimal attribute subset which keeps the decision rule invariant in inconsistent decision systems. Here, we solve this problem in inconsistent covering decision systems.

A new nonprobabilistic entropy of a vague set is proposed by means of the intersection and union of the membership degree and nonmembership degree of the vague set. The concept called vague cross-entropy of vague sets will also be discussed and its definition is given by analogy with the cross-entropy of probability distributions. Finally, two numeric examples are presented to illustrate the applications of vague cross-entropy to pattern recognition and medical diagnosis.

Keeping in view the non-probabilistic nature of experiments, two new measures of weighted fuzzy entropy have been introduced and to check their authenticity, the essential properties of these measures have been studied. Under the fact that measures of entropy can be used for the study of optimization principles when certain partial information is available, we have applied the existing as well as the newly introduced weighted measures of fuzzy entropy to study the maximum entropy principle.

The measure-theoretic definition of Kullback-Leibler relative-entropy (or simply KL-entropy) plays a basic role in defining various classical information measures on general spaces. Entropy, mutual information and conditional forms of entropy can be expressed in terms of KL-entropy and hence properties of their measure-theoretic analogs will follow from those of measure-theoretic KL-entropy. These measure-theoretic definitions are key to extending the ergodic theorems of information theory to non-discrete cases. A fundamental theorem in this respect is the Gelfand-Yaglom-Perez (GYP) Theorem [M.S. Pinsker, Information and Information Stability of Random Variables and Process, 1960, Holden-Day, San Francisco, CA (English ed., 1964, translated and edited by Amiel Feinstein), Theorem. 2.4.2] which states that measure-theoretic relative-entropy equals the supremum of relative-entropies over all measurable partitions. This paper states and proves the GYP-theorem for Renyi relative-entropy of order greater than one. Consequently, the result can be easily extended to Tsallis relative-entropy.

A Function of Degree of Disagreement (FDOD) function has been introduced in the study of information discrepancies among multiple information sources [W. Fang, Mathematical Social Sciences 28 (2) (1994) 85–111; J. Global Optim. 11 (1997) 387–408]. It possesses some peculiar properties and can be applied in some areas, in which the traditional measures of information discrepancy are not valid, for example, in the study of multiple DNA sequence comparison. In this paper, based on the ideas concerned with Shannon entropy and measures of difference, we investigate its characterization. The final results and even intermediate results show a close connection between the FDOD function and Shannon entropy.

Shannon's measure of entropy and Gini-Simpson's diversity index are usually used by social scientists for measuring diversity of a multinomial population. In this paper entropy of order r (Renyi, A., 1961. In: Proceedings of Fourth Berkeley Symposium on Mathematics, Statistics and Probability 1, 547–561) (Renyi, 1961) is presented as a generalization of Shannon's entropy to measure and analyze the diversity. Asymptotic properties of r order index in a simple-stage cluster sampling are investigated. Also, applications to testing hypotheses are discussed. Finite sample results are computed and, to finish, a comparison of family members via moments is given.

In this paper the concept of entropy is used to construct multivariate quality-control charts to monitor multivariate variability in a process. It is assumed that the data parent population is multivariate normal with covariance structure Σ and estimator S . The asymptotic distribution of the sample entropy is obtained. It is shown that the use of | S | as an overall measure of variability is not satisfactory and it is proposed a conditional entropy approach when the correlation matrix P 0 is known or fixed. Under these assumptions, two new overall variability measures are defined based on the sample variances and ranges on the variables under consideration. Control charts are developed using these concepts and are applied to a multivariate quality-control example.

We suggest some new measures for the calculation of the entropy of a probability distribution. One motivation for this is to provide tools for the application of the maximum entropy principle. The first class of entropy measures is based on the ordered weighted averaging operator. Another class of these measures is based on the distance of the probability distribution from the uniform distribution. A third class of entropy measures is based on the t-conorms. We then suggest some measures of fuzziness associated with a fuzzy set.

A new definition of probabilistic entropy based on the exponential behavior of gain function has recently been introduced by the authors 1. [1]. It has also been extended to define various image entropy measures 2. [2]and hybrid and higher order entropy of fuzzy sets 3. [3]. The present work investigates some additional properties (in the light of both probability theory and fuzzy set theory) of this new entropy definition to ensure its greater range of applications.

New entropy measures such as higher order fuzzy entropy and hybrid entropy of a set have been introduced. Various properties along with their proofs have been included. Applicability of these new measures to various problems has been highlighted.