Search

The quality of sensor placement in water distribution systems (WDSs) has considerable impact on leak detection accuracy and identification efficiency. Sensor-placement optimization has long been a major issue in the research of WDS. Conventional hierarchical algorithms based on joint entropy have been widely applied to sensor-placement optimization in the engineering-safety monitoring field. Although such algorithms can effectively reduce information redundancy and are computationally very efficient, they fail to fully account for various practical failure scenarios. Sensor failures introduce bias into monitored data and disrupt the monitoring process, thereby compromising the monitoring performance of a given sensor network. To overcome this drawback, this study proposes a new optimal sensor-placement scheme for WDSs based on an improved hierarchical algorithm. The new optimization scheme works by sequentially selecting sensors using joint entropy as a sensor-placement objective function and fully considers various scenarios of a single-sensor failure to ensure minimal information entropy loss during sensor failure. Simulation is performed on an example network (i.e., Net 3 from EPANET), revealing that when a sensor fails, information entropy loss is 0.011 for conventional sensor-placement schemes, but it is only 0.007 for the improved scheme. Statistical analysis shows that the improved scheme has a higher robustness and adaptability for leak detection and identification, owing to the full consideration of various scenarios of a single-sensor failure.

Feature selection based on the fuzzy neighborhood rough set model (FNRS) is highly popular in data mining. However, the dependent function of FNRS only considers the information present in the lower approximation of the decision while ignoring the information present in the upper approximation of the decision. This construction method may lead to the loss of some information. To solve this problem, this paper proposes a fuzzy neighborhood joint entropy model based on fuzzy neighborhood self-information measure (FNSIJE) and applies it to feature selection. First, to construct four uncertain fuzzy neighborhood self-information measures of decision variables, the concept of self-information is introduced into the upper and lower approximations of FNRS from the algebra view. The relationships between these measures and their properties are discussed in detail. It is found that the fourth measure, named tolerance fuzzy neighborhood self-information, has better classification performance. Second, an uncertainty measure based on the fuzzy neighborhood joint entropy has been proposed from the information view. Inspired by both algebra and information views, the FNSIJE is proposed. Third, the K–S test is used to delete features with weak distinguishing performance, which reduces the dimensionality of high-dimensional gene datasets, thereby reducing the complexity of high-dimensional gene datasets, and then, a forward feature selection algorithm is provided. Experimental results show that compared with related methods, the presented model can select less important features and have a higher classification accuracy.

In the recent past, human action recognition is inviting increased attention in the automated video surveillance systems. An efficient human action classification technique in an unconstrained environment is proposed in this paper. A novel descriptor relying on joint entropy of difference in magnitude and orientation of the optical flow feature is developed in order to model human actions. Initially, flow feature is computed using Pyramid–Warping–Cost volume Network (PWCNet), considering every two consecutive frames. Then, the feature descriptor is formed based on the joint entropy of difference in flow magnitude and orientation collected from the regular grid of each frame in the action sequence. Finally, in order to incorporate long-term temporal dependency, a spiking neural network is embedded to aggregate the information across the frames. Different optimization techniques and different types of hidden nodes are utilized in the spiking neural network to analyze the performance of the proposed work. Extensive experiments on the benchmark datasets for human action recognition show the efficacy of the proposed method.

Image segmentation is the initial task in image processing which is extensively utilized in object recognition and detection. In the field of image segmentation, multilevel thresholding is one of the leading methods. Though, the computational cost of this method scales exponentially as the number of the threshold value increases, which directs to exercise of optimization method to determine the optimal value of the thresholds. In this article, a newly modified algorithm called quasi opposite multiverse optimization (QOMVO) is proposed. The proposed QOMVO is based on quasi opposite based learning and multiverse optimization (MVO) algorithm. The quasi opposite based learning helps to improve the exploration phase of QOMVO. QOMVO is coupled with a new proposed entropy called Joint entropy (Renyi-Tsalii) to perform image segmentation by finding the optimal threshold value. The outcome of the proposed algorithm is compared with other evolutionary algorithm based on objective function value, feature similarity index, structural similarity index , quality index based on local variance, uniformity, normalized absolute error and computational time. A non-parametric test called the Wilcoxon test is done to justify the response of these parameters. Along with comparing with other algorithms, a comparison has also been made with other entropy, i.e. Renyi’s and Tsallis. The experimental outcomes confirmed that the proposed algorithm provides more reliable results than other existing methods.

Intensity measure (IM) selection is a crucial step in regional seismic risk assessment (RSRA) of spatially distributed structural portfolios. In order to facilitate more confident regional seismic risk estimates, this study proposes an entropy-based IM selection methodology, offering the first systematic and quantitative regional-level IM selection approach. By conceptualizing the spatially distributed structural portfolio as an integrated multi-response structural system, the joint entropy of the system’s unconditional seismic demands is leveraged as an IM evaluation criterion. Owing to the adaptation of a newly developed advanced IM co-simulation method and multivariate surrogate demand modeling techniques, this entropy-based IM selection approach is able to holistically incorporate uncertainties rising from the spatial IM random field, structural parameters, and surrogate demand models, during the course of uncertainty propagation in RSRA. The efficacy of the proposed methodology is demonstrated along with practical heuristics for alleviating the computational burden, based on a hypothetical highway bridge portfolio. Different application cases in the context of RSRA are considered, including pre-event RSRA considering a single scenario-earthquake as well as a stochastic earthquake catalog, and post-event RSRA considering record updating. The results consistently highlight the significance of the proposed IM selection method in facilitating more confident regional seismic risk estimates. Moreover, this study also provides valuable insights into record updating in reducing the level of uncertainty of the spatial IM random field, and its implication on IM selection in post-event RSRA.

Uncertainty measures are important tools for analyzing various data. However, there are relatively few studies on the uncertainty measures for covering rough set models. In this paper, from the viewpoint of the lower and upper approximations, we propose new uncertainty measures, the lower rough entropy and the upper rough entropy, for covering rough set models. Then, we define the concepts of the joint entropy and the conditional entropy in the covering rough set models. Some important properties of these measures are obtained, and their relationships are investigated. Furthermore, we provide a certain characterization of reducible element of a covering by means of the proposed measures, and apply the proposed rough entropy to evaluate the significance of covering granules of a covering. Finally, we apply these rough entropies to measure a dual degree between covering lower and upper approximations. The theoretical analysis and examples show that the proposed uncertainty measures for covering rough set models are reasonable and useful.

Kernel density estimation, which is a non-parametric method about estimating probability density distribution of random variables, has been used in feature selection. However, existing feature selection methods based on kernel density estimation seldom consider interval-valued data. Actually, interval-valued data exist widely. In this paper, a feature selection method based on kernel density estimation for interval-valued data is proposed. Firstly, the kernel function in kernel density estimation is defined for interval-valued data. Secondly, the interval-valued kernel density estimation probability structure is constructed by the defined kernel function, including kernel density estimation conditional probability, kernel density estimation joint probability and kernel density estimation posterior probability. Thirdly, kernel density estimation entropies for interval-valued data are proposed by the constructed probability structure, including information entropy, conditional entropy and joint entropy of kernel density estimation. Fourthly, we propose a feature selection approach based on kernel density estimation entropy. Moreover, we improve the proposed feature selection algorithm and propose a fast feature selection algorithm based on kernel density estimation entropy. Finally, comparative experiments are conducted from three perspectives of computing time, intuitive identifiability and classification performance to show the feasibility and the effectiveness of the proposed method.

Inspired on the classical Bayesian method also called mutual information, by generalizing the classical case to quantum approach to study the multipartite correlation in quantum image by using the flexible representation of quantum images (FRQI) method. Some important entanglement measures are calculated including von Neumann entropy, conditional entropy, joint entropy, discord and mutual information. We take two typical images of size 2 × 2 pixels and 8 × 8 pixels and study their entanglement measures from different classical and quantum point of view. The quantum image of size 2 × 2 is studied under the change of color for one pixel, but the quantum image of 8 × 8 is treated under translation transform. We find that the classical joint entropy remains invariant under the change of color for one pixel but the quantum entropy is sensitive to such change. It is shown that the total correlation IT could arrive to the double amount of the classical joint entropy. The discord in the classical case is zero, but it varies much in the quantum case.

This article considers the problem of estimation of the parameter of DUS-exponential distribution based on upper records through classical as well as Bayesian procedures. Maximum likelihood estimator is calculated under the classical scheme and Bayes estimator is obtained under the squared error loss function by using MCMC technique. The performances of both the estimators are compared on the basis of their estimated mean squared errors. The asymptotic and HPD intervals for the unknown parameter is also calculated. In addition to that, the entropy of ith upper record and joint entropy based on m upper records are obtained for considered distribution. A real data set is considered to illustrate the suitability of the proposed methodology.

Myanmar language also known as Burmese language is a free order of word language and syntactic patterns of one word can vary based on the position and the structure in the sentence. There are many ambiguous part of speech (POS) tags on one word in the sentence of Myanmar text. This research work presents disambiguation for the POS of the words in written Myanmar text. We aim at removing this ambiguity on Myanmar word and assigning single POS to each word of sentence. This is demonstrated on the following ideas: (i) input the sentence and segmented into words using syllable segmentation rules and forward maximum matching approach with monolingual Myanmar dictionary and (ii) apply the Joint Entropy (JE) for POS ambiguous for each word in the sentence with monolingual Myanmar tagged corpus. Joint probability value could be given the useful and accurate disambiguation of POS for free order and structure of words in Myanmar text. The monolingual Myanmar tagged corpus and tagged dictionary are created including 620 sentences and 15,000 words, respectively. This study attempts practical word segmentation and POS tagging system which can really overcome bottleneck of the machine translation system for Myanmar to other languages and research activities related to natural language processing (NLP).

The social network plays a significant role in the exchange of information in the system. Information sharing includes the exchange of ideas, thoughts, opinions about some product, politics, etc. It is essential to find the amount of information shared between the nodes in the network. Therefore, we introduced a quantum mechanical model for information sharing in the social network. The model considers different states of network components (nodes) and encodes the state of mind of individuals in the network. The node’s importance in the information flow is evaluated through the quantum walk approach. Besides, we also consider some classical mechanics parameters like different centrality measures to find mutual information shared between a node with its neighbor through the concept of marginal entropy and joint entropy. In the end, the results of the classical approach and quantum mechanical approach are compared and analyzed for a network constructed using different network models and validated through some real-world datasets.

Computer Vision (CV) domain is widely used in the current era of automation and visual surveillance for the detection and classification of different objects in a diverse environment. The automatic machine inspection of different objects in the scenes is based on internal and external parameters like features that provide a huge amount of information related to the nature of an object in the scene. In this work, we propose a new automated method based on classical and deep learning feature selection. The proposed object classification method follows three steps. The data augmentation is performed in the first step to make the balance database. Later, Pyramid HOG (PHOG) and Central Symmetric LBP (CS-LBP) features are serially fused along with deep learning-based extracted features. The deep learning features are extracted from the pre-trained CNN model name Inception V3. In the third step, a new technique name Joint Entropy along with KNN (JEKNN) is employed to select the best features. The best-selected features are finally classified by well-known supervised learning methods and choose the best one based on higher accuracy. The proposed method is evaluated on Caltech101 balanced dataset and achieved maximum accuracy of 90.4% on Ensemble classifier which outperforms as compare to existing techniques.

Cloud computing provides enormous storage capacity and huge computation power. Cloud enables users to deploy data-intensive applications without substructure investment. Intermediate data are generated under the cloud applications and stored again in cloud. An opponent may analyse multiple intermediate data sets to access the privacy sensitive information.The proposed technique to have good privacy and utility trade off, joint entropy with adaptive optimization process is used to maintain the privacy in cloud. Here optimal entropy value process using adaptive particle swarm optimization (APSO) process. After get the optimal entropy database difference model was processed, Entropy and database difference ratio is taken as the evaluation matrices and is tested using various datasets. The technique is also compared to the existing PSO optimization process on privacy preservation in cloud. An adaption feature of Particle swarm optimization show that APSO enhances privacy and preserving cost of intermediate data sets can be significantly saved.

In this article, we propose high order, semi-discrete, entropy stable, finite difference schemes for Ten-Moment Gaussian Closure equations. The crucial components of these schemes are an entropy conservative flux and suitable high order entropy dissipative operators to ensure entropy stability. We design two numerical fluxes, one is approximately entropy conservative, and another is entropy conservative flux. For the construction of appropriate entropy dissipative operators, we also derive entropy scaled right eigenvectors. This is used for sign preserving reconstruction of scaled entropy variables, which results in second and third order entropy stable schemes. We also extend these schemes to a plasma flow model with source term. Several numerical results are presented for homogeneous and non-homogeneous cases to demonstrate stability and performance of these schemes.

This paper proposes some new distance measures between interval-valued neutrosophic sets (IvNSs) and their similarity measures. Then, some entropy measures of IvNS based on the distances are proposed as the extension of the entropy measures of interval-valued intuitionistic fuzzy sets (IvIFSs). Also, we investigate the relationship between the presented entropy measures and the similarity measures for IvNSs. Finally, the comparison of the new entropy measures with existing entropy measures for IvNSs is given by the numerical and decision-making examples to demonstrate that the proposed new entropy measures for IvNSs are effective and reasonable and more intelligible in representing the degree of fuzziness of IvNSs than the existing ones.

Partial entropy is a device to measure how much of entropy of an uncertain random variable belongs to uncertain variables. However, partial entropy may fail to measure the uncertainty of an uncertain random variable. In this paper, for refining this problem, a definition of partial triangular entropy is presented. In addition, several properties of partial triangular entropy are obtained. Moreover, partial triangular entropy of an uncertain random variable is applied to mean-variance portfolio selection.

In this paper, we introduce a quantity measure which is called (R, S)-norm entropy and discuss some of its major properties with Shannon’s and other entropies in the literature. Based on this (R, S)-norm entropy, we have proposed a new (R, S)-norm fuzzy information measure and discussed its validity and properties. Further, we have given its comparison with other fuzzy information measures to prove its effectiveness. Attribute weights play an important role in multiple-attribute decision-making problems. In the present communication, two methods of determining the attribute weights are introduced. First is the case when the information regarding attribute weights is incompletely known or completely unknown and second is when we have partial information about attribute weights. For the first case, the extension of ordinary entropy weight method is used to calculate attribute weights and minimum entropy principle method based on solving a linear programming model is used in the second case. Finally, two methods are explained through numerical examples.

Evolutionary algorithms are used widely in optimization studies on water distribution networks. The optimization algorithms use simulation models that analyse the networks under various operating conditions. The solution process typically involves cost minimization along with reliability constraints that ensure reasonably satisfactory performance under abnormal operating conditions also. Flow entropy has been employed previously as a surrogate reliability measure. While a body of work exists for a single operating condition under steady state conditions, the effectiveness of flow entropy for systems with multiple operating conditions has received very little attention. This paper describes a multi-objective genetic algorithm that maximizes the flow entropy under multiple operating conditions for any given network. The new methodology proposed is consistent with the maximum entropy formalism that requires active consideration of all the relevant information. Furthermore, an alternative but equivalent flow entropy model that emphasizes the relative uniformity of the nodal demands is described. The flow entropy of water distribution networks under multiple operating conditions is discussed with reference to the joint entropy of multiple probability spaces, which provides the theoretical foundation for the optimization methodology proposed. Besides the rationale, results are included that show that the most robust or failure-tolerant solutions are achieved by maximizing the sum of the entropies.

Uncertain variables, applied to modelling imprecise quantities, are fundamental concepts in uncertainty theory. In order to characterize the information deficiency of an uncertain variable, this paper proposes a definition of quadratic entropy. Compared with the traditional entropy, it is much easier to compute and has a wider range. Furthermore, the principle of maximum entropy is applied to quadratic entropy, and two theorems of maximum quadratic entropy with moment constraint are proved.

In this paper, a type of entropy of action of a semi-group on a measure space is introduced. Then we apply this concept to specific semi-groups to extract metric entropy of a dynamical system as a special case.

Kolmogorov–Sinai entropy-based irregularity measures such as approximate entropy (ApEn), sample entropy and fuzzy entropy are widely used for short-term heart rate variability analysis. These entropy statistics are estimated for a specific value of the tolerance parameter (r) that is mostly chosen from a common recommended range. Entropy measurement on short-term signals is highly sensitive to the choice of r. An incorrect selection of r results in an inaccurate entropy value, thereby leading to unreliable information retrieval. By addressing this inaccuracy due to r selection, the quality and reliability of information retrieval can be improved. Thus, we hypothesize that generating a complete entropy profile using all potential r values will give a more complete and useful information about signal irregularity in contrast to the case of finding entropy at a single selected value of r. In order to do so, one must be able to accurately select all potential r candidates. In this paper, we use a data-driven algorithm based on cumulative histograms to automatically select potential r values for an individual signal based on its dynamics. An appropriate set of r values is designated by the algorithm for generating a series of ApEn values (ApEn profile) instead of a single value of ApEn. ApEn profile- based secondary measures such as TotalApEn and SDApEn have been used as features to classify sets of synthetic and physiologic data. Our study proves that secondary measures obtained from an ApEn profile are more efficient in indicating irregularity levels in comparison with the traditional measure of ApEn evaluated at a single r value, specially in the case of short length data.

Dempster-Shafer evidence theory is widely used to deal with uncertainty in intelligent systems. However, the application of this theory is constrained by the failure to balance multiple conflict evidence. The existing studies have primarily focused on investigating similarity of evidence. However, the similarity measurement is highly dependent on the capability of distance functions and will substantially increase the computational complexity. So, the efficient method with acceptable expense should be intensively investigated. In this paper, we propose a new method based on the variance of information entropy to handle the conflict of evidence. First, the fuzzy preference relations based on the variance of information entropy are constructed for multiple pieces of evidence. Next, credible values of alternative evidence are calculated. Finally, according to the Dempster's rule of combination, the weighted average combination result can be obtained. Typical example and several actual data are used to demonstrate that the proposed method is more reasonable than some existing methods both in managing conflict and reducing computational complexity.

Recently, quantile-based study of entropy measures found greater interest among researchers as an alternative method of measuring uncertainty of a random variable. The quantile-based entropy measures possess some unique properties than its distribution function approach. Motivated by this, in the present paper, we introduce a quantile-based entropy of order statistics and study its properties. We also propose a quantile-based residual entropy of order statistics, an alternative method to measure the uncertainty of ordered observations for used items. Unlike distribution function approach, we derive an explicit relationship between the quantile density function and quantile-based residual entropy of order statistics.

We review the notion of weighted quantum entropy and consider the weighted quantum entropy for bipartite and noncomposite quantum systems. We extend the subadditivity condition, the inequality known for the weighted entropy information, to the case of indivisible qudit system, such as a qutrit. We discuss the new inequality for the qutrit density matrix for different weights and states, as well as the role of weighted entropy with respect to nonlinear quantum channels.

This paper addresses the joint entropy maximization algorithm constrained by the variable leaky factor (JEM-VL) aiming at mitigating the feedback filter (FBF) mismatch effects characterizing the operation of the decision feedback equalizer which, at the start of adaptation, swaps positions of feedforward and feedback filters so that the latter acts as a linear all-pole whitener of the received signal. The FBF mismatch is a result of disparity between the FBF setup achieved in the blind mode by observing channel outputs and an excepted FBF setup in the tracking mode which is driven by detected data symbols. Depending on the given signal complexity and inter-symbol interference severity, the FBF filter mismatch is typically manifested by the equalizer convergence instability or even the catastrophic error propagation effects arising at the time of the equalizer structure-criterion switching from the blind to the tracking operation mode. The constraint of superfluous coefficients of the FBF filter by means of the JEM-VL algorithm eliminates the equalizer convergence instability at the time of its switching and, consequently, increases equalization successfulness. The efficiency of the JEM-VL algorithm is verified by simulations using the 64-QAM signal.

With the recent advancement of data acquisition techniques, 3D animation is becoming a new challenging subject for data processing. In this paper, we present a joint entropy-based key-frame extraction method, which further derives a motion segmentation method for 3D animations. We start by applying an existing deformation-based feature descriptor to measure the degree of deformation of each triangle within each frame, from which we compute the statistical joint probability distribution of triangles’ deformation between two consecutive subsequences of frames with a fixed length. Then, we further compute joint entropy between the two subsequences. This allows us to extract key-frames by taking the local maximal from the joint entropy curve (or energy curve) of a given 3D animation. Furthermore, we classify the extracted key-frames by grouping the key-frames with similar motions into the same cluster. Finally, we compute a boundary frame between each of the two neighboring frames with different motions, which is achieved by minimizing the variance of energy between the two motions. The experimental results show that our method successfully extracts representative key-frames of different motions, and the comparisons with existing methods show the effectiveness and the efficiency of our method.

Partially rank-ordered set (PROS) sampling is a generalization of ranked set sampling in which rankers are not required to fully rank the sampling units in each set, hence having more flexibility to perform the necessary judgemental ranking process. The PROS sampling has a wide range of applications in different fields ranging from environmental and ecological studies to medical research and it has been shown to be superior over ranked set sampling and simple random sampling for estimating the population mean. We study Fisher information content and uncertainty structure of the PROS samples and compare them with those of simple random sample (SRS) and ranked set sample (RSS) counterparts of the same size from the underlying population. We study uncertainty structure in terms of the Shannon entropy, Renyi entropy and Kullback–Leibler (KL) discrimination measures.

We prove that if a measure-preserving automorphism has a scaling entropy sequence, then this sequence can be chosen nondecreasing and subadditive.

In this expository article, we study optimization problems specified via linear and relative entropy inequalities. Such relative entropy programs (REPs) are convex optimization problems as the relative entropy function is jointly convex with respect to both its arguments. Prominent families of convex programs such as geometric programs (GPs), second-order cone programs, and entropy maximization problems are special cases of REPs, although REPs are more general than these classes of problems. We provide solutions based on REPs to a range of problems such as permanent maximization, robust optimization formulations of GPs, and hitting-time estimation in dynamical systems. We survey previous approaches to some of these problems and the limitations of those methods, and we highlight the more powerful generalizations afforded by REPs. We conclude with a discussion of quantum analogs of the relative entropy function, including a review of the similarities and distinctions with respect to the classical case. We also describe a stylized application of quantum relative entropy optimization that exploits the joint convexity of the quantum relative entropy function.

Wireless networks appeal to privacy eavesdroppers that secretly capture packets from the open medium and read the metadata and data content in search of any type of information through data mining and statistical analysis. Addressing the problems of personal privacy for wireless users will require flexible privacy protection mechanisms adaptively to the frequently varying context. Privacy quantification is prerequisite for enabling a context-aware privacy protection in wireless networks. This paper proposes to quantify the data privacy during wireless communication processes. We give a computational quantification method of data privacy through introducing the concepts of privacy entropy and privacy joint entropy, which permits users and applications to on-demand customize their preferential sensitivity extent of privacy leakage and protection strength. Accordingly, we put forward a data-privacy protection scheme in order to explain how to utilize the proposed computational quantification method of data privacy against the excessive disclosure of data privacy. The results show that the computational quantitation method can effectively characterize the real-time fluctuation of data privacy during communication processes and provide the reliable judgment to context-aware privacy protection, which enables to real-timely control the present data privacy to an anticipated target and to balance a tradeoff between the data privacy and communication efficiency.

The optimal design of data acquisition is not obvious in Bayesian network models. The dependency structure may vary dramatically, which makes learning and information evaluation complicated and sometimes non-intuitive. The motivation for working on this topic is petroleum exploration, and the application of this paper is prospect selection in the North Sea. Here, the data gathering is often carried out during seasonal campaigns, and it is useful to plan the experimentation and to understand which data are likely to be most informative. Information measures are used to compare possible future observation sets. Four information measures are studied: Shannon Entropy, sum of variances, Node-wise Entropy and overall prediction error. The Shannon Entropy is commonly considered the standard measure of information, and the Node-wise Entropy measure can be interpreted as an approximation to the former. The variance measure links uncertainty and variance. The prediction error measure is tied to decision-making rules. The results lead to new insight about prospect selection. For example, the Node-wise Entropy and the variance measure behave similarly, and the optimal observation set of Shannon Entropy does not correspond to what one intuitively would consider as minimizing unknown information in this case.

The resultant measures of the entropy/information content in complex elec- tronic states are discussed in the canonical position and momentum representations of molecular quantum mechanics. The nonclassical (phase/current) supplements of the classical (probability) descriptors of the overall entropy/information content in electronic states are identified and the associated entropy deficiency (information dis- tance) quantities are introduced. The Shannon (global, logarithmic) and Fisher (local, gradient) information descriptors in both spaces are summarized, and the momentum continuity equation is used to establish the associated probability source. General relations between global and local information densities are examined and the etropic principles determining molecular phase equilibria are investigated.

We consider the relative operator entropy constructed by the spectral geometric mean and see its properties analogous to those of the Tsallis relative operator entropy by the usual geometric mean. Furthermore, we define the quantum relative entropy constructed by the spectral geometric mean and derive its subadditivity under tensor product.

For any bipartite state, how strongly can one subsystem be quantum correlated with another? Using the Koashi---Winter relation, we study the upper bound of purified quantum discord, which is given by the sum of the von Neumann entropy of the unmeasured subsystem and the entanglement of formation shared between the unmeasured subsystem with the environment. In particular, we find that Luo et al.'s conjecture on the quantum correlations and the Lindblad conjecture are true under the condition that the entanglement of formation vanishes. Let the discord distance be defined as the difference between the left discord and the right discord. If the Lindblad conjecture is true, we show that the joint entropy is a tight upper bound for the discord distance. Further, we obtain a necessary and sufficient condition for saturating upper bounds of purified quantum discord and discord distance separately with the equality conditions for the Araki---Lieb inequality and the Lindblad conjecture.

The second law of thermodynamics is fundamental in landscape ecology, and Shannon entropy has been employed as an important means of analyzing landscape patterns. However, the thermodynamic basis of Shannon entropy has been recently questioned because such entropy considers only probability and not configurational information. As a result, Boltzmann entropy (also called configurational entropy), which is the basic measure in thermodynamics, has been revisited, and some thoughts on its calculation have been put forward. Nevertheless, a comprehensive calculation method is still lacking. The objective of this study is to propose a feasible solution for the calculation of configurational entropy for landscape gradients. To calculate the configurational entropy, the first step is to define a good macrostate and then to determine the number of microstates. The macrostate of a landscape gradient is defined as its abstract (i.e., upscaled) representation. The number of microstates is calculated by determining all the possible ways of downscaling from the macrostate to the original. Both simulated and real-life landscape patterns were used for experimental validation. The results show that the entropy calculated using the proposed method successfully captures the disorder of landscape gradients in terms of both composition and configuration. Configurational entropy, calculated using the proposed method, can serve as a thermodynamics-based metric to describe gradient-based landscapes and their changes across space and through time. With this metric, it becomes possible to interpret landscape ecological processes based on thermodynamic insights.

It is an interesting subject to analyze the coupling dependence between time series. Many information-theoretic methods have been proposed for this purpose. In this article, we propose a new permutation-based entropy for the detection of coupling structures between stock markets. It is inspired by inner composition alignment method that we use the number of crossing points instead of mode $$\pi $$ in permutation entropy definition to define the probability distribution. The measure is named as cross-permutation entropy (CPE), and can be used to infer the cross-correlation properties as well as the direction of couplings. The capabilities of CPE to correctly infer couplings together with their directionality are compared against transfer entropy for artificial signals as well as stock market time series. The results show that CPE is efficient in the analysis of cross-correlation between signals. Besides, in the analysis of financial time series, we found the coupling between stock markets in a same country are generally stronger than what would be expected from different countries.

In the present work we focus on the generalization of two types of measures of information namely divergence-type and entropy-type. Kullback–Leibler discrimination measure and Shannon entropy have been considered in this context for truncated random variables. We propose a generalized discrimination measure between two residual and past lifetime distributions along a similar line of Varma’s entropy. Some properties of this measure are studied and a characterization of the proportional (reversed) hazards model is given. Furthermore, Shannon entropy is generalized on the basis of Varma’s entropy for past lifetime distribution. These results generalize and enhance the related existing results that are developed based on Kullback–Leibler information and Shannon entropy.

Entropy is used as a measure to characterize the uncertainty. So far, entropy for uncertain variables in the forms of logarithm function and triangular function has been proposed. This paper mainly studies the concept of triangular entropy, and verifies its properties such as translation invariance and positive linearity. As an application, this paper also considers a mean-variance portfolio selection problem with triangular entropy as a constraint.

Uncertainty measure is an important tool for analyzing imprecise and ambiguous data. Some information entropy models in rough set theory have been defined for various information systems. However, there are relatively few studies on evaluating uncertainty in fuzzy rough set. In this paper, we propose a new complement information entropy model in fuzzy rough set based on arbitrary fuzzy relation, which takes inner-class and outer-class information into consideration. The corresponding definitions of complement conditional entropy, complement joint entropy, complement mutual information and complement information granularity are also presented. The properties of these definitions are analyzed, which show complement information entropy shares some similar properties with Shannon's entropy. Moreover, a generalized information entropy model is proposed by introducing probability distribution into fuzzy approximate space. This model can be used to measure uncertainty of data with the different sample distributions. Applications of the proposed entropy measures in feature importance evaluation and feature selection are studied with data set experiments. Experimental results show that the proposed method is effective and adaptable to different classifiers.

The cumulative residual entropy (CRE) is a new measure of information and an alternative to the Shannon differential entropy in which the density function is replaced by the survival function. This new measure overcomes deficiencies of the differential entropy while extending the Shannon entropy from the discrete random variable cases to the continuous counterpart. Some properties of the cumulative residual entropy, its estimation and applications has been studied by many researchers. The objective of this paper is twofold. In the first part, we give a central limit theorem result for the empirical cumulative residual entropy based on a right censored random sample from an unknown distribution. In the second part, we use the CRE of the comparison distribution function to propose a goodness-of-fit test for the exponential distribution. The performance of the test statistic is evaluated using a simulation study. Finally, some numerical examples illustrating the theory are also given.

Multispectral geospatial image sets retain the scene’s spatial and spectral information. To jointly use both of them for analysis purposes, we propose to extend the concept of wavelet packets, by introducing a new integrated multispectral entropy function. Each spectral band is individually decomposed by the wavelet packets transform, and then the entropy term is jointly guided by information from all bands, simultaneously. Finally, the wavelet packets coefficients undergo a dimension reduction process. We present examples of this theory applied to hyperspectral satellite imagery.

Pattern synchronization (PS) can capture one aspect of the dynamic interactions between bivariate physiological systems. It can be tested by several entropy-based measures, e.g., cross sample entropy (X-SampEn), cross fuzzy entropy (X-FuzzyEn), multivariate multiscale entropy (MMSE), etc. A comprehensive comparison on their distinguishability is currently missing. Besides, they are highly dependent on several pre-defined parameters, the threshold value r in particular. Thus, their consistency also needs further elucidation. Based on the well-accepted assumption that a tight coupling necessarily leads to a high PS, we performed a couple of evaluations over several simulated coupled models in this study. All measures were compared to each other with respect to their consistency and distinguishability, which were quantified by two pre-defined criteria-degree of crossing (DoC) and degree of monotonicity (DoM). Results indicated that X-SampEn and X-FuzzyEn could only work well over coupled stochastic systems with meticulously selected r. It is thus not recommended to apply them to the intrinsic complex physiological systems. However, MMSE was suitable for both, indicating by relatively higher DoC and DoM values. Final analysis on the cardiorespiratory coupling validated our results.

The joint entropy of a non-dissipative driven harmonic oscillator for both the single and the double Gaussian states is determined. It is shown that, a non-random driving force does not influence the evolution of the system’s joint entropy. For a random driving force however, the joint entropy increases almost monotonically with time, while exhibiting oscillatory behavior. The increase is a further proof of the decoherence mechanism in the system, for which we determine the duration and other characteristics.

In nonlinear error growth dynamics, the initial error cannot be accurately determined, and the forecast error, which is also uncertain, can be considered to be a random variable. Entropy in information theory is a natural measure of the uncertainty of a random variable associated with a probability distribution. This paper effectively combines statistical information theory and nonlinear error growth dynamics, and introduces some fundamental concepts of entropy in information theory for nonlinear error growth dynamics. Entropy based on nonlinear error can be divided into time entropy and space entropy, which are used to estimate the predictabilities of the whole dynamical system and each of its variables. This is not only applicable for investigating the dependence between any two variables of a multivariable system, but also for measuring the influence of each variable on the predictability of the whole system. Taking the Lorenz system as an example, the entropy of nonlinear error is applied to estimate predictability. The time and space entropies are used to investigate the spatial distribution of predictability of the whole Lorenz system. The results show that when moving around two chaotic attractors or near the edge of system space, a Lorenz system with lower sensitivity to the initial field behaves with higher predictability and a longer predictability limit. The example analysis of predictability of the Lorenz system demonstrates that the predictability estimated by the entropy of nonlinear error is feasible and effective, especially for estimation of predictability of the whole system. This provides a theoretical foundation for further work in estimating real atmospheric multivariable joint predictability.

Uncertain set is a set-valued function on an uncertainty space, and attempts to model unsharp concepts. Firstly, a definition of quadratic entropy to characterize the uncertainty of uncertain sets resulting from information deficiency is proposed. Secondly, some properties of quadratic entropy for uncertain sets are given, and the relation between quadratic entropy and Liu's entropy of uncertain sets is discussed. Finally, a quadratic cross entropy for uncertain sets is investigated.

The entropic perspective on the molecular electronic structure is investigated. Information-theoretic description of electron probabilities is extended to cover the complex amplitudes (wave functions) of quantum mechanics. This analysis emphasizes the entropic concepts due to the phase part of electronic states, which generates the probability current density, thus allowing one to distinguish the information content of states generating the same electron density and differing in their current densities. The classical information measures of Fisher and Shannon, due to the probability/density distributions themselves, are supplemented by the nonclassical terms generated by the wave-function phase or the associated probability current. A complementary character of the Fisher and Shannon information measures is explored and the relationship between these classical information densities is derived. It is postulated to characterize also their nonclassical (phase/current-dependent) contributions. The continuity equations of the generalized information densities are examined and the associated nonclassical information sources are identified. The variational rules involving the quantum-generalized Shannon entropy, which generate the stationary and time-dependent Schrodinger equations from the relevant maximum entropy principles, are discussed and their implications for the system “thermodynamic” equilibrium states are examined. It is demonstrated that the lowest, stationary “thermodynamic” state differs from the true ground state of the system, by exhibiting the space-dependent phase, linked to the modulus part of the wave function, and hence also a nonvanishing probability current.

The Shannon entropy of a random variable has become a very useful tool in Probability Theory. In this paper we extend the concept of cumulative residual entropy introduced by Rao et al. (in IEEE Trans Inf Theory 50:1220–1228, 2004). The new concept called generalized cumulative residual entropy (GCRE) is related with the record values of a sequence of i.i.d. random variables and with the relevation transform. We also consider a dynamic GCRE obtained using the residual lifetime. For these concepts we obtain some characterization results, stochastic ordering and aging classes properties and some relationships with other entropy concepts.

Probabilistic belief contraction has been a much neglected topic in the field of probabilistic reasoning. This is due to the difficulty in establishing a reasonable reversal of the effect of Bayesian conditionalization on a probabilistic distribution. We show that indifferent contraction, a solution proposed by Ramer to this problem through a judicious use of the principle of maximum entropy, is a probabilistic version of a full meet contraction. We then propose variations of indifferent contraction, using both the Shannon entropy measure as well as the Hartley entropy measure, with an aim to avoid excessive loss of beliefs that full meet contraction entails.

In this paper, we propose entropy-based spectrum sensing schemes to detect the existence of a primary user in Cognitive Radio (CR). To support this proposal, we have studied four types of entropies [Approximate Entropy (ApEn), Bispectral Entropy (BispEn), Sample Entropy (SamEn) and Renyi Entropy (RenyiEn)] and their applications for spectrum sensing. The reason for investigating these entropies comes from the fact that different types of entropies have different characteristics which make them more or less suitable for specific applications. Monte Carlo simulations were executed to find out how suitable these measures are for CR. Averaged value curves, boxplots and Analysis of Variance (ANOVA) tests document the performance of these types of entropies. The results show that BispEn outperformed the other three entropy measures. The ANOVA test shows that BispEn can sense modulated signals when the (SNR) is as low as ?15 dB. This is at least a 5 dB improvement compared to the other entropies studied in this paper.

Disagreements over the meaning of the thermodynamic entropy and how it should be defined in statistical mechanics have endured for well over a century. In an earlier paper, I showed that there were at least nine essential properties of entropy that are still under dispute among experts. In this paper, I examine the consequences of differing definitions of the thermodynamic entropy of macroscopic systems. Two proposed definitions of entropy in classical statistical mechanics are (1) defining entropy on the basis of probability theory (first suggested by Boltzmann in 1877), and (2) the traditional textbook definition in terms of a volume in phase space (also attributed to Boltzmann). The present paper demonstrates the consequences of each of these proposed definitions of entropy and argues in favor of a definition based on probabilities.