Search

In hyperspectral anomaly detection, anomalies are rare targets that exhibit distinct spectral signatures from the background. Thus, anomalies are with low probabilities of occurrence in hyperspectral images. In this article, we develop a new technique for hyperspectral anomaly detection that adopts a new information theory perspective, to fully utilize the aforementioned concepts. Our goal is to transform system entropy into quantitative metrics of anomaly conspicuousness of pixels. To do so, two tasks are first completed: first, the construction of occurrence probability of pixels based on the density peak clustering algorithm, and second, the valid system definitions for pixels in specific anomaly detection problems with multiviews. Specifically, three types of systems are separately established by pixel pairs to conform to the definitions of three entropy definitions in information theory, i.e., Shannon entropy, joint entropy, and relative entropy. Then, three individual entropy-based metrics that assess the anomaly conspicuousness are defined. In addition, we design a standard deviation-based ensemble strategy for the integrated representation of the three individual metrics, which considers both logic ``or'' and ``and'' operations to simultaneously improve the detection rate and reduce the false alarm rate. Our experimental results obtained on two publicly available datasets with anomalies of different sizes and shapes demonstrate the superiority of our newly proposed anomaly detection method.

Feature selection based on the fuzzy neighborhood rough set model (FNRS) is highly popular in data mining. However, the dependent function of FNRS only considers the information present in the lower approximation of the decision while ignoring the information present in the upper approximation of the decision. This construction method may lead to the loss of some information. To solve this problem, this paper proposes a fuzzy neighborhood joint entropy model based on fuzzy neighborhood self-information measure (FNSIJE) and applies it to feature selection. First, to construct four uncertain fuzzy neighborhood self-information measures of decision variables, the concept of self-information is introduced into the upper and lower approximations of FNRS from the algebra view. The relationships between these measures and their properties are discussed in detail. It is found that the fourth measure, named tolerance fuzzy neighborhood self-information, has better classification performance. Second, an uncertainty measure based on the fuzzy neighborhood joint entropy has been proposed from the information view. Inspired by both algebra and information views, the FNSIJE is proposed. Third, the K–S test is used to delete features with weak distinguishing performance, which reduces the dimensionality of high-dimensional gene datasets, thereby reducing the complexity of high-dimensional gene datasets, and then, a forward feature selection algorithm is provided. Experimental results show that compared with related methods, the presented model can select less important features and have a higher classification accuracy.

Image segmentation is the initial task in image processing which is extensively utilized in object recognition and detection. In the field of image segmentation, multilevel thresholding is one of the leading methods. Though, the computational cost of this method scales exponentially as the number of the threshold value increases, which directs to exercise of optimization method to determine the optimal value of the thresholds. In this article, a newly modified algorithm called quasi opposite multiverse optimization (QOMVO) is proposed. The proposed QOMVO is based on quasi opposite based learning and multiverse optimization (MVO) algorithm. The quasi opposite based learning helps to improve the exploration phase of QOMVO. QOMVO is coupled with a new proposed entropy called Joint entropy (Renyi-Tsalii) to perform image segmentation by finding the optimal threshold value. The outcome of the proposed algorithm is compared with other evolutionary algorithm based on objective function value, feature similarity index, structural similarity index , quality index based on local variance, uniformity, normalized absolute error and computational time. A non-parametric test called the Wilcoxon test is done to justify the response of these parameters. Along with comparing with other algorithms, a comparison has also been made with other entropy, i.e. Renyi’s and Tsallis. The experimental outcomes confirmed that the proposed algorithm provides more reliable results than other existing methods.

Intensity measure (IM) selection is a crucial step in regional seismic risk assessment (RSRA) of spatially distributed structural portfolios. In order to facilitate more confident regional seismic risk estimates, this study proposes an entropy-based IM selection methodology, offering the first systematic and quantitative regional-level IM selection approach. By conceptualizing the spatially distributed structural portfolio as an integrated multi-response structural system, the joint entropy of the system’s unconditional seismic demands is leveraged as an IM evaluation criterion. Owing to the adaptation of a newly developed advanced IM co-simulation method and multivariate surrogate demand modeling techniques, this entropy-based IM selection approach is able to holistically incorporate uncertainties rising from the spatial IM random field, structural parameters, and surrogate demand models, during the course of uncertainty propagation in RSRA. The efficacy of the proposed methodology is demonstrated along with practical heuristics for alleviating the computational burden, based on a hypothetical highway bridge portfolio. Different application cases in the context of RSRA are considered, including pre-event RSRA considering a single scenario-earthquake as well as a stochastic earthquake catalog, and post-event RSRA considering record updating. The results consistently highlight the significance of the proposed IM selection method in facilitating more confident regional seismic risk estimates. Moreover, this study also provides valuable insights into record updating in reducing the level of uncertainty of the spatial IM random field, and its implication on IM selection in post-event RSRA.

Uncertainty measures are important tools for analyzing various data. However, there are relatively few studies on the uncertainty measures for covering rough set models. In this paper, from the viewpoint of the lower and upper approximations, we propose new uncertainty measures, the lower rough entropy and the upper rough entropy, for covering rough set models. Then, we define the concepts of the joint entropy and the conditional entropy in the covering rough set models. Some important properties of these measures are obtained, and their relationships are investigated. Furthermore, we provide a certain characterization of reducible element of a covering by means of the proposed measures, and apply the proposed rough entropy to evaluate the significance of covering granules of a covering. Finally, we apply these rough entropies to measure a dual degree between covering lower and upper approximations. The theoretical analysis and examples show that the proposed uncertainty measures for covering rough set models are reasonable and useful.

The purpose of this study is to propose novel and efficient competitiveness indexes to measure the level of competition among container shipping operators based on a specific region. These indexes should require only basic data, which is full container throughput on the basis of terminal/ hinterland and ship operator. This study takes advantages of two methods to propose novel indexes as alternatives to Herfindahl Hirschman Index (HHI), which is very popular to measure level of competition. Originally named Competition-based Overall Similarity Measurement Index (COSMI) and Entropy Competitiveness Index (ECI) utilize overall similarity measure from clustering analysis and entropy methodologies, respectively. Both indexes have been proposed with two variants for each. COSMI200+ ignores the throughput of each SO having an amount less than 200 Twentyfoot Equivalent Units (TEUs), but COSMITOP5 takes into account only the top 5 SOs in terms of local throughput in a hinterland. ECI-JOINT includes a joint entropy coefficient which is constant for each hinterland, but ECI-VAR takes into account a variable entropy coefficient defined by the number of ship operators in each hinterland. Analyzing a dataset for the terminals located in Turkey, the Entropy Competitiveness Index (by means of ECI-JOINT variant) has been exhibited as a good alternative to HHI with a great correlation coefficient with it: 0.97. Theoretically, Competitionbased Overall Similarity Measurement Index (by means of COSMITOP5 variant) seems a promising method, but it is highly affected by outliers and inconstant numbers of ship operators per route, indicating a moderate correlation coefficient with HHI: 0.45., Bu çalışmanın amacı, belli bir bölge bazlı olarak konteyner gemi operatörleri arasındaki rekabet seviyesini ölçmeye yarayan yeni ve etkin rekabet indeksleri oluşturmaktır. Bu indeksler terminal/bölge ve gemi operatörü bazlı olarak, sadece başlıca verilerden olan dolu konteyner sevkiyat verilerine ihtiyaç duymalıdır. Bu çalışma, rekabet seviyesi ölçümünde popüler olan Herfindahl Hirschman İndeksine (HHI) alternatif yeni indeksler oluşturabilmek için iki farklı yöntemden yararlanmaktadır. Özgün olarak Rekabet Bazlı Toplam Benzerlik Ölçüsü İndeksi (COSMI) ve Entropi Rekabet İndeksi (ECI) olarak adlandırılan bu indeksler, sırasıyla kümeleme analizi toplam benzerlik ölçüsü ve entropi yöntemlerinden yararlanmaktadırlar. Çalışmada her iki indeksin ikişer uyarlaması incelenmiştir. COSMI200+, bir lokal bölgedeki 200 TEU’dan daha az yükleme-tahliye performansı gösteren gemi operatörlerini gözardı ederken, COSMITOP5 sadece en yüksek performansa sahip 5 gemi operatörünü dikkate almaktadır. ECI-JOINT uyarlamasında sabit olan ortak bir entropi katsayısı kullanılırken, ECI-VAR uyarlamasında her bir lokal bölgedeki gemi operatörü sayısına göre değişen entropi katsayısı kullanılmaktadır. Türkiye’de mukim terminallere ait verilerin analizi, Entropi Rekabet İndeksi’nin (ECI-JOINT uyarlaması ile) HHI ile 0,97 değerinde korelasyon katsayısına sahip olduğunu ve bu indekse iyi bir alternatif olduğunu göstermektedir. Teorik olarak Rekabet Bazlı Toplam Rekabet Ölçüsü İndeksi (COSMITOP5 uyarlaması ile) umut veren bir yöntemi barındırsa da, dışadüşenler ve rota bazlı değişken gemi operatörü sayısı nedeniyle HHI ile 0,45 değerinde orta dereceli bir korelasyon katsayısına sahiptir.

Kernel density estimation, which is a non-parametric method about estimating probability density distribution of random variables, has been used in feature selection. However, existing feature selection methods based on kernel density estimation seldom consider interval-valued data. Actually, interval-valued data exist widely. In this paper, a feature selection method based on kernel density estimation for interval-valued data is proposed. Firstly, the kernel function in kernel density estimation is defined for interval-valued data. Secondly, the interval-valued kernel density estimation probability structure is constructed by the defined kernel function, including kernel density estimation conditional probability, kernel density estimation joint probability and kernel density estimation posterior probability. Thirdly, kernel density estimation entropies for interval-valued data are proposed by the constructed probability structure, including information entropy, conditional entropy and joint entropy of kernel density estimation. Fourthly, we propose a feature selection approach based on kernel density estimation entropy. Moreover, we improve the proposed feature selection algorithm and propose a fast feature selection algorithm based on kernel density estimation entropy. Finally, comparative experiments are conducted from three perspectives of computing time, intuitive identifiability and classification performance to show the feasibility and the effectiveness of the proposed method.

In this paper, some new results and inequalities of dynamic failure entropy (DFE) defined by Abbasnejad (2011) will be obtained. Then weighted versions of failure and survival entropies of order β introduced and some properties and inequalities based on these uncertainty measures found. Then weighted DFE will be introduced and some properties investigated. Finally, empirical cumulative entropy is applied to estimate the new information measures.

© 2013 IEEE. Distributed Denial of Service (DDoS) attacks are increasingly harmful to the cyberspace nowadays. The attackers can now easily launch a bigger and more challenging DDoS attack both towards and with Internet-of-Things (IoT) devices, due to the fast popularization of them. Because of the characteristic of fast overwhelming, it is important to make fast as well as accurate response to DDoS attacks, and the real-time performance can be even more important to prevent and legitimate the attacks. Among the methods proposed by researchers, the entropy-based detection method provides a sensitive and reliable performance. However, the balance between computational complexity and recognition accuracy remains a challenge. In this paper, we propose a detection method that consists of 3 main parts in different aspects: a sliding time window to fasten the entropy calculation, a single-directional filter to realize early detection during the DDoS progress but not after the crash, and a quintile deviation check algorithm to optimize the detection result. These will eventually lead to a real-time and high-efficient performance to recognize IoT DDoS attacks as soon as possible.

Cross-frequency phase–amplitude coupling (PAC) plays an important role in neuronal oscillations network, reflecting the interaction between the phase of low-frequency oscillation (LFO) and amplitude of the high-frequency oscillations (HFO). Thus, we applied four methods based on permutation analysis to measure PAC, including multiscale permutation mutual information (MPMI), permutation conditional mutual information (PCMI), symbolic joint entropy (SJE), and weighted-permutation mutual information (WPMI). To verify the ability of these four algorithms, a performance test including the effects of coupling strength, signal-to-noise ratios (SNRs), and data length was evaluated by using simulation data. It was shown that the performance of SJE was similar to that of other approaches when measuring PAC strength, but the computational efficiency of SJE was the highest among all these four methods. Moreover, SJE can also accurately identify the PAC frequency range under the interference of spike noise. All in all, the results demonstrate that SJE is better for evaluating PAC between neural oscillations.

Number: 11 Publisher: Multidisciplinary Digital Publishing Institute; Information theory is a unifying mathematical theory to measure information content, which is key for research in cryptography, statistical physics, and quantum computing [...]

The social network plays a significant role in the exchange of information in the system. Information sharing includes the exchange of ideas, thoughts, opinions about some product, politics, etc. It is essential to find the amount of information shared between the nodes in the network. Therefore, we introduced a quantum mechanical model for information sharing in the social network. The model considers different states of network components (nodes) and encodes the state of mind of individuals in the network. The node’s importance in the information flow is evaluated through the quantum walk approach. Besides, we also consider some classical mechanics parameters like different centrality measures to find mutual information shared between a node with its neighbor through the concept of marginal entropy and joint entropy. In the end, the results of the classical approach and quantum mechanical approach are compared and analyzed for a network constructed using different network models and validated through some real-world datasets.

In this paper, we propose the selection of most suitable mother wavelet function for Turkish phonemes using discrete wavelet transform. The determination of most similar mother wavelet function to the signal has been a challenge in speech processing. The optimum mother wavelet function for Turkish phonemes have been determined by using quantitative measures which are energy and Shannon entropy, information theoretic measures which are joint entropy, conditional entropy, mutual information, and relative entropy from wavelet coefficients of the phonemes. In this study, 101 potential functions were investigated to determine the most appropriate mother wavelet. Experimental results show that the most appropriate wavelet functions for /ç/ and /ş/ phonemes which are unvoiced fricatives have been found as Bi-orthogonal 3.9 and Bi-orthogonal 5.5, respectively. By considering all the results, it is seen that the Bi-orthogonal 3.1 and Discrete Meyer wavelet functions are the most suitable mother wavelets for all other phonemes.

Inspired on the classical Bayesian method also called mutual information, by generalizing the classical case to quantum approach to study the multipartite correlation in quantum image by using the flexible representation of quantum images (FRQI) method. Some important entanglement measures are calculated including von Neumann entropy, conditional entropy, joint entropy, discord and mutual information. We take two typical images of size 2 × 2 pixels and 8 × 8 pixels and study their entanglement measures from different classical and quantum point of view. The quantum image of size 2 × 2 is studied under the change of color for one pixel, but the quantum image of 8 × 8 is treated under translation transform. We find that the classical joint entropy remains invariant under the change of color for one pixel but the quantum entropy is sensitive to such change. It is shown that the total correlation IT could arrive to the double amount of the classical joint entropy. The discord in the classical case is zero, but it varies much in the quantum case.

This article considers the problem of estimation of the parameter of DUS-exponential distribution based on upper records through classical as well as Bayesian procedures. Maximum likelihood estimator is calculated under the classical scheme and Bayes estimator is obtained under the squared error loss function by using MCMC technique. The performances of both the estimators are compared on the basis of their estimated mean squared errors. The asymptotic and HPD intervals for the unknown parameter is also calculated. In addition to that, the entropy of ith upper record and joint entropy based on m upper records are obtained for considered distribution. A real data set is considered to illustrate the suitability of the proposed methodology.

Blind image quality assessment (BIQA) aims to evaluate the quality of an image without pristine image objectively, which is highly desired in many perception-oriented image processing systems. Distortions degrade the visual contents and cause image quality degradation. Moreover, the visual contents of an image suffer from individual degradations by different types and different levels of distortions, which makes us difficult to analyze the quality degradation. From the perspective of information theory, there is a decrease in the amount of the visual contents when images are distorted. Therefore, in this paper, the image quality is assessed through its visual entropy degradation. Researches on the neuroscience indicate that the simple cells in the local receptive field can be characterized as being spatially localized and oriented, then the local intensity, gradient, and orientation features are extracted to represent the visual contents of an image. By deducing the joint entropy equation, the joint entropy is related to the statistical distributions. Next, in order to measure the visual entropy, the joint statistical distributions of those features are calculated. Finally, by measuring the degradations on these distributions of distorted images, a novel BIQA method is proposed. The experimental results on the databases of LIVE, CSIQ, and TID2013 show that the proposed method has superior performance than other state-of-the-art BIQA methods.

Multi-label feature selection that identifies important features from the original feature set of multi-labeled datasets has been attracting considerable attention owing to its generality compared to conventional single-label feature selection. The unimportant features are filtered by scoring the dependency of features to labels. In conventional multi-label feature filter studies, the score function is obtained by approximating a dependency measure such as joint entropy because direct calculation is often impractical due to the presence of multiple labels with limited training patterns. Although the efficacy of approximation can differ depending on the characteristics of the multi-label dataset, conventional methods presume a certain approximation method, leading to a degenerated feature subset if the presumed approximation is inappropriate for the given dataset. In this study, we propose a strategy for selecting an approximation among a series of approximations depending on the number of involved variables and consequently instantiate a score function based on the chosen approximation. The experimental results demonstrate that the proposed strategy and score function outperform conventional multi-label feature selection methods.

Meditation, a self-regulation practice, attracts attention on its physiological and psychological effects, and its effects on nonlinear complexity of heartbeats are analyzed in our contribution. Heartbeats of Chinese Chi, Kundalini Yoga practitioners and non-meditation metronomic breathing subjects, all from physionet, contribute to our research. We introduce the meditative heart rates in time and frequency domains, and extract their nonlinear dynamics by local dynamic and global static symbolic time-series analysis. Shannon entropy for each individual symbolization and joint entropy for the two kinds of symbolic transformations are applied to detect heartbeats’ nonlinear complexity. Test results prove that symbolic entropy is an exquisite quantitative analysis to characterize and distinguish nonlinear complexity of the physiological signals. Heart rates, during meditation, show highest regularities and lowest nonlinear complexities, and meditators, even in pre-meditation status, have lower heartbeats nonlinear complexity than the non-meditation volunteers, which suggest that meditation enhances cardiac self-regulation and reduces the nonlinear dynamic complexity of heart rates.

A microsleep is a brief and an involuntary sleep-related loss of consciousness of up to 15 s. We investigated the performances of seven pairwise inter-channel relationships–covariance, Pearson’s correlation coefficient, wavelet cross-spectral power, wavelet coherence, joint entropy, mutual information, and phase synchronization index–in continuous prediction of microsleep states from EEG. These relationships were used as the feature sets of a linear discriminant analysis (LDA) and a linear support vector machine classifiers. Priors for both classifiers were incorporated to address the class imbalance in the training data sets. Each feature set was extracted from a 5-s window of EEG with the step of 0.25 s and was demeaned with respect to the mean of first 2 min. The sequential forward selection (SFS) method, based on a serial combination of the correlation coefficient, Fisher score-based filter, and an LDA-based wrapper, was used to select features from each training set. The comparison was based on 16-channel EEG data from eight subjects who had performed a 1-D visuomotor task for two 1-h sessions. The prediction performances were evaluated using leave-one-subject-out cross-validation. For both classifiers, non-normalized feature sets were found to perform better than normalized feature sets. Furthermore, demeaning the non-normalized features considerably improved the prediction performance. Overall, the LDA classifier with joint entropy features resulted in the best average prediction performances (phi, AUCPR, and AUCROC) of (0.47, 0.50, and 0.95). Joint entropy between O1 and O2 from theta frequency band was the most informative feature.

Cyber Physical Systems (CPS) represent a technology that connects the physical world and the virtual world to intelligently enhance the operational environment they are operating in. CPS represent complex systems that potentially offer a variety of features like improving the reliability, cost efficiency or maintainability of physical systems. Nevertheless, these improvements are still accompanied by unprecedented complexity, high uncertainty, and overall vulnerability of the CPS. This paper provides an analysis of CPS as sources of dynamic complexity in Cyber-Physical-System of Systems (CPSS) via networks of information transferring, storing, and generating agents. Dynamic system complexity and the impact on the CPS and CPSS is explored via Shannon Entropy and joint entropy. As a final step, a trade-off relationship between the benefits and drawbacks of increased dynamic complexity in CPS and CPSS is presented and leads to the conclusion that both are effective tools for value generation in which dynamic complexity leads to decreased system efficiency.

A new solution to overcome the constraints of multimodality medical intra-subject image registration is proposed, using the mutual information (MI) of image histogram-oriented gradients as a new matching criterion. We present a rigid, multi-modal image registration algorithm based on linear transformation and oriented gradients for the alignment of T2-weighted (T2w) images (as a fixed reference) and diffusion tensor imaging (DTI) (b-values of 500 and 1250 s/mm2) as floating images of three patients to compensate for the motion during the acquisition process. Diffusion MRI is very sensitive to motion, especially when the intensity and duration of the gradient pulses (characterized by the b-value) increases. The proposed method relies on the whole brain surface and addresses the variability of anatomical features into an image stack. The sparse features refer to corners detected using the Harris corner detector operator, while dense features use all image pixels through the image histogram of oriented gradients (HOG) as a measure of the degree of statistical dependence between a pair of registered images. HOG as a dense feature is focused on the structure and extracts the oriented gradient image in the x and y directions. MI is used as an objective function for the optimization process. The entropy functions and joint entropy function are determined using the HOGs data. To determine the best image transformation, the fiducial registration error (FRE) measure is used. We compare the results against the MI-based intensities results computed using a statistical intensity relationship between corresponding pixels in source and target images. Our approach, which is devoted to the whole brain, shows improved registration accuracy, robustness, and computational cost compared with the registration algorithms, which use anatomical features or regions of interest areas with specific neuroanatomy. Despite the supplementary HOG computation task, the computation time is comparable for MI-based intensities and MI-based HOG methods.

This paper uses quantitative eye tracking indicators to analyze the relationship between images of paintings and human viewing. First, we build the eye tracking fixation sequences through areas of interest (AOIs) into an information channel, the gaze channel. Although this channel can be interpreted as a generalization of a first-order Markov chain, we show that the gaze channel is fully independent of this interpretation, and stands even when first-order Markov chain modeling would no longer fit. The entropy of the equilibrium distribution and the conditional entropy of a Markov chain are extended with additional information-theoretic measures, such as joint entropy, mutual information, and conditional entropy of each area of interest. Then, the gaze information channel is applied to analyze a subset of Van Gogh paintings. Van Gogh artworks, classified by art critics into several periods, have been studied under computational aesthetics measures, which include the use of Kolmogorov complexity and permutation entropy. The gaze information channel paradigm allows the information-theoretic measures to analyze both individual gaze behavior and clustered behavior from observers and paintings. Finally, we show that there is a clear correlation between the gaze information channel quantities that come from direct human observation, and the computational aesthetics measures that do not rely on any human observation at all This work is supported by the National Natural Science Foundation of China under grant No. 61702359, and by grant TIN2016-75866-C3-3-R from Spanish Government.

First, information theory is reviewed for readers who are unfamiliar with basic concepts and ideas. Information of events and entropy of sample spaces and/or random variables are introduced and the properties are discussed. To measure the differences between distributions, the Kullback–Leibler (KL) information is explained through a theoretical consideration. In order to treat the association between random variables, the joint entropy and the conditional entropy are also introduced. Second, basic statistics are considered from a viewpoint of entropy. The log-likelihood ratio test statistic is interpreted in entropy. By introducing entropy of continuous variables, t and F statistics are also discussed through KL information.

This paper proposes an efficient method to compute entropy and joint entropy of images. Entropy of images is used to determine its quality. It is defined as the randomness or uncertainty present in the image. Similarly, joint entropy is a measure of the uncertainty present in the overlapped region of two images. Entropy and joint entropy computations are vital in several image processing applications. Intensity based image registration is done by maximizing the mutual information between two images. Mutual information is nothing but the difference between sum of individual entropies and joint entropy of two images. Image registration has applications, especially in the medical field, e.g. diagnosis and treatment of diseases. The entropy and joint entropy computation methods proposed in this paper are computationally less expensive than the standard methods. Entropy computation takes 78.60% less time than the standard method while computational time of joint entropy is reduced by 83.59%. This increase in efficiency comes at the cost of an error of 1.52% in entropy and 4.54% in joint entropy.

We propose a new definition of entropy of basic probability assignments (BPAs) in the Dempster–Shafer (DS) theory of belief functions, which is interpreted as a measure of total uncertainty in the BPA. Our definition is different from those proposed by Hohle, Smets, Yager, Nguyen, Dubois–Prade, Lamata–Moral, Klir–Ramer, Klir–Parviz, Pal et al., Maeda–Ichihashi, Harmanec–Klir, Abellan–Moral, Jousselme et al., Pouly et al., and Deng. We state a list of six desired properties of entropy for DS belief functions theory, four of which are motivated by Shannon's definition of entropy of probability functions, and the remaining two are requirements that adapt this measure to the philosophy of the DS theory. Three of our six desired properties are different from the five properties proposed by Klir and Wierman. We demonstrate that our definition satisfies all six properties in our list, whereas none of the existing definitions do. Our new definition has two components. The first component is Shannon's entropy of an equivalent probability mass function obtained using the plausibility transform, which constitutes the conflict measure of entropy. The second component is Dubois–Prade's definition of entropy of basic probability assignments in the DS theory, which constitutes the non-specificity measure of entropy. Our new definition is the sum of these two components. Our definition does not satisfy the subadditivity property. Whether there exists a definition that satisfies our six properties plus subadditivity remains an open question.

Shannon's entropy and its variants have been applied to measure uncertainty in a variety of special binary relations. However, few studies have been conducted on uncertainty of general binary relations. In this study, we present a unified form of uncertainty measures for general binary relations. We redefine the concepts of entropy, joint entropy, conditional entropy, and mutual information. These uncertainty measures are generalizations of corresponding measures of special relations. We study the relationship between these measures and examine important properties. Finally, numerical experiments are performed to identify applications of the proposed uncertainty measures. Comparing with existing uncertainty measures, the proposed method not only addresses the uncertainty of heterogeneous data sets, but also exhibit better performance in attribute reduction. This study can provide a fundamental framework for uncertainty theories of special rough set models.

Cloud computing provides enormous storage capacity and huge computation power. Cloud enables users to deploy data-intensive applications without substructure investment. Intermediate data are generated under the cloud applications and stored again in cloud. An opponent may analyse multiple intermediate data sets to access the privacy sensitive information.The proposed technique to have good privacy and utility trade off, joint entropy with adaptive optimization process is used to maintain the privacy in cloud. Here optimal entropy value process using adaptive particle swarm optimization (APSO) process. After get the optimal entropy database difference model was processed, Entropy and database difference ratio is taken as the evaluation matrices and is tested using various datasets. The technique is also compared to the existing PSO optimization process on privacy preservation in cloud. An adaption feature of Particle swarm optimization show that APSO enhances privacy and preserving cost of intermediate data sets can be significantly saved.

In this paper, we perform a multiscale entropy analysis on the Dow Jones Industrial Average Index using the Shannon entropy. The stock index shows the characteristic of multi-scale entropy that caused by noise in the market. The entropy is demonstrated to have significant predictive ability for the stock index in both long-term and short-term, and empirical results verify that noise does exist in the market and can affect stock price. It has important implications on market participants such as noise traders.

Extreme precipitation events have been increasing in recent decades and have been causing impacts to human life and property, which requires expansion of existing networks to provide essential information for hydrometeorological analysis. This study developed an entropy and copula-based approach for precipitation monitoring network expansion, aiming at adding stations in ungauged areas with high value of monitoring (VOM), which was estimated through information content and redundancy. The approach was applied to a network in the Taihu Lake basin, China. We grouped the adjacent stations based on Thiessen polygons, computed VOM for each group and identified candidate locations for additional stations. The entropy measures, e.g., joint entropy and total correlation, required to compute VOM were estimated through the principle of maximum entropy (POME) and copula-based joint distribution fitting. We evaluated the performance of five copula families and adopted the Student t-copula as the optimal copula model. Results showed that orographic effect and network density were the main factors that influence VOM variation, i.e., high VOM appeared in the southwest mountain area and the east area with sparsely distributed stations. We also compared the results with those derived from three data quantization methods, among which the rounding technique produced consistent results. The final network schemes were generated by adding stations following a strategy that supports decision preference on the positioning scheme and station number, and were evaluated through the Kriging interpolation error. The proposed approach provides an efficient and a practical way for strengthening a precipitation monitoring system.

This paper proposes some new distance measures between interval-valued neutrosophic sets (IvNSs) and their similarity measures. Then, some entropy measures of IvNS based on the distances are proposed as the extension of the entropy measures of interval-valued intuitionistic fuzzy sets (IvIFSs). Also, we investigate the relationship between the presented entropy measures and the similarity measures for IvNSs. Finally, the comparison of the new entropy measures with existing entropy measures for IvNSs is given by the numerical and decision-making examples to demonstrate that the proposed new entropy measures for IvNSs are effective and reasonable and more intelligible in representing the degree of fuzziness of IvNSs than the existing ones.

Background The Electroencephalogram (EEG) signal is a time series depictive signal that contains the useful knowledge about the state of the brain. It has high temporal resolution for detection of chronic brain disorders such as epilepsy/seizure, dementia, sleep apnea, schizophrenia, etc. In this work, EEG is a prime concern for seizure/epilepsy detection and localization. Methods Entropy estimator is a good solution to this problem. Here, the time series complexity analysis of brain signal is carried using five different entropy estimators: Shannon Entropy, Renyi Entropy, Approximate Entropy, Sample Entropy, and Fuzzy Entropy. The average entropy values of EEG signal is significantly found lower for epileptic data sets compared to non-epileptic EEG. Results Experimental results evaluated for discriminating ability of each entropy measure demonstrated that among all entropies, Fuzzy Entropy discriminates between epileptic EEG and non-epileptic EEG more efficiently with a minimum p-value (0.001) compared to other four entropy estimators. Fuzzy Entropy defines the similarity between two vectors fuzzily on the basis of exponential function. Unlike to Approximate and Sample Entropy, the Fuzzy Entropy is free from parameter limitations and offers efficient results even for the small tolerance (r Conclusion The Fuzzy entropy gives better stability and consistency with highest discriminating ability compared to other entropy estimators. Eventually, results for detection and localization of epilepsy for affected channel and region through the variance and FuzzyEn are cross-validated by expert Neuro-physician.

Anomaly-based Intrusion Detection is a key research topic in network security due to its ability to face unknown attacks and new security threats. For this reason, many works on the topic have been proposed in the last decade. Nonetheless, an ultimate solution, able to provide a high detection rate with an acceptable false alarm rate, has still to be identified. In this paper we propose a novel intrusion detection system that performs anomaly detection by studying the variation in the entropy associated to the network traffic. To this aim, the traffic is first aggregated by means of random data structures (namely three-dimension reversible sketches) and then the entropy of different traffic descriptors is computed by using several definitions. The experimental results obtained over the MAWILab dataset validate the system and demonstrate the effectiveness of our proposal for a proper set of entropy definitions.

Partial entropy is a device to measure how much of entropy of an uncertain random variable belongs to uncertain variables. However, partial entropy may fail to measure the uncertainty of an uncertain random variable. In this paper, for refining this problem, a definition of partial triangular entropy is presented. In addition, several properties of partial triangular entropy are obtained. Moreover, partial triangular entropy of an uncertain random variable is applied to mean-variance portfolio selection.

In this paper, the exponential weighted entropy (EWE) and exponential weighted mutual information (EWMI) are proposed as the more generalized forms of Shannon entropy and mutual information (MI), respectively. They are position-related and causal systems that redefine the foundations of information-theoretic metrics. As the special forms of the weighted entropy and the weighted mutual information, EWE and EWMI have been proved that they preserve nonnegativity and concavity properties similar to Shannon frameworks. They can be adopted as the information measures in spatial interaction modeling. Paralleling with the normalized mutual information (NMI), the normalized exponential weighted mutual information (NEWMI) is also investigated. Image registration experiments demonstrate that EWMI and NEWMI algorithms can achieve higher aligned accuracy than MI and NMI algorithms.

In this paper, we introduce a quantity measure which is called (R, S)-norm entropy and discuss some of its major properties with Shannon’s and other entropies in the literature. Based on this (R, S)-norm entropy, we have proposed a new (R, S)-norm fuzzy information measure and discussed its validity and properties. Further, we have given its comparison with other fuzzy information measures to prove its effectiveness. Attribute weights play an important role in multiple-attribute decision-making problems. In the present communication, two methods of determining the attribute weights are introduced. First is the case when the information regarding attribute weights is incompletely known or completely unknown and second is when we have partial information about attribute weights. For the first case, the extension of ordinary entropy weight method is used to calculate attribute weights and minimum entropy principle method based on solving a linear programming model is used in the second case. Finally, two methods are explained through numerical examples.

Adequate and accurate hydrologic information from optimal hydrometric networks is an essential part of effective water resources management. Although the key hydrologic processes in the water cycle are interconnected, hydrometric networks (e.g., streamflow, precipitation, groundwater level) have been routinely designed individually. A decision support framework is proposed for integrated design of multivariable hydrometric networks. The proposed method is applied to design optimal precipitation and streamflow networks simultaneously. The epsilon-dominance hierarchical Bayesian optimization algorithm was combined with Shannon entropy of information theory to design and evaluate hydrometric networks. Specifically, the joint entropy from the combined networks was maximized to provide the most information, and the total correlation was minimized to reduce redundant information. To further optimize the efficiency between the networks, they were designed by maximizing the conditional entropy of the streamflow network given the information of the precipitation network. Compared to the traditional individual variable design approach, the integrated multivariable design method was able to determine more efficient optimal networks by avoiding the redundant stations. Additionally, four quantization cases were compared to evaluate their effects on the entropy calculations and the determination of the optimal networks. The evaluation results indicate that the quantization methods should be selected after careful consideration for each design problem since the station rankings and the optimal networks can change accordingly.

Evolutionary algorithms are used widely in optimization studies on water distribution networks. The optimization algorithms use simulation models that analyse the networks under various operating conditions. The solution process typically involves cost minimization along with reliability constraints that ensure reasonably satisfactory performance under abnormal operating conditions also. Flow entropy has been employed previously as a surrogate reliability measure. While a body of work exists for a single operating condition under steady state conditions, the effectiveness of flow entropy for systems with multiple operating conditions has received very little attention. This paper describes a multi-objective genetic algorithm that maximizes the flow entropy under multiple operating conditions for any given network. The new methodology proposed is consistent with the maximum entropy formalism that requires active consideration of all the relevant information. Furthermore, an alternative but equivalent flow entropy model that emphasizes the relative uniformity of the nodal demands is described. The flow entropy of water distribution networks under multiple operating conditions is discussed with reference to the joint entropy of multiple probability spaces, which provides the theoretical foundation for the optimization methodology proposed. Besides the rationale, results are included that show that the most robust or failure-tolerant solutions are achieved by maximizing the sum of the entropies.

In this article, the concept of cumulative residual entropy (CRE) given by Rao et al. (2004) is extended to Tsallis entropy function and dynamic version, both residual and past of it. We study some properties and characterization results for these generalized measures. In addition, we provide some characterization results of the first order statistic based on the Tsallis survival entropy.

SYNOPTIC ABSTRACTAbbasnejad (2011) proposed new measures of information known as failure and dynamic failure entropy of order α based on generalization of the well-known cumulative entropy. In this article, we propose the weighted forms of failure entropy (WFE) and dynamic failure entropy (DWFE) of order α. Some relationships of these measures with well-known reliability measures and ageing classes are studied and some characterization results for reflected Weibull distribution are provided. We also propose an empirical estimator for the WDFE of order α. We demonstrate the performance of the proposed estimator with a real life data set and further carried out a comprehensive Monte-Carlo simulation study.

Considering the effects of environment disturbances and instrument systems, the actual detecting signals always are carrying different trends, which result in that it is difficult to accurately catch signals complexity. So choosing steady and effective analysis methods is very important. In this paper, we applied entropy measures—the base-scale entropy and approximate entropy to analyze signal complexity, and studied the effect of trends on the ideal signal and the heart rate variability (HRV) signals, that is, linear, periodic, and power-law trends which are likely to occur in actual signals. The results show that approximate entropy is unsteady when we embed different trends into the signals, so it is not suitable to analyze signal with trends. However, the base-scale entropy has preferable stability and accuracy for signal with different trends. So the base-scale entropy is an effective method to analyze the actual signals.

Shannon entropy is being increasingly used in biomedical research as an index of complexity and information content in sequences of symbols, e.g. languages, amino acid sequences, DNA methylation patterns and animal vocalizations. Yet, distributional properties of information entropy as a random variable have seldom been the object of study, leading to researchers mainly using linear models or simulation-based analytical approach to assess differences in information content, when entropy is measured repeatedly in different experimental conditions. Here a method to perform inference on entropy in such conditions is proposed. Building on results coming from studies in the field of Bayesian entropy estimation, a symmetric Dirichlet-multinomial regression model, able to deal efficiently with the issue of mean entropy estimation, is formulated. Through a simulation study the model is shown to outperform linear modeling in a vast range of scenarios and to have promising statistical properties. As a practical example, the method is applied to a data set coming from a real experiment on animal communication.