Search

We feature papers on recent developments in the areas of adaptive signal processing, machine learning, and deep learning using information theory and entropy to improve performance in widespread and popular problems, and also to provide effective solutions to emerging problems. Our goal is to publish recent developments in the areas of adaptive signal processing, machine learning, and deep learning using information theory and entropy to improve performance in widespread and popular problems, and also to provide effective solutions to emerging problems. This Special Issue on "Adaptive Signal Processing and Machine Learning Using Entropy and Information Theory" was birthed from observations of the recent trend in the literature. Entropy-based cost functions have replaced mean-square-error (MSE)-based ones and have been widely used in adaptive signal processing and machine learning to improve performance by designing and optimizing effective and specific models that fit the data, even in noisy and adverse conditions. [Extracted from the article]

Medical image diagnosis using deep learning has shown significant promise in clinical medicine. However, it often encounters two major difficulties in real-world applications: (1) domain shift, which invalidates the trained model on new datasets, and (2) class imbalance problems leading to model biases towards majority classes. To address these challenges, this paper proposes a transfer learning solution, named Dynamic Weighting Translation Transfer Learning (DTTL), for imbalanced medical image classification. The approach is grounded in information and entropy theory and comprises three modules: Cross-domain Discriminability Adaptation (CDA), Dynamic Domain Translation (DDT), and Balanced Target Learning (BTL). CDA connects discriminative feature learning between source and target domains using a synthetic discriminability loss and a domain-invariant feature learning loss. The DDT unit develops a dynamic translation process for imbalanced classes between two domains, utilizing a confidence-based selection approach to select the most useful synthesized images to create a pseudo-labeled balanced target domain. Finally, the BTL unit performs supervised learning on the reassembled target set to obtain the final diagnostic model. This paper delves into maximizing the entropy of class distributions, while simultaneously minimizing the cross-entropy between the source and target domains to reduce domain discrepancies. By incorporating entropy concepts into our framework, our method not only significantly enhances medical image classification in practical settings but also innovates the application of entropy and information theory within deep learning and medical image processing realms. Extensive experiments demonstrate that DTTL achieves the best performance compared to existing state-of-the-art methods for imbalanced medical image classification tasks. [ABSTRACT FROM AUTHOR]

Underwriters play a pivotal role in the IPO process. Information entropy, a tool for measuring the uncertainty and complexity of information, has been widely applied to various issues in complex networks. Information entropy can quantify the uncertainty and complexity of nodes in the network, providing a unique analytical perspective and methodological support for this study. This paper employs a bipartite network analysis method to construct the relationship network between underwriters and accounting firms, using the centrality of underwriters in the network as a measure of their influence to explore the impact of underwriters' influence on the distribution of interests and audit outcomes. The findings indicate that a more pronounced influence of underwriters significantly increases the ratio of underwriting fees to audit fees. Higher influence often accompanies an increase in abnormal underwriting fees. Further research reveals that companies underwritten by more influential underwriters experience a decline in audit quality. Finally, the study reveals that a well-structured audit committee governance and the rationalization of market sentiments can mitigate the negative impacts of underwriters' influence. The innovation of this paper is that it enriches the content related to underwriters by constructing the relationship network between underwriters and accounting firms for the first time using a bipartite network through the lens of information entropy. This conclusion provides new directions for thinking about the motives and possibilities behind financial institutions' cooperation, offering insights for market regulation and policy formulation. [ABSTRACT FROM AUTHOR]

Traffic state classification and relevance calculation at intersections are both difficult problems in traffic control. In this paper, we propose an intersection relevance model based on a temporal graph attention network, which can solve the above two problems at the same time. First, the intersection features and interaction time of the intersections are regarded as input quantities together with the initial labels of the traffic data. Then, they are inputted into the temporal graph attention (TGAT) model to obtain the classification accuracy of the target intersections in four states—free, stable, slow moving, and congested—and the obtained neighbouring intersection weights are used as the correlation between the intersections. Finally, it is validated by VISSIM simulation experiments. In terms of classification accuracy, the TGAT model has a higher classification accuracy than the three traditional classification models and can cope well with the uneven distribution of the number of samples. The information gain algorithm from the information entropy theory was used to derive the average delay as the most influential factor on intersection status. The correlation from the TGAT model positively correlates with traffic flow, making it interpretable. Using this correlation to control the division of subareas improves the road network's operational efficiency more than the traditional correlation model does. This demonstrates the effectiveness of the TGAT model's correlation. [ABSTRACT FROM AUTHOR]

This paper presents a multivariate textual analysis of more than 1300 papers on entropy in ecology. There are six main insights that emerged. First, there is a large body of literature that has addressed some aspect of entropy in ecology, most of which has been published in the last 5–10 years. Second, the vast majority of these papers focus on species distribution, species richness, relative abundance or trophic structure and not landscape-scale patterns or processes, pe se. Third, there have been few papers addressing landscape-level questions related to entropy. Fourth, the quantitative analysis with hierarchical clustering identified a strongly nested structure among papers that addressed entropy in ecology. Fifth, there is clear differentiation of papers focused on landscape-level applications of entropy from other papers, with landscape focused papers clustered together at each level of the hierarchy in a relatively small and closely associated group. Sixth, this group of landscape-focused papers was substructured between papers that explicitly adopted entropy measures to quantify the spatial pattern of landscape mosaics, often using variations on Boltzmann entropy, versus those that utilize Shannon entropy measures from information theory, which are not generally explicit in their assessment of spatial configuration. This review provides a comprehensive, quantitative assessment of the scope, trends and relationships among a large body of literature related to entropy in ecology and for the first time puts landscape ecological research on entropy into that context. [ABSTRACT FROM AUTHOR]

This paper highlights that metrics from the machine learning field (e.g., entropy and information gain) used to qualify a classifier model can be used to evaluate the effectiveness of separation systems. To evaluate the efficiency of separation systems and their operation units, entropy- and information gain-based metrics were developed. The receiver operating characteristic (ROC) curve is used to determine the optimal cut point in a separation system. The proposed metrics are verified by simulation experiments conducted on the stochastic model of a waste-sorting system. [ABSTRACT FROM AUTHOR]

With the development of image recovery models, especially those based on adversarial and perceptual losses, the detailed texture portions of images are being recovered more naturally. However, these restored images are similar but not identical in detail texture to their reference images. With traditional image quality assessment methods, results with better subjective perceived quality often score lower in objective scoring. Assessment methods suffer from subjective and objective inconsistencies. This paper proposes a regional differential information entropy (RDIE) method for image quality assessment to address this problem. This approach allows better assessment of similar but not identical textural details and achieves good agreement with perceived quality. Neural networks are used to reshape the process of calculating information entropy, improving the speed and efficiency of the operation. Experiments conducted with this study's image quality assessment dataset and the PIPAL dataset show that the proposed RDIE method yields a high degree of agreement with people's average opinion scores compared with other image quality assessment metrics, proving that RDIE can better quantify the perceived quality of images. [ABSTRACT FROM AUTHOR]

In this paper, we provide geometric insights with visualization into the multivariate Gaussian distribution and its entropy and mutual information. In order to develop the multivariate Gaussian distribution with entropy and mutual information, several significant methodologies are presented through the discussion, supported by illustrations, both technically and statistically. The paper examines broad measurements of structure for the Gaussian distributions, which show that they can be described in terms of the information theory between the given covariance matrix and correlated random variables (in terms of relative entropy). The content obtained allows readers to better perceive concepts, comprehend techniques, and properly execute software programs for future study on the topic's science and implementations. It also helps readers grasp the themes' fundamental concepts to study the application of multivariate sets of data in Gaussian distribution. The simulation results also convey the behavior of different elliptical interpretations based on the multivariate Gaussian distribution with entropy for real-world applications in our daily lives, including information coding, nonlinear signal detection, etc. Involving the relative entropy and mutual information as well as the potential correlated covariance analysis, a wide range of information is addressed, including basic application concerns as well as clinical diagnostics to detect the multi-disease effects. [ABSTRACT FROM AUTHOR]

Entropy and the second law of thermodynamics are the central organizing principles of nature, but the ideas and implications of the second law are poorly developed in landscape ecology. The purpose of this Special Issue "Entropy in Landscape Ecology" in Entropy is to bring together current research on applications of thermodynamics in landscape ecology, to consolidate current knowledge and identify key areas for future research. The special issue contains six articles, which cover a broad range of topics including relationships between entropy and evolution, connections between fractal geometry and entropy, new approaches to calculate configurational entropy of landscapes, example analyses of computing entropy of landscapes, and using entropy in the context of optimal landscape planning. Collectively these papers provide a broad range of contributions to the nascent field of ecological thermodynamics. Formalizing the connections between entropy and ecology are in a very early stage, and that this special issue contains papers that address several centrally important ideas, and provides seminal work that will be a foundation for the future development of ecological and evolutionary thermodynamics. [ABSTRACT FROM AUTHOR]

In the theory of special relativity, energy can be found in two forms: kinetic energy and rest mass. The potential energy of a body is actually stored in the form of rest mass, the interaction energy too, but temperature is not. Information acquired about a dynamical system can be potentially used to extract useful work from it. Hence, the "mass–energy–information equivalence principle" that has been recently proposed. In this paper, it is first recalled that for a thermodynamic system made of non-interacting entities at constant temperature, the internal energy is also constant. So, the energy involved in a variation in entropy ( T Δ S ) differs from a change in the potential energy stored or released and cannot be associated to a corresponding variation in mass of the system, even if it is expressed in terms of the quantity of information. This debate gives us the opportunity to deepen the notion of entropy seen as a quantity of information, to highlight the difference between logical irreversibility (a state-dependent property) and thermodynamical irreversibility (a path-dependent property), and to return to the nature of the link between energy and information that is dynamical. [ABSTRACT FROM AUTHOR]

Mobile robot olfaction of toxic and hazardous odor sources is of great significance in anti-terrorism, disaster prevention, and control scenarios. Aiming at the problems of low search efficiency and easily falling into a local optimum of the current odor source localization strategies, the paper proposes the adaptive space-aware Infotaxis II algorithm. To improve the tracking efficiency of robots, a new reward function is designed by considering the space information and emphasizing the exploration behavior of robots. Considering the enhancement in exploratory behavior, an adaptive navigation-updated mechanism is proposed to adjust the movement range of robots in real time through information entropy to avoid an excessive exploration behavior during the search process, which may lead the robot to fall into a local optimum. Subsequently, an improved adaptive cosine salp swarm algorithm is applied to confirm the optimal information adaptive parameter. Comparative simulation experiments between ASAInfotaxis II and the classical search strategies are carried out in 2D and 3D scenarios regarding the search efficiency and search behavior, which show that ASAInfotaxis II is competent to improve the search efficiency to a larger extent and achieves a better balance between exploration and exploitation behaviors. [ABSTRACT FROM AUTHOR]

Recent studies on watermarking techniques based on image carriers have demonstrated new approaches that combine adversarial perturbations against steganalysis with embedding distortions. However, while these methods successfully counter convolutional neural network-based steganalysis, they do not adequately protect the data of the carrier itself. Recognizing the high sensitivity of Deep Neural Networks (DNNs) to small perturbations, we propose HAG-NET, a method based on image carriers, which is jointly trained by the encoder, decoder, and attacker. In this paper, the encoder generates Adversarial Steganographic Examples (ASEs) that are adversarial to the target classification network, thereby providing protection for the carrier data. Additionally, the decoder can recover secret data from ASEs. The experimental results demonstrate that ASEs produced by HAG-NET achieve an average success rate of over 99% on both the MNIST and CIFAR-10 datasets. ASEs generated with the attacker exhibit greater robustness in terms of attack ability, with an average increase of about 3.32%. Furthermore, our method, when compared with other generative stego examples under similar perturbation strength, contains significantly more information according to image information entropy measurements. [ABSTRACT FROM AUTHOR]

Few-shot learning aims to solve the difficulty in obtaining training samples, leading to high variance, high bias, and over-fitting. Recently, graph-based transductive few-shot learning approaches supplement the deficiency of label information via unlabeled data to make a joint prediction, which has become a new research hotspot. Therefore, in this paper, we propose a novel ensemble semi-supervised few-shot learning strategy via transductive network and Dempster–Shafer (D-S) evidence fusion, named ensemble transductive propagation networks (ETPN). First, we present homogeneity and heterogeneity ensemble transductive propagation networks to better use the unlabeled data, which introduce a preset weight coefficient and provide the process of iterative inferences during transductive propagation learning. Then, we combine the information entropy to improve the D-S evidence fusion method, which improves the stability of multi-model results fusion from the pre-processing of the evidence source. Third, we combine the L2 norm to improve an ensemble pruning approach to select individual learners with higher accuracy to participate in the integration of the few-shot model results. Moreover, interference sets are introduced to semi-supervised training to improve the anti-disturbance ability of the mode. Eventually, experiments indicate that the proposed approaches outperform the state-of-the-art few-shot model. The best accuracy of ETPN increases by 0.3% and 0.28% in the 5-way 5-shot, and by 3.43% and 7.6% in the 5-way 1-shot on miniImagNet and tieredImageNet, respectively. [ABSTRACT FROM AUTHOR]

Identifying influential nodes is a key research topic in complex networks, and there have been many studies based on complex networks to explore the influence of nodes. Graph neural networks (GNNs) have emerged as a prominent deep learning architecture, capable of efficiently aggregating node information and discerning node influence. However, existing graph neural networks often ignore the strength of the relationships between nodes when aggregating information about neighboring nodes. In complex networks, neighboring nodes often do not have the same influence on the target node, so the existing graph neural network methods are not effective. In addition, the diversity of complex networks also makes it difficult to adapt node features with a single attribute to different types of networks. To address the above problems, the paper constructs node input features using information entropy combined with the node degree value and the average degree of the neighbor, and proposes a simple and effective graph neural network model. The model obtains the strength of the relationships between nodes by considering the degree of neighborhood overlap, and uses this as the basis for message passing, thereby effectively aggregating information about nodes and their neighborhoods. Experiments are conducted on 12 real networks, using the SIR model to verify the effectiveness of the model with the benchmark method. The experimental results show that the model can identify the influence of nodes in complex networks more effectively. [ABSTRACT FROM AUTHOR]

The safe and comfortable operation of high-speed trains has attracted extensive attention. With the operation of the train, the performance of high-speed train bogie components inevitably degrades and eventually leads to failures. At present, it is a common method to achieve performance degradation estimation of bogie components by processing high-speed train vibration signals and analyzing the information contained in the signals. In the face of complex signals, the usage of information theory, such as information entropy, to achieve performance degradation estimations is not satisfactory, and recent studies have more often used deep learning methods instead of traditional methods, such as information theory or signal processing, to obtain higher estimation accuracy. However, current research is more focused on the estimation for a certain component of the bogie and does not consider the bogie as a whole system to accomplish the performance degradation estimation task for several key components at the same time. In this paper, based on soft parameter sharing multi-task deep learning, a multi-task and multi-scale convolutional neural network is proposed to realize performance degradation state estimations of key components of a high-speed train bogie. Firstly, the structure takes into account the multi-scale characteristics of high-speed train vibration signals and uses a multi-scale convolution structure to better extract the key features of the signal. Secondly, considering that the vibration signal of high-speed trains contains the information of all components, the soft parameter sharing method is adopted to realize feature sharing in the depth structure and improve the utilization of information. The effectiveness and superiority of the structure proposed by the experiment is a feasible scheme for improving the performance degradation estimation of a high-speed train bogie. [ABSTRACT FROM AUTHOR]

This paper proposes an encryption scheme for high pixel density images. Based on the application of the quantum random walk algorithm, the long short-term memory (LSTM) can effectively solve the problem of low efficiency of the quantum random walk algorithm in generating large-scale pseudorandom matrices, and further improve the statistical properties of the pseudorandom matrices required for encryption. The LSTM is then divided into columns and fed into the LSTM in order for training. Due to the randomness of the input matrix, the LSTM cannot be trained effectively, so the output matrix is predicted to be highly random. The LSTM prediction matrix of the same size as the key matrix is generated based on the pixel density of the image to be encrypted, which can effectively complete the encryption of the image. In the statistical performance test, the proposed encryption scheme achieves an average information entropy of 7.9992, an average number of pixels changed rate (NPCR) of 99.6231%, an average uniform average change intensity (UACI) of 33.6029%, and an average correlation of 0.0032. Finally, various noise simulation tests are also conducted to verify its robustness in real-world applications where common noise and attack interference are encountered. [ABSTRACT FROM AUTHOR]

This paper proposes a new string transformation technique called Move with Interleaving (MwI). Four possible ways of rearranging 2D raster images into 1D sequences of values are applied, including scan-line, left-right, strip-based, and Hilbert arrangements. Experiments on 32 benchmark greyscale raster images of various resolutions demonstrated that the proposed transformation reduces information entropy to a similar extent as the combination of the Burrows–Wheeler transform followed by the Move-To-Front or the Inversion Frequencies. The proposed transformation MwI yields the best result among all the considered transformations when the Hilbert arrangement is applied. [ABSTRACT FROM AUTHOR]

In a paper published in this journal, I addressed the following problem: under which conditions will two scientists, observing the same system and sharing the same initial information, reach the same probabilistic description upon the application of the Maximum Entropy inference principle (MaxEnt) independent of the probability distribution chosen to set up the MaxEnt procedure. This is a minimal objectivity requirement which is generally asked for scientific investigation. In the same paper, I applied the findings to a critical examination of the application of MaxEnt made in Harte's Maximum Entropy Theory of Ecology (METE). Prof. Harte published a comment to my paper and this is my reply. For the sake of the reader who may be unaware of the content of the papers, I have tried to make this reply self-contained and to skip technical details. However, I invite the interested reader to consult the previously published papers. [ABSTRACT FROM AUTHOR]

In this paper, a new combinatorial structure is introduced for image encryption, which has an excellent encryption effect on security and efficiency. An n-transversal in a Latin square has the function of classifying all the matrix's positions, and it can provide a pair of orthogonal Latin squares. Employing an n-transversal of a Latin square, we can permutate all the pixels of an image group by group for the first time, then use two Latin squares for auxiliary diffusion based on a chaotic sequence, and finally, make use of a pair of orthogonal Latin squares to perform the second scrambling. The whole encryption process is "scrambling–diffusion–scrambling". The experimental results indicated that this algorithm passed various tests and achieved a secure and fast encryption effect, which outperformed many of the latest papers. The final information entropy was very close to 8, and the correlation coefficient was approximately 0. All these tests verified the robustness and practicability of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Outlier detection is an important task in the field of data mining and a highly active area of research in machine learning. In industrial automation, datasets are often high-dimensional, meaning an effort to study all dimensions directly leads to data sparsity, thus causing outliers to be masked by noise effects in high-dimensional spaces. The "curse of dimensionality" phenomenon renders many conventional outlier detection methods ineffective. This paper proposes a new outlier detection algorithm called EOEH (Ensemble Outlier Detection Method Based on Information Entropy-Weighted Subspaces for High-Dimensional Data). First, random secondary subsampling is performed on the data, and detectors are run on various small-scale sub-samples to provide diverse detection results. Results are then aggregated to reduce the global variance and enhance the robustness of the algorithm. Subsequently, information entropy is utilized to construct a dimension-space weighting method that can discern the influential factors within different dimensional spaces. This method generates weighted subspaces and dimensions for data objects, reducing the impact of noise created by high-dimensional data and improving high-dimensional data detection performance. Finally, this study offers a design for a new high-precision local outlier factor (HPLOF) detector that amplifies the differentiation between normal and outlier data, thereby improving the detection performance of the algorithm. The feasibility of this algorithm is validated through experiments that used both simulated and UCI datasets. In comparison to popular outlier detection algorithms, our algorithm demonstrates a superior detection performance and runtime efficiency. Compared with the current popular, common algorithms, the EOEH algorithm improves the detection performance by 6% on average. In terms of running time for high-dimensional data, EOEH is 20% faster than the current popular algorithms. [ABSTRACT FROM AUTHOR]

Market uncertainty has a significant impact on market performance. Previous studies have dedicated much effort towards investigations into market uncertainty related to information asymmetry and risk. However, they have neglected the uncertainty inherent in market transactions, which is also an important aspect of market performance, besides the quantity of transactions and market efficiency. In this paper, we put forward a concept of transaction entropy to measure market uncertainty and see how it changes with price. Transaction entropy is defined as the ratio of the total information entropy of all traders to the quantity of transactions, reflecting the level of uncertainty in making successful transactions. Based on the computational and simulated results, our main finding is that transaction entropy is the lowest at equilibrium, it will decrease in a shortage market, and increase in a surplus market. Additionally, we make a comparison of the total entropy of the centralized market with that of the decentralized market, revealing that the price-filtering mechanism could effectively reduce market uncertainty. Overall, the introduction of transaction entropy enriches our understanding of market uncertainty and facilitates a more comprehensive assessment of market performance. [ABSTRACT FROM AUTHOR]

Vibrations of defective gearboxes show great complexities. Therefore, dynamics and noise levels of vibrations of gearboxes vary with operation of gearboxes. As a result, nonlinearity and determinism of data can serve to describe running conditions of gearboxes. However, measuring of nonlinearity and determinism of data is challenging. This paper defines a two-dimensional measure for simultaneously quantifying nonlinearity and determinism of data by comparing generalized Hurst exponents of original, shuffle and surrogate data. Afterwards, this paper proposes a novel method for fault diagnosis of gearboxes using the two-dimensional measure. Robustness of the proposed method was validated numerically by analyzing simulative signals with different noise levels. Moreover, the performance of the proposed method was benchmarked against Approximate Entropy, Sample Entropy, Permutation Entropy and Delay Vector Variance by conducting two independent gearbox experiments. The results show that the proposed method achieves superiority over the others in fault diagnosis of gearboxes. [ABSTRACT FROM AUTHOR]

Click-through rate (CTR) prediction is a research point for measuring recommendation systems and calculating AD traffic. Existing studies have proved that deep learning performs very well in prediction tasks, but most of the existing studies are based on deterministic models, and there is a big gap in capturing uncertainty. Modeling uncertainty is a major challenge when using machine learning solutions to solve real-world problems in various domains. In order to quantify the uncertainty of the model and achieve accurate and reliable prediction results. This paper designs a CTR prediction framework combining feature selection and feature interaction. In this framework, a CTR prediction model based on Bayesian deep learning is proposed to quantify the uncertainty in the prediction model. On the squeeze network and DNN parallel prediction model framework, the approximate posterior parameter distribution of the model is obtained using the Monte Carlo dropout, and obtains the integrated prediction results. Epistemic and aleatoric uncertainty are defined and adopt information entropy to calculate the sum of the two kinds of uncertainties. Epistemic uncertainty could be measured by mutual information. Experimental results show that the model proposed is superior to other models in terms of prediction performance and has the ability to quantify uncertainty. [ABSTRACT FROM AUTHOR]

In underground industries, practitioners frequently employ argots to communicate discreetly and evade surveillance by investigative agencies. Proposing an innovative approach using word vectors and large language models, we aim to decipher and understand the myriad of argots in these industries, providing crucial technical support for law enforcement to detect and combat illicit activities. Specifically, positional differences in semantic space distinguish argots, and pre-trained language models' corpora are crucial for interpreting them. Expanding on these concepts, the article assesses the semantic coherence of word vectors in the semantic space based on the concept of information entropy. Simultaneously, we devised a labeled argot dataset, MNGG, and developed an argot recognition framework named CSRMECT, along with an argot interpretation framework called LLMResolve. These frameworks leverage the MECT model, the large language model, prompt engineering, and the DBSCAN clustering algorithm. Experimental results demonstrate that the CSRMECT framework outperforms the current optimal model by 10% in terms of the F1 value for argot recognition on the MNGG dataset, while the LLMResolve framework achieves a 4% higher accuracy in interpretation compared to the current optimal model.The related experiments undertaken also indicate a potential correlation between vector information entropy and model performance. [ABSTRACT FROM AUTHOR]

As a popular research subject in the field of computer vision, knowledge distillation (KD) is widely used in semantic segmentation (SS). However, based on the learning paradigm of the teacher–student model, the poor quality of teacher network feature knowledge still hinders the development of KD technology. In this paper, we investigate the output features of the teacher–student network and propose a feature condensation-based KD network (FCKDNet), which reduces pseudo-knowledge transfer in the teacher–student network. First, combined with the pixel information entropy calculation rule, we design a feature condensation method to separate the foreground feature knowledge from the background noise of the teacher network outputs. Then, the obtained feature condensation matrix is applied to the original outputs of the teacher and student networks to improve the feature representation capability. In addition, after performing feature condensation on the teacher network, we propose a soft enhancement method of features based on spatial and channel dimensions to improve the dependency of pixels in the feature maps. Finally, we divide the outputs of the teacher network into spatial condensation features and channel condensation features and perform distillation loss calculation with the student network separately to assist the student network to converge faster. Extensive experiments on the public datasets Pascal VOC and Cityscapes demonstrate that our proposed method improves the baseline by 3.16% and 2.98% in terms of mAcc, and 2.03% and 2.30% in terms of mIoU, respectively, and has better segmentation performance and robustness than the mainstream methods. [ABSTRACT FROM AUTHOR]

As a measure of complexity, information entropy is frequently used to categorize time series, such as machinery failure diagnostics, biological signal identification, etc., and is thought of as a characteristic of dynamic systems. Many entropies, however, are ineffective for multivariate scenarios due to correlations. In this paper, we propose a local structure entropy (LSE) based on the idea of a recurrence network. Given certain tolerance and scales, LSE values can distinguish multivariate chaotic sequences between stochastic signals. Three financial market indices are used to evaluate the proposed LSE. The results show that the L S E F S T E 100 and L S E S & P 500 are higher than L S E S Z I , which indicates that the European and American stock markets are more sophisticated than the Chinese stock market. Additionally, using decision trees as the classifiers, LSE is employed to detect bearing faults. LSE performs higher on recognition accuracy when compared to permutation entropy. [ABSTRACT FROM AUTHOR]

Compared with the strong background noise, the energy entropy of early fault signals of bearings are weak under actual working conditions. Therefore, extracting the bearings' early fault features has always been a major difficulty in fault diagnosis of rotating machinery. Based on the above problems, the masking method is introduced into the Local Mean Decomposition (LMD) decomposition process, and a weak fault extraction method based on LMD and mask signal (MS) is proposed. Due to the mode mixing of the product function (PF) components decomposed by LMD in the noisy background, it is difficult to distinguish the authenticity of the fault frequency. Therefore, the MS method is introduced to deal with the PF components that are decomposed by the LMD and have strong correlation with the original signal, so as to suppress the modal aliasing phenomenon and extract the fault frequencies. In this paper, the actual fault signal of the rolling bearing is analyzed. By combining the MS method with the LMD method, the fault signal mixed with the noise is processed. The kurtosis value at the fault frequency is increased by eight-fold, and the signal-to-noise ratio (SNR) is increased by 19.1%. The fault signal is successfully extracted by the proposed composite method. [ABSTRACT FROM AUTHOR]

The complex nature of the interlacement of economic actors is quite evident at the level of the Stock market, where any company may actually interact with the other companies buying and selling their shares. In this respect, the companies populating a Stock market, along with their connections, can be effectively modeled through a directed network, where the nodes represent the companies, and the links indicate the ownership. This paper deals with this theme and discusses the concentration of a market. A cross-shareholding matrix is considered, along with two key factors: the node out-degree distribution which represents the diversification of investments in terms of the number of involved companies, and the node in-degree distribution which reports the integration of a company due to the sales of its own shares to other companies. While diversification is widely explored in the literature, integration is most present in literature on contagions. This paper captures such quantities of interest in the two frameworks and studies the stochastic dependence of diversification and integration through a copula approach. We adopt entropies as measures for assessing the concentration in the market. The main question is to assess the dependence structure leading to a better description of the data or to market polarization (minimal entropy) or market fairness (maximal entropy). In so doing, we derive information on the way in which the in- and out-degrees should be connected in order to shape the market. The question is of interest to regulators bodies, as witnessed by specific alert threshold published on the US mergers guidelines for limiting the possibility of acquisitions and the prevalence of a single company on the market. Indeed, all countries and the EU have also rules or guidelines in order to limit concentrations, in a country or across borders, respectively. The calibration of copulas and model parameters on the basis of real data serves as an illustrative application of the theoretical proposal. [ABSTRACT FROM AUTHOR]

This paper constructs a continuous dual-channel closed-loop supply chain (DCLSC) model with delayed decision under government intervention. The existence conditions of the local stability of the equilibrium point are discussed. We analyze the influence of delay parameters, the adjustment speed of wholesale price, recovery rate of waste products, direct price, carbon quota subsidy, and carbon tax on the stability and complexity of model by using bifurcation diagram, entropy diagram, attractor, and time series diagram and so on. Besides, the delay feedback control method is adopted to control the unstable or chaotic system effectively. The main conclusions of this paper show that the variables mentioned above must be within a reasonable range. Otherwise, the model will lose stability or enter chaos. The government can effectively adjust manufacturers' profit through carbon tax and carbon quota subsidy, and encourage manufacturers to reduce carbon emissions and increase the remanufacturing of waste products. [ABSTRACT FROM AUTHOR]

As we move into the information age, the amount of data in various fields has increased dramatically, and data sources have become increasingly widely distributed. The corresponding phenomenon of missing data is increasingly common, and it leads to the generation of incomplete multi-source information systems. In this context, this paper's proposal aims to address the limitations of rough set theory. We study the method of multi-source fusion in incomplete multi-source systems. This paper presents a method for fusing incomplete multi-source systems based on information entropy; in particular, by comparison with another method, our fusion method is validated. Furthermore, extensive experiments are conducted on six UCI data sets to verify the performance of the proposed method. Additionally, the experimental results indicate that multi-source information fusion approaches significantly outperform other approaches to fusion. [ABSTRACT FROM AUTHOR]

Due to their wide application in many disciplines, how to make an efficient ranking for nodes, especially for nodes in graph data, has aroused lots of attention. To overcome the shortcoming that most traditional ranking methods only consider the mutual influence between nodes but ignore the influence of edges, this paper proposes a self-information weighting-based method to rank all nodes in graph data. In the first place, the graph data are weighted by regarding the self-information of edges in terms of node degree. On this base, the information entropy of nodes is constructed to measure the importance of each node and in which case all nodes can be ranked. To verify the effectiveness of this proposed ranking method, we compare it with six existing methods on nine real-world datasets. The experimental results show that our method performs well on all of these nine datasets, especially for datasets with more nodes. [ABSTRACT FROM AUTHOR]

In this paper, we present the concept of the logical entropy of order m, logical mutual information, and the logical entropy for information sources. We found upper and lower bounds for the logical entropy of a random variable by using convex functions. We show that the logical entropy of the joint distributions X 1 and X 2 is always less than the sum of the logical entropy of the variables X 1 and X 2 . We define the logical Shannon entropy and logical metric permutation entropy to an information system and examine the properties of this kind of entropy. Finally, we examine the amount of the logical metric entropy and permutation logical entropy for maps. [ABSTRACT FROM AUTHOR]

Complexity is an important factor throughout the software life cycle. It is increasingly difficult to guarantee software quality, cost and development progress with the increase in complexity. Excessive complexity is one of the main reasons for the failure of software projects, so effective recognition, measurement and control of complexity becomes the key of project management. At first, this paper analyzes the current research situation of software complexity systematically and points out existing problems in current research. Then, it proposes a WSR framework of software complexity, which divides the complexity of software into three levels of Wuli (WL), Shili (SL) and Renli (RL), so that the staff in different roles may have a better understanding of complexity. Man is the main source of complexity, but the current research focuses on WL complexity, and the research of RL complexity is extremely scarce, so this paper emphasizes the research of RL complexity of software projects. This paper not only analyzes the composing factors of RL complexity, but also provides the definition of RL complexity. Moreover, it puts forward a quantitative measurement method of the complexity of personnel organization hierarchy and the complexity of personnel communication information based on information entropy first and analyzes and validates the scientificity and rationality of this measurement method through a large number of cases. [ABSTRACT FROM AUTHOR]

This paper proposes an image encryption scheme based on a discrete-time alternating quantum walk (AQW) and the advanced encryption standard (AES). We use quantum properties to improve the AES algorithm, which uses a keystream generator related to AQW parameters to generate a probability distribution matrix. Some singular values of the matrix are extracted as the key to the AES algorithm. The Rcon of the AES algorithm is replaced with the elements of the probability distribution matrix. Then, the ascending order of the size of the clone probability distribution matrix scrambles the mapping rules of the S-box and ShiftRow transformations in the AES algorithm. The algorithm uses a probability distribution matrix and plaintext XOR operation to complete the preprocessing and uses the modified AES algorithm to complete the encryption process. The technology is based on simulation verification, including pixel correlation, histograms, differential attacks, noise attacks, information entropy, key sensitivity, and space. The results demonstrate a remarkable encryption effect. Compared with other improved AES algorithms, this algorithm has the advantages of the original AES algorithm and improves the ability to resist correlation attacks. [ABSTRACT FROM AUTHOR]

Landslides are a common type of natural disaster in mountainous areas. As a result of the comprehensive influences of geology, geomorphology and climatic conditions, the susceptibility to landslide hazards in mountainous areas shows obvious regionalism. The evaluation of regional landslide susceptibility can help reduce the risk to the lives of mountain residents. In this paper, the Shannon entropy theory, a fuzzy comprehensive method and an analytic hierarchy process (AHP) have been used to demonstrate a variable type of weighting for landslide susceptibility evaluation modeling, combining subjective and objective weights. Further, based on a single factor sensitivity analysis, we established a strict criterion for landslide susceptibility assessments. Eight influencing factors have been selected for the study of Zhen'an County, Shan'xi Province: the lithology, relief amplitude, slope, aspect, slope morphology, altitude, annual mean rainfall and distance to the river. In order to verify the advantages of the proposed method, the landslide index, prediction accuracy P, the R-index and the area under the curve were used in this paper. The results show that the proposed model of landslide hazard susceptibility can help to produce more objective and accurate landslide susceptibility maps, which not only take advantage of the information from the original data, but also reflect an expert's knowledge and the opinions of decision-makers. [ABSTRACT FROM AUTHOR]

In the research and data analysis of the differences involved in group preferences, conventional statistical methods cannot reflect the integrity and preferences of human minds; in particular, it is difficult to exclude humans' irrational factors. This paper introduces a preference amount model based on relative entropy theory. A related expansion is made based on the characteristics of the questionnaire data, and we also construct the parameters to measure differences in the data distribution of different groups on the whole. In this paper, this parameter is called the center distance, and it effectively reflects the preferences of human minds. Using the survey data of securities market participants as an example, this paper analyzes differences in market participants' attitudes toward the effectiveness of securities regulation. Based on this method, differences between groups that were overlooked by analysis of variance are found, and certain aspects obscured by general data characteristics are also found. [ABSTRACT FROM AUTHOR]

Based on the research of domestic and foreign scholars, this paper has improved and established a double oligopoly market model of renewable energy, and analyzed the complex dynamic characteristics of a system based on entropy theory and chaos theory, such as equilibrium point, stability, Hopf bifurcation conditions, etc. This paper also studied and simulated the effects of the natural growth rate of energy and the single delay decision on the renewable energy system by minimizing the entropy of the system and reducing the system instability to a minimum, so that the degree of disorder within the system was reduced. The results show that with the increase of the natural growth rate of energy, the stability of the system is not affected, but the market demand of the oligopoly 1 is gradually reducing and the market demand of the oligopoly 2 is gradually increasing. At the same time, a single oligopoly making the time delay decision will affect the stability of the two oligopolies. With the increase of delay, the time required to reach the stable state will grow, and the system will eventually enter the Hopf bifurcation, thus the system will have its entropy increased and fall into an unstable state. Therefore, in the actual market of renewable energy, oligopolies should pay attention to the natural growth rate of energy and time delay, ensuring the stability of the game process and the orderliness of the system. [ABSTRACT FROM AUTHOR]

Let us begin by considering two book titles: A provocative title, What Is a StatisticalModel? McCullagh (2002) and an alternative title, In a Search for Structure. The Fisher Information. Gromov (2012). It is the richness in open problems and the links with other research domains that make a research topic exciting. Information geometry has both properties. Differential information geometry is the differential geometry of statistical models. The topology of information is the topology of statistical models. This highlights the importance of both questions raised by Peter McCullagh and Misha Gromov. The title of this paper looks like a list of key words. However, the aim is to emphasize the links between those topics. The theory of homology of Koszul-Vinberg algebroids and their modules (KV homology in short) is a useful key for exploring those links. In Part A we overview three constructions of the KV homology. The first construction is based on the pioneering brute formula of the coboundary operator. The second construction is based on the theory of semi-simplicial objects. The third construction is based on the anomaly functions of abstract algebras and their abstract modules. We use the KV homology for investigating links between differential information geometry and differential topology. For instance, "dualistic relation of Amari" and "Riemannian or symplectic Foliations"; "Koszul geometry" and "linearization of webs"; "KV homology" and "complexity of models". Regarding the complexity of a model, the challenge is to measure how far from being an exponential family is a given model. In Part A we deal with the classical theory of models. Part B is devoted to answering both questions raised by McCullagh and B Gromov. A few criticisms and examples are used to support our criticisms and to motivate a new approach. In a given category an outstanding challenge is to find an invariant which encodes the points of a moduli space. In Part B we face four challenges. (1) The introduction of a new theory of statistical models. This re-establishment must answer both questions of McCullagh and Gromov; (2) The search for an characteristic invariant which encodes the points of the moduli space of isomorphism class of models; (3) The introduction of the theory of homological statistical models. This is a pioneering notion. We address its links with Hessian geometry; (4) We emphasize the links between the classical theory of models, the new theory and Vanishing Theorems in the theory of homological statistical models. Subsequently, the differential information geometry has a homological nature. That is another notable feature of our approach. This paper is dedicated to our friend and colleague Alexander Grothendieck. [ABSTRACT FROM AUTHOR]

Person authentication, based on electroencephalography (EEG) signals, is one of the directions possible in the study of EEG signals. In this paper, a method for the selection of EEG electrodes and features in a discriminative manner is proposed. Given that EEG signals are unstable and non-linear, a non-linear analysis method, i.e., fuzzy entropy, is more appropriate. In this paper, unlike other methods using different signal sources and patterns, such as rest state and motor imagery, a novel paradigm using the stimuli of self-photos and non-self-photos is introduced. Ten subjects are selected to take part in this experiment, and fuzzy entropy is used as a feature to select the minimum number of electrodes that identifies individuals. The experimental results show that the proposed method can make use of two electrodes (FP1 and FP2) in the frontal area, while the classification accuracy is greater than 87.3%. The proposed biometric system, based on EEG signals, can provide each subject with a unique key and is capable of human recognition. [ABSTRACT FROM AUTHOR]

With the rapid development of electronic commerce, spam reviews are rapidly growing on the Internet to manipulate online customers' opinions on goods being sold. This paper proposes a novel approach, called CoFea (Co-training by Features), to identify spam reviews, based on entropy and the co-training algorithm. After sorting all lexical terms of reviews by entropy, we produce two views on the reviews by dividing the lexical terms into two subsets. One subset contains odd-numbered terms and the other contains even-numbered terms. Using SVM (support vector machine) as the base classifier, we further propose two strategies, CoFea-T and CoFea-S, embedded with the CoFea approach. The CoFea-T strategy uses all terms in the subsets for spam review identification by SVM. The CoFea-S strategy uses a predefined number of terms with small entropy for spam review identification by SVM. The experiment results show that the CoFea-T strategy produces better accuracy than the CoFea-S strategy, while the CoFea-S strategy saves more computing time than the CoFea-T strategy with acceptable accuracy in spam review identification. [ABSTRACT FROM AUTHOR]

In several applications, the assumption of normality is often violated in data with some level of skewness, so skewness affects the mean's estimation. The class of skew–normal distributions is considered, given their flexibility for modeling data with asymmetry parameter. In this paper, we considered two location parameter (μ) estimation methods in the skew–normal setting, where the coefficient of variation and the skewness parameter are known. Specifically, the least square estimator (LSE) and the best unbiased estimator (BUE) for μ are considered. The properties for BUE (which dominates LSE) using classic theorems of information theory are explored, which provides a way to measure the uncertainty of location parameter estimations. Specifically, inequalities based on convexity property enable obtaining lower and upper bounds for differential entropy and Fisher information. Some simulations illustrate the behavior of differential entropy and Fisher information bounds. [ABSTRACT FROM AUTHOR]

This paper describes a new model for portfolio optimization (PO), using entropy and mutual information instead of variance and covariance as measurements of risk. We also compare the performance in and out of sample of the original Markowitz model against the proposed model and against other state of the art shrinkage methods. It was found that ME (mean-entropy) models do not always outperform their MV (mean-variance) and robust counterparts, although presenting an edge in terms of portfolio diversity measures, especially for portfolio weight entropy. It further shows that when increasing return constraints on portfolio optimization, ME models were more stable overall, showing dampened responses in cumulative returns and Sharpe indexes in comparison to MV and robust methods, but concentrated their portfolios more rapidly as they were more evenly spread initially. Finally, the results suggest that it was also shown that, depending on the market, increasing return constraints may have positive or negative impacts on the out-of-sample performance. [ABSTRACT FROM AUTHOR]

The Stock Exchange is basically ruled by the extreme market sentiments of euphoria and fear. The type of sentiment is given by the color of the candlestick (white = bullish sentiments, black = bearish sentiments), meanwhile the intensity of the sentiment is given by the size of it. In this paper you will see that the intensity of any sentiment is astonishingly distributed in a robust, systematic and universal way, according to a law of exponential decay, the conclusion of which is supported by the analysis of the Lyapunov exponent, the information entropy and the frequency distribution of candlestick size. [ABSTRACT FROM AUTHOR]

Many graph invariants have been used for the construction of entropy-based measures to characterize the structure of complex networks. The starting point has been always based on assigning a probability distribution to a network when using Shannon's entropy. In particular, Cao et al. (2014 and 2015) defined special graph entropy measures which are based on degrees powers. In this paper, we obtain some lower and upper bounds for these measures and characterize extremal graphs. Moreover we resolve one part of a conjecture stated by Cao et al. [ABSTRACT FROM AUTHOR]

Because of the general lack of multi-level hospital management collaboration performance effectiveness research, this paper proposes a multi-level hospital management Synergy Entropy-House of Quality (HoQ) Measurement Model by innovatively combining the House of Quality (HoQ) measure model with a Synergy Entropy computing principle. Triangular fuzzy functions are used to determine the importance degree parameter of each hospital management element which combined with the results from the Synergy Entropy evaluation of the hospital management elements, arrive at a comprehensive collaborative computation result for the various elements, ensuring results objectivity. Finally, the analysis of the collaborative research on multi-level hospital management demonstrated the scientific effectiveness of the hospital management Synergy Entropy-House of Quality (HoQ) Measurement Model. [ABSTRACT FROM AUTHOR]

This paper offers sufficient conditions for the Miller-Madow estimator and the jackknife estimator of entropy to have respective asymptotic normalities on countably infinite alphabets. [ABSTRACT FROM AUTHOR]

This paper is focused on f-divergences, consisting of three main contributions. The first one introduces integral representations of a general f-divergence by means of the relative information spectrum. The second part provides a new approach for the derivation of f-divergence inequalities, and it exemplifies their utility in the setup of Bayesian binary hypothesis testing. The last part of this paper further studies the local behavior of f-divergences. [ABSTRACT FROM AUTHOR]

Entropy and relative entropy measures play a crucial role in mathematical information theory. The relative entropies are also widely used in statistics under the name of divergence measures which link these two fields of science through the minimum divergence principle. Divergence measures are popular among statisticians as many of the corresponding minimum divergence methods lead to robust inference in the presence of outliers in the observed data; examples include the φ-divergence, the density power divergence, the logarithmic density power divergence and the recently developed family of logarithmic super divergence (LSD). In this paper, we will present an alternative information theoretic formulation of the LSD measures as a two-parameter generalization of the relative a-entropy, which we refer to as the general (α, β)-entropy. We explore its relation with various other entropies and divergences, which also generates a two-parameter extension of Renyi entropy measure as a by-product. This paper is primarily focused on the geometric properties of the relative (α, β)-entropy or the LSD measures; we prove their continuity and convexity in both the arguments along with an extended Pythagorean relation under a power-transformation of the domain space. We also derive a set of sufficient conditions under which the forward and the reverse projections of the relative (α, β)-entropy exist and are unique. Finally, we briefly discuss the potential applications of the relative (α, β)-entropy or the LSD measures in statistical inference, in particular, for robust parameter estimation and hypothesis testing. Our results on the reverse projection of the relative (α, β)-entropy establish, for the first time, the existence and uniqueness of the minimum LSD estimators. Numerical illustrations are also provided for the problem of estimating the binomial parameter. [ABSTRACT FROM AUTHOR]

Strong background noise and complicated interfering signatures when implementing vibration-based monitoring make it difficult to extract the weak diagnostic features due to incipient faults in a multistage gearbox. This can be more challenging when multiple faults coexist. This paper proposes an effective approach to extract multi-fault features of a wind turbine gearbox based on an integration of minimum entropy deconvolution (MED) and multipoint optimal minimum entropy deconvolution adjusted (MOMEDA). By using simulated periodic transient signals with different noise to signal ratios (SNR), it evaluates the outstanding performance of MED in noise suppression and reveals the deficient in extract multiple impulses. On the other hand, MOMEDA can performs better in extracting multiple pulses but not robust to noise influences. To compromise the merits of them, therefore the diagnostic approach is formalized by extracting the multiple weak features with MOMEDA based on the MED denoised signals. Experimental verification based on vibrations from a wind turbine gearbox test bed shows that the approach allows successful identification of multiple faults occurring simultaneously on the shaft and bearing in the high speed transmission stage of the gearbox. [ABSTRACT FROM AUTHOR]

In order to solve the high-precision motion control problem of the n-degree-of-freedom (n-DOF) manipulator driven by large amount of real-time data, a motion control algorithm based on self-organizing interval type-2 fuzzy neural network error compensation (SOT2-FNNEC) is proposed. The proposed control framework can effectively suppress various types of interference such as base jitter, signal interference, time delay, etc., during the movement of the manipulator. The fuzzy neural network structure and self-organization method are used to realize the online self-organization of fuzzy rules based on control data. The stability of the closed-loop control systems are proved by Lyapunov stability theory. Simulations show that the algorithm is superior to a self-organizing fuzzy error compensation network and conventional sliding mode variable structure control methods in control performance. [ABSTRACT FROM AUTHOR]