Search

We feature papers on recent developments in the areas of adaptive signal processing, machine learning, and deep learning using information theory and entropy to improve performance in widespread and popular problems, and also to provide effective solutions to emerging problems. Our goal is to publish recent developments in the areas of adaptive signal processing, machine learning, and deep learning using information theory and entropy to improve performance in widespread and popular problems, and also to provide effective solutions to emerging problems. This Special Issue on "Adaptive Signal Processing and Machine Learning Using Entropy and Information Theory" was birthed from observations of the recent trend in the literature. Entropy-based cost functions have replaced mean-square-error (MSE)-based ones and have been widely used in adaptive signal processing and machine learning to improve performance by designing and optimizing effective and specific models that fit the data, even in noisy and adverse conditions. [Extracted from the article]

Medical image diagnosis using deep learning has shown significant promise in clinical medicine. However, it often encounters two major difficulties in real-world applications: (1) domain shift, which invalidates the trained model on new datasets, and (2) class imbalance problems leading to model biases towards majority classes. To address these challenges, this paper proposes a transfer learning solution, named Dynamic Weighting Translation Transfer Learning (DTTL), for imbalanced medical image classification. The approach is grounded in information and entropy theory and comprises three modules: Cross-domain Discriminability Adaptation (CDA), Dynamic Domain Translation (DDT), and Balanced Target Learning (BTL). CDA connects discriminative feature learning between source and target domains using a synthetic discriminability loss and a domain-invariant feature learning loss. The DDT unit develops a dynamic translation process for imbalanced classes between two domains, utilizing a confidence-based selection approach to select the most useful synthesized images to create a pseudo-labeled balanced target domain. Finally, the BTL unit performs supervised learning on the reassembled target set to obtain the final diagnostic model. This paper delves into maximizing the entropy of class distributions, while simultaneously minimizing the cross-entropy between the source and target domains to reduce domain discrepancies. By incorporating entropy concepts into our framework, our method not only significantly enhances medical image classification in practical settings but also innovates the application of entropy and information theory within deep learning and medical image processing realms. Extensive experiments demonstrate that DTTL achieves the best performance compared to existing state-of-the-art methods for imbalanced medical image classification tasks. [ABSTRACT FROM AUTHOR]

Underwriters play a pivotal role in the IPO process. Information entropy, a tool for measuring the uncertainty and complexity of information, has been widely applied to various issues in complex networks. Information entropy can quantify the uncertainty and complexity of nodes in the network, providing a unique analytical perspective and methodological support for this study. This paper employs a bipartite network analysis method to construct the relationship network between underwriters and accounting firms, using the centrality of underwriters in the network as a measure of their influence to explore the impact of underwriters' influence on the distribution of interests and audit outcomes. The findings indicate that a more pronounced influence of underwriters significantly increases the ratio of underwriting fees to audit fees. Higher influence often accompanies an increase in abnormal underwriting fees. Further research reveals that companies underwritten by more influential underwriters experience a decline in audit quality. Finally, the study reveals that a well-structured audit committee governance and the rationalization of market sentiments can mitigate the negative impacts of underwriters' influence. The innovation of this paper is that it enriches the content related to underwriters by constructing the relationship network between underwriters and accounting firms for the first time using a bipartite network through the lens of information entropy. This conclusion provides new directions for thinking about the motives and possibilities behind financial institutions' cooperation, offering insights for market regulation and policy formulation. [ABSTRACT FROM AUTHOR]

Traffic state classification and relevance calculation at intersections are both difficult problems in traffic control. In this paper, we propose an intersection relevance model based on a temporal graph attention network, which can solve the above two problems at the same time. First, the intersection features and interaction time of the intersections are regarded as input quantities together with the initial labels of the traffic data. Then, they are inputted into the temporal graph attention (TGAT) model to obtain the classification accuracy of the target intersections in four states—free, stable, slow moving, and congested—and the obtained neighbouring intersection weights are used as the correlation between the intersections. Finally, it is validated by VISSIM simulation experiments. In terms of classification accuracy, the TGAT model has a higher classification accuracy than the three traditional classification models and can cope well with the uneven distribution of the number of samples. The information gain algorithm from the information entropy theory was used to derive the average delay as the most influential factor on intersection status. The correlation from the TGAT model positively correlates with traffic flow, making it interpretable. Using this correlation to control the division of subareas improves the road network's operational efficiency more than the traditional correlation model does. This demonstrates the effectiveness of the TGAT model's correlation. [ABSTRACT FROM AUTHOR]

This paper presents a multivariate textual analysis of more than 1300 papers on entropy in ecology. There are six main insights that emerged. First, there is a large body of literature that has addressed some aspect of entropy in ecology, most of which has been published in the last 5–10 years. Second, the vast majority of these papers focus on species distribution, species richness, relative abundance or trophic structure and not landscape-scale patterns or processes, pe se. Third, there have been few papers addressing landscape-level questions related to entropy. Fourth, the quantitative analysis with hierarchical clustering identified a strongly nested structure among papers that addressed entropy in ecology. Fifth, there is clear differentiation of papers focused on landscape-level applications of entropy from other papers, with landscape focused papers clustered together at each level of the hierarchy in a relatively small and closely associated group. Sixth, this group of landscape-focused papers was substructured between papers that explicitly adopted entropy measures to quantify the spatial pattern of landscape mosaics, often using variations on Boltzmann entropy, versus those that utilize Shannon entropy measures from information theory, which are not generally explicit in their assessment of spatial configuration. This review provides a comprehensive, quantitative assessment of the scope, trends and relationships among a large body of literature related to entropy in ecology and for the first time puts landscape ecological research on entropy into that context. [ABSTRACT FROM AUTHOR]

This paper highlights that metrics from the machine learning field (e.g., entropy and information gain) used to qualify a classifier model can be used to evaluate the effectiveness of separation systems. To evaluate the efficiency of separation systems and their operation units, entropy- and information gain-based metrics were developed. The receiver operating characteristic (ROC) curve is used to determine the optimal cut point in a separation system. The proposed metrics are verified by simulation experiments conducted on the stochastic model of a waste-sorting system. [ABSTRACT FROM AUTHOR]

With the development of image recovery models, especially those based on adversarial and perceptual losses, the detailed texture portions of images are being recovered more naturally. However, these restored images are similar but not identical in detail texture to their reference images. With traditional image quality assessment methods, results with better subjective perceived quality often score lower in objective scoring. Assessment methods suffer from subjective and objective inconsistencies. This paper proposes a regional differential information entropy (RDIE) method for image quality assessment to address this problem. This approach allows better assessment of similar but not identical textural details and achieves good agreement with perceived quality. Neural networks are used to reshape the process of calculating information entropy, improving the speed and efficiency of the operation. Experiments conducted with this study's image quality assessment dataset and the PIPAL dataset show that the proposed RDIE method yields a high degree of agreement with people's average opinion scores compared with other image quality assessment metrics, proving that RDIE can better quantify the perceived quality of images. [ABSTRACT FROM AUTHOR]

In this paper, we provide geometric insights with visualization into the multivariate Gaussian distribution and its entropy and mutual information. In order to develop the multivariate Gaussian distribution with entropy and mutual information, several significant methodologies are presented through the discussion, supported by illustrations, both technically and statistically. The paper examines broad measurements of structure for the Gaussian distributions, which show that they can be described in terms of the information theory between the given covariance matrix and correlated random variables (in terms of relative entropy). The content obtained allows readers to better perceive concepts, comprehend techniques, and properly execute software programs for future study on the topic's science and implementations. It also helps readers grasp the themes' fundamental concepts to study the application of multivariate sets of data in Gaussian distribution. The simulation results also convey the behavior of different elliptical interpretations based on the multivariate Gaussian distribution with entropy for real-world applications in our daily lives, including information coding, nonlinear signal detection, etc. Involving the relative entropy and mutual information as well as the potential correlated covariance analysis, a wide range of information is addressed, including basic application concerns as well as clinical diagnostics to detect the multi-disease effects. [ABSTRACT FROM AUTHOR]

In the theory of special relativity, energy can be found in two forms: kinetic energy and rest mass. The potential energy of a body is actually stored in the form of rest mass, the interaction energy too, but temperature is not. Information acquired about a dynamical system can be potentially used to extract useful work from it. Hence, the "mass–energy–information equivalence principle" that has been recently proposed. In this paper, it is first recalled that for a thermodynamic system made of non-interacting entities at constant temperature, the internal energy is also constant. So, the energy involved in a variation in entropy ( T Δ S ) differs from a change in the potential energy stored or released and cannot be associated to a corresponding variation in mass of the system, even if it is expressed in terms of the quantity of information. This debate gives us the opportunity to deepen the notion of entropy seen as a quantity of information, to highlight the difference between logical irreversibility (a state-dependent property) and thermodynamical irreversibility (a path-dependent property), and to return to the nature of the link between energy and information that is dynamical. [ABSTRACT FROM AUTHOR]

Mobile robot olfaction of toxic and hazardous odor sources is of great significance in anti-terrorism, disaster prevention, and control scenarios. Aiming at the problems of low search efficiency and easily falling into a local optimum of the current odor source localization strategies, the paper proposes the adaptive space-aware Infotaxis II algorithm. To improve the tracking efficiency of robots, a new reward function is designed by considering the space information and emphasizing the exploration behavior of robots. Considering the enhancement in exploratory behavior, an adaptive navigation-updated mechanism is proposed to adjust the movement range of robots in real time through information entropy to avoid an excessive exploration behavior during the search process, which may lead the robot to fall into a local optimum. Subsequently, an improved adaptive cosine salp swarm algorithm is applied to confirm the optimal information adaptive parameter. Comparative simulation experiments between ASAInfotaxis II and the classical search strategies are carried out in 2D and 3D scenarios regarding the search efficiency and search behavior, which show that ASAInfotaxis II is competent to improve the search efficiency to a larger extent and achieves a better balance between exploration and exploitation behaviors. [ABSTRACT FROM AUTHOR]

Recent studies on watermarking techniques based on image carriers have demonstrated new approaches that combine adversarial perturbations against steganalysis with embedding distortions. However, while these methods successfully counter convolutional neural network-based steganalysis, they do not adequately protect the data of the carrier itself. Recognizing the high sensitivity of Deep Neural Networks (DNNs) to small perturbations, we propose HAG-NET, a method based on image carriers, which is jointly trained by the encoder, decoder, and attacker. In this paper, the encoder generates Adversarial Steganographic Examples (ASEs) that are adversarial to the target classification network, thereby providing protection for the carrier data. Additionally, the decoder can recover secret data from ASEs. The experimental results demonstrate that ASEs produced by HAG-NET achieve an average success rate of over 99% on both the MNIST and CIFAR-10 datasets. ASEs generated with the attacker exhibit greater robustness in terms of attack ability, with an average increase of about 3.32%. Furthermore, our method, when compared with other generative stego examples under similar perturbation strength, contains significantly more information according to image information entropy measurements. [ABSTRACT FROM AUTHOR]

Few-shot learning aims to solve the difficulty in obtaining training samples, leading to high variance, high bias, and over-fitting. Recently, graph-based transductive few-shot learning approaches supplement the deficiency of label information via unlabeled data to make a joint prediction, which has become a new research hotspot. Therefore, in this paper, we propose a novel ensemble semi-supervised few-shot learning strategy via transductive network and Dempster–Shafer (D-S) evidence fusion, named ensemble transductive propagation networks (ETPN). First, we present homogeneity and heterogeneity ensemble transductive propagation networks to better use the unlabeled data, which introduce a preset weight coefficient and provide the process of iterative inferences during transductive propagation learning. Then, we combine the information entropy to improve the D-S evidence fusion method, which improves the stability of multi-model results fusion from the pre-processing of the evidence source. Third, we combine the L2 norm to improve an ensemble pruning approach to select individual learners with higher accuracy to participate in the integration of the few-shot model results. Moreover, interference sets are introduced to semi-supervised training to improve the anti-disturbance ability of the mode. Eventually, experiments indicate that the proposed approaches outperform the state-of-the-art few-shot model. The best accuracy of ETPN increases by 0.3% and 0.28% in the 5-way 5-shot, and by 3.43% and 7.6% in the 5-way 1-shot on miniImagNet and tieredImageNet, respectively. [ABSTRACT FROM AUTHOR]

Identifying influential nodes is a key research topic in complex networks, and there have been many studies based on complex networks to explore the influence of nodes. Graph neural networks (GNNs) have emerged as a prominent deep learning architecture, capable of efficiently aggregating node information and discerning node influence. However, existing graph neural networks often ignore the strength of the relationships between nodes when aggregating information about neighboring nodes. In complex networks, neighboring nodes often do not have the same influence on the target node, so the existing graph neural network methods are not effective. In addition, the diversity of complex networks also makes it difficult to adapt node features with a single attribute to different types of networks. To address the above problems, the paper constructs node input features using information entropy combined with the node degree value and the average degree of the neighbor, and proposes a simple and effective graph neural network model. The model obtains the strength of the relationships between nodes by considering the degree of neighborhood overlap, and uses this as the basis for message passing, thereby effectively aggregating information about nodes and their neighborhoods. Experiments are conducted on 12 real networks, using the SIR model to verify the effectiveness of the model with the benchmark method. The experimental results show that the model can identify the influence of nodes in complex networks more effectively. [ABSTRACT FROM AUTHOR]

The safe and comfortable operation of high-speed trains has attracted extensive attention. With the operation of the train, the performance of high-speed train bogie components inevitably degrades and eventually leads to failures. At present, it is a common method to achieve performance degradation estimation of bogie components by processing high-speed train vibration signals and analyzing the information contained in the signals. In the face of complex signals, the usage of information theory, such as information entropy, to achieve performance degradation estimations is not satisfactory, and recent studies have more often used deep learning methods instead of traditional methods, such as information theory or signal processing, to obtain higher estimation accuracy. However, current research is more focused on the estimation for a certain component of the bogie and does not consider the bogie as a whole system to accomplish the performance degradation estimation task for several key components at the same time. In this paper, based on soft parameter sharing multi-task deep learning, a multi-task and multi-scale convolutional neural network is proposed to realize performance degradation state estimations of key components of a high-speed train bogie. Firstly, the structure takes into account the multi-scale characteristics of high-speed train vibration signals and uses a multi-scale convolution structure to better extract the key features of the signal. Secondly, considering that the vibration signal of high-speed trains contains the information of all components, the soft parameter sharing method is adopted to realize feature sharing in the depth structure and improve the utilization of information. The effectiveness and superiority of the structure proposed by the experiment is a feasible scheme for improving the performance degradation estimation of a high-speed train bogie. [ABSTRACT FROM AUTHOR]

This paper proposes an encryption scheme for high pixel density images. Based on the application of the quantum random walk algorithm, the long short-term memory (LSTM) can effectively solve the problem of low efficiency of the quantum random walk algorithm in generating large-scale pseudorandom matrices, and further improve the statistical properties of the pseudorandom matrices required for encryption. The LSTM is then divided into columns and fed into the LSTM in order for training. Due to the randomness of the input matrix, the LSTM cannot be trained effectively, so the output matrix is predicted to be highly random. The LSTM prediction matrix of the same size as the key matrix is generated based on the pixel density of the image to be encrypted, which can effectively complete the encryption of the image. In the statistical performance test, the proposed encryption scheme achieves an average information entropy of 7.9992, an average number of pixels changed rate (NPCR) of 99.6231%, an average uniform average change intensity (UACI) of 33.6029%, and an average correlation of 0.0032. Finally, various noise simulation tests are also conducted to verify its robustness in real-world applications where common noise and attack interference are encountered. [ABSTRACT FROM AUTHOR]

This paper proposes a new string transformation technique called Move with Interleaving (MwI). Four possible ways of rearranging 2D raster images into 1D sequences of values are applied, including scan-line, left-right, strip-based, and Hilbert arrangements. Experiments on 32 benchmark greyscale raster images of various resolutions demonstrated that the proposed transformation reduces information entropy to a similar extent as the combination of the Burrows–Wheeler transform followed by the Move-To-Front or the Inversion Frequencies. The proposed transformation MwI yields the best result among all the considered transformations when the Hilbert arrangement is applied. [ABSTRACT FROM AUTHOR]

In this paper, a new combinatorial structure is introduced for image encryption, which has an excellent encryption effect on security and efficiency. An n-transversal in a Latin square has the function of classifying all the matrix's positions, and it can provide a pair of orthogonal Latin squares. Employing an n-transversal of a Latin square, we can permutate all the pixels of an image group by group for the first time, then use two Latin squares for auxiliary diffusion based on a chaotic sequence, and finally, make use of a pair of orthogonal Latin squares to perform the second scrambling. The whole encryption process is "scrambling–diffusion–scrambling". The experimental results indicated that this algorithm passed various tests and achieved a secure and fast encryption effect, which outperformed many of the latest papers. The final information entropy was very close to 8, and the correlation coefficient was approximately 0. All these tests verified the robustness and practicability of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Outlier detection is an important task in the field of data mining and a highly active area of research in machine learning. In industrial automation, datasets are often high-dimensional, meaning an effort to study all dimensions directly leads to data sparsity, thus causing outliers to be masked by noise effects in high-dimensional spaces. The "curse of dimensionality" phenomenon renders many conventional outlier detection methods ineffective. This paper proposes a new outlier detection algorithm called EOEH (Ensemble Outlier Detection Method Based on Information Entropy-Weighted Subspaces for High-Dimensional Data). First, random secondary subsampling is performed on the data, and detectors are run on various small-scale sub-samples to provide diverse detection results. Results are then aggregated to reduce the global variance and enhance the robustness of the algorithm. Subsequently, information entropy is utilized to construct a dimension-space weighting method that can discern the influential factors within different dimensional spaces. This method generates weighted subspaces and dimensions for data objects, reducing the impact of noise created by high-dimensional data and improving high-dimensional data detection performance. Finally, this study offers a design for a new high-precision local outlier factor (HPLOF) detector that amplifies the differentiation between normal and outlier data, thereby improving the detection performance of the algorithm. The feasibility of this algorithm is validated through experiments that used both simulated and UCI datasets. In comparison to popular outlier detection algorithms, our algorithm demonstrates a superior detection performance and runtime efficiency. Compared with the current popular, common algorithms, the EOEH algorithm improves the detection performance by 6% on average. In terms of running time for high-dimensional data, EOEH is 20% faster than the current popular algorithms. [ABSTRACT FROM AUTHOR]

Market uncertainty has a significant impact on market performance. Previous studies have dedicated much effort towards investigations into market uncertainty related to information asymmetry and risk. However, they have neglected the uncertainty inherent in market transactions, which is also an important aspect of market performance, besides the quantity of transactions and market efficiency. In this paper, we put forward a concept of transaction entropy to measure market uncertainty and see how it changes with price. Transaction entropy is defined as the ratio of the total information entropy of all traders to the quantity of transactions, reflecting the level of uncertainty in making successful transactions. Based on the computational and simulated results, our main finding is that transaction entropy is the lowest at equilibrium, it will decrease in a shortage market, and increase in a surplus market. Additionally, we make a comparison of the total entropy of the centralized market with that of the decentralized market, revealing that the price-filtering mechanism could effectively reduce market uncertainty. Overall, the introduction of transaction entropy enriches our understanding of market uncertainty and facilitates a more comprehensive assessment of market performance. [ABSTRACT FROM AUTHOR]

Click-through rate (CTR) prediction is a research point for measuring recommendation systems and calculating AD traffic. Existing studies have proved that deep learning performs very well in prediction tasks, but most of the existing studies are based on deterministic models, and there is a big gap in capturing uncertainty. Modeling uncertainty is a major challenge when using machine learning solutions to solve real-world problems in various domains. In order to quantify the uncertainty of the model and achieve accurate and reliable prediction results. This paper designs a CTR prediction framework combining feature selection and feature interaction. In this framework, a CTR prediction model based on Bayesian deep learning is proposed to quantify the uncertainty in the prediction model. On the squeeze network and DNN parallel prediction model framework, the approximate posterior parameter distribution of the model is obtained using the Monte Carlo dropout, and obtains the integrated prediction results. Epistemic and aleatoric uncertainty are defined and adopt information entropy to calculate the sum of the two kinds of uncertainties. Epistemic uncertainty could be measured by mutual information. Experimental results show that the model proposed is superior to other models in terms of prediction performance and has the ability to quantify uncertainty. [ABSTRACT FROM AUTHOR]

In underground industries, practitioners frequently employ argots to communicate discreetly and evade surveillance by investigative agencies. Proposing an innovative approach using word vectors and large language models, we aim to decipher and understand the myriad of argots in these industries, providing crucial technical support for law enforcement to detect and combat illicit activities. Specifically, positional differences in semantic space distinguish argots, and pre-trained language models' corpora are crucial for interpreting them. Expanding on these concepts, the article assesses the semantic coherence of word vectors in the semantic space based on the concept of information entropy. Simultaneously, we devised a labeled argot dataset, MNGG, and developed an argot recognition framework named CSRMECT, along with an argot interpretation framework called LLMResolve. These frameworks leverage the MECT model, the large language model, prompt engineering, and the DBSCAN clustering algorithm. Experimental results demonstrate that the CSRMECT framework outperforms the current optimal model by 10% in terms of the F1 value for argot recognition on the MNGG dataset, while the LLMResolve framework achieves a 4% higher accuracy in interpretation compared to the current optimal model.The related experiments undertaken also indicate a potential correlation between vector information entropy and model performance. [ABSTRACT FROM AUTHOR]

As a popular research subject in the field of computer vision, knowledge distillation (KD) is widely used in semantic segmentation (SS). However, based on the learning paradigm of the teacher–student model, the poor quality of teacher network feature knowledge still hinders the development of KD technology. In this paper, we investigate the output features of the teacher–student network and propose a feature condensation-based KD network (FCKDNet), which reduces pseudo-knowledge transfer in the teacher–student network. First, combined with the pixel information entropy calculation rule, we design a feature condensation method to separate the foreground feature knowledge from the background noise of the teacher network outputs. Then, the obtained feature condensation matrix is applied to the original outputs of the teacher and student networks to improve the feature representation capability. In addition, after performing feature condensation on the teacher network, we propose a soft enhancement method of features based on spatial and channel dimensions to improve the dependency of pixels in the feature maps. Finally, we divide the outputs of the teacher network into spatial condensation features and channel condensation features and perform distillation loss calculation with the student network separately to assist the student network to converge faster. Extensive experiments on the public datasets Pascal VOC and Cityscapes demonstrate that our proposed method improves the baseline by 3.16% and 2.98% in terms of mAcc, and 2.03% and 2.30% in terms of mIoU, respectively, and has better segmentation performance and robustness than the mainstream methods. [ABSTRACT FROM AUTHOR]

As a measure of complexity, information entropy is frequently used to categorize time series, such as machinery failure diagnostics, biological signal identification, etc., and is thought of as a characteristic of dynamic systems. Many entropies, however, are ineffective for multivariate scenarios due to correlations. In this paper, we propose a local structure entropy (LSE) based on the idea of a recurrence network. Given certain tolerance and scales, LSE values can distinguish multivariate chaotic sequences between stochastic signals. Three financial market indices are used to evaluate the proposed LSE. The results show that the L S E F S T E 100 and L S E S & P 500 are higher than L S E S Z I , which indicates that the European and American stock markets are more sophisticated than the Chinese stock market. Additionally, using decision trees as the classifiers, LSE is employed to detect bearing faults. LSE performs higher on recognition accuracy when compared to permutation entropy. [ABSTRACT FROM AUTHOR]

Due to their wide application in many disciplines, how to make an efficient ranking for nodes, especially for nodes in graph data, has aroused lots of attention. To overcome the shortcoming that most traditional ranking methods only consider the mutual influence between nodes but ignore the influence of edges, this paper proposes a self-information weighting-based method to rank all nodes in graph data. In the first place, the graph data are weighted by regarding the self-information of edges in terms of node degree. On this base, the information entropy of nodes is constructed to measure the importance of each node and in which case all nodes can be ranked. To verify the effectiveness of this proposed ranking method, we compare it with six existing methods on nine real-world datasets. The experimental results show that our method performs well on all of these nine datasets, especially for datasets with more nodes. [ABSTRACT FROM AUTHOR]

In this paper, we present the concept of the logical entropy of order m, logical mutual information, and the logical entropy for information sources. We found upper and lower bounds for the logical entropy of a random variable by using convex functions. We show that the logical entropy of the joint distributions X 1 and X 2 is always less than the sum of the logical entropy of the variables X 1 and X 2 . We define the logical Shannon entropy and logical metric permutation entropy to an information system and examine the properties of this kind of entropy. Finally, we examine the amount of the logical metric entropy and permutation logical entropy for maps. [ABSTRACT FROM AUTHOR]

This paper proposes an image encryption scheme based on a discrete-time alternating quantum walk (AQW) and the advanced encryption standard (AES). We use quantum properties to improve the AES algorithm, which uses a keystream generator related to AQW parameters to generate a probability distribution matrix. Some singular values of the matrix are extracted as the key to the AES algorithm. The Rcon of the AES algorithm is replaced with the elements of the probability distribution matrix. Then, the ascending order of the size of the clone probability distribution matrix scrambles the mapping rules of the S-box and ShiftRow transformations in the AES algorithm. The algorithm uses a probability distribution matrix and plaintext XOR operation to complete the preprocessing and uses the modified AES algorithm to complete the encryption process. The technology is based on simulation verification, including pixel correlation, histograms, differential attacks, noise attacks, information entropy, key sensitivity, and space. The results demonstrate a remarkable encryption effect. Compared with other improved AES algorithms, this algorithm has the advantages of the original AES algorithm and improves the ability to resist correlation attacks. [ABSTRACT FROM AUTHOR]

In several applications, the assumption of normality is often violated in data with some level of skewness, so skewness affects the mean's estimation. The class of skew–normal distributions is considered, given their flexibility for modeling data with asymmetry parameter. In this paper, we considered two location parameter (μ) estimation methods in the skew–normal setting, where the coefficient of variation and the skewness parameter are known. Specifically, the least square estimator (LSE) and the best unbiased estimator (BUE) for μ are considered. The properties for BUE (which dominates LSE) using classic theorems of information theory are explored, which provides a way to measure the uncertainty of location parameter estimations. Specifically, inequalities based on convexity property enable obtaining lower and upper bounds for differential entropy and Fisher information. Some simulations illustrate the behavior of differential entropy and Fisher information bounds. [ABSTRACT FROM AUTHOR]

This paper describes a new model for portfolio optimization (PO), using entropy and mutual information instead of variance and covariance as measurements of risk. We also compare the performance in and out of sample of the original Markowitz model against the proposed model and against other state of the art shrinkage methods. It was found that ME (mean-entropy) models do not always outperform their MV (mean-variance) and robust counterparts, although presenting an edge in terms of portfolio diversity measures, especially for portfolio weight entropy. It further shows that when increasing return constraints on portfolio optimization, ME models were more stable overall, showing dampened responses in cumulative returns and Sharpe indexes in comparison to MV and robust methods, but concentrated their portfolios more rapidly as they were more evenly spread initially. Finally, the results suggest that it was also shown that, depending on the market, increasing return constraints may have positive or negative impacts on the out-of-sample performance. [ABSTRACT FROM AUTHOR]

In order to solve the high-precision motion control problem of the n-degree-of-freedom (n-DOF) manipulator driven by large amount of real-time data, a motion control algorithm based on self-organizing interval type-2 fuzzy neural network error compensation (SOT2-FNNEC) is proposed. The proposed control framework can effectively suppress various types of interference such as base jitter, signal interference, time delay, etc., during the movement of the manipulator. The fuzzy neural network structure and self-organization method are used to realize the online self-organization of fuzzy rules based on control data. The stability of the closed-loop control systems are proved by Lyapunov stability theory. Simulations show that the algorithm is superior to a self-organizing fuzzy error compensation network and conventional sliding mode variable structure control methods in control performance. [ABSTRACT FROM AUTHOR]

In the era of big data, it is challenging to efficiently retrieve the required images from the vast amount of data. Therefore, a content-based image retrieval system is an important research direction to address this problem. Furthermore, a multi-feature-based image retrieval system can compensate for the shortage of a single feature to a certain extent, which is essential for improving retrieval system performance. Feature selection and feature fusion strategies are critical in the study of multi-feature fusion image retrieval. This paper proposes a multi-feature fusion image retrieval strategy with adaptive features based on information entropy theory. Firstly, we extract the image features, construct the distance function to calculate the similarity using the information entropy proposed in this paper, and obtain the initial retrieval results. Then, we obtain the precision of single feature retrieval based on the correlation feedback as the retrieval trust and use the retrieval trust to select the effective features automatically. After that, we initialize the weights of selected features using the average weights, construct the probability transfer matrix, and use the PageRank algorithm to update the initialized feature weights to obtain the final weights. Finally, we calculate the comprehensive similarity based on the final weights and output the detection results. This has two advantages: (1) the proposed strategy uses multiple features for image retrieval, which has better performance and more substantial generalization than the retrieval strategy based on a single feature; (2) compared with the fixed-feature retrieval strategy, our method selects the best features for fusion in each query, which takes full advantages of each feature. The experimental results show that our proposed method outperforms other methods. In the datasets of Corel1k, UC Merced Land-Use, and RSSCN7, the top10 retrieval precision is 99.55%, 88.02%, and 88.28%, respectively. In the Holidays dataset, the mean average precision (mAP) was 92.46%. [ABSTRACT FROM AUTHOR]

Effective diagnosis of vibration fault is of practical significance to ensure the safe and stable operation of power transformers. Aiming at the traditional problems of transformer vibration fault diagnosis, a novel feature extraction method based on complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and multi-scale dispersion entropy (MDE) was proposed. In this paper, CEEMDAN method is used to decompose the original transformer vibration signal. Additionally, then MDE is used to capture multi-scale fault features in the decomposed intrinsic mode functions (IMFs). Next, the principal component analysis (PCA) method is employed to reduce the feature dimension and extract the effective information in vibration signals. Finally, the simplified features are sent into density peak clustering (DPC) to get the fault diagnosis results. The experimental data analysis shows that CEEMDAN-MDE can effectively extract the information of the original vibration signals and DPC can accurately diagnose the types of transformer faults. By comparing different algorithms, the practicability and superiority of this proposed method are verified. [ABSTRACT FROM AUTHOR]

This document introduces methods for describing long texts as groups of repeating symbols or patterns. The process converts a series of real-number values into texts. Developed tailored algorithms for identifying repeated sequences in the text are applied to decompose the text into nested tree-like structures of repeating symbols and is called the Nested Repeated Sequence Decomposition Model (NRSDM). The NRSDM is especially valuable for extracting repetitive behaviors in oscillatory but non-periodic and chaotic processes where the classical Fourier transform has limited application. The NRSDM along with the two graphical representations proposed here form a promising tool for characterizing long texts configured to represent the behavior of unidimensional processes. [ABSTRACT FROM AUTHOR]

The complementary combination of emphasizing target objects in infrared images and rich texture details in visible images can effectively enhance the information entropy of fused images, thereby providing substantial assistance for downstream composite high-level vision tasks, such as nighttime vehicle intelligent driving. However, mainstream fusion algorithms lack specific research on the contradiction between the low information entropy and high pixel intensity of visible images under harsh light nighttime road environments. As a result, fusion algorithms that perform well in normal conditions can only produce low information entropy fusion images similar to the information distribution of visible images under harsh light interference. In response to these problems, we designed an image fusion network resilient to harsh light environment interference, incorporating entropy and information theory principles to enhance robustness and information retention. Specifically, an edge feature extraction module was designed to extract key edge features of salient targets to optimize fusion information entropy. Additionally, a harsh light environment aware (HLEA) module was proposed to avoid the decrease in fusion image quality caused by the contradiction between low information entropy and high pixel intensity based on the information distribution characteristics of harsh light visible images. Finally, an edge-guided hierarchical fusion (EGHF) module was designed to achieve robust feature fusion, minimizing irrelevant noise entropy and maximizing useful information entropy. Extensive experiments demonstrate that, compared to other advanced algorithms, the method proposed fusion results contain more useful information and have significant advantages in high-level vision tasks under harsh nighttime lighting conditions. [ABSTRACT FROM AUTHOR]

To tackle the issue of the traditional intelligent diagnostic algorithm's insufficient utilization of correlation characteristics within the time series of fault signals and to meet the challenges of accuracy and computational complexity in rotating machinery fault diagnosis, a novel approach based on a recurrence binary plot (RBP) and a lightweight, deep, separable, dilated convolutional neural network (DSD-CNN) is proposed. Firstly, a recursive encoding method is used to convert the fault vibration signals of rotating machinery into two-dimensional texture images, extracting feature information from the internal structure of the fault signals as the input for the model. Subsequently, leveraging the excellent feature extraction capabilities of a lightweight convolutional neural network embedded with attention modules, the fault diagnosis of rotating machinery is carried out. The experimental results using different datasets demonstrate that the proposed model achieves excellent diagnostic accuracy and computational efficiency. Additionally, compared with other representative fault diagnosis methods, this model shows better anti-noise performance under different noise test data, and it provides a reliable and efficient reference solution for rotating machinery fault-classification tasks. [ABSTRACT FROM AUTHOR]

The influence of the collective and quantum effects on the Shannon information entropy for atomic states in dense nonideal plasma was investigated. The interaction potential, which takes into account the effect of quantum non-locality as well as electronic correlations, was used to solve the Schrödinger equation for the hydrogen atom. It is shown that taking into account ionic screening leads to an increase in entropy, while taking into account only electronic screening does not lead to significant changes. [ABSTRACT FROM AUTHOR]

Since the reliability of the avionics module is crucial for aircraft safety, the fault diagnosis and health management of this module are particularly significant. While deep learning-based prognostics and health management (PHM) methods exhibit highly accurate fault diagnosis, they have disadvantages such as inefficient data feature extraction and insufficient generalization capability, as well as a lack of avionics module fault data. Consequently, this study first employs fault injection to simulate various fault types of the avionics module and performs data enhancement to construct the P2020 communications processor fault dataset. Subsequently, a multichannel fault diagnosis method, the Hybrid Attention Adaptive Multi-scale Temporal Convolution Network (HAAMTCN) for the integrated functional circuit module of the avionics module, is proposed, which adaptively constructs the optimal size of the convolutional kernel to efficiently extract features of avionics module fault signals with large information entropy. Further, the combined use of the Interaction Channel Attention (ICA) module and the Hierarchical Block Temporal Attention (HBTA) module results in the HAAMTCN to pay more attention to the critical information in the channel dimension and time step dimension. The experimental results show that the HAAMTCN achieves an accuracy of 99.64% in the avionics module fault classification task which proves our method achieves better performance in comparison with existing methods. [ABSTRACT FROM AUTHOR]

To construct a chaotic system with complex characteristics and to improve the security of image data, a five-dimensional tri-valued memristor chaotic system with high complexity is innovatively constructed. Firstly, a pressure-controlled tri-valued memristor on Liu's pseudo-four-wing chaotic system is introduced. Through analytical methods, such as Lyapunov exponential map, bifurcation map and attractor phase diagram, it is demonstrated that the new system has rich dynamical behaviors with periodic limit rings varying with the coupling parameter of the system, variable airfoil phenomenon as well as transient chaotic phenomenon of chaos-periodic depending on the system parameter and chaos-quasi-periodic depending on the memristor parameter. The system is simulated with dynamic circuits based on Simulink. Secondly, the differently structured synchronous controls of chaotic systems are realized using a nonlinear feedback control method. Finally, based on the newly constructed five-dimensional chaotic system, a variable parameter color image encryption scheme is proposed to iteratively generate varying chaotic pseudo-random sequences by varying the system parameters, which will be used for repetition-free disambiguation, additive modulo left-shift diffusion and DNA encryption for the three components of RGB of the color image after chunking. The simulation results are analyzed by histogram, information entropy, adjacent pixel correlation, etc., and the images are tested using differential attack, noise attack and geometric attack, as well as analyzing the PSNR and SSIM of the decrypted image quality. The results show that the encryption method has a certain degree of security and can be applied to medical, military and financial fields with more complex environmental requirements. [ABSTRACT FROM AUTHOR]

In most silent speech research, continuously observing tongue movements is crucial, thus requiring the use of ultrasound to extract tongue contours. Precisely and in real-time extracting ultrasonic tongue contours presents a major challenge. To tackle this challenge, the novel end-to-end lightweight network DAFT-Net is introduced for ultrasonic tongue contour extraction. Integrating the Convolutional Block Attention Module (CBAM) and Attention Gate (AG) module with entropy-based optimization strategies, DAFT-Net establishes a comprehensive attention mechanism with dual functionality. This innovative approach enhances feature representation by replacing traditional skip connection architecture, thus leveraging entropy and information-theoretic measures to ensure efficient and precise feature selection. Additionally, the U-Net's encoder and decoder layers have been streamlined to reduce computational demands. This process is further supported by information theory, thus guiding the reduction without compromising the network's ability to capture and utilize critical information. Ablation studies confirm the efficacy of the integrated attention module and its components. The comparative analysis of the NS, TGU, and TIMIT datasets shows that DAFT-Net efficiently extracts relevant features, and it significantly reduces extraction time. These findings demonstrate the practical advantages of applying entropy and information theory principles. This approach improves the performance of medical image segmentation networks, thus paving the way for real-world applications. [ABSTRACT FROM AUTHOR]

Link prediction is recognized as a crucial means to analyze dynamic social networks, revealing the principles of social relationship evolution. However, the complex topology and temporal evolution characteristics of dynamic social networks pose significant research challenges. This study introduces an innovative fusion framework that incorporates entropy, causality, and a GCN model, focusing specifically on link prediction in dynamic social networks. Firstly, the framework preprocesses the raw data, extracting and recording timestamp information between interactions. It then introduces the concept of "Temporal Information Entropy (TIE)", integrating it into the Node2Vec algorithm's random walk to generate initial feature vectors for nodes in the graph. A causality analysis model is subsequently applied for secondary processing of the generated feature vectors. Following this, an equal dataset is constructed by adjusting the ratio of positive and negative samples. Lastly, a dedicated GCN model is used for model training. Through extensive experimentation in multiple real social networks, the framework proposed in this study demonstrated a better performance than other methods in key evaluation indicators such as precision, recall, F1 score, and accuracy. This study provides a fresh perspective for understanding and predicting link dynamics in social networks and has significant practical value. [ABSTRACT FROM AUTHOR]

Salient object detection (SOD) aims to accurately identify significant geographical objects in remote sensing images (RSI), providing reliable support and guidance for extensive geographical information analyses and decisions. However, SOD in RSI faces numerous challenges, including shadow interference, inter-class feature confusion, as well as unclear target edge contours. Therefore, we designed an effective Global Semantic-aware Aggregation Network (GSANet) to aggregate salient information in RSI. GSANet computes the information entropy of different regions, prioritizing areas with high information entropy as potential target regions, thereby achieving precise localization and semantic understanding of salient objects in remote sensing imagery. Specifically, we proposed a Semantic Detail Embedding Module (SDEM), which explores the potential connections among multi-level features, adaptively fusing shallow texture details with deep semantic features, efficiently aggregating the information entropy of salient regions, enhancing information content of salient targets. Additionally, we proposed a Semantic Perception Fusion Module (SPFM) to analyze map relationships between contextual information and local details, enhancing the perceptual capability for salient objects while suppressing irrelevant information entropy, thereby addressing the semantic dilution issue of salient objects during the up-sampling process. The experimental results on two publicly available datasets, ORSSD and EORSSD, demonstrated the outstanding performance of our method. The method achieved 93.91% Sα, 98.36% Eξ, and 89.37% Fβ on the EORSSD dataset. [ABSTRACT FROM AUTHOR]

The Sustainable Development Goals have been criticized for not providing sufficient balance between human well-being and environmental well-being. By contrast, joint agent-environment systems theory is focused on reciprocal synchronous generative development. The purpose of this paper is to extend this theory towards practical application in sustainable development projects. This purpose is fulfilled through three interrelated contributions. First, a practitioner description of the theory is provided. Then, the theory is extended through reference to research concerned with multilevel pragmatics, competing signals, commitment processes, technological mediation, and psychomotor functioning. In addition, the theory is related to human-driven biosocial-technical innovation through the example of digital twins for agroecological urban farming. Digital twins being digital models that mirror physical processes; that are connected to physical processes through, for example, sensors and actuators; and which carry out analyses of physical processes in order to improve their performance. Together, these contributions extend extant theory towards application for synchronous generative development that balances human well-being and environmental well-being. However, the practical examples in the paper indicate that counterproductive complexity can arise from situated entropy amidst biosocial-technical innovations: even when those innovations are compatible with synchronous generative development. [ABSTRACT FROM AUTHOR]

The multi-disc wet clutch is widely used in transmission systems as it transfers the torque and power between the gearbox and the driving engine. During service, the buckling of the friction components in the wet clutch is inevitable, which can shorten the lifetime of the wet clutch and decrease the vehicle performance. Therefore, fault diagnosis and online monitoring are required to identify the buckling state of the friction components. However, unlike in other rotating machinery, the time-domain features of the vibration signal lack efficiency in fault diagnosis for the wet clutch. This paper aims to present a new fault diagnosis method based on multi-speed Hilbert spectrum entropy to classify the buckling state of the wet clutch. Firstly, the wet clutch is classified depending on the buckling degree of the disks, and then a bench test is conducted to obtain vibration signals of each class at varying speeds. By comparing the accuracy of different classifiers with and without entropy, Hilbert spectrum entropy shows higher efficiency than time-domain features for the wet clutch diagnosis. Thus, the classification results based on multi-speed entropy achieve even better accuracy. [ABSTRACT FROM AUTHOR]

Multilevel thresholding segmentation of color images plays an important role in many fields. The pivotal procedure of this technique is determining the specific threshold of the images. In this paper, a hybrid preaching optimization algorithm (HPOA) for color image segmentation is proposed. Firstly, the evolutionary state strategy is adopted to evaluate the evolutionary factors in each iteration. With the introduction of the evolutionary state, the proposed algorithm has more balanced exploration-exploitation compared with the original POA. Secondly, in order to prevent premature convergence, a randomly occurring time-delay is introduced into HPOA in a distributed manner. The expression of the time-delay is inspired by particle swarm optimization and reflects the history of previous personal optimum and global optimum. To better verify the effectiveness of the proposed method, eight well-known benchmark functions are employed to evaluate HPOA. In the interim, seven state-of-the-art algorithms are utilized to compare with HPOA in the terms of accuracy, convergence, and statistical analysis. On this basis, an excellent multilevel thresholding image segmentation method is proposed in this paper. Finally, to further illustrate the potential, experiments are respectively conducted on three different groups of Berkeley images. The quality of a segmented image is evaluated by an array of metrics including feature similarity index (FSIM), peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and Kapur entropy values. The experimental results reveal that the proposed method significantly outperforms other algorithms and has remarkable and promising performance for multilevel thresholding color image segmentation. [ABSTRACT FROM AUTHOR]

During the last few decades, the notion of entropy has become omnipresent in many scientific disciplines, ranging from traditional applications in statistical physics and chemistry, information theory, and statistical estimation to more recent applications in biology, astrophysics, geology, financial markets, or social networks. This approach incorporates many popular generalized entropies and gives a general recipe for the thermodynamics of such generalized entropies. The main result of the sixth contribution [[6]] was to introduce a generalization the concept of Shannon's entropy power based on Rényi entropy. [Extracted from the article]

In order to deal with the new threat of low altitude slow small (LSS) targets in air defense operations and provide support for LSS target interception decision, we propose a simple and reliable LSS target threat assessment method. Based on the detection capability of LSS targets and their threat characteristics, this paper proposes a threat evaluation factor and threat degree quantization function in line with the characteristics of LSS targets. LSS targets not only have the same threat characteristics as traditional air targets but also have the unique characteristics of flexible mobility and dynamic mission planning. Therefore, we use analytic hierarchy process (AHP) and information entropy to determine the subjective and objective threat factor weights of LSS targets and use the optimization model to combine them to obtain more reliable evaluation weights. Finally, the effectiveness and credibility of the proposed method are verified by experimental simulation. [ABSTRACT FROM AUTHOR]

Kinetic theory provides modeling of open quantum systems subject to Markovian noise via the Wigner–Fokker–Planck equation, which is an alternate of the Lindblad master equation setting, having the advantage of great physical intuition as it is the quantum equivalent of the classical phase space description. We perform a numerical inspection of the Wehrl entropy for the benchmark problem of a harmonic potential, since the existence of a steady state and its analytical formula have been proven theoretically in this case. When there is friction in the noise terms, no theoretical results on the monotonicity of absolute entropy are available. We provide numerical results of the time evolution of the entropy in the case with friction using a stochastic (Euler–Maruyama-based Monte Carlo) numerical solver. For all the chosen initial conditions studied (all of them Gaussian states), up to the inherent numerical error of the method, one cannot disregard the possibility of monotonic behavior even in the case under study, where the noise includes friction terms. [ABSTRACT FROM AUTHOR]

The Ultimatum Game is a simplistic representation of bargaining processes occurring in social networks. In the standard version of this game, the first player, called the proposer, makes an offer on how to split a certain amount of money. If the second player, called the responder, accepts the offer, the money is divided according to the proposal; if the responder declines the offer, both players receive no money. In this article, an agent-based model is employed to evaluate the performance of five distinct strategies of playing a modified version of this game. A strategy corresponds to instructions on how a player must act as the proposer and as the responder. Here, the strategies are inspired by the following basic emotions: anger, fear, joy, sadness, and surprise. Thus, in the game, each interacting agent is a player endowed with one of these five basic emotions. In the modified version explored in this article, the spatial dimension is taken into account and the survival of the players depends on successful negotiations. Numerical simulations are performed in order to determine which basic emotion dominates the population in terms of prevalence and accumulated money. Information entropy is also computed to assess the time evolution of population diversity and money distribution. From the obtained results, a conjecture on the emergence of the sense of fairness is formulated. [ABSTRACT FROM AUTHOR]

Sustainable development is a practical path to optimize industrial structures and enhance investment efficiency. Investigating risk contagion within ESG industries is a crucial step towards reducing systemic risks and fostering the green evolution of the economy. This research constructs ESG industry indices, taking into account the possibility of extreme tail risks, and employs VaR and CoVaR as measures of tail risk. The TENET network approach is integrated to to capture the structural evolution and direction of information flow among ESG industries, employing information entropy to quantify the topological characteristics of the network model, exploring the risk transmission paths and evolution patterns of ESG industries in an extreme tail risk event. Finally, Mantel tests are conducted to examine the existence of significant risk spillover effects between ESG and traditional industries. The research finds strong correlations among ESG industry indices during stock market crash, Sino–US trade frictions, and the COVID-19 pandemic, with industries such as the COAL, CMP, COM, RT, and RE playing key roles in risk transmission within the network, transmitting risks to other industries. Affected by systemic risk, the information entropy of the TENET network significantly decreases, reducing market information uncertainty and leading market participants to adopt more uniform investment strategies, thus diminishing the diversity of market behaviors. ESG industries show resilience in the face of extreme risks, demonstrating a lack of significant risk contagion with traditional industries. [ABSTRACT FROM AUTHOR]

This research offers a solution to a highly recognized and controversial problem within the composite indicator literature: sub-indicators weighting. The research proposes a novel hybrid weighting method that maximizes the discriminating power of the composite indicator with objectively defined weights. It considers the experts' uncertainty concerning the conceptual importance of sub-indicators in the multidimensional phenomenon, setting maximum and minimum weights (constraints) in the optimization function. The hybrid weighting scheme, known as the SAW-Max-Entropy method, avoids attributing weights that are incompatible with the multidimensional phenomenon's theoretical framework. At the same time, it reduces the influence of assessment errors and judgment biases on composite indicator scores. The research results show that the SAW-Max-Entropy weighting scheme achieves greater discriminating power than weighting schemes based on the Entropy Index, Expert Opinion, and Equal Weights. The SAW-Max-Entropy method has high application potential due to the increasing use of composite indicators across diverse areas of knowledge. Additionally, the method represents a robust response to the challenge of constructing composite indicators with superior discriminating power. [ABSTRACT FROM AUTHOR]

This study presents a novel approach to predicting price fluctuations for U.S. sector index ETFs. By leveraging information-theoretic measures like mutual information and transfer entropy, we constructed threshold networks highlighting nonlinear dependencies between log returns and trading volume rate changes. We derived centrality measures and node embeddings from these networks, offering unique insights into the ETFs' dynamics. By integrating these features into gradient-boosting algorithm-based models, we significantly enhanced the predictive accuracy. Our approach offers improved forecast performance for U.S. sector index futures and adds a layer of explainability to the existing literature. [ABSTRACT FROM AUTHOR]