Search

We feature papers on recent developments in the areas of adaptive signal processing, machine learning, and deep learning using information theory and entropy to improve performance in widespread and popular problems, and also to provide effective solutions to emerging problems. Our goal is to publish recent developments in the areas of adaptive signal processing, machine learning, and deep learning using information theory and entropy to improve performance in widespread and popular problems, and also to provide effective solutions to emerging problems. This Special Issue on "Adaptive Signal Processing and Machine Learning Using Entropy and Information Theory" was birthed from observations of the recent trend in the literature. Entropy-based cost functions have replaced mean-square-error (MSE)-based ones and have been widely used in adaptive signal processing and machine learning to improve performance by designing and optimizing effective and specific models that fit the data, even in noisy and adverse conditions. [Extracted from the article]

Recent studies on watermarking techniques based on image carriers have demonstrated new approaches that combine adversarial perturbations against steganalysis with embedding distortions. However, while these methods successfully counter convolutional neural network-based steganalysis, they do not adequately protect the data of the carrier itself. Recognizing the high sensitivity of Deep Neural Networks (DNNs) to small perturbations, we propose HAG-NET, a method based on image carriers, which is jointly trained by the encoder, decoder, and attacker. In this paper, the encoder generates Adversarial Steganographic Examples (ASEs) that are adversarial to the target classification network, thereby providing protection for the carrier data. Additionally, the decoder can recover secret data from ASEs. The experimental results demonstrate that ASEs produced by HAG-NET achieve an average success rate of over 99% on both the MNIST and CIFAR-10 datasets. ASEs generated with the attacker exhibit greater robustness in terms of attack ability, with an average increase of about 3.32%. Furthermore, our method, when compared with other generative stego examples under similar perturbation strength, contains significantly more information according to image information entropy measurements. [ABSTRACT FROM AUTHOR]

This paper studies contrastive divergence (CD) learning algorithm and proposes a new algorithm for training restricted Boltzmann machines (RBMs). We derive that CD is a biased estimator of the log-likelihood gradient method and make an analysis of the bias. Meanwhile, we propose a new learning algorithm called average contrastive divergence (ACD) for training RBMs. It is an improved CD algorithm, and it is different from the traditional CD algorithm. Finally, we obtain some experimental results. The results show that the new algorithm is a better approximation of the log-likelihood gradient method and outperforms the traditional CD algorithm. [ABSTRACT FROM AUTHOR]

We propose a numerical method to learn maximum entropy (MaxEnt) distributions with spatio-temporal constraints from experimental spike trains. This is an extension of two papers, [10] and [4], which proposed the estimation of parameters where only spatial constraints were taken into account. The extension we propose allows one to properly handle memory effects in spike statistics, for large-sized neural networks. [ABSTRACT FROM AUTHOR]

Recently, the accuracy of voice authentication system has increased significantly due to the successful application of the identity vector (i-vector) model. This paper proposes a new method for i-vector extraction. In the method, a perceptual wavelet packet transform (PWPT) is designed to convert speech utterances into wavelet entropy feature vectors, and a Convolutional Neural Network (CNN) is designed to estimate the frame posteriors of the wavelet entropy feature vectors. In the end, i-vector is extracted based on those frame posteriors. TIMIT and VoxCeleb speech corpus are used for experiments and the experimental results show that the proposed method can extract appropriate i-vector which reduces the equal error rate (EER) and improve the accuracy of voice authentication system in clean and noisy environment. [ABSTRACT FROM AUTHOR]

The sustainability of regional water resources has important supporting data needed for establishing policies on the sustainable development of the social economy. The purpose of this paper is to propose an assessment method to accurately reflect the sustainability of regional water resources in various areas. The method is based on the relative entropy of the information entropy theory. The steps are as follows. Firstly, the pretreatment of the evaluation sample data is required, before the relative entropy of each standard evaluation sample and evaluation grade (SEG) is calculated to obtain the entropy weight of each evaluation index. After this, the entropy weighted comprehensive index (WCI) of the standard evaluation grade sample is obtained. The function relation between WCI and SEG can be fitted by the cubic polynomial to construct the evaluation function. Using the above steps, a generalized entropy method (GEM) for the sustainable assessment of regional water resources is established and it is used to evaluate the sustainability of water resources in the Pingba and Huai River areas in China. The results show that the proposed GEM model can accurately reflect the sustainable water resources in the two regions. Compared with the other evaluation models, such as the Shepherd method, Artificial Neural Network and Fuzzy comprehensive evaluation, the GEM model has larger differences in its evaluation results, which are more reasonable. Thus, the proposed GEM model can provide scientific data support for coordinating the relationship between the sustainable development and utilization of regional water resources in order to improve the development of regional population, society and economy. [ABSTRACT FROM AUTHOR]

Understanding how ensembles of neurons collectively interact will be a key step in developing a mechanistic theory of cognitive processes. Recent progress in multineuron recording and analysis techniques has generated tremendous excitement over the physiology of living neural networks. One of the key developments driving this interest is a new class of models based on the principle of maximum entropy. Maximum entropy models have been reported to account for spatial correlation structure in ensembles of neurons recorded from several different types of data. Importantly, these models require only information about the firing rates of individual neurons and their pairwise correlations. If this approach is generally applicable, it would drastically simplify the problem of understanding how neural networks behave. Given the interest in this method, several groups now have worked to extend maximum entropy models to account for temporal correlations. Here, we review how maximum entropy models have been applied to neuronal ensemble data to account for spatial and temporal correlations. We also discuss criticisms of the maximum entropy approach that argue that it is not generally applicable to larger ensembles of neurons. We conclude that future maximum entropy models will need to address three issues: temporal correlations, higher-order correlations, and larger ensemble sizes. Finally, we provide a brief list of topics for future research. [ABSTRACT FROM AUTHOR]

The modeling and forecasting of BBS (Bulletin Board System) posts time series is crucial for government agencies, corporations and website operators to monitor public opinion. Accurate prediction of the number of BBS posts will assist government agencies or corporations in making timely decisions and estimating the future number of BBS posts will help website operators to allocate resources to deal with the possible hot events pressure. By combining sample entropy (SampEn) and deep neural networks (DNN), an approach (SampEn-DNN) is proposed for BBS posts time series modeling and forecasting. The main idea of SampEn-DNN is to utilize SampEn to decide the input vectors of DNN with smallest complexity, and DNN to enhance the prediction performance of time series. Selecting Tianya Zatan new posts as the data source, the performances of SampEn-DNN were compared with auto-regressive integrated moving average (ARIMA), seasonal ARIMA, polynomial regression, neural networks, etc. approaches for prediction of the daily number of new posts. From the experimental results, it can be found that the proposed approach SampEn-DNN outperforms the state-of-the-art approaches for BBS posts time series modeling and forecasting. [ABSTRACT FROM AUTHOR]