Search

The accurate identification of a driver's braking intention is crucial to the formulation of regenerative braking control strategies for electric vehicles. In this paper, a braking intention recognition model based on the sample entropy of the braking signal and a probabilistic neural network (PNN) is proposed to achieve the accurate recognition of different braking intentions. Firstly, the brake pedal travel signal is decomposed to extract the effective components via variational modal decomposition (VMD); then, the features of the decomposed signal are extracted using sample entropy to obtain the multidimensional feature vector of the braking signal; finally, the sparrow search algorithm (SSA) and probabilistic neural network are combined to optimize the smoothing factor with the sparrow search algorithm and the cross-entropy loss function as the fitness function to establish a braking intention recognition model. The experimental validation results show that combining the sample entropy features of the braking signal with the probabilistic neural network can effectively identify the braking intention, and the SSA-PNN algorithm has higher recognition accuracy compared with the traditional machine learning algorithm. [ABSTRACT FROM AUTHOR]

The adoption of deep neural networks for profiling side-channel attacks opened new perspectives for leakage detection. Recent publications showed that cryptographic implementations featuring different countermeasures could be broken without feature selection or trace preprocessing. This success comes with a high price: an extensive hyperparameter search to find optimal deep learning models. As deep learning models usually suffer from overfitting due to their high fitting capacity, it is crucial to avoid over-training regimes, which require a correct number of epochs. For that, early stopping is employed as an efficient regularization method that requires a consistent validation metric. Although guessing entropy is a highly informative metric for profiling side-channel attacks, it is time-consuming, especially if computed for all epochs during training, and the number of validation traces is significantly large. This paper shows that guessing entropy can be efficiently computed during training by reducing the number of validation traces without affecting the efficiency of early stopping decisions. Our solution significantly speeds up the process, impacting the performance of the hyperparameter search and overall profiling attack. Our fast guessing entropy calculation is up to 16× faster, resulting in more hyperparameter tuning experiments and allowing security evaluators to find more efficient deep learning models. [ABSTRACT FROM AUTHOR]

This paper presents a method for the transparent, robust, and highly capacitive watermarking of video signals using an information mapper. The proposed architecture is based on the use of deep neural networks to embed the watermark in the luminance channel in the YUV color space. An information mapper was used to enable the transformation of a multi-bit binary signature of varying capacitance reflecting the entropy measure of the system into a watermark embedded in the signal frame. To confirm the effectiveness of the method, tests were carried out for video frames with a resolution of 256 × 256 pixels, with a watermark capacity of 4 to 16,384 bits. Transparency metrics (SSIM and PSNR) and a robustness metric—the bit error rate (BER)—were used to assess the performance of the algorithms. [ABSTRACT FROM AUTHOR]

Adversarial examples easily mislead vision systems based on deep neural networks (DNNs) trained with softmax cross entropy (SCE) loss. The vulnerability of DNN comes from the fact that SCE drives DNNs to fit on the training examples, whereas the resultant feature distributions between the training and adversarial examples are unfortunately misaligned. Several state-of-the-art methods start from improving the inter-class separability of training examples by modifying loss functions, where we argue that the adversarial examples are ignored, thus resulting in a limited robustness to adversarial attacks. In this paper, we exploited the inference region, which inspired us to apply margin-like inference information to SCE, resulting in a novel inference-softmax cross entropy (I-SCE) loss, which is intuitively appealing and interpretable. The inference information guarantees that it is difficult for neural networks to cross the decision boundary under an adversarial attack, and guarantees both the inter-class separability and the improved generalization to adversarial examples, which was further demonstrated and proved under the min-max framework. Extensive experiments show that the DNN models trained with the proposed I-SCE loss achieve a superior performance and robustness over the state-of-the-arts under different prevalent adversarial attacks; for example, the accuracy of I-SCE is 63% higher than SCE under the P G D 50 u n attack on the MNIST dataset. These experiments also show that the inference region can effectively solve the misaligned distribution. [ABSTRACT FROM AUTHOR]

The spiking neural network (SNN) is regarded as a promising candidate to deal with the great challenges presented by current machine learning techniques, including the high energy consumption induced by deep neural networks. However, there is still a great gap between SNNs and the online meta-learning performance of artificial neural networks. Importantly, existing spike-based online meta-learning models do not target the robust learning based on spatio-temporal dynamics and superior machine learning theory. In this invited article, we propose a novel spike-based framework with minimum error entropy, called MeMEE, using the entropy theory to establish the gradient-based online meta-learning scheme in a recurrent SNN architecture. We examine the performance based on various types of tasks, including autonomous navigation and the working memory test. The experimental results show that the proposed MeMEE model can effectively improve the accuracy and the robustness of the spike-based meta-learning performance. More importantly, the proposed MeMEE model emphasizes the application of the modern information theoretic learning approach on the state-of-the-art spike-based learning algorithms. Therefore, in this invited paper, we provide new perspectives for further integration of advanced information theory in machine learning to improve the learning performance of SNNs, which could be of great merit to applied developments with spike-based neuromorphic systems. [ABSTRACT FROM AUTHOR]

Distributed denial-of-service (DDoS) attack is one of the major threats to the web server. The rapid increase of DDoS attacks on the Internet has clearly pointed out the limitations in current intrusion detection systems or intrusion prevention systems (IDS/IPS), mostly caused by application-layer DDoS attacks. Within this context, the objective of the paper is to detect a DDoS attack using a multilayer perceptron (MLP) classification algorithm with genetic algorithm (GA) as learning algorithm. In this work, we analyzed the standard EPA-HTTP (environmental protection agency-hypertext transfer protocol) dataset and selected the parameters that will be used as input to the classifier model for differentiating the attack from normal profile. The parameters selected are the HTTP GET request count, entropy, and variance for every connection. The proposed model can provide a better accuracy of 98.31%, sensitivity of 0.9962, and specificity of 0.0561 when compared to other traditional classification models. [ABSTRACT FROM AUTHOR]

With the rapid development of deep neural networks, salient object detection has achieved great success in natural images. However, detecting salient objects from optical remote sensing images still remains a challenging task due to the diversity of object types, scale, shape and orientation variations, as well as cluttered backgrounds. Therefore, it is impractical to directly leverage methods designed for natural images to detect salient objects in optical remote sensing images. In this work, we present an end-to-end deep neural network for salient object detection in optical remote sensing images via global context relation-guided feature aggregation. Since the objects in remote sensing images often have a scattered distribution, we design a global context relation module to capture the global relationships between different spatial positions. In order to effectively integrate low-level appearance features as well as high-level semantic features for enhancing the final performance, we develop a feature aggregation module with the global context relation information as guidance and embed it into the backbone network to refine the deep features in a progressive manner. Instead of using traditional binary cross entropy as a training loss which treats all pixels equally, we design a weighted binary cross entropy to capture local surrounding information of different pixels. Extensive experiments on three public datasets are conducted to validate the efficiency of the proposed network and the results demonstrate that our proposed method consistently outperforms other competitors. [ABSTRACT FROM AUTHOR]

The use of a deep neural network scheme is proposed to help clinicians solve a difficult diagnosis problem in neurology. The proposed multilayer architecture includes a feature engineering step (from time-frequency transformation), a double compressing stage trained by unsupervised learning, and a classification stage trained by supervised learning. After fine-tuning, the deep network is able to discriminate well the class of patients from controls with around 90% sensitivity and specificity. This deep model gives better classification performance than some other standard discriminative learning algorithms. As in clinical problems there is a need for explaining decisions, an effort has been carried out to qualitatively justify the classification results. The main novelty of this paper is indeed to give an entropic interpretation of how the deep scheme works and reach the final decision. [ABSTRACT FROM AUTHOR]

The complexity of deep neural network models (DNNs) severely limits their application on devices with limited computing and storage resources. Knowledge distillation (KD) is an attractive model compression technology that can effectively alleviate this problem. Multi-teacher knowledge distillation (MKD) aims to leverage the valuable and diverse knowledge distilled by multiple teacher networks to improve the performance of the student network. Existing approaches typically rely on simple methods such as averaging the prediction logits or using sub-optimal weighting strategies to fuse distilled knowledge from multiple teachers. However, employing these techniques cannot fully reflect the importance of teachers and may even mislead student's learning. To address this issue, we propose a novel Decoupled Multi-Teacher Knowledge Distillation based on Entropy (DE-MKD). DE-MKD decouples the vanilla knowledge distillation loss and assigns adaptive weights to each teacher to reflect its importance based on the entropy of their predictions. Furthermore, we extend the proposed approach to distill the intermediate features from multiple powerful but cumbersome teachers to improve the performance of the lightweight student network. Extensive experiments on the publicly available CIFAR-100 image classification benchmark dataset with various teacher-student network pairs demonstrated the effectiveness and flexibility of our approach. For instance, the VGG8|ShuffleNetV2 model trained by DE-MKD reached 75.25%|78.86% top-one accuracy when choosing VGG13|WRN40-2 as the teacher, setting new performance records. In addition, surprisingly, the distilled student model outperformed the teacher in both teacher-student network pairs. [ABSTRACT FROM AUTHOR]

An introduction is presented in which the editor discusses various reports within the issue on topics including DNA encoding based encryption scheme, fuzzy linear system and synchronization of neural networks.

Rolling bearings are the most important parts in rotating machinery, and one of the most vulnerable parts to failure. The rolling bearing is a cyclic symmetrical structure that is stable under normal operating conditions. However, when the rolling bearing fails, its symmetry is destroyed, resulting in unstable performance and causing major accidents. If the performance of rolling bearings can be monitored and evaluated in real time, maintenance strategies can be implemented promptly. In this paper, by using wavelet packet energy entropy (WPEE), the early fault-free features of bearing and the failure samples of similar bearings are decomposed firstly, and the energy value is extracted as the original feature, simultaneously. Secondly, a radial basis function (RBF) neural network model is established by using early fault-free features and similar bearing failure characteristics. The bearing full-life data characteristics of the extracted features are added into the RBF model in an iterative manner to obtain performance degradation Indicator. Boxplot was introduced as an adaptive threshold method to determine the failure threshold. Finally, the results are verified by empirical mode decomposition and Hilbert envelope demodulation. A bearing accelerated life experiment is performed to validate the feasibility and validity of the proposed method. The experimental results show that the method can diagnose early fault points in time and evaluate the degree of bearing degradation, which is of great significance for industrial practical applications. [ABSTRACT FROM AUTHOR]

Analyzing deep neural networks (DNNs) via information plane (IP) theory has gained tremendous attention recently to gain insight into, among others, DNNs' generalization ability. However, it is by no means obvious how to estimate the mutual information (MI) between each hidden layer and the input/desired output to construct the IP. For instance, hidden layers with many neurons require MI estimators with robustness toward the high dimensionality associated with such layers. MI estimators should also be able to handle convolutional layers while at the same time being computationally tractable to scale to large networks. Existing IP methods have not been able to study truly deep convolutional neural networks (CNNs). We propose an IP analysis using the new matrix-based Rényi's entropy coupled with tensor kernels, leveraging the power of kernel methods to represent properties of the probability distribution independently of the dimensionality of the data. Our results shed new light on previous studies concerning small-scale DNNs using a completely new approach. We provide a comprehensive IP analysis of large-scale CNNs, investigating the different training phases and providing new insights into the training dynamics of large-scale neural networks. [ABSTRACT FROM AUTHOR]

Handwritten signatures are widely utilized as a form of personal recognition. However, they have the unfortunate shortcoming of being easily abused by those who would fake the identification or intent of an individual which might be very harmful. Therefore, the need for an automatic signature recognition system is crucial. In this paper, a signature recognition approach based on a probabilistic neural network (PNN) and wavelet transform average framing entropy (AFE) is proposed. The system was tested with a wavelet packet (WP) entropy denoted as a WP entropy neural network system (WPENN) and with a discrete wavelet transform (DWT) entropy denoted as a DWT entropy neural network system (DWENN). Our investigation was conducted over several wavelet families and different entropy types. Identification tasks, as well as verification tasks, were investigated for a comprehensive signature system study. Several other methods used in the literature were considered for comparison. Two databases were used for algorithm testing. The best recognition rate result was achieved by WPENN whereby the threshold entropy reached 92%. [ABSTRACT FROM AUTHOR]

In this research, a neural network (NN) model for metal price forecasting based on an evolutionary approach is proposed. Both the neural network model's network parameters and network architecture are selected automatically. The time series metal price data set is used to construct a novel fitness function that takes into account both error minimizations and the reproduction of the auto-correlation function. Calculating the average entropy values allowed the selection of the input parameter count for the neural network model. Gold price forecasting was performed using the proposed methodology. The optimal hidden node number, learning rate, and momentum are 9, 0.026, and 0.76, respectively, according to the evolutionary-based NN model. The proposed strategy is shown to reduce estimation error while also reproducing the auto-correlation function of the time series data set by the validation results with gold price data. The performance of the proposed method is better than other current methods, according to a comparison study. [ABSTRACT FROM AUTHOR]

This paper proposes a deep convolutional neural network model with encoder-decoder architecture to extract road network from satellite images. We employ ResNet-18 and Atrous Spatial Pyramid Pooling technique to trade off between the extraction precision and running time. A modified cross entropy loss function is proposed to train our deep model. A PointRend algorithm is used to recover a smooth, clear and sharp road boundary. The augmentated DeepGlobe dataset is used to train our deep model and the asynchronous training method is applied to accelerate the training process. Five salellite images covering Xiaomu village are taken as input to evaluate our model. The proposed E-Road model has fewer number of parameters and shorter training time. The experiments show E-Road outperforms other state-of-the-art deep models with 5.84% to 59.09% improvement, and can give the accurate predictions for the images with complex environment. [ABSTRACT FROM AUTHOR]

Image retrieval based on a convolutional neural network (CNN) has attracted great attention among researchers because of the high performance. The pooling method has become a research hotpot in the task of image retrieval in recent years. In this paper, we propose the feature distribution entropy (FDE) to measure the difference of regional distribution information in the feature maps from CNNs. We propose a novel pooling method, which fuses our proposed FDE with region maximum activations of convolutions (R-MAC) features to improve the performance of image retrieval, as it takes the advantage of regional distribution information in the feature maps. Compared with the descriptors computed by R-MAC pooling, our proposed method considers not only the most significant feature values of each region in feature map, but also the distribution difference in different regions. We utilize the histogram of feature values to calculate regional distribution entropy and concatenate the regional distribution entropy into FDE, which is further normalized and fused with R-MAC feature vectors by weighted summation to generate the final feature descriptors. We have conducted experiments on public datasets and the results demonstrate that our proposed method could produce better retrieval performances than existing state-of-the-art algorithms. Further, higher performance could be achieved by performing these post-processing on the improved feature descriptors. [ABSTRACT FROM AUTHOR]

Dynamic Light Scattering is a well-established technique used in particle sizing. An alternative procedure for Dynamic Light Scattering time series processing based on spectral entropy computation and Artificial Neural Networks is described. An error analysis of the proposed method was carried out and the results on both the simulated and on the experimental DLS time series are presented in detail. The results reveal the possibility of designing an advanced sensor capable of detecting particles with a size bigger than a threshold using this alternative for processing the DLS time series. [ABSTRACT FROM AUTHOR]

Purpose: In this work, we propose an implementation of the Bienenstock–Cooper–Munro (BCM) model, obtained by a combination of the classical framework and modern deep learning methodologies. The BCM model remains one of the most promising approaches to modeling the synaptic plasticity of neurons, but its application has remained mainly confined to neuroscience simulations and few applications in data science. Methods: To improve the convergence efficiency of the BCM model, we combine the original plasticity rule with the optimization tools of modern deep learning. By numerical simulation on standard benchmark datasets, we prove the efficiency of the BCM model in learning, memorization capacity, and feature extraction. Results: In all the numerical simulations, the visualization of neuronal synaptic weights confirms the memorization of human-interpretable subsets of patterns. We numerically prove that the selectivity obtained by BCM neurons is indicative of an internal feature extraction procedure, useful for patterns clustering and classification. The introduction of competitiveness between neurons in the same BCM network allows the network to modulate the memorization capacity of the model and the consequent model selectivity. Conclusions: The proposed improvements make the BCM model a suitable alternative to standard machine learning techniques for both feature selection and classification tasks. [ABSTRACT FROM AUTHOR]

Measuring the predictability and complexity of 2D data (image) series using entropy is an essential tool for evaluation of systems' irregularity and complexity in remote sensing and geophysical mapping. However, the existing methods have some drawbacks related to their strong dependence on method parameters and image rotation. To overcome these difficulties, this study proposes a new method for estimating two-dimensional neural network entropy (NNetEn2D) for evaluating the regularity or predictability of images using the LogNNet neural network model. The method is based on an algorithm for converting a 2D kernel into a 1D data series followed by NNetEn2D calculation. An artificial test image was created for the study. We demonstrate the advantage of using circular instead of square kernels through comparison of the invariance of the NNetEn2D distribution after image rotation. Highest robustness was observed for circular kernels with a radius of R = 5 and R = 6 pixels, with a NNetEn2D calculation error of no more than 10%, comparable to the distortion of the initial 2D data. The NNetEn2D entropy calculation method has two main geometric parameters (kernel radius and its displacement step), as well as two neural network hyperparameters (number of training epochs and one of six reservoir filling techniques). We evaluated our method on both remote sensing and geophysical mapping images. Remote sensing imagery (Sentinel-2) shows that brightness of the image does not affect results, which helps keep a rather consistent appearance of entropy maps over time without saturation effects being observed. Surfaces with little texture, such as water bodies, have low NNetEn2D values, while urban areas have consistently high values. Application to geophysical mapping of rocks to the northwest of southwest Australia is characterized by low to medium entropy and highlights aspects of the geology. These results indicate the success of NNetEn2D in providing meaningful entropy information for 2D in remote sensing and geophysical applications. [ABSTRACT FROM AUTHOR]

In this paper, we study Tsallis' fractional entropy (TFE) in a complex domain by applying the definition of the complex probability functions. We study the upper and lower bounds of TFE based on some special functions. Moreover, applications in complex neural networks (CNNs) are illustrated to recognize the accuracy of CNNs. [ABSTRACT FROM AUTHOR]

The performance evaluation of wind power forecasting under commercially operating circumstances is critical to a wide range of decision-making situations, yet difficult because of its stochastic nature. This paper firstly introduces a novel TRSWA-BP neural network, of which learning process is based on an efficiency tabu, real-coded, small-world optimization algorithm (TRSWA). In order to deal with the strong volatility and stochastic behavior of the wind power sequence, three forecasting models of the TRSWA-BP are presented, which are combined with EMD (empirical mode decomposition), PSR (phase space reconstruction), and EMD-based PSR. The error sequences of the above methods are then proved to have non-Gaussian properties, and a novel criterion of normalized Renyi’s quadratic entropy (NRQE) is proposed, which can evaluate their dynamic predicted accuracy. Finally, illustrative predictions of the next 1, 4, 6, and 24 h time-scales are examined by historical wind power data, under different evaluations. From the results, we can observe that not only do the proposed models effectively revise the error due to the fluctuation and multi-fractal property of wind power, but also that the NRQE can reserve its feasible assessment upon the stochastic predicted error. [ABSTRACT FROM AUTHOR]

Artificial neural networks are currently one of the most commonly used classifiers and over the recent years they have been successfully used in many practical applications, including banking and finance, health and medicine, engineering and manufacturing. A large number of error functions have been proposed in the literature to achieve a better predictive power. However, only a few works employ Tsallis statistics, although the method itself has been successfully applied in other machine learning techniques. This paper undertakes the effort to examine the q -generalized function based on Tsallis statistics as an alternative error measure in neural networks. In order to validate different performance aspects of the proposed function and to enable identification of its strengths and weaknesses the extensive simulation was prepared based on the artificial benchmarking dataset. The results indicate that Tsallis entropy error function can be successfully introduced in the neural networks yielding satisfactory results and handling with class imbalance, noise in data or use of non-informative predictors. [ABSTRACT FROM AUTHOR]

This study developed a structural health monitoring (SHM) system based on refined composite multiscale cross-sample entropy (RCMCSE) and an artificial neural network for monitoring structures under ambient vibrations. RCMCSE was applied to enhance the reliability of entropy estimations. First, RCMCSE was implemented to extract damage features, and finite element analysis software was then used to generate training samples, which included stiffness reductions to achieve various damage patterns. A neural network model was constructed and trained using entropy values for these damage patterns. An experiment was conducted on a seven-story steel benchmark structure to validate the performance of the proposed system. Additionally, a confusion matrix was established to evaluate the performance of the proposed system. The results obtained for a scaled-down benchmark structure indicated that 89.8% of the floors were accurately classified, and 90% of the practical damaged floors were correctly diagnosed. The performance evaluation demonstrated that the proposed SHM system exhibited increased damage location accuracy. [ABSTRACT FROM AUTHOR]

Recently, the Entropy Ensemble Filter (EEF) method was proposed to mitigate the computational cost of the Bootstrap AGGregatING (bagging) method. This method uses the most informative training data sets in the model ensemble rather than all ensemble members created by the conventional bagging. In this study, we evaluate, for the first time, the application of the EEF method in Neural Network (NN) modeling of El Nino-southern oscillation. Specifically, we forecast the first five principal components (PCs) of sea surface temperature monthly anomaly fields over tropical Pacific, at different lead times (from 3 to 15 months, with a three-month increment) for the period 1979-2017. We apply the EEF method in a multiple-linear regression (MLR) model and two NN models, one using Bayesian regularization and one Levenberg-Marquardt algorithm for training, and evaluate their performance and computational efficiency relative to the same models with conventional bagging. All models perform equally well at the lead time of 3 and 6 months, while at higher lead times, the MLR model's skill deteriorates faster than the nonlinear models. The neural network models with both bagging methods produce equally successful forecasts with the same computational efficiency. It remains to be shown whether this finding is sensitive to the dataset size. [ABSTRACT FROM AUTHOR]

Over the past two decades, the Bootstrap AGGregatING (bagging) method has been widely used for improving simulation. The computational cost of this method scales with the size of the ensemble, but excessively reducing the ensemble size comes at the cost of reduced predictive performance. The novel procedure proposed in this study is the Entropy Ensemble Filter (EEF), which uses the most informative training data sets in the ensemble rather than all ensemble members created by the bagging method. The results of this study indicate efficiency of the proposed method in application to synthetic data simulation on a sinusoidal signal, a sawtooth signal, and a composite signal. The EEF method can reduce the computational time of simulation by around 50% on average while maintaining predictive performance at the same level of the conventional method, where all of the ensemble models are used for simulation. The analysis of the error gradient (root mean square error of ensemble averages) shows that using the 40% most informative ensemble members of the set initially defined by the user appears to be most effective. [ABSTRACT FROM AUTHOR]

Response surface methodology (RSM) and artificial neural networks (ANN) were evaluated and compared in order to decide which method was the most appropriate to predict and optimize total phenolic content (TPC) and oleuropein yields in olive tree leaf (Olea europaea) extracts, obtained after solvent-free microwave-assisted extraction (SFMAE). The SFMAE processing conditions were: microwave irradiation power 250-350 W, extraction time 2-3 min, and the amount of sample 5-10 g. The calculation of the standard enthalpies of vaporization, sublimation and solvation of organic molecules is presented using a common computer algorithm on the basis of a group-additivity method. The same algorithm is also shown to enable the calculation of their entropy of fusion as well as the total phase-change entropy of liquid crystals. The present method is based on the complete breakdown of the molecules into their constituting atoms and their immediate neighbourhood; the respective calculations of the contribution of the atomic groups by means of the Gauss-Seidel fitting method is based on experimental data collected from literature. The feasibility of the calculations for each of the mentioned descriptors was verified by means of a 10-fold cross-validation procedure proving the good to high quality of the predicted values for the three mentioned enthalpies and for the entropy of fusion, whereas the predictive quality for the total phase-change entropy of liquid crystals was poor. The goodness of fit (Q²) and the standard deviation (≤) of the cross-validation calculations for the five descriptors was as follows: 0.9641 and 4.56 kJ/mol (N = 3386 test molecules) for the enthalpy of vaporization, 0.8657 and 11.39 kJ/mol (N = 1791) for the enthalpy of sublimation, 0.9546 and 4.34 kJ/mol (N = 373) for the enthalpy of solvation, 0.8727 and 17.93 J/mol/K (N = 2637) for the entropy of fusion and 0.5804 and 32.79 J/mol/K (N = 2643) for the total phase-change entropy of liquid crystals. The large discrepancy between the results of the two closely related entropies is discussed in detail. Molecules for which both the standard enthalpies of vaporization and sublimation were calculable, enabled the estimation of their standard enthalpy of fusion by simple subtraction of the former from the latter enthalpy. For 990 of them the experimental enthalpy-of-fusion values are also known, allowing their comparison with predictions, yielding a correlation coefficient R² of 0.6066. [ABSTRACT FROM AUTHOR]

Measuring the predictability and complexity of time series using entropy is essential tool designing and controlling a nonlinear system. However, the existing methods have some drawbacks related to the strong dependence of entropy on the parameters of the methods. To overcome these difficulties, this study proposes a new method for estimating the entropy of a time series using the LogNNet neural network model. The LogNNet reservoir matrix is filled with time series elements according to our algorithm. The accuracy of the classification of images from the MNIST-10 database is considered as the entropy measure and denoted by NNetEn. The novelty of entropy calculation is that the time series is involved in mixing the input information in the reservoir. Greater complexity in the time series leads to a higher classification accuracy and higher NNetEn values. We introduce a new time series characteristic called time series learning inertia that determines the learning rate of the neural network. The robustness and efficiency of the method is verified on chaotic, periodic, random, binary, and constant time series. The comparison of NNetEn with other methods of entropy estimation demonstrates that our method is more robust and accurate and can be widely used in practice. [ABSTRACT FROM AUTHOR]

Diabetic retinopathy (DR) is a common complication of diabetes mellitus (DM), and it is necessary to diagnose DR in the early stages of treatment. With the rapid development of convolutional neural networks in the field of image processing, deep learning methods have achieved great success in the field of medical image processing. Various medical lesion detection systems have been proposed to detect fundus lesions. At present, in the image classification process of diabetic retinopathy, the fine-grained properties of the diseased image are ignored and most of the retinopathy image data sets have serious uneven distribution problems, which limits the ability of the network to predict the classification of lesions to a large extent. We propose a new non-homologous bilinear pooling convolutional neural network model and combine it with the attention mechanism to further improve the network's ability to extract specific features of the image. The experimental results show that, compared with the most popular fundus image classification models, the network model we proposed can greatly improve the prediction accuracy of the network while maintaining computational efficiency. [ABSTRACT FROM AUTHOR]

Samples of triticale seeds of various qualities were assessed in the study. The seeds were obtained during experiments, reflecting the actual sowing conditions. The experiments were conducted on an original test facility designed by the authors of this study. The speed of the air (15, 20, 25 m/s) transporting seeds in the pneumatic conduit was adjusted to sowing. The resulting graphic database enabled the distinction of six classes of seeds according to their quality and sowing speed. The database was prepared to build training, validation and test sets. The neural model generation process was based on multi-layer perceptron networks (MLPN) and statistical (machine training). When the MLPN was used to identify contaminants in seeds sown at a speed of 15 m/s, the lowest RMS error of 0.052 was noted, whereas the classification correctness coefficient amounted to 0.99. [ABSTRACT FROM AUTHOR]

The discrimination of non-focal class (NFC) and focal class (FC), is vital in localizing the epileptogenic zone (EZ) during neurosurgery. In the conventional diagnosis method, the neurologist has to visually examine the long hour electroencephalogram (EEG) signals, which consumes time and is prone to error. Hence, in this present work, automated diagnosis of FC EEG signals from NFC EEG signals is developed using the Fast Walsh–Hadamard Transform (FWHT) method, entropies, and artificial neural network (ANN). The FWHT analyzes the EEG signals in the frequency domain and decomposes it into the Hadamard coefficients. Five different nonlinear features, namely approximate entropy (ApEn), log-energy entropy (LogEn), fuzzy entropy (FuzzyEn), sample entropy (SampEn), and permutation entropy (PermEn) are extracted from the decomposed Hadamard coefficients. The extracted features detail the nonlinearity in the NFC and the FC EEG signals. The judicious entropy features are supplied to the ANN classifier, with a 10-fold cross-validation method to classify the NFC and FC classes. Two publicly available datasets such as the University of Bonn and Bern-Barcelona dataset are used to evaluate the proposed approach. A maximum sensitivity of 99.70%, the accuracy of 99.50%, and specificity of 99.30% with the 3750 pairs of NFC and FC signal are achieved using the Bern-Barcelona dataset, while the accuracy of 92.80%, the sensitivity of 91%, and specificity of 94.60% is achieved using University of Bonn dataset. Compared to the existing technique, the proposed approach attained a maximum classification performance in both the dataset. [ABSTRACT FROM AUTHOR]

Analysis of electroencephalogram (EEG) signals is essential because it is an efficient method to diagnose neurological brain disorders. In this work, a single system is developed to diagnose one or two neurological diseases at the same time (two-class mode and three-class mode). For this purpose, different EEG feature-extraction and classification techniques are investigated to aid in the accurate diagnosis of neurological brain disorders: epilepsy and autism spectrum disorder (ASD). Two different modes, single-channel and multi-channel, of EEG signals are analyzed for epilepsy and ASD. The independent components analysis (ICA) technique is used to remove the artifacts from EEG dataset. Then, the EEG dataset is segmented and filtered to remove noise and interference using an elliptic band-pass filter. Next, the EEG signal features are extracted from the filtered signal using a discrete wavelet transform (DWT) to decompose the filtered signal to its sub-bands delta, theta, alpha, beta and gamma. Subsequently, five statistical methods are used to extract features from the EEG sub-bands: the logarithmic band power (LBP), standard deviation, variance, kurtosis, and Shannon entropy (SE). Further, the features are fed into four different classifiers, linear discriminant analysis (LDA), support vector machine (SVM), k-nearest neighbor (KNN), and artificial neural networks (ANNs), to classify the features corresponding to their classes. The combination of DWT with SE and LBP produces the highest accuracy among all the classifiers. The overall classification accuracy approaches 99.9% using SVM and 97% using ANN for the three-class single-channel and multi-channel modes, respectively. [ABSTRACT FROM AUTHOR]

The image quality evaluation method, based on the convolutional neural network (CNN), achieved good evaluation performance. However, this method can easily lead the visual quality of image sub-blocks to change with the spatial position after the image is processed by various distortions. Consequently, the visual quality of the entire image is difficult to reflect objectively. On this basis, this study combines wavelet transform and CNN method to propose an image quality evaluation method based on wavelet CNN. The low-frequency, horizontal, vertical, and diagonal sub-band images decomposed by wavelet transform are selected as the inputs of convolution neural network. The feature information in multiple directions is extracted by convolution neural network. Then, the information entropy of each sub-band image is calculated and used as the weight of each sub-band image quality. Finally, the quality evaluation values of four sub-band images are weighted and fused to obtain the visual quality values of the entire image. Experimental results show that the proposed method gains advantage from the global and local information of the image, thereby further improving its effectiveness and generalization. [ABSTRACT FROM AUTHOR]

Irregular spatial dependency is one of the major characteristics of remote sensing images, which brings about challenges for classification tasks. Deep supervised models such as convolutional neural networks (CNNs) have shown great capacity for remote sensing image classification. However, they generally require a huge labeled training set for the fine tuning of a deep neural network. To handle the irregular spatial dependency of remote sensing images and mitigate the conflict between limited labeled samples and training demand, we design a superpixel-guided layer-wise embedding CNN (SLE-CNN) for remote sensing image classification, which can efficiently exploit the information from both labeled and unlabeled samples. With the superpixel-guided sampling strategy for unlabeled samples, we can achieve an automatic determination of the neighborhood covering for a spatial dependency system and thus adapting to real scenes of remote sensing images. In the designed network, two types of loss costs are combined for the training of CNN, i.e., supervised cross entropy and unsupervised reconstruction cost on both labeled and unlabeled samples, respectively. Our experimental results are conducted with three types of remote sensing data, including hyperspectral, multispectral, and synthetic aperture radar (SAR) images. The designed SLE-CNN achieves excellent classification performance in all cases with a limited labeled training set, suggesting its good potential for remote sensing image classification. [ABSTRACT FROM AUTHOR]

This study used Xilinx Field Programmable Gate Arrays (FPGAs) to implement a functional neuro-fuzzy network (FNFN) for solving nonlinear control problems. A functional link neural network (FLNN) was used as the consequent part of the proposed FNFN model. This study adopted the linear independent functions and the orthogonal polynomials in a functional expansion of the FLNN. Thus, the design of the FNFN model could improve the control accuracy. The learning algorithm of the FNFN model was divided into structure learning and parameter learning. The entropy measurement was adopted in the structure learning to determine the generated new fuzzy rule, whereas the gradient descent method in the parameter learning was used to adjust the parameters of the membership functions and the weights of the FLNN. In order to obtain high speed operation and real-time application, a very high speed integrated circuit hardware description language (VHDL) was used to design the FNFN controller and was implemented on FPGA. Finally, the experimental results demonstrated that the proposed hardware implementation of the FNFN model confirmed the viability in the temperature control of a water bath and the backing control of a car. [ABSTRACT FROM AUTHOR]

Contemplating the importance of studying current–voltage curves in superconductivity, it has been recently and rightly argued that their approximation, rather than incessant measurements, seems to be a more viable option. This especially becomes bona fide when the latter needs to be recorded for a wide range of critical parameters including temperature and magnetic field, thereby becoming a tedious monotonous procedure. Artificial neural networks have been recently put forth as one methodology for approximating these so-called electrical measurements for various geometries of antidots on a superconducting thin film. In this work, we demonstrate that the prediction accuracy, in terms of mean-squared error, achieved by artificial neural networks is rather constrained, and, due to their immense credence on randomly generated networks' coefficients, they may result in vastly varying prediction accuracies for different geometries, experimental conditions, and their own tunable parameters. This inconsistency in prediction accuracies is resolved by controlling the uncertainty in networks' initialization and coefficients' generation by means of a novel entropy based genetic algorithm. The proposed method helps in achieving a substantial improvement and consistency in the prediction accuracy of current–voltage curves in comparison to existing works, and is amenable to various geometries of antidots, including rectangular, square, honeycomb, and kagome, on a superconducting thin film. [ABSTRACT FROM AUTHOR]