Search

De-mining operations are of critical importance for humanitarian efforts and safety in conflict-affected regions. In this paper, we address the challenge of enhancing the accuracy and efficiency of mine detection systems. We present an innovative Deep Learning architecture tailored for pulse induction-based Metallic Mine Detectors (MMD), so called DL-MMD. Our methodology leverages deep neural networks to distinguish amongst nine distinct materials with an exceptional validation accuracy of 93.5%. This high level of precision enables us not only to differentiate between anti-personnel mines, without metal plates but also to detect minuscule 0.2-g vertical paper pins in both mineralized soil and non-mineralized environments. Moreover, through comparative analysis, we demonstrate a substantial 3% and 7% improvement (approx.) in accuracy performance compared to the traditional K-Nearest Neighbors and Support Vector Machine classifiers, respectively. The fusion of deep neural networks with the pulse induction-based MMD not only presents a cost-effective solution but also significantly expedites decision-making processes in de-mining operations, ultimately contributing to improved safety and effectiveness in these critical endeavors. [ABSTRACT FROM AUTHOR]

This research paper aims to ensure the reliable and efficient operation of grid-connected solid-state transformers (SSTs) by detecting and evaluating various undesirable operating conditions. The study considers different types of faults, including internal faults like open switches and open capacitors, external faults such as symmetrical and asymmetrical faults occurring at various locations of the SST, and abnormalities on the grid side known as sympathetic inrush conditions. To analyze these operating conditions, the secondary current of the high-frequency transformer is normalized and decomposed using the discrete wavelet transform (DWT) and wavelet packet transform (WPT). From the DWT and WPT decomposition at multiple levels, several statistical parameters are calculated. These statistical parameters are carefully selected from different decomposition levels to enhance the effectiveness of the detection algorithm utilizing DWT and WPT. In order to quickly identify and classify all operating conditions that impact the performance of the grid-connected SST, a three-layer feedforward artificial neural network (ANN) is employed, using the selected statistical features. The accuracy and efficiency of the ANN-based classification approach are evaluated by assessing the effectiveness of the statistical features obtained from DWT and WPT. Simulation results have been altered by introducing various noise levels to systematically assess the performance of the proposed algorithms. The average accuracy of the DWT-ANN algorithm stands at 97.89%, while the WPT-ANN algorithm achieves a slightly elevated accuracy level of 98.01%. This notable similarity in accuracy curtails from the careful selection of the wavelet function, decomposition level, and feature sets. [ABSTRACT FROM AUTHOR]

The paper studies the effectiveness of machine learning techniques in predicting the microhardness of composite materials manufactured from a mixture of iron-copper (Fe-Cu) nanopowders and hydroxyapatite (HA) particles. It is shown that the different proportion of Fe-Cu and HA powders and the polymer fraction in the composite significantly affect its hardness. A surrogate model based on the artificial neural network (ANN) and the analytical model, is proposed to quantitatively evaluate the microhardness, depending on the powder/polymer ratio. The ANN models show the probability distribution of indentation values after microhardness testing, which is the input data for the analytical model to compute the hardness. This approach can reduce the time and cost of the material research and minimizes the reliance on expensive materials and experimental equipment. [ABSTRACT FROM AUTHOR]

In industrial applications, three-phase induction motors (IMs) are widely used, and they are subjected to many types of faults. One such fault is the loss of a motor phase caused by a blown fuse, a broken wire, mechanical damage, etc. When this fault occurs during motor operation, it continues to rotate but experiences rapid heating, which can ultimately lead to motor failure. Therefore, various protection devices are available to protect the motor against this fault, but most traditional protection devices do not offer a comprehensive classification of such a fault. So, in this paper, an efficient neural network model is presented for detecting and classifying 12 types of phase loss faults for a three-phase induction motor (IM) based on factors such as the unhealthy phase, fault location, and motor action modes (standstill, transient, and steady-state modes). Thus, the main goal of this work is to determine the motor mode during the fault, the defective phase, and its location to help the maintenance team repair the fault quickly. The system is simulated and tested using the "MATLAB/Simulink" software, employing a feed-forward neural network. The simulation results demonstrate that the proposed network achieves correct detection and classification of phase loss faults within a short time frame from the occurrence of the fault. Therefore, the proposed network model proves to be a simple and reliable solution for integration into the protection system of a three-phase IM, enabling the detection and classification of various phase loss faults. [ABSTRACT FROM AUTHOR]

The paper examines the implementation of neural network methods for predicting peak water levels during the period of spring ice drift by the example of the Sukhona, Northern Dvina, and Pechora rivers. All considered neural network methods have shown high efficiency according to the criteria recommended by the Hydrometcenter of Russia and surpassed regression dependencies in the skill of forecasts. When using the method of training artificial neural networks, the standard error of prediction is reduced by approximately 10–20% as compared with regression dependencies. [ABSTRACT FROM AUTHOR]

This paper introduces a measure, called Lipschitz widths, of the optimal performance possible of certain nonlinear methods of approximation. Notably, those Lipschitz widths provide a theoretical benchmark for the approximation quality achieved via deep neural networks. The paper also discusses basic properties of the Lipschitz widths and their relation to entropy numbers and other well-known widths such as the Kolmogorov and the stable manifold widths. We show that Lipschitz widths with fixed Lipschitz constant and entropy numbers decay very similar, while when the Lipschitz constant grows with n, the Lipschitz width could be much smaller than the entropy numbers. [ABSTRACT FROM AUTHOR]

To reliably and accurately classify complicated "big" datasets, machine learning models must be continually improved. This research proposes straightforward yet competitive neural networks for text classification, even though graph neural networks (GNN) have reignited interest in graph-based text classification models. Convolutional neural networks (CNN), artificial neural networks (ANN), and their refined "fine-tuned" models (denoted as FT-CNN and FT-ANN) are the names given to our proposed models. The models presented in this paper demonstrate that our simple models like (CNN, ANN, FT-CNN, and FT-ANN) can perform better than more complex GNN ones such as (SGC, SSGC, and TextGCN) and are comparable to others (i.e., HyperGAT and Bert). The process of fine-tuning is also highly recommended because it improves the performance and reliability of models. The performance of our suggested models on five benchmark datasets (namely, Reuters (R8), R52, 20NewsGroup, Ohsumed, and Mr) is vividly illustrated. According to the experimental findings, on the majority of the target datasets, these models—especially those that have been fine-tuned—perform surprisingly better than SOTA approaches, including GNN-based models. [ABSTRACT FROM AUTHOR]

This paper presents the results of experimental study of the AZ31 magnesium alloy milling process. Dry milling was carried out under high-speed machining conditions. First, a stability lobe diagram was determined using CutPro software. Next, experimental studies were carried out to verify the stability lobe diagram. The tests were carried out for different feed per tooth and cutting speed values using two types of tools. During the experimental investigations, cutting forces in three directions were recorded. The obtained time series were subjected to general analysis and analysis using composite multiscale entropy. Modelling and prediction were performed using Statistica Neural Network software, in which two types of neural networks were applied: multi-layered perceptron and radial basis function. It was observed that milling with high cutting speed values allows for component values of cutting force to be lowered as a result of the transition into the high-speed machining conditions range. In most cases, the highest values for the analysed parameters were recorded for the component Fx, whereas the lowest were recorded for Fy. Additionally, the paper shows that a prediction (with the use of artificial neural networks) of the components of cutting force can be made, both for the amplitudes of components of cutting force Famp and for root mean square Frms. [ABSTRACT FROM AUTHOR]

The possibility of solving problems of chemical kinetics using artificial neural networks is investigated. The main laboriousness of solving problems of chemical kinetics lies in solving a rigid system of balance equations, whose right side contains the component mass production intensity. This problem can be singled out as a separate stage of solving a system of ordinary differential equations within a common time step of the global problem, and this stage is considered in this paper. A fairly simple model is developed that can solve this problem, which makes it possible to achieve a threefold acceleration of calculations as compared to numerical methods. The resulting neural network operates in a recurrent mode and can predict the behavior of a chemical multicomponent dynamic system many steps ahead. [ABSTRACT FROM AUTHOR]

In machine learning models, one of the most popular models is artificial neural networks. The activation function is one of the important parameters of neural networks. In this paper, the sigmoid function is used as an activation function with a fractional derivative approach to minimize the convergence error in backpropagation and to maximize the generalization performance of neural networks. The proportional Caputo definition is considered a fractional derivative. We evaluated three neural network models on the usage of the proportional Caputo derivative. The results show that the proportional Caputo derivative approach has higher classification accuracy than traditional derivative models in backpropagation for neural networks with and without L2 regularization. [ABSTRACT FROM AUTHOR]

The popular directions for automatic modulation classification algorithms investigate how to develop feature extraction methods for further signal classification. In this paper, we propose a mapping algorithm for a manually designed feature extraction method by using the reconstruction component of principal component analysis (PCA), which further extracts discrimination between signal features via a PCA reconstruction component. Two supervised neural network models are studied to achieve the limits of the learning matrix in modulation signal classification. Some experimental results show that different modulation schemes can be obviously classified using matrix mapping for feature extraction. Moreover, the modulation classification accuracy based on the mapping extraction feature, which has a lower SNR requirement for training, is slightly improved compared with some triradial deep learning methods. [ABSTRACT FROM AUTHOR]

Oxygen content in the flue gas system of power plants is an essential factor affecting boiler efficiency. Accurate oxygen content measurement is vital in evaluating boiler combustion efficiency. The device measuring oxygen content in flue gases at an oil refinery uses a Zirconia oxygen analyzer. This sensor utilization without sensor redundancy makes the oxygen content measurement conducted manually. Workers' manual measurement is risky because it is a high-risk work area. In addition, the oxygen content in flue gas also indicates boiler combustion efficiency and the amount of other harmful gases produced by the boiler. This paper proposes a soft sensor using artificial neural networks (ANN) and extreme gradient boosting (XGBoost) to predict oxygen content. The dataset used is collected from the historical data of the distributed control system of an oil refinery system boiler. The experimental results show that the one hidden layer ANN model achieves an MAE of 0.0715 and RMSE of 0.0935, while the XGBoost model with hyperparameter tuning and seven features achieves an MAE of 0.0452 and RMSE of 0.0642. The results suggest that the XGBoost model with hyperparameter tuning and seven features outperforms the one hidden layer ANN model. The use of the seven features of the XGBoost model is the result of optimization between computational complexity and system performance. [ABSTRACT FROM AUTHOR]

The proposed paper presents the analysis, design, implementation and evaluation of an ultra-short-term frequency trading system for the foreign exchange (FOREX) market, which features all stages of the trading process (Pretrade Analysis, Trend Forecasting, Transaction Execution) substantially exploiting artificial intelligence techniques. Our goal is to simulate the judgment and decision making of the human expert (technical analyst or broker) with a system that responds in a timely manner to changes in market conditions, thus allowing the optimization of ultra-short-term transactions. We designed and implemented a series of technical indicator simulators, which are fed to a novel artificial neural network architecture, to eventually generate the trend forecasting signal. We also designed and implemented a series of customizable ultra-short-term automated trading machines, which receive as inputs the generated forecasting signals and perform real-time virtual transactions. A comparative analysis of the results of both automated trading machines and each machine is carried out for a comprehensive variety of trend forecasting sources. [ABSTRACT FROM AUTHOR]

In recent years, various depth sensors that are small enough to be used with mobile hardware have been introduced. They provide important information for use cases like 3D reconstruction or in the context of augmented reality where tracking and camera data alone would be insufficient. However, depth sensors may not always be available due to hardware limitations or when simulating augmented reality applications for prototyping purposes. In these cases, different approaches like stereo matching or depth estimation using neural networks may provide a viable alternative. In this paper, we therefore explore the imitation of depth sensors using deep neural networks. For this, we use a state-of-the-art network for depth estimation and adapt it in order to mimic a Structure Sensor as well as an iPad LiDAR sensor. We evaluate the network which was pre-trained on NYU V2 directly as well as several variations where transfer learning is applied in order to adapt the network to different depth sensors while using various data preprocessing and augmentation techniques. We show that a transfer learning approach together with appropriate data processing can enable an accurate modeling of the respective depth sensors. [ABSTRACT FROM AUTHOR]

The smarter world needs more efforts to purposeful manage and usage of technologies, science, artificial intelligence, and artificial neural networks, as their product. One of the main tools to facilitate collecting beneficial information is reviewing the publications of the target domain. To this end, the present paper attempts to categorize recent trends and innovations in the application of artificial neural networks in water quality and pollution research. The most prevalent methods for water pollution and quality prediction modeling during 2011–2020 were BPNN and normal MLP.. Conducting and comparing other analytical methods such as ANN, RBFNN, LSTM, and CNN to find accurate results are in the next orders. Moreover, along with the emerging and development of the internet of things (IoT), it was observed that IoT as a real-time technology for collecting relevant data, which appeared in the water quality analysis since 2016–2017, was used increasingly in 2019 and 2020. Furthermore, reviewed papers revealed that pH, chemical oxygen demand, and biochemical oxygen demand were the most water quality indicators. Regionally, drivers in China were the most investigated sites during the last decade, and then India and Turkey were the second and third place. [ABSTRACT FROM AUTHOR]

Facial emotion recognition extracts the human emotions from the images and videos. As such, it requires an algorithm to understand and model the relationships between faces and facial expressions and to recognize human emotions. Recently, deep learning models are utilized to improve the performance of facial emotion recognition. However, the deep learning models suffer from the overfitting issue. Moreover, deep learning models perform poorly for images which have poor visibility and noise. Therefore, in this paper, an efficient deep learning-based facial emotion recognition model is proposed. Initially, contrast-limited adaptive histogram equalization (CLAHE) is applied to improve the visibility of input images. Thereafter, a modified joint trilateral filter is applied to the obtained enhanced images to remove the impact of impulsive noise. Finally, an efficient deep convolutional neural network is designed. Adam optimizer is also utilized to optimize the cost function of deep convolutional neural networks. Experiments are conducted by using the benchmark dataset and competitive human emotion recognition models. Comparative analysis demonstrates that the proposed facial emotion recognition model performs considerably better compared to the competitive models [ABSTRACT FROM AUTHOR]

In order to manufacture functional parts using filament deposition modelling (FDM), an understanding of the machine's capabilities is necessary. Eliciting this understanding poses a significant challenge due to a lack of knowledge relating manufacturing process parameters to mechanical properties of the manufactured part. Prior work has proposed that this could be overcome through the creation of capability profiles for FDM machines. However, such an approach has yet to be implemented and incorporated into the overall design process. Correspondingly, the aim of this paper is two-fold and includes the creation of a comprehensive capability profile for FDM and the implementation of the profile and evaluation of its utility within a generative design methodology. To provide the foundations for the capability profile, this paper first reports an experimental testing programme to characterise the influence of five manufacturing parameters on a part's ultimate tensile strength (UTS) and tensile modulus (E). This characterisation is used to train an artificial neural network (ANN). This ANN forms the basis of a capability profile that is shown to be able to represent the mechanical properties with RMSEP of 1.95 MPa for UTS and 0.82 GPa for E. To validate the capability profile, it is incorporated into a generative design methodology enabling its application to the design and manufacture of functional parts. The resulting methodology is used to create two load bearing components where it is shown to be able to generate parts with satisfactory performance in only a couple of iterations. The novelty of the reported work lies in demonstrating the practical application of capability profiles in the FDM design process and how, when combined with generative approaches, they can make effective design decisions in place of the user. [ABSTRACT FROM AUTHOR]

The present paper intends to draw the conception of language implied in the technique of word embeddings that supported the recent development of deep neural network models in computational linguistics. After a preliminary presentation of the basic functioning of elementary artificial neural networks, we introduce the motivations and capabilities of word embeddings through one of its pioneering models, word2vec. To assess the remarkable results of the latter, we inspect the nature of its underlying mechanisms, which have been characterized as the implicit factorization of a word-context matrix. We then discuss the ordinary association of the "distributional hypothesis" with a "use theory of meaning," often justifying the theoretical basis of word embeddings, and contrast them to the theory of meaning stemming from those mechanisms through the lens of matrix models (such as vector space models and distributional semantic models). Finally, we trace back the principles of their possible consistency through Harris's original distributionalism up to the structuralist conception of language of Saussure and Hjelmslev. Other than giving access to the technical literature and state of the art in the field of natural language processing to non-specialist readers, the paper seeks to reveal the conceptual and philosophical stakes involved in the recent application of new neural network techniques to the computational treatment of language. [ABSTRACT FROM AUTHOR]

Nowadays, sensors play a major role in several fields, such as science, industry and everyday technology. Therefore, the information received from the sensors must be reliable. If the sensors present any anomalies, serious problems can arise, such as publishing wrong theories in scientific papers, or causing production delays in industry. One of the most common anomalies are uncalibrations. An uncalibration occurs when the sensor is not adjusted or standardized by calibration according to a ground truth value. In this work, an online machine-learning based uncalibration detector for temperature, humidity and pressure sensors is presented. This development integrates an artificial neural network as the main component which learns from the behavior of the sensors under calibrated conditions. Then, after being trained and deployed, it detects uncalibrations once they take place. The obtained results show that the proposed system is able to detect the 100% of the presented uncalibration events, although the time response in the detection depends on the resolution of the model for the specific location, i.e., the minimum statistically significant variation in the sensor behavior that the system is able to detect. This architecture can be adapted to different contexts by applying transfer learning, such as adding new sensors or having different environments by re-training the model with minimum amount of data. [ABSTRACT FROM AUTHOR]

In this paper, we investigate Hebbian learning strategies applied to Convolutional Neural Network (CNN) training. We consider two unsupervised learning approaches, Hebbian Winner-Takes-All (HWTA), and Hebbian Principal Component Analysis (HPCA). The Hebbian learning rules are used to train the layers of a CNN in order to extract features that are then used for classification, without requiring backpropagation (backprop). Experimental comparisons are made with state-of-the-art unsupervised (but backprop-based) Variational Auto-Encoder (VAE) training. For completeness,we consider two supervised Hebbian learning variants (Supervised Hebbian Classifiers—SHC, and Contrastive Hebbian Learning—CHL), for training the final classification layer, which are compared to Stochastic Gradient Descent training. We also investigate hybrid learning methodologies, where some network layers are trained following the Hebbian approach, and others are trained by backprop. We tested our approaches on MNIST, CIFAR10, and CIFAR100 datasets. Our results suggest that Hebbian learning is generally suitable for training early feature extraction layers, or to retrain higher network layers in fewer training epochs than backprop. Moreover, our experiments show that Hebbian learning outperforms VAE training, with HPCA performing generally better than HWTA. [ABSTRACT FROM AUTHOR]

The paper presents the application of the artificial neural network algorithm for the retrieval of total precipitable water in the atmosphere over water and land from the measurements of MTVZA-GYa microwave radiometer on board the Meteor-M No. 2-2 satellite. Satellite-based estimates of total precipitable water were compared with radiosonde and AERONET data, as well as with the ECMWF numerical weather prediction model output. According to the comparison, the root-mean-square error (RMSE) does not exceed 4.5 mm for radiosonde data and is less than 4 mm for the ECMWF and AERONET data. The best accuracy is provided over water with the RMSE not exceeding 3 mm. The total precipitable water estimates retrieved from MTVZA-GYa and NOAA-20/ATMS radiometer data are consistent over water, while the MTVZA-GYa based estimates are more accurate over land. [ABSTRACT FROM AUTHOR]

The paper investigates hyperparameter optimization problem. Hyperparameters are the parameters of model parameter distribution. The adequate choice of hyperparameter values prevents model overfit and allows it to obtain higher predictive performance. Neural network models with large amount of hyperparameters are analyzed. The hyperparameter optimization for models is computationally expensive. The paper proposes modifications of various gradient-based methods to simultaneously optimize many hyperparameters. The paper compares the experiment results with the random search. The main impact of the paper is hyperparameter optimization algorithms analysis for the models with high amount of parameters. To select precise and stable models the authors suggest to use two model selection criteria: cross-validation and evidence lower bound. The experiments show that the models optimized using the evidence lower bound give higher error rate than the models obtained using cross-validation. These models also show greater stability when data is noisy. The evidence lower bound usage is preferable when the model tends to overfit or when the cross-validation is computationally expensive. The algorithms are evaluated on regression and classification datasets. [ABSTRACT FROM AUTHOR]

Autonomous underwater vehicles (AUVs) are underwater robots which are able to perform certain tasks without the help of a human operator. The key skill of each AUV is the capability to avoid collisions. To this end, appropriate devices and software are necessary with the potential to detect obstacles and to take proper decisions from the point of view of both the task and safety of the vehicle. The paper presents a neural collision avoidance system (NCAS) designed for the biomimetic autonomous underwater vehicle (BAUV). The NCAS is a component of the path following and collision avoidance system (PFCAS), which as the name implies is responsible for safely leading the vehicle along a desired path with collision avoidance. The task of NCAS is to make decisions regarding vehicle maneuvers in the horizontal plane, but only in the close proximity of the obstacles. It is implemented as an evolutionary artificial neural network designed by means of a neuro-evolutionary technique called assembler encoding with evolvable operations (AEEO). The paper outlines operation and construction of the BAUV as well as the PFCAS, the role of the NCAS in the entire system, and briefly presents AEEO as well as reporting on the experiments performed in simulation. [ABSTRACT FROM AUTHOR]

This paper provides an overview of recent approaches to deep learning as applied to speech processing tasks, primarily for automatic speech recognition, but also text-to-speech and speaker, language and emotion recognition. The focus is on efficient methods, addressing issues of accuracy, computation, storage, and delay. The discussion puts the speech processing tasks in the broader context of pattern recognition, comparing with signals other than speech. It also compares machine learning with other recent methods of speech analysis, e.g., hidden Markov models. The paper emphasizes a thorough understanding of the choices made in analyzing and interpreting speech signals. It minimizes use of mathematics and is aimed at non-experts; the references provide needed detail for those interested. [ABSTRACT FROM AUTHOR]

Cold metal transfer (CMT) welding is a sophisticated version of fusion welding process available with advanced features incorporated in it. During the process of joining metallic parts with CMT, the bead shape and size have significant effects on the weld quality. Typically, bead geometry is characterized by the three important parameters namely weld width, weld depth and reinforcement height. For rapid and intelligent welding, it is imperative to monitor the bead shape and size during the process. However, the measurement system of three parameters is often time consuming and complex. In this paper, an attempt is made to reduce the computational time of artificial intelligence models that are used for intelligent welding processes. Bead coefficient is modelled as an alternative for bead geometry properties. Computationally efficient artificial neural network models are developed for examining the feasibility of bead coefficient over bead geometry properties with and without online-temperature measurements. The predicted results of both forward and reverse neural network models are in good coherence with the experimental values. The outcome of this study is expected to provide a detailed understanding of effects of process parameters on bead geometry and bead coefficient, which facilitates online monitoring of welding processes. [ABSTRACT FROM AUTHOR]

In this paper, the application of artificial neural network in ship course control systems is investigated. Two-multilayered feed-forward neural network course control system is proposed. The first neural network plays the role of ship forward dynamic approximator. The second one is the course controller. Both neural networks are trained in a quasi-online regime using training data acquired from system functional process to cope with changing ship dynamics. A cost function is used in control action calculation. The performance of the proposed system is evaluated in different conditions. The system stability is verified via simulation. The simulation results show that the course control system is able to keep the predefined direction in various sea conditions and the proposed approach serves the consideration on developing and applying in designing real ship autopilot systems. [ABSTRACT FROM AUTHOR]

The field of Music Emotion Recognition has become and established research sub-domain of Music Information Retrieval. Less attention has been directed towards the counterpart domain of Audio Emotion Recognition, which focuses upon detection of emotional stimuli resulting from non-musical sound. By better understanding how sounds provoke emotional responses in an audience, it may be possible to enhance the work of sound designers. The work in this paper uses the International Affective Digital Sounds set. A total of 76 features are extracted from the sounds, spanning the time and frequency domains. The features are then subjected to an initial analysis to determine what level of similarity exists between pairs of features measured using Pearson's r correlation coefficient before being used as inputs to a multiple regression model to determine their weighting and relative importance. The features are then used as the input to two machine learning approaches: regression modelling and artificial neural networks in order to determine their ability to predict the emotional dimensions of arousal and valence. It was found that a small number of strong correlations exist between the features and that a greater number of features contribute significantly to the predictive power of emotional valence, rather than arousal. Shallow neural networks perform significantly better than a range of regression models and the best performing networks were able to account for 64.4% of the variance in prediction of arousal and 65.4% in the case of valence. These findings are a major improvement over those encountered in the literature. Several extensions of this research are discussed, including work related to improving data sets as well as the modelling processes. [ABSTRACT FROM AUTHOR]

The design of gas turbines is a challenging area of cyber-physical systems where complex model-based simulations across multiple disciplines (e.g., performance, aerothermal) drive the design process. As a result, a continuously increasing amount of data is derived during system design. Finding new insights in such data by exploiting various machine learning (ML) techniques is a promising industrial trend since better predictions based on real data result in substantial product quality improvements and cost reduction. This paper presents a method that generates data from multi-paradigm simulation tools, develops and trains ML models for prediction, and deploys such prediction models into an active control system operating at runtime with limited computational power. We explore the replacement of existing traditional prediction modules with ML counterparts with different architectures. We validate the effectiveness of various ML models in the context of three (real) gas turbine bearings using over 150,000 data points for training, validation, and testing. We introduce code generation techniques for automated deployment of neural network models to industrial off-the-shelf programmable logic controllers. [ABSTRACT FROM AUTHOR]

This paper reports an intelligent approach for modeling and optimisation of filling system design (FSD) in the case of sand casting process of aluminium alloy. In order to achieve this purpose, physics-based process modeling using finite element method (FEM) has been integrated with artificial neural networks (ANN) and genetic algorithm (GA) soft computing techniques. A three-dimensional FE model of the studied process has been developed and validated, using experimental literature data, to predict two melt flow behaviour (MFB) indexes named ingate velocity and jet high. Two feed-forward back-propagation ANN-based process models were developed and optimised to establish the relationship between the FSD input parameters and each studied MFB index. Both ANN models were trained, tested and tuned by using database generated from FE computations. It was found that both ANN models could independently predict, with a high accuracy, the values of the ingate velocity and the jet high for training and test data. The developed ANN models were coupled with an evolutionary GA to select the optimal FSD for each one. The validity of the found solutions was tested by comparing ANN-GA prediction with FE computation for both studied MFB indexes. It was found that error between predicted and simulated values does not exceed 5.61% and 6.31% respectively for the ingate velocity and the jet high, which proves that the proposed approach is reliable and robust for FSD optimisation. [ABSTRACT FROM AUTHOR]

Wind energy is getting more and more integrated into power grids, giving rise to some challenges because of its inherent intermittent and irregular nature. Wind speed forecasting plays a fundamental role in overcoming such challenging issues and, thus, assisting the power utility manager in optimizing the supply–demand balancing through wind energy generation. This paper suggests a new hybrid scheme WNN, based on discrete wavelet transform (DWT) combined with artificial neural network (ANN), for wind speed forecasting. More specifically, this work aims at designing the most appropriate discrete wavelet filters, best adapted to a one day ahead wind speed forecasting. The optimized DWT filters are intended to effectively preprocess the wind speed time series data in order to enhance the prediction accuracy. Using wind speed data collected from three different locations in the Magherbian region, the obtained simulation results indicate that the proposed approach outperforms other conventional wavelet-based forecasting structures regarding the wind speed prediction precision. Moreover, compared to the standard wavelet 'db4' based approach, the optimized wavelet filter-based structure leads to a forecasting accuracy improvement, in terms of RMSE and MAPE index errors, that amounts to nearly 13% and 19%, respectively. [ABSTRACT FROM AUTHOR]

Wireless mesh networks are a special class of wireless networks that are implemented as a collection of radio nodes in a mesh pattern or topology. Unlike MANETs, the mobility of nodes is very less in the topology. Quality of service is an essential metric in the performance of mesh networks which are attributed to several parameters including optimal routing through shortest path, ability for other nodes to communicate even if a particular node in the mesh fails, minimization of packet loss and time delay, computational complexity and cost, energy. This research paper is focused toward minimization of energy taken as the objective function and a five-stage neural network is used and trained after optimizing with a genetic algorithm. The experiments have been conducted in NS2 and Qualnet environment with a varying number of mesh routers and energy computed. The performance of energy savings has been compared against conventional routing techniques like AODV and a bee colony optimization technique presented in the literature. An energy savings of 51% have been reported in the paper justifying the superiority of the hybrid G-ANN algorithm. [ABSTRACT FROM AUTHOR]

Transient stability problem is very important for the secure operation of today's heavily loaded power systems. In this paper, an auxiliary (supplementary) controller is employed for static synchronous compensator (STATCOM) to improve the transient stability limit of multimachine power systems. In the proposed controller, the angle and speed of a severely disturbed machine are used to change the shunt susceptance of STATCOM in the transient period. The developed control method is successfully applied to the New England 10-machine, 39-bus test system. The simulation results obtained, however, show that to have a desired critical clearing time for a particular fault scenario, the gain of the STATCOM auxiliary controller needs to be adjusted according to the system operating point. On the other hand, calculating the required gain of the STATCOM controller via a trial-and-error approach and performing several time-domain simulations is very time-consuming. To speed up the method of calculating that gain in online applications, a multilayered perceptron (MLP) neural network (NN)-based approach is proposed in this paper. The pre-fault system operating conditions are chosen as the inputs of the MLP NN. In addition, an intuitive-based method is represented to reduce the number of neural network inputs. The simulation results obtained by the proposed MLP NN-based method are also presented and discussed in detail. [ABSTRACT FROM AUTHOR]

The role of prognostics and health management is ever more prevalent with advanced techniques of estimation methods. However, data processing and remaining useful life prediction algorithms are often very different. Some difficulties in accurate prediction can be tackled by redefining raw data parameters into more meaningful and comprehensive health level indicators that will then provide performance information. Proper data processing has a significant importance on remaining useful life predictions, for example, to deal with data limitations or/and multi-regime operating conditions. The framework proposed in this paper considers a similarity-based prognostic algorithm that is fed by the use of data normalisation and filtering methods for operational trajectories of complex systems. This is combined with a data-driven prognostic technique based on feed-forward neural networks with multi-regime normalisation. In particular, the paper takes a close look at how pre-processing methods affect algorithm performance. The work presented herein shows a conceptual prognostic framework that overcomes challenges presented by short-term test datasets and that increases the prediction performance with regards to prognostic metrics. [ABSTRACT FROM AUTHOR]

Misclassification is a critical problem in many machine learning applications. Since even the classifier models with high accuracy (e.g., > 95%) still introduce some misclassification error, it may not be possible to rely on the output of a classifier. In this paper, we introduce trustable learning, which prompts the learning model to yield only the true output, thus avoiding misclassifications. Whenever the model cannot decide the output accurately, the learning model should indicate that there could be a misclassification error if it is forced to classify, and hence, it should reject to make a decision or defer it to a human expert. Therefore, we develop a methodology for trustable learning and apply it to artificial neural networks and show that it is possible to develop a classifier with 0% misclassification error. We propose a novel neural network architecture named WisdomNet that could provide zero prediction error by introducing an additional neuron named as conjugate neuron that would indicate whether the network is able to classify the data correctly or not. The WisdomNet architecture can be applied to any previously built model, and we have evaluated WisdomNet with several network architectures such as multilayer perceptron, convolutional neural network, and deep network on different data sets. The results show that the WisdomNet is able to reduce the classification error rate to 0%, while labeling the data is difficult to classify as 'reject' at a low percentage of within around 10%. [ABSTRACT FROM AUTHOR]

Lithology identification of geological beds in the subsurface is fundamental in reservoir characterization. Recently, automated log analysis has an increasing demand in reservoir research and the oil industry. In this context, Machine Learning (ML) techniques arise as a surrogate model to provide lithology identification in a fast way. However, to achieve suitable performance, ML techniques require the adjustment of some parameters, and that can become a hard task, depending on the difficulty of the problem to be solved. This paper presents an Artificial Neural Network (ANN), assisted by an adaptive Differential Evolution (DE) algorithm to classify petrophysical data in the Southern Provence Basin. The main contribution is searching for a competent ANN configuration, including architecture, activation functions, regularization, and training algorithms. The proposed approach outperformed four classifiers and two results previously published. The computational methodology proposed here is able to assist in the classification of petrophysical data, helping to improve the procedure of reservoir characterization and the idealization of the development of production. [ABSTRACT FROM AUTHOR]

Models based on neural networks (NN) have been used widely and successfully in traffic prediction resulting in improved accuracy and efficiency in traffic flow, speed, passenger flow, and delay. Input data include continuous and discrete variables and these impact traffic changes both internally and externally. However, few studies have focused on discrete traffic-related variables in NN-based forecasting models. Inappropriate utilization of discrete variables may cause useful factors to become insignificant and lead to an inefficient forecasting model. In this paper, a NN-based model is used to predict traffic flow of a bike-sharing system in Suzhou, China. The model only uses external and discrete variables like weather, places of interest (POIs), and holiday periods. We applied both entity embedding and one-hot encoding for the data preprocessing of these variables. The results show that (1) Entity embedding can effectively increase the continuity of categorical variables and slightly improve the prediction efficiency for the NN model; and (2) The hidden relationship in variables can be identified through visual analysis, and the trained embedding vectors can also be used in traffic-related tasks. [ABSTRACT FROM AUTHOR]

In this paper, a particular discrete-time nonlinear and time-invariant system represented as a vector difference equation is analyzed for its stability properties. The motivation for analyzing this particular system is that it models gated recurrent unit neural networks commonly used and well known in machine learning applications. From the technical perspective, the analyses exploit the systems similarities to a convex combination of discrete-time systems, where one of the systems is trivial, and thus, the overall properties are mostly dependent on the other one. Stability results are formulated for the nonlinear system and its linearization with respect to the systems, in general, multiple equilibria. To motivate and illustrate the potential of these results in applications, some particular results are derived for the gated recurrent unit neural network models and a connection between local stability analysis and learning is provided. [ABSTRACT FROM AUTHOR]

Stock price prediction is a rich research topic that has attracted interest from various areas of science. The recent success of machine learning in speech and image recognition has prompted researchers to apply these methods to asset price prediction. The majority of literature has been devoted to predicting either the actual asset price or the direction of price movement. In this paper, we study a hitherto little explored question of predicting significant changes in stock price based on previous changes using machine learning algorithms. We are particularly interested in the performance of neural network classifiers in the given context. To this end, we construct and test three neural network models including multilayer perceptron, convolutional net, and long short-term memory net. As benchmark models, we use random forest and relative strength index methods. The models are tested using 10-year daily stock price data of four major US public companies. Test results show that predicting significant changes in stock price can be accomplished with a high degree of accuracy. In particular, we obtain substantially better results than similar studies that forecast the direction of price change. [ABSTRACT FROM AUTHOR]

This article addresses why the neural network model has been selected by nature against other computational models to generate behavior in complex multicellular clades. This question, which has not yet been addressed in research, should not be ignored because understanding this issue is necessary to have a complete picture of the evolutionary process of the nervous system. The starting point to discuss the issue is a proposal made 30 years ago: the free-moving hypothesis. This proposal establishes prediction as the main function of the brain and that all multicellular organisms that move require a brain in order to make predictions. This article contains a review contrasting this hypothesis with the discoveries made in the last 30 years within different biological kingdoms. Although none of these discoveries contradict the free-moving hypothesis, it still does not answer the main question. Alternative hypotheses about the origin of the nervous system are discussed in this paper, but they also are not able to answer the question. Six hypotheses are proposed as possible answers, and each of them is discussed by comparing neural processing systems with three other alternative processing systems. The result is that the neural processing system is selected against other kinds of processing systems because it has computational robustness to damage, allowing its function of prediction to be more durable. While this result, called the first neural processing principle, answers the initial question and permits a finished proof of the free-moving hypothesis, it gives rise to the question of how computationally robust a system must be to be selected by nature. This paper claims that the system selected must be computationally robust enough to have enough offspring to allow variation. This answer, named the second neural principle, determines the minimum amount of neurons that a neural processing system must have, but not the maximum. To address this issue, the third and fourth neural processing principles are stated, which determine that the maximum number of neurons is limited by energetic restrictions and body size, respectively. The results presented in this paper show that computational robustness is an important parameter to understand the evolution of nervous system. [ABSTRACT FROM AUTHOR]

There are species of carbon steel in the industry suffering from corrosion phenomena under seawater environment. In this paper, for purposes of the prediction of 3C steel corrosion rate, the proposed methodology here adopts a hybrid model based on neural network (NN) and imperialist competitive algorithm (ICA). Additionally, to validate the suggested method, we have managed to apply a procedure, namely leaving-one-out cross-validation (LOOCV). Thus, the model in this paper is abbreviated as NN-ICA_LOOCV. In the case study, the model is implemented on 46 experimental samples. This dataset is included within five parameters, namely temperature, dissolved oxygen, salinity, PH value and oxidation-reduction potential as inputs and corrosion rate(s) as an output parameter. The dataset was divided into two parts: one for training and the other for testing with 42 and 4 data number, respectively. For an evaluation purpose, the performance of NN-ICA_LOOCV is compared with other models on the basis of indicators such as the coefficient of determination ( R ), root-mean-square error (RMSE) and mean absolute error (MAE). The model was successfully tested, yielding a prediction of corrosion rate with a RMSE of around 0.01, MAE of 0.011 and a correlation factor of 0.99 to the test data. The results demonstrate that the carefully designed hybrid model further succeeded to denote lower modeling error and higher accuracy. Hence, this model is an applicable and reliable offer to engineers in order to online and safe prediction of corrosion rate in 3C steel under seawater environment. [ABSTRACT FROM AUTHOR]

Current software tools for computer generation of Persian calligraphy can be mostly described as conventional fonts and typesetting software, which basically neglect the 'variations' of real calligraphy performed by hand, in terms of personalization to different calligraphers' styles, as well as their statistical characteristics. In this paper, we address the problem of natural-looking Persian calligraphy synthesis via a machine learning based approach, at the level of subwords. Given images of samples written by a calligrapher, we train a parametric model to imitate the style. The core idea is to make use of templates (fonts) as a source of background knowledge, and learn a probabilistic mapping from them to personal styles of calligraphers, which is posed as transformation of attributed graphs using neural networks with sliding windows. This can be understood as adding 'naturalness' to a Persian calligraphy font, in essence. We report both objective and subjective evaluations, including the model performance in writer (calligrapher) identification task and Visual Turing Test. The results of the latter suggest that humans are unable to distinguish the calligraphy synthesized by our approach from real calligraphy in many cases. [ABSTRACT FROM AUTHOR]

The prediction of convective clouds formation is a very important problem in different areas such as agriculture, natural hazards prevention or transport-related facilities. In this paper, we evaluate the capacity of different types of evolutionary artificial neural networks to predict the formation of convective clouds, tackling the problem as a classification task. We use data from Madrid-Barajas airport, including variables and indices derived from the Madrid-Barajas airport radiosonde station. As objective variable, we use the cloud information contained in the METAR and SPECI meteorological reports from the same airport and we consider a prediction time horizon of 12 h. The performance of different types of evolutionary artificial neural networks has been discussed and analysed, including three types of basis functions (sigmoidal unit, product unit and radial basis function) and two types of models, a mono-objective evolutionary algorithm with two objective functions and a multi-objective evolutionary algorithm optimised by the two objective functions simultaneously. We show that some of the developed neuro-evolutionary models obtain high quality solutions to this problem, due to its high unbalance characteristic. [ABSTRACT FROM AUTHOR]

In this paper, the concept of Virtual Weather Stations (VWS) is introduced. A VWS is an integration of algorithms to download meteorological data, process and use them with the main objective of estimate data in nearby locations with no meteorological stations. To develop the VWS, the performances of different interpolation methods were evaluated to test the accuracy. Daily data from an automatic weather station network, such as precipitation (Precip), air temperature (Temp), air relative humidity, mean wind speed, total solar irradiation, and reference evapotranspiration were interpolated using artificial neural networks (ANNs) with the hardlim, sigmoid, hyperbolic tangent (tanh), softsign, and rectified linear unit (relu) activations functions were employed. To contrast the ANNs interpolations, alternatives methods such as inverse distance weighting, inverse-squared distance weighting, multilinear regression, and random forest regression were used. To validate the models, a randomly selected weather station was removed from the daily datasets, and the interpolated values were compared with the actual station records. Additionally, interpolations in the summer and winter months were performed to check the capability of the models during periods with more extreme phenomena. The results showed that the interpolation methods have an R2 up to 0.98 for variables such as temperatures for the period of 1 year. Meanwhile, during the summer and winter, the models presented lower accuracy. From a practical perspective, the methods here described could be useful to produce meteorological data with the VWS to record temperatures and dose the irrigation in crops. [ABSTRACT FROM AUTHOR]

Searching optimal parameters for neural networks can be formulated as a multi-modal optimization problem. This paper proposes a novel water wave optimization (WWO)-based memetic algorithm to identify the optimal weights for neural networks. In the proposed water wave optimization-based memetic algorithm (WWOMA), we employ WWO to perform global search by both individual improvement and population co-evolution and then employ several local search components to enhance its local refinement ability. Moreover, an effective Meta-Lamarckian learning strategy is utilized to choose a proper local search component to concentrate computational efforts on more promising solutions. We carry out simulation experiments on six well-known neural network designing benchmark problems, both the simulation results and statistical comparisons demonstrate the feasibility, effectiveness and efficiency of applying WWOMA to design neural networks. Furthermore, we apply WWOMA to design neural networks and use well-trained neural networks to predict tensile strength of micro-alloyed steels. Evaluation on a practical industrial case with 2489 sample data shows that, in comparison with other algorithms, WWOMA-based neural networks can obtain notable and robust prediction accuracy, which further demonstrates that WWOMA is a promising and efficient algorithm for designing neural networks. It is worth mentioning that, to the best of our knowledge, this is the first report about applying water wave optimization to train neural networks. [ABSTRACT FROM AUTHOR]

The selection of the initial network weights has been a known key aspect affecting the convergence of sigmoidal activation function-based artificial neural networks. In this paper, a new network initialization scheme has been proposed that initializes the network weights such that activation functions in the network are not saturated initially. The proposed method ensures that the initial outputs of the hidden neurons are in the active region which positively impacts the network's rate of convergence. Unlike most of the earlier proposed initialization schemes, this method does not depend on architectural parameters like the size of the input layer or the hidden layer. The performance of the proposed scheme has been compared against eight well-known weight initialization routines over six benchmark real-world problems. Results show that the proposed weight initialization routine enables the network to achieve better performance within the same count of network training epochs. A right-tailed t-test also shows that our proposed scheme is significantly better in most of the cases against the other techniques or statistically similar in a few cases but never underperforms. Hence, it may be considered as a strong alternative to the conventional neural network initialization techniques. [ABSTRACT FROM AUTHOR]

In this paper we address the issue of automatically learning parameters of spiking neural networks. Biological neurons are formalized as timed automata and synaptical connections are represented as shared channels among these automata. Such a formalism allows us to take into account several time-related aspects, such as the influence of past inputs in the computation of the potential value of each neuron, or the presence of the refractory period, a lapse of time immediately following the spike emission in which the neuron cannot emit. The proposed model is then formally validated: more precisely, we ensure that some relevant properties expressed as temporal logical formulae hold in the model. Once the validation step is accomplished, we take advantage of the proposed model to write an algorithm for learning synaptical weight values such that an expected behavior can be displayed. The technique we present takes inspiration from supervised learning ones: we compare the effective output of the network to the expected one and backpropagate proper corrective actions in the network. We develop several case studies including a mutual inhibition network. [ABSTRACT FROM AUTHOR]

The Internet of Things (IoT) has emerged as a disruptive technology for the current and future of computing and communication. IoT is characterized by a variety of heterogeneous technologies and devices able to be connected to the Internet. Current and future research and development efforts aim at adding artificial intelligence to IoT systems, enabling devices to become smart and thus make autonomous decisions individually or collectively. Additionally, such smart devices have the ability to interact not only with other smart devices but also with humans. Thus, the aim of this paper is to investigate the usability of the artificial intelligence in the IoT paradigm. To achieve the approach, a system called smart-IoT is built based on artificial neural networks, namely, neural networks have been learned by back-propagation algorithm. The system is tested using mobile devices under Android as smart objects. Experiments with neural networks were carried on certain services (such as auto set alarms for a specific event, or estimating the time to return home). These experiments showed the feasibility of embedding neural networks techniques into the IoT system. The approach allows also for easy adding of new services, which in turn means that smart IoT is a modular and full-fledged system. [ABSTRACT FROM AUTHOR]

In this paper, the event-triggered non-fragile state estimation problem is considered for a class of uncertain discrete-time neural networks with random delays and state-dependent stochastic disturbances. The norm-bounded uncertainty goes into all the parameters of the addressed neural network model. A Bernoulli stochastic variable is introduced to characterize the probability distribution of the occurrence of the time delays. The phenomenon of estimator gain variations is considered to account for the potential implementation error of the state estimator. An event-triggered mechanism is proposed with hope to mitigate the network load and orchestrate the bandwidth utilization. We aim to design a non-fragile state estimator such that the error dynamics of the neuron state estimation is globally asymptotically stable in the mean square. By means of the Lyapunov stability theory, delay-distribution-dependent sufficient conditions are established to ensure the existence of the desired event-triggered non-fragile state estimators and the estimator parameters are then derived in terms of the solution to a set of linear matrix inequalities. An illustrative example is utilized to show the usefulness and applicability of the proposed algorithm. [ABSTRACT FROM AUTHOR]

The "boost-diffusion" low-pressure nitriding used to low-frictional coatings manufacturing of aircraft engines' piston rings is a nonsteady-state process; therefore, designing and prediction of the process' kinetics by analytical solutions of Fick's equations or numerical methods of diffusion are difficult, due to the nonlinear relationship between the diffusion coefficient and the rate of diffusion as well as nonsteady-state boundary conditions. The best solution in this case, as the practice and theory indicate, is computer-aided design based on neural networks. The paper describes neural network model and its training procedures based on data mining in the application to the monitoring and control of low-pressure nitriding process for creation of low-frictional coatings on gray irons and steels used for the piston rings manufacturing. The goal was to study the usefulness of the multilayer feed-forward perceptrons and radial basis function of neural networks for modeling of multiphase kinetic diffusion for low-pressure nitriding. As it was shown, the use of specialist networks that designate single features gives more accurate prediction results than the use of general networks that design several features at the same time. It has been proved that it is possible to construct an industrial application of the low-pressure nitriding based on artificial neural networks. The results of the research will be the basis for the development of innovative, specialized software supporting the design of gradient low-friction layers based on the FineLPN low-pressure nitriding and consequently the design of intelligent supervision over their manufacturing technology. [ABSTRACT FROM AUTHOR]

Long-term monthly streamflow forecasting has great importance in the water resource system planning. However, its modelling in extreme cases is difficult, especially in semiarid regions. The main purpose of this paper is to evaluate the accuracy of artificial neural networks (ANNs) and hybrid wavelet-artificial neural networks (WA-ANNs) for multi-step monthly streamflow forecasting in two different hydro-climatic regions in Northern Algeria. Different issues have been addressed, both those related to the model's structure and those related to wavelet transform. The discrete wavelet transform has been used for the preprocessing of the input variables of the hybrid models, and the multi-step streamflow forecast was carried out by means of a recursive approach. The study demonstrated that WA-ANN models outperform the single ANN models for the two hydro-climatic regions. According to the performance criteria used, the results highlighted the ability of WA-ANN models with lagged streamflows, precipitations and evapotranspirations to forecast up to 19 months for the humid region with good accuracy [Nash–Sutcliffe criterion (Ns) equal 0.63], whereas, for the semiarid region, the introduction of evapotranspirations does not improve the model's accuracy for long lead time (Ns less than 0.6 for all combinations used). The maximum lead time achieved, for the semiarid region, was about 13 months, with only lagged streamflows as inputs. [ABSTRACT FROM AUTHOR]