Search

Solving partial differential equations using neural networks is mostly a proof of concept approach. In the case of direct function approximation, a single neural network is constructed to be the solution of a particular boundary value problem. Independent variables are fed into the input layer, and a single output is considered as the solution’s value. The network is substituted into the equation, and the residual is then minimized with respect to the weights of the network using a gradient-based method. Our previous work showed that by minimizing all derivatives of the residual up to the third order one can obtain a machine precise solution for 2D boundary value problem using very sparse grids. The goal of this paper is to use this grid complexity advantage in order to obtain a solution faster than finite differences. However, the number of all possible high-order derivatives (and therefore the training time) increases with the number of dimensions and it was unclear whether this goal can be achieved. Here, we demonstrate that this increase can be compensated by using random directional derivatives instead. In 2D case neural networks are slower than finite differences, but for each additional dimension the complexity increases approximately 4 times for neural networks and 125 times for finite differences. This allows neural networks to catch up in 3D case for memory complexity and in 5D case for time complexity. For the first time a machine precise solution was obtained with neural network faster than with finite differences method.

In real-world application of affective brain–computer interface (aBCI), individual differences across subjects and non-stationary characteristics of electroencephalogram (EEG) signals can cause data bias. Moreover, for new specific subject, the size of sample data is very small compared to that of existing subjects, which easily leads to overfitting in deep neural network training and reduces generalization performance of the network. In this paper, the deep multi-source adaptation transfer network (DMATN) is proposed for the new subjects in aBCI. In DMATN, the multi-source selection is employed to obtain the portion of existing EEG data mostly correlated with new subject and to decrease by two-fifth source data. To explore domain-invariant structures, deep adaptation network is used to map correlated source domain and the target domain (new subject) into reproducing kernel Hilbert space (RKHS) optimized by the multiple kernel variant of maximum mean discrepancies (MK-MMD). To more precisely predict the emotional state of the new subject, domain discriminator is applied in DMATN to make the data distribution of the two domains more similar. Finally, across-subject experiments on SEED dataset are conducted to evaluate the proposed method. The experimental results show that DMATN model can achieve the state-of-the-art performance of 84.46%, 83.32% and 84.90% in three sessions, respectively. It also shows great time efficiency in applications of aBCI.

Taking the hot-rolling products made by an iron and steel company as the research object, this paper builds the inverse model reflecting the relationship between the hot-rolling steel product performance indicators, the chemical composition of steel and the rolling technological parameters by using the BP neural network. So the purpose of getting technological parameters is achieved, according to the given steel performance indicators. Combining the BP neural network, adaptive inverse control with internal model control theory, this paper builds the BP neural network inverse model with multiple input and single output based on internal model control. Therefore, it realizes the inverse mapping between the output and the input variables of the BP neural network. And the output variables can be obtained according to the input variables. Besides, this paper also gives the detailed steps to solve the inverse model. Then, the model is applied to the hot-rolling steel products quality control system. The performance indicators of the hot-rolling products are set up, and the rolling technological parameters—the rolling crimp temperature—are solved. The model realizes the controllability of rolling technological parameters. Finally, through the verification of hot-rolling products quality control positive system, the error is in line with the enterprise production requirements.

Fast and effective recommender systems are fundamental to fulfill the growing requirements of the e-commerce industry. The strength of matrix factorization procedure based on stochastic gradient descent (SGD) algorithm is exploited widely to solve the recommender system problem. Modern computing paradigms are designed by utilizing the concept of fractional gradient in standard SGD and outperform the standard counterpart. The performance of fractional SGD improves considerably by adaptively tuning the learning rate parameter. A nonlinear computing paradigm based on normalized version of fractional SGD is developed in this paper to investigate the adaptive behavior of learning rate with novel application to recommender systems. The accuracy of the proposed approach is verified through root mean square error metric by using different latent features, learning rates, fractional orders and datasets. The superiority of the designed method is validated through comparison with the state-of-the-art counterparts.

This paper introduces a scheme for semi-supervised data representation. It proposes a flexible nonlinear embedding model that imitates the principle of spectral graph convolutions. Structured data are exploited in order to determine nonlinear and linear models. The introduced scheme takes advantage of data graphs at two different levels. First, it incorporates manifold regularization that is naturally encoded by the graph itself. Second, the regression model is built on the convolved data samples that are obtained by the joint use of the data and their associated graph. The proposed semi-supervised embedding can tackle challenges related to over-fitting in image data spaces. The proposed graph convolution-based semi-supervised embedding paves the way to new theoretical and application perspectives related to the nonlinear embedding. Indeed, building flexible models that adopt convolved data samples can enhance both the data representation and the final performance of the learning system. Several experiments are conducted on six image datasets for comparing the introduced scheme with many state-of-art semi-supervised approaches. These experimental results show the effectiveness of the introduced data representation scheme.

The performance of conventional nonlinear model predictive control $$\left( {{\text{NMPC}}} \right)$$ system relies heavily on the accuracy of the prediction model. In cases of significant plant-model mismatch, non-desirable responses may be observed in the controlled outputs. This paper proposes a parameter adaptation technique for tackling this problem. In the proposed approach, the output disturbance is selected as the adaptation parameter while the adaptation law is modelled as a function of the tracking error using a first-order difference equation. The adaptation law is integrated into a $$\mathrm{NMPC}$$ algorithm to achieve offset-free tracking. The effectiveness of the proposed scheme is demonstrated on two simulation case studies—a pH system and a continuously stirred tank reactor (CSTR); and an experimental cascaded two tank process. The simulation results obtained showed that the proposed scheme achieves zero offset in the face of significant plant-model mismatch arising from uncertainties in model parameters, unmeasured disturbances, and measurement noise and compared favourably with existing methods. The experimental results obtained during real-time implementation of the proposed control scheme corroborate this assertion and show its industrial applicability.

In this paper we propose a new approach for dynamic selection of ensembles of classifiers. Based on the concept named multistage organizations, the main objective of which is to define a multi-layer fusion function adapted to each recognition problem, we propose dynamic multistage organization (DMO), which defines the best multistage structure for each test sample. By extending Dos Santos et al.’s approach, we propose two implementations for DMO, namely DSA m and DSA c . While the former considers a set of dynamic selection functions to generalize a DMO structure, the latter considers contextual information, represented by the output profiles computed from the validation dataset, to conduct this task. The experimental evaluation, considering both small and large datasets, demonstrated that DSA c dominated DSA m on most problems, showing that the use of contextual information can reach better performance than other existing methods. In addition, the performance of DSA c can also be enhanced in incremental learning. However, the most important observation, supported by additional experiments, is that dynamic selection is generally preferred over static approaches when the recognition problem presents a high level of uncertainty.

We demonstrate several techniques to encourage practical uses of neural networks for fluid flow estimation. In the present paper, three perspectives which are remaining challenges for applications of machine learning to fluid dynamics are considered: 1. interpretability of machine-learned results, 2. bulking out of training data, and 3. generalizability of neural networks. For the interpretability, we first demonstrate two methods to observe the internal procedure of neural networks, i.e., visualization of hidden layers and application of gradient-weighted class activation mapping (Grad-CAM), applied to canonical fluid flow estimation problems -- $(1)$ drag coefficient estimation of a cylinder wake and $(2)$ velocity estimation from particle images. It is exemplified that both approaches can successfully tell us evidences of the great capability of machine learning-based estimations. We then utilize some techniques to bulk out training data for super-resolution analysis and temporal prediction for cylinder wake and NOAA sea surface temperature data to demonstrate that sufficient training of neural networks with limited amount of training data can be achieved for fluid flow problems. The generalizability of machine learning model is also discussed by accounting for the perspectives of inter/extrapolation of training data, considering super-resolution of wakes behind two parallel cylinders. We find that various flow patterns generated by complex interaction between two cylinders can be reconstructed well, even for the test configurations regarding the distance factor. The present paper can be a significant step toward practical uses of neural networks for both laminar and turbulent flow problems., 24 pages, 24 figures

In this paper, a general class of memristive neural networks with variable delays is studied. By utilizing control theory and nonsmoooth analysis, two sufficient criteria ensuring input-to-state stability of memristive neural networks with variable delays are firstly obtained which are novel and more practical than the previous works in the literature. Finally, a numerical example is given to demonstrate the effectiveness of our results.

Microarrays are capable of detecting the expression levels of thousands of genes simultaneously. So, gene expression data from DNA microarray are characterized by many measured variables (genes) on only a few samples. One important application of gene expression data is to classify the samples. In statistical terms, the very large number of predictors or variables compared to small number of samples makes most of classical “class prediction” methods unemployable. Generally, this problem can be avoided by selecting only the relevant features or extracting new features containing the maximal information about the class label from the original data. In this paper, a new method for gene selection based on independent variable group analysis is proposed. In this method, we first used t-statistics method to select a part of genes from the original data. Then, we selected the key genes from the selected genes for tumor classification using IVGA. Finally, we used SVM to classify tumors based on the key genes selected using IVGA. To validate the efficiency, the proposed method is applied to classify three different DNA microarray data sets. The prediction results show that our method is efficient and feasible.

The quality and size of training set have a great impact on the results of deep learning-based face-related tasks. However, collecting and labeling adequate samples with high-quality and balanced distributions still remains a laborious and expensive work, and various data augmentation techniques have thus been widely used to enrich the training dataset. In this paper, we review the existing works of face data augmentation from the perspectives of the transformation types and methods, with the state-of-the-art approaches involved. Among all these approaches, we put the emphasis on the deep learning-based works, especially the generative adversarial networks which have been recognized as more powerful and effective tools in recent years. We present their principles, discuss the results and show their applications as well as limitations. Different evaluation metrics for evaluating these approaches are also introduced. We point out the challenges and opportunities in the field of face data augmentation and provide brief yet insightful discussions.

A new algorithm for ranking the input features and obtaining the best feature subset is developed and illustrated in this paper. The asymptotic formula for mutual information and the expectation maximisation (EM) algorithm are used to developing the feature selection algorithm in this paper. We not only consider the dependence between the features and the class, but also measure the dependence among the features. Even for noisy data, this algorithm still works well. An empirical study is carried out in order to compare the proposed algorithm with the current existing algorithms. The proposed algorithm is illustrated by application to a variety of problems.

This paper proposes using neural networks (NN) to implement a real coded genetic algorithm (GA) with the center of gravity crossover (CGX) and the minimal generation gap (MGG) model. With all genetic operations of GA including selection, crossover, mutation and evaluation implemented with NN modules, this approach can realize in parallel genetic operations on the whole chromosome to achieve the maximum parallel realization potential of the MGG model of the GA. At the same time expensive hardware for field programmable gate arrays (FPGA) and the high speed memory of hardware for GA can be avoided. The performance of our solution is validated with a suite of benchmark test functions. This paper suggests that implementing GA with NN is a promising research direction for greatly reducing the running time of GA.

This paper aims to serve two main objectives; one is to demonstrate the modelling capabilities of a neuro-fuzzy approach, namely ANFIS (adaptive-network based fuzzy inference system) to a nonlinear system; and the other is to design a fuzzy controller to control such a system. The nonlinear system, which is a liquid-level system, is represented first by its mathematical model and then by ANFIS architecture. The ANFIS model is formed by means of input–output data set taken from the mathematical model. Then a PID-type fuzzy controller, which linguistically approximates the classical three-term compensation, was designed to control the system represented by both its mathematical and ANFIS models in order to perform an agreement comparison between them. It is shown that the ANFIS architecture can model a nonlinear system very accurately by means of input–output pairs obtained either from the actual system or its mathematical model. It is also shown that such a system can be controlled effectively by a fuzzy controller.

The problems associated with training feedforward artificial neural networks (ANNs) such as the multilayer perceptron (MLP) network and radial basis function (RBF) network have been well documented. The solutions to these problems have inspired a considerable amount of research, one particular area being the application of evolutionary search algorithms such as the genetic algorithm (GA). To date, the vast majority of GA solutions have been aimed at the MLP network. This paper begins with a brief overview of feedforward ANNs and GAs followed by a review of the current state of research in applying evolutionary techniques to training RBF networks.

Many attempts have been made to track web usage patterns and provide dynamic suggestions that might help users reach the information they seek more quickly. These mechanisms rely on mining the web log files for usage patterns. While this is easy for a single server, it would be much more difficult for multiple servers. We propose a mechanism that uses agent-based technologies and dynamic self-organizing maps to combine usage patterns in multiple web server logs and structures of multiple web sites to provide a more efficient and informative set of suggested links. This paper includes results of an ongoing project, which builds upon previously published results.

Effective multilingual information filtering is required to alleviate users’ burden of information overload resulting from the increasing flood of multilingual textual content available extensively over the World-Wide Web. This paper proposes a content-based self-organizing approach to multilingual information filtering using fuzzy logic and the self-organizing map. This approach screens and evaluates multilingual documents based on their semantic contents. Correlated multilingual documents are disseminated according to their corresponding themes or topics, thus enabling language-independent content-based information access efficiently and effectively. A Web-based multilingual online news-filtering system is developed to illustrate how the approach works.

The growing self-organising map (GSOM) has recently been proposed as an alternative neural network architecture based on the traditional self-organising map (SOM). The GSOM provides the user with the ability to control the spread of the map by defining a parameter called the spread factor (SF), which results in enhanced data mining and hierarchical clustering opportunities. When experimenting with the SOM, the grid size (number of rows and columns of nodes) can be changed until a suitable cluster distribution is achieved. In this paper we highlight the effect of the spread factor on the GSOM and contrast this effect with grid size change (increase and decrease) in the SOM. We also present experimental results in support of our claims regarding differences between GSOM and SOM.

The problem of obtaining the error at the output of a neuro sliding mode controller is analyzed in this paper. The controller operates in discrete time and the method presented describes an error measure that can be used if the task to be achieved is to drive the system under control to a predefined sliding regime. Once the task-specific output error is calculated, the neurocontroller parameters can be tuned so that the task is achieved. The paper postulates the strategy for discrete time representation of uncertain nonlinear systems belonging to a particular class. The performance of the proposed technique has been clarified on a third order nonlinear system, and the parameters of the controller are adjusted by using the error backpropagation algorithm. It is observed that the prescribed behavior can be achieved with a simple network configuration.

This paper compares a feature transformation method using a genetic algorithm (GA) with two conventional methods for artificial neural networks (ANNs). In this study, the GA is incorporated to improve the learning and generalizability of ANNs for stock market prediction. Daily predictions are conducted and prediction accuracy is measured. In this study, three feature transformation methods for ANNs are compared. Comparison of the results achieved by a feature transformation method using the GA to the other two feature transformation methods shows that the performance of the proposed model is better. Experimental results show that the proposed approach reduces the dimensionality of the feature space and decreases irrelevant factors for stock market prediction.

The grade of textile yarns is an important index in evaluating the yarn’s market value. This paper uses the backpropagation neural network (BNN) and Karhunen-Loeve (K-L) expansion method to construct a new and highly accurate grading system. Outcomes show that a highly accurate and neutral grading system can be obtained if the BNN learning sample is comprehensive or by adopting the BNN with a relearning technique (self-healing). Considering the possibility of reducing the dimension of BNN input vectors without losing the accuracy, this paper preprocesses the BNN grading system using the K-L expansion. Experiments demonstrate that the K-L expansion provides a way to reduce the input dimensions, and that a single principle axis value of the BNN with the K-L expansion grading system is able to grade textile yarns. In addition, the experiment demonstrates that as the input dimensions are reduced to four in a self-healing neural network with the K-L expansion, the grading system provides the high accuracy and robustness.

This paper proposes an information system that classifies Web pages according a taxonomy, which is mainly used from seven search engines/directories. The proposed classifier is a four-layer generalised regression neural network (GRNN) that aims to perform the information segmentation according to information filtering techniques using content descriptor vectors. Eight categories of Web pages were used in order to evaluate the robustness of the method, while no restrictions were imposed except for the language of the content, which is English. The system can be used as an assistant and consultative tool for classification purposes as well as for estimating the population of Web pages at any given point in time.

This paper proposes a pursuit system that utilizes the artificial life concept where autonomous mobile agents emulate the social behavior of animals and insects and realize their group behavior. Each agent contains sensors to perceive other agents in several directions, and decides its behavior based on the information obtained by these sensors. In this paper, a neural network is used for behavior decision controlling. The input of the neural network is decided by the existence of other agents, and the distance to the other agents. The output determines the directions in which the agent moves. The connection weight values of this neural network are encoded as genes, and the fitness individuals are determined using a genetic algorithm. Here, the fitness values imply how much group behavior adequately fit the goal and can express group behavior. The validity of the system is verified through simulation. Also in this paper, we have observed the agents’ emergent behavior during simulation.

Changing the resolution of digital images and video is needed image processing systems. In this paper, we present nonlinear interpolation schemes for still image resolution enhancement. The proposed neural network interpolation method is based on wavelet reconstruction. With the wavelet decomposition, the image signals can be divided into several time–frequency portions. In this work, the wavelet decomposition signal is used to train the neural networks. The pixels in the low-resolution image are used as the input signal of the neural network to estimate all the wavelet sub-images of the corresponding high-resolution image. The image of increased resolution is finally produced by the synthesis procedure of wavelet transform. In the simulation, the proposed method obtains much better performance than other traditional methods. Moreover, the easy implementation and high flexibility of the proposed algorithm also make it applicable to various other related problems.

A novel neural-network-based method of time series forecasting is presented in this paper. The method combines the optimal partition algorithm (OPA) with the radial basis function (RBF) neural network. OPA for ordered samples is used to perform the clustering for the samples. The centers and widths of the RBF neural network are determined based on the clustering. The difference of the objective functions of the clustering is used to adjust the structure of the neural network dynamically. Thus, the number of the hidden nodes is selected adaptively. The method is applied to stock price prediction. The results of numerical simulations demonstrate the effectiveness of the method. Comparisons with the hard c-means (HCM) algorithm show that the proposed OPA method possesses obvious advantages in the precision of forecasting, generalization, and forecasting trends. Simulations also show that the OPA–orthogonal least squares (OPA–OLS) algorithm, which combines OPA with the OLS algorithm, results in better performance for forecasting trends.

A new supervision system consisting of three modules is presented. The main novelty is the first module that corresponds to a modelling task. This module, which uses the auto-adaptive and dynamical clustering (AUDyC) neural network, allows us to continuously analyse and classify the functioning state of the monitored system using a dynamical modelling of all known modes (good/bad functioning modes represent different classes). The second module exploits these models of the functioning modes in order to detect “fast” and “slow” deviations. From membership degrees and from the information extracted by the monitoring module, the third module, dedicated to the diagnostics, informs the user about the functioning conditions of the system. In this paper, the main characteristics of the AUDyC and its abilities to model on-line non-stationary data are presented. Then, the description of the supervision system is given and some experimental results stemmed from a supervision application of a hydraulic system are discussed.

Principal component analysis (PCA) by neural networks is one of the most frequently used feature extracting methods. To process huge data sets, many learning algorithms based on neural networks for PCA have been proposed. However, traditional algorithms are not globally convergent. In this paper, a new PCA learning algorithm based on cascade recursive least square (CRLS) neural network is proposed. This algorithm can guarantee the network weight vector converges to an eigenvector associated with the largest eigenvalue of the input covariance matrix globally. A rigorous mathematical proof is given. Simulation results show the effectiveness of the algorithm.

One of the major tasks in some human–computer interface applications, such as face recognition and video telephony, is to localize a human face in an image. In this paper, we propose to use hierarchical neural networks with local recurrent connectivity to solve this task not only in unambiguous situations, but also in the presence of complex backgrounds, difficult lighting, and noise. The networks are trained using a database of gray-scale still images and manually determined eye coordinates. They are able to produce reliable and accurate eye coordinates for unknown images by iteratively refining initial solutions. Because the networks process entire images, there is no need for any time-consuming scanning across positions and scales. Furthermore, the fast network updates allow for real-time face tracking. In this case, the networks are trained using still images that move in random directions. The trained networks are able to accurately track the eye positions in the test image sequences.

Real-time embedded systems are spreading to more and more new fields and their scope and complexity have grown dramatically in the last few years. Nowadays, real-time embedded computers or controllers can be found everywhere, both in very simple devices used in everyday life and in professional environments. Real-time embedded systems have to take into account robustness, safety and timeliness. The most-used schedulability analysis is the worst-case response time proposed by Joseph and Pandya (Comput J 29:390–395,1986). This test provides a bivaluated response (yes/no) indicating whether the processes will meet their corresponding deadlines or not. Nevertheless, sometimes the real-time designer might want to know, more exactly, the probability of the processes meeting their deadlines, in order to assess the risk of a failed scheduling depending on critical requirements of the processes. This paper presents RealNet, a neural network architecture that will generate schedules from timing requirements of a real-time system. The RealNet simulator will provide the designer, after iterating and averaging over some trials, an estimation of the probability that the system will not meet the deadlines. Moreover, the knowledge of the critical processes in these schedules will allow the designer to decide whether changes in the implementation are required.

In this paper, a hybrid neural network that is capable of incremental learning and classification of patterns with incomplete data is proposed. Fuzzy ARTMAP (FAM) is employed as the constituting network for pattern classification while fuzzy c-means (FCM) clustering is used as the underlying algorithm for processing training as well as test samples with missing features. To handle an incomplete training set, FAM is first trained using complete samples only. Missing features of the training samples are estimated and replaced using two FCM-based strategies. Then, network training is conducted using all the complete and estimated samples. To handle an incomplete test set, a non-substitution FCM-based strategy is employed so that a predicted output can be produced rapidly. The performance of the proposed hybrid network is evaluated using a benchmark problem, and its practical applicability is demonstrated using a medical diagnosis task. The results are compared, analysed and quantified statistically with the bootstrap method. Implications of the proposed network for pattern classification tasks with incomplete data are discussed.

The paper presents a neural network based multi-classifier system for the identification of Escherichia coli promoter sequences in strings of DNA. As each gene in DNA is preceded by a promoter sequence, the successful location of an E. coli promoter leads to the identification of the corresponding E. coli gene in the DNA sequence. A set of 324 known E. coli promoters and a set of 429 known non-promoter sequences were encoded using four different encoding methods. The encoded sequences were then used to train four different neural networks. The classification results of the four individual neural networks were then combined through an aggregation function, which used a variation of the logarithmic opinion pool method. The weights of this function were determined by a genetic algorithm. The multi-classifier system was then tested on 159 known promoter sequences and 171 non-promoter sequences not contained in the training set. The results obtained through this study proved that the same data set, when presented to neural networks in different forms, can provide slightly varying results. It also proves that when different opinions of more classifiers on the same input data are integrated within a multi-classifier system, we can obtain results that are better than the individual performances of the neural networks. The performances of our multi-classifier system outperform the results of other prediction systems for E. coli promoters developed so far.

In real systems, fault diagnosis is performed by a human diagnostician, and it encounters complex knowledge associations, both for normal and faulty behaviour of the target system. The human diagnostician relies on deep knowledge about the structure and the behaviour of the system, along with shallow knowledge on fault-to-manifestation patterns acquired from practice. This paper proposes a general approach to embed deep and shallow knowledge in neural network models for fault diagnosis by abduction, using neural sites for logical aggregation of manifestations and faults. All types of abduction problems were considered. The abduction proceeds by plausibility and relevance criteria multiply applied. The neural network implements plausibility by feed-forward links between manifestations and faults, and relevance by competition links between faults. Abduction by plausibility and relevance is also used for decision on the next best test along the diagnostic refinement. A case study on an installation in a rolling mill plant is presented.

Some medical and epidemiological surveys have been designed to predict a nominal response variable with several levels. With regard to the type of pregnancy there are four possible states: wanted, unwanted by wife, unwanted by husband and unwanted by couple. In this paper, we have predicted the type of pregnancy, as well as the factors influencing it using two different models and comparing them. Regarding the type of pregnancy with several levels, we developed a multinomial logistic regression and a neural network based on the data and compared their results using three statistical indices: sensitivity, specificity and kappa coefficient. Based on these three indices, neural network proved to be a better fit for prediction on data in comparison to multinomial logistic regression. When the relations among variables are complex, one can use neural networks instead of multinomial logistic regression to predict the nominal response variables with several levels in order to gain more accurate predictions.

The main objective of this paper is to propose a Neuro-Fuzzy network, which can model a system from input–output data by automatically dividing the input–output space and extracting fuzzy if-then rules from numerical data. The structure of the network is simple with input, rule and output layers only. The connections between input and rule layer is used to derive the membership functions of the fuzzified part. Kohonen’s self-organizing learning algorithm is applied to partition the pattern space. Using this algorithm, similar rules are mapped close by and extraction of if-then rules is made easy. It can also adapt to a number of rules automatically. The proposed network is verified for three benchmark applications. Experimental results show that the adaptive method discussed in this paper not only trains in a few hundred iterations but also provides better performance measures when compared to conventional methods.

Neural networks have attracted much interest in the last two decades for their potential to realistically describe brain functions, but so far they have failed to provide models that can be simulated in a reasonable time on computers; rather they have been limited to toy models. Quantum computing is a possible candidate for improving the computational efficiency of neural networks. In this framework of quantum computing, the Qubit neuron model, proposed by Matsui and Nishimura, has shown a high efficiency in solving problems such as data compression. Simulations have shown that the Qubit model solves learning problems with significantly improved efficiency as compared to the classical model. In this paper, we confirm our previous results in further detail and investigate what contributes to the efficiency of our model through 4-bit and 6-bit parity check problems, which are known as basic benchmark tests. Our simulations suggest that the improved performance is due to the use of superposition of neural states and the use of probability interpretation in the observation of the output states of the model.

An adaptive direct recurrent neural network model is developed for nonlinear dynamic system modelling in this paper. The model adaptation is achieved with the extended Kalman filter (EKF). A novel recursive algorithm is proposed to calculate the Jacobian matrix in the model adaptation so that the algorithm is simple and converges fast. The effectiveness of the developed adaptive model is demonstrated by applying to modelling a simulated continuous stirred tank reactor (CSTR). The model converges to the new process dynamics very quickly after a constant disturbance is added, and therefore can be used as an adaptive model in the adaptive model predictive control or internal model control for time-varying systems or fault tolerant control of nonlinear systems.

This paper proposes a new codebook generation algorithm for image data compression using a combined scheme of principal component analysis (PCA) and genetic algorithm (GA). The combined scheme makes full use of the near global optimal searching ability of GA and the computation complexity reduction of PCA to compute the codebook. The experimental results show that our algorithm outperforms the popular LBG algorithm in terms of computational efficiency and image compression performance.

Condition monitoring of machine tool inserts is important for increasing the reliability and quality of machining operations. Various methods have been proposed for effective tool condition monitoring (TCM), and currently it is generally accepted that the indirect sensor-based approach is the best practical solution to reliable TCM. Furthermore, in recent years, neural networks (NNs) have been shown to model successfully, the complex relationships between input feature sets of sensor signals and tool wear data. NNs have several properties that make them ideal for effectively handling noisy and even incomplete data sets. There are several NN paradigms which can be combined to model static and dynamic systems. Another powerful method of modeling noisy dynamic systems is by using hidden Markov models (HMMs), which are commonly employed in modern speech-recognition systems. The use of HMMs for TCM was recently proposed in the literature. Though the results of these studies were quite promising, no comparative results of competing methods such as NNs are currently available. This paper is aimed at presenting a comparative evaluation of the performance of NNs and HMMs for a TCM application. The methods are employed on exactly the same data sets obtained from an industrial turning operation. The advantages and disadvantages of both methods are described, which will assist the condition-monitoring community to choose a modeling method for other applications.

Contemporary design process requires the development of a new computational intelligence or soft computing methodology that involves intelligence integration and hybrid intelligent systems for design, analysis and evaluation, and optimization. This paper first presents a discussion of the need to incorporate “intelligence” into an automated design process and the various constraints that designers face when embarking on industrial design projects. Then, it presents the design problem as optimizing the design output against constraints and the use of soft computing and hybrid intelligent systems techniques. In this paper, a soft-computing-integrated intelligent design framework is developed. A hybrid dual cross-mapping neural network (HDCMNN) model is proposed using the hybrid soft computing technique based on “cross-mapping” between a back-propagation network (BPNN) and a recurrent Hopfield network (HNN) for supporting modeling, analysis and evaluation, and optimization tasks in the design process. The two networks perform different but complementary tasks—the BPNN “decides” if the design problem is a “type 0” (rational) or “type 1” (non-rational) problem, and the output layer weights are then used as the energy function for the HNN. The BPNN is used for representing design patterns, training classification boundaries, and outputting network weight values to the HNN, and then the HNN uses the calculated network weight values to evaluate and modify or re-design the design patterns. The developed system provides a unified soft-computing-integrated intelligent design framework with both symbolic and computational intelligence. The system has self-modifying and self-learning functions. Within the system, only one network training is needed for accomplishing the evaluation, rectification/modification, and optimization tasks in the design process. Finally, two case studies are provided to illustrate and validate the developed model and system.

This paper reviews the potential benefits that can be obtained by the implementation of data fusion in a multi-sensor environment. A thorough review of the commonly used data fusion frameworks is presented together with important factors that need to be considered during the development of an effective data fusion problem-solving strategy. A system-based approach is defined for the application of data fusion systems within engineering. Structured guidelines for users are proposed.

Fuzzy logic can bring about inappropriate inferences as a result of ignoring some information in the reasoning process. Neural networks are powerful tools for pattern processing, but are not appropriate for the logical reasoning needed to model human knowledge. The use of a neural logic network derived from a modified neural network, however, makes logical reasoning possible. In this paper, we construct a fuzzy inference network by extending the rule–inference network based on an existing neural logic network. The propagation rule used in the existing rule–inference network is modified and applied. In order to determine the belief value of a proposition pertaining to the execution part of the fuzzy rules in a fuzzy inference network, the nodes connected to the proposition to be inferenced should be searched for. The search costs are compared and evaluated through application of sequential and priority searches for all the connected nodes.

Information security has been a critical issue in the field of information systems. One of the key factors in the security of a computer system is how to identify the authorization of users. Password-based user authentication is widely used to authenticate a legitimate user in the current system. In conventional password-based user authentication schemes, a system has to maintain a password table or verification table which stores the information of users’ IDs and passwords. Although the one-way hash functions and encryption algorithms are applied to prevent the passwords from being disclosed, the password table or verification table is still vulnerable. In order to solve this problem, in this paper, we apply the technique of back-propagation network instead of the functions of the password table and verification table. Our proposed scheme is useful in solving the security problems that occurred in systems using the password table and verification table. Furthermore, our scheme also allows each user to select a username and password of his/her choice.

Dunis and Williams (Derivatives: use, trading and regulation 8(3):211–239, 2002; Applied quantitative methods for trading and investment. Wiley, Chichester, 2003) have shown the superiority of a Multi-layer perceptron network (MLP), outperforming its benchmark models such as a moving average convergence divergence technical model (MACD), an autoregressive moving average model (ARMA) and a logistic regression model (LOGIT) on a Euro/Dollar (EUR/USD) time series. The motivation for this paper is to investigate the use of different neural network architectures. This is done by benchmarking three different neural network designs representing a level estimator, a classification model and a probability distribution predictor. More specifically, we present the Mulit-layer perceptron network, the Softmax cross entropy model and the Gaussian mixture model and benchmark their respective performance on the Euro/Dollar (EUR/USD) time series as reported by Dunis and Williams. As it turns out, the Multi-layer perceptron does best when used without confirmation filters and leverage, while the Softmax cross entropy model and the Gaussian mixture model outperforms the Multi-layer perceptron when using more sophisticated trading strategies and leverage. This might be due to the ability of both models using probability distributions to identify successfully trades with a high Sharpe ratio.

Most of the cost functions used for blind equalization are nonconvex and nonlinear functions of tap weights, when implemented using linear transversal filter structures. Therefore, a blind equalization scheme with a nonlinear structure that can form nonconvex decision regions is desirable. The efficacy of complex-valued feedforward neural networks for blind equalization of linear and nonlinear communication channels has been confirmed by many studies. In this paper we present a complex valued neural network for blind equalization with M-ary phase shift keying (PSK) signals. The complex nonlinear activation functions used in the neural network are especially defined for handling the M-ary PSK signals. The training algorithm based on constant modulus algorithm (CMA) cost function is derived. The improved performance of the proposed neural network in both, stationary and nonstationary environments, is confirmed through computer simulations.

In this paper, the radial basis function-based Gaussian mixture model (GMM) is applied to model and predict drug dissolution profiles in a time-series approach. The Parzen-window method is embedded into the GMM for determining whether the network predictions are derived from interpolation or extrapolation of the training data. A benchmark study on time-series prediction is first used to evaluate and compare the GMM performance with those from other models. The GMM is then used to predict dissolution profiles of a matrix-controlled release theophylline pellet preparation. Performance of the GMM is assessed using the difference and similarity factors, as recommended by the United States Food and Drug Administration for dissolution profile comparison. In addition, bootstrapping is employed to estimate the confidence intervals of the network predictions. The experimental results are analyzed and compared, and implications of the GMM for pharmaceutical product formulation tasks are discussed.

As a generalization of the multi-layer perceptron (MLP), the circular back-propagation neural network (CBP) possesses better adaptability. An improved version of the CBP (the ICBP) is presented in this paper. Despite having less adjustable weights, the ICBP has better adaptability than the CBP, which quite equals the famous Occam’s razor principle for model selection. In its application to time series, considering both structural changes and correlations of time series itself, we introduce the principle of the discounted least squares (DLS) in CBP and ICBP, respectively, and investigate their predicting capacity further. Introduction of DLS improves the predicting performance of both on a benchmark time series data set. Finally, the comparison of experimental results shows that ICBP with DLS (DLS-ICBP) has better predicting performance than DLS-CBP.

This paper investigates the efficacy of cross-entropy and square-error objective functions used in training feed-forward neural networks to estimate posterior probabilities. Previous research has found no appreciable difference between neural network classifiers trained using cross-entropy or squared-error. The approach employed here, though, shows cross-entropy has significant, practical advantages over squared-error.

The Kohonen self-organising feature map (SOM) has several important properties that can be used within the data mining/knowledge discovery and exploratory data analysis process. A key characteristic of the SOM is its topology preserving ability to map a multi-dimensional input into a two-dimensional form. This feature is used for classification and clustering of data. However, a great deal of effort is still required to interpret the cluster boundaries. In this paper we present a technique which can be used to extract propositional IF..THEN type rules from the SOM network’s internal parameters. Such extracted rules can provide a human understandable description of the discovered clusters.

Scene analysis is an important area of research with the aim of identifying objects and their relationships in natural scenes. MINERVA benchmark has been recently introduced in this area for testing different image processing and classification schemes. In this paper we present results on the classification of eight natural objects in the complete set of 448 natural images using neural networks. An exhaustive set of experiments with this benchmark has been conducted using four different segmentation methods and five texture-based feature extraction methods. The results in this paper show the performance of a neural network classifier on a tenfold cross-validation task. On the basis of the results produced, we are able to rank how well different image segmentation algorithms are suited to the task of region of interest identification in these images, and we also see how well texture extraction algorithms rank on the basis of classification results.

The behavior of a multivariable predictive control scheme based on neural networks applied to a model of a nonlinear multivariable real process, consisting of a pressurized tank is investigated in this paper. The neural scheme consists of three neural networks; the first is meant for the identification of plant parameters (identifier), the second one is for the prediction of future control errors (predictor) and the third one, based on the two previous, compute the control input to be applied to the plant (controller). The weights of the neural networks are updated on-line, using standard and dynamic backpropagation. The model of the nonlinear process is driven to an operation point and it is then controlled with the proposed neural control scheme, analyzing the maximum range over the neural control works properly.