Search

WOS: 000521878300002, Up to date technological implementations of deep convolutional neural networks are at the forefront of many issues, such as autonomous device control, effective image and pattern recognition solutions. Deep neural networks generally utilize a hybrid topology of a feature extractor containing convolutional layers followed by a fully connected classifier network. The characteristic and quality of the produced features differ according to the deep learning structure. In order to get high performance, it is necessary to choose an effective topology. In this study, a novel topology based hybrid structure named as Deep Convolutional Generalized Classifier Neural Network and its learning algoritm are introduced. This novel structure allows the deep learning network to extract features with the desired characteristics. This ensures high performance classification, even for relatively small deep learning networks. This has led to many novelties such as principal feature analysis, better learning ability, one-pass learning for classifier part, new error computation and backpropagation approach for filter weights. Two experiment sets were performed to measure the performance of DC-GCNN. In the first experiment set, DC-GCNN was compared with clasical approach on 10 different datasets. DC-GCNN performed better up to 44.45% for precision, 39.69% for recall and 42.57% for F1-score. In the second experiment set, DC-GCNN's performance was compared with alternative methods on larger datasets. Proposed structure performed better than alternative deep learning based classifier structures on CIFAR-10 and MNIST datasets with 89.12% and 99.28% accuracy values.

This paper presents a brief overview of vibration-based damage identification studies based on Deep Learning (DL) in civil engineering structures. The presence, type, size, and propagation of structural damage on civil infrastructure have always been a topic of research. In the last couple of decades, there has been a significant shift in the damage detection paradigm when the advancements in sensing and computing technologies met with the ever-expanding use of artificial neural network algorithms. Machine-Learning (ML) tools enabled researchers to implement more feasible and faster tools in damage detection applications. When an artificial neural network has more than three layers, it is typically considered as a ?deep? learning network. Being an important accomplishment of the ML era, DL tools enable complex systems which are made of several layers to learn implementations of data with outstanding categorization and compartmentalization capability. In fact, with proper training, a DL tool can operate directly with the unprocessed raw data and help the algorithm produce output data. Competitive capabilities like this led DL algorithms perform very well in complicated problems by dividing a relatively large problem into much smaller and more manageable portions. Specifically for damage identification and localization on civil infrastructure, Convolutional Neural Networks (CNNs) and Unsupervised Pretrained Networks (UPNs) are the known DL tools published in the literature. This paper presents an overview of these studies. Scopus

Many deep neural networks are built by using stacked convolutional layers of fixed and single size (often 3$\times$3) kernels. This paper describes a method for training the size of convolutional kernels to provide varying size kernels in a single layer. The method utilizes a differentiable, and therefore backpropagation-trainable Gaussian envelope which can grow or shrink in a base grid. Our experiments compared the proposed adaptive layers to ordinary convolution layers in a simple two-layer network, a deeper residual network, and a U-Net architecture. The results in the popular image classification datasets such as MNIST, MNIST-CLUTTERED, CIFAR-10, Fashion, and ``Faces in the Wild'' showed that the adaptive kernels can provide statistically significant improvements on ordinary convolution kernels. A segmentation experiment in the Oxford-Pets dataset demonstrated that replacing a single ordinary convolution layer in a U-shaped network with a single 7$\times$7 adaptive layer can improve its learning performance and ability to generalize., 25 pages

Machine learning (ML) techniques play a crucial role in today's modern world. Over the last years, road traffic safety is one of the applications where ML‐methods have been successfully employed to prevent road users from being killed or seriously injured. A reliable data‐driven predictive model is essential for this purpose. This could be achieved by successfully applying an intelligent transportation system to identify a driver at a higher risk of crashes. This study investigates the capabilities of different deep learning techniques to predict motorcycle crash severity. This study is based on 2,430 motorcycle crashes in a mountainous area in the United States over a 10‐year period. Different deep networks (DNNs), including deep belief network, standard recurrent neural network (RNN), multilayer neural network, and single‐layer neural network, were considered and compared in terms of prediction accuracy of motorcycle crash severity. Before conducting any analysis, feature reduction was performed to identify the optimal number of variables to include in the models by minimizing the error rate. Different metrics including the area under the curve and confusion matrix were used to compare the different models. Although the analyses were conducted on a relatively small dataset, the results indicate that almost all the DNN models better perform in predicting the severity of motorcycle crashes, compared with the single layer neural network. Finally, the RNN outperforms the other three neural network models. A comprehensive discussion has been made about the methodological approach implemented in this study.

The motivation of this study is to develop a compact offline recognition model for Khmer handwritten text that would be successfully applied under limited access to high-performance computational hardware. Such a task aims to ease the ad-hoc digitization of vast handwritten archives in many spheres. Data collected for previous experiments were used in this work. The oneagainst-all classification was completed with state-of-the-art techniques. A compact deep learning model (2+1CNN), with two convolutional layers and one fully connected layer, was proposed. The recognition rate came out to be within 93-98%. The compact model is performed on par with the state-of-theart models. It was discovered that computational capacity requirements usually associated with deep learning can be alleviated, therefore allowing applications under limited computational power.

In general, the imbalanced dataset is a problem often found in health applications. In medical data classification, we often face the imbalanced number of data samples where at least one of the classes constitutes only a very small minority of the data. In the same time, it represent a difficult problem in most of machine learning algorithms. There have been many works dealing with classification of imbalanced dataset. In this paper, we proposed a learning method based on a cost sensitive extension of Least Mean Square (LMS) algorithm that penalizes errors of different samples with different weights and some rules of thumb to determine those weights. After the balancing phase, we apply the different techniques (Support Vector Machine [SVM], K- Nearest Neighbor [K-NN] and Multilayer perceptron [MLP]) for the balanced datasets. We have also compared the obtained results before and after balancing method. We have obtained best results compared to literature with a classification accuracy of 100%.

WOS: 000386750300004, 28110727, Congestive heart failure (CHF) is a degree of cardiac disease occurring as a result of the heart's inability to pump enough blood for the human body. In recent studies, coronary artery disease (CAD) is accepted as the most important cause of CHF. This study focuses on the diagnosis of both the CHF and the CAD. The Hilbert-Huang transform (HHT), which is effective on nonlinear and non-stationary signals, is used to extract the features from R-R intervals obtained from the raw electrocardiogram data. The statistical features are extracted from instinct mode functions that are obtained applying the HHT to R-R intervals. Classification performance is examined with extracted statistical features using a multilayer perceptron neural network. The designed model classified the CHF, the CAD patients and a normal control group with rates of 97.83%, 93.79% and 100%, accuracy, specificity and sensitivity, respectively. Also, early diagnosis of the CHF was performed by interpretation of the CAD with a classification accuracy rate of 97.53%, specificity of 98.18% and sensitivity of 97.13%. As a result, a single system having the ability of both diagnosis and early diagnosis of CHF is performed by integrating the CAD diagnosis method to the CHF diagnosis method. (C) 2016 Elsevier Ireland Ltd. All rights reserved.

Deep neural networks show high accuracy in theproblem of semantic and instance segmentation of biomedicaldata. However, this approach is computationally expensive. Thecomputational cost may be reduced with network simplificationafter training or choosing the proper architecture, which providessegmentation with less accuracy but does it much faster. In thepresent study, we analyzed the accuracy and performance ofUNet and ENet architectures for the problem of semantic imagesegmentation. In addition, we investigated the ENet architecture by replacing of some convolution layers with box-convolutionlayers. The analysis performed on the original dataset consisted of histology slices with mast cells. These cells provide a region forsegmentation with different types of borders, which vary fromclearly visible to ragged. ENet was less accurate than UNet byonly about 1-2%, but ENet performance was 8-15 times faster than UNet one., Comment: 4 pages, 5 figures, 1 table

Polychronous Wavefront Computation (PWC) is an abstraction of spiking neural networks that has been shown to be capable of basic computational functions and simple pattern recognition through multilayer configurations. The objective of this work is to apply unsupervised learning methods to multilayer PWC configurations to improve performance providing a basis for more advanced applications and deep learning. Previous work on defining multilayer PWC configurations is extended by applying biologically inspired learning methods to dynamically suppress unneeded transponders and improve configuration performance. Simple learning approaches based on concepts from spike-timing-dependent plasticity and potentiation decay models are adapted to PWC transponders and combined with training sequences to optimize the transponder configurations for recognition. Learning is further enhanced by configuring transponders in recurrent structures to activate hidden layer transponders creating reverberations that reinforce learning. A means to classify multiple input patterns into general concepts is also introduced to further enhance the recognition capabilities of the configurations. The concepts are experimentally validated and analyzed through application to a 7-segment display digit recognition problem showing that the approach can improve PWC configuration performance and reduce complexity.

The main objective of this work is to develop a method to study the contour of crater wear and measure it in three dimensions. A new technique for the measurement and visualization of tool wear has been presented in this paper. This method provides visualization of the tool wear geometry using a pair of stereo images. In addition, prediction of tool wear using artificial neural network is presented. A multilayered perceptron with back-propagation algorithm has been used for tool wear estimation, which could be trained using much less data than that is required in a normal mathematical simulation. Speed, feed, depth of cut and cutting time were used as input parameters and flank wear width and crater wear depth were output parameters. Training and testing of the network were carried out and the results are presented and analyzed in this work. ? 2001 Elsevier Science B.V.

In this paper, we present a review of some recent works on approximation by feedforward neural networks. A particular emphasis is placed on the computational aspects of the problem, i.e. we discuss the possibility of realizing a feedforward neural network which achieves a prescribed degree of accuracy of approximation, and the determination of the number of hidden layer neurons required to achieve this accuracy. Furthermore, a unifying framework is introduced to understand existing approaches to investigate the universal approximation problem using feedforward neural networks. Some new results are also presented. Finally, two training algorithms are introduced which can determine the weights of feedforward neural networks, with sigmoidal activation neurons, to any degree of prescribed accuracy. These training algorithms are designed so that they do not suffer from the problems of local minima which commonly affect neural network learning algorithms.

One of the major challenges of providing reliable healthcare services is to diagnose and treat diseases in an accurate and timely manner. Recently, many researchers have successfully used artificial neural networks as a diagnostic assessment tool. In this study, the validation of such an assessment tool has been developed for treatment of the femoral peripheral arterial disease using a radial basis function neural network (RBFNN). A data set for training the RBFNN has been prepared by analyzing records of patients who had been treated by the thoracic and cardiovascular surgery clinic of a university hospital. The data set includes 186 patient records having 16 characteristic features associated with a binary treatment decision, namely, being a medical or a surgical one. K-means clustering algorithm has been used to determine the parameters of radial basis functions and the number of hidden nodes of the RBFNN is determined experimentally. For performance evaluation, the proposed RBFNN was compared to three different multilayer perceptron models having Pareto optimal hidden layer combinations using various performance indicators. Results of comparison indicate that the RBFNN can be used as an effective assessment tool for femoral peripheral arterial disease treatment.

An investigation of the applicability of neural network-based methods in predicting the values of multiple parameters, given the value of a single parameter within a particular problem domain is presented. In this context, the input parameter may be an important source of variation that is related with a complex mapping function to the remaining sources of variation within a multivariate distribution. The definition of the relationship between the variables of a multivariate distribution and a single source of variation allows the estimation of the values of multiple variables given the value of the single variable, addressing in that way an ill-conditioned one-to-many mapping problem. As part of our investigation, two problem domains are considered: predicting the values of individual stock shares, given the value of the general index, and predicting the grades received by high school pupils, given the grade for a single course or the average grade. With our work, the performance of standard neural network-based methods and in particular multilayer perceptrons (MLPs), radial basis functions (RBFs), mixture density networks (MDNs) and a latent variable method, the general topographic mapping (GTM), is compared. According to the results, MLPs and RBFs outperform MDNs and the GTM for these one-to-many mapping problems. © 2010 Springer-Verlag London Limited. 20 6 775 785 Cited By :5

This paper proposes the use of hybrid Hidden Markov Model (HMM)/Artificial Neural Network (ANN) models for recognizing unconstrained offline handwritten texts. The structural part of the optical models has been modeled with Markov chains, and a Multilayer Perceptron is used to estimate the emission probabilities. This paper also presents new techniques to remove slope and slant from handwritten text and to normalize the size of text images with supervised learning methods. Slope correction and size normalization are achieved by classifying local extrema of text contours with Multilayer Perceptrons. Slant is also removed in a nonuniform way by using Artificial Neural Networks. Experiments have been conducted on offline handwritten text lines from the IAM database, and the recognition rates achieved, in comparison to the ones reported in the literature, are among the best for the same task. © 2006 IEEE., The authors acknowledge the valuable help provided by Moises Pastor, Juan Miguel Vilar, Alex Graves, and Marcus Liwicki. Thanks are also due to the reviewers and the Editor-in-Chief for their many valuable comments and suggestions. This work has been partially supported by the Spanish Ministerio de Educacion y Ciencia (TIN2006-12767) and by the BPFI 06/250 Scholarship from the Conselleria d'Empresa, Universitat i Ciencia, Generalitat Valenciana.

A learning process of a single neural network (SNN) to improve prediction accuracy of protein secondary structure is optimized. The protein secondary structures are predicted using a multiple alignment of amino acid as the input data. A multi-modal neural network (MNN) has been proposed to improve the precision of prediction. This method uses five independent neural networks, and the final decision is made by averaging all outputs of five SNNs. In the proposed method, the same prediction accuracy can be achieved by using only a single NN and optimizing a learning process. In a learning process of protein structure prediction, over learning is easily occurred. So, the learning process is optimized so as to avoid the over learning. For this purpose, small learning rates, adding small random noise to the input data, and updating the connection weights by the average in some group are useful. The prediction accuracy 58% obtained by using the conventional SNN is improved to 66%, which is the same accuracy of the MNN, which needs five SNNs.

In training neural networks, it is important to reduce input variables for saving memory, reducing network size, and achieving fast training. The paper proposes two kinds of selecting methods for useful input variables. One of them is to use information of connection weights after training. If a sum of absolute value of the connection weights related to the input node is large, then this input variable is selected. In some cases, only positive connection weights are taken into account. The other method is based on correlation coefficients among the input variables. If a time series of the input variable can be obtained by amplifying and shifting that of another input variable, then the former can be absorbed in the latter. These analysis methods are applied to predicting cutting error caused by thermal expansion and compression in machine tools. The input variables are reduced from 32 points to 16 points, while maintaining good prediction within 6/spl mu/m, which can be applicable to real machine tools.

In this research, neural network (NN) and genetic algorithm (GA) are used together to design optimal NN structure. The proposed approach combines the characteristics of GA and NN to reduce the computational complexity of artificial intelligence applications in design and manufacturing. Genetic input selection approach is introduced to obtain optimal NN topology. Experimental results are given to evaluate the performance of the proposed system.

Many problems solved by multilayer neural networks (MLNNs) are reduced into pattern mapping. If the mapping includes several different rules, it is difficult to solve these problems by using a single MLNN with linear connection weights and continuous activation functions. In this paper, a structure trainable neural network has been proposed. The gate units are embedded, which can be trained together with the connection weights. Pattern mapping problems, which include several different mapping rules, can be realized using a single new network. Since, some parts of the network can be commonly used for different mapping rules, the network size can be reduced compared with the modular neural networks, which consists of several independent expert networks.

A training data selection method for multi-class data is proposed. This method can be used for multilayer neural networks (MLNN). The MLNN can be applied to pattern classification, signal process, and other problems that can be considered as the classification problem. The proposed data selection algorithm selects the important data to achieve a good classification performance. However, the training using the selected data converges slowly, so we also propose an acceleration method. The proposed training method adds the randomly selected data to the boundary data. The validity of the proposed methods is confirmed through the computer simulation.