Search

While perception tasks such as visual object recognition and text understanding play an important role in human intelligence, subsequent tasks that involve inference, reasoning, and planning require an even higher level of intelligence. The past few years have seen major advances in many perception tasks using deep learning models. For higher-level inference, however, probabilistic graphical models with their Bayesian nature are still more powerful and flexible. To achieve integrated intelligence that involves both perception and inference, it is naturally desirable to tightly integrate deep learning and Bayesian models within a principled probabilistic framework, which we call Bayesian deep learning. In this unified framework, the perception of text or images using deep learning can boost the performance of higher-level inference and in return, the feedback from the inference process is able to enhance the perception of text or images. This paper proposes a general framework for Bayesian deep learning and reviews its recent applications on recommender systems, topic models, and control. In this paper, we also discuss the relationship and differences between Bayesian deep learning and other related topics such as the Bayesian treatment of neural networks. [ABSTRACT FROM PUBLISHER]

Health care organizations are leveraging machine-learning techniques, such as artificial neural networks (ANN), to improve delivery of care at a reduced cost. Applications of ANN to diagnosis are well-known; however, ANN are increasingly used to inform health care management decisions. We provide a seminal review of the applications of ANN to health care organizational decision-making. We screened 3,397 articles from six databases with coverage of Health Administration, Computer Science and Business Administration. We extracted study characteristics, aim, methodology and context (including level of analysis) from 80 articles meeting inclusion criteria. Articles were published from 1997–2018 and originated from 24 countries, with a plurality of papers (26 articles) published by authors from the United States. Types of ANN used included ANN (36 articles), feed-forward networks (25 articles), or hybrid models (23 articles); reported accuracy varied from 50% to 100%. The majority of ANN informed decision-making at the micro level (61 articles), between patients and health care providers. Fewer ANN were deployed for intra-organizational (meso- level, 29 articles) and system, policy or inter-organizational (macro- level, 10 articles) decision-making. Our review identifies key characteristics and drivers for market uptake of ANN for health care organizational decision-making to guide further adoption of this technique. [ABSTRACT FROM AUTHOR]

Effort-aware just-in-time (JIT) defect prediction is to rank source code changes based on the likelihood of detects as well as the effort to inspect such changes. Accurate defect prediction algorithms help to find more defects with limited effort. To improve the accuracy of defect prediction, in this paper, we propose a deep learning based approach for effort-aware just-in-time defect prediction. The key idea of the proposed approach is that neural network and deep learning could be exploited to select useful features for defect prediction because they have been proved excellent at selecting useful features for classification and regression. First, we preprocess ten numerical metrics of code changes, and then feed them to a neural network whose output indicates how likely the code change under test contains bugs. Second, we compute the benefit cost ratio for each code change by dividing the likelihood by its size. Finally, we rank code changes according to their benefit cost ratio. Evaluation results on a well-known data set suggest that the proposed approach outperforms the state-of-the-art approaches on each of the subject projects. It improves the average recall and popt by 15.6% and 8.1%, respectively. [ABSTRACT FROM AUTHOR]

Big data classification is very slow when using traditional machine learning classifiers, particularly when using a lazy and slow-by-nature classifier such as the k-nearest neighbors algorithm (KNN). This paper proposes a new approach which is based on sorting the feature vectors of training data in a binary search tree to accelerate big data classification using the KNN approach. This is done using two methods, both of which utilize two local points to sort the examples based on their similarity to these local points. The first method chooses the local points based on their similarity to the global extreme points, while the second method chooses the local points randomly. The results of various experiments conducted on different big datasets show reasonable accuracy rates compared to state-of-the-art methods and the KNN classifier itself. More importantly, they show the high classification speed of both methods. This strong trait can be used to further improve the accuracy of the proposed methods. [ABSTRACT FROM AUTHOR]

Advanced Metering Infrastructure (AMI) realizes a two-way communication of electricity data through by interconnecting with a computer network as the core component of the smart grid. Meanwhile, it brings many new security threats and the traditional intrusion detection method can’t satisfy the security requirements of AMI. In this paper, an intrusion detection system based on Online Sequence Extreme Learning Machine (OS-ELM) is established, which is used to detecting the attack in AMI and carrying out the comparative analysis with other algorithms. Simulation results show that, compared with other intrusion detection methods, intrusion detection method based on OS-ELM is more superior in detection speed and accuracy. [ABSTRACT FROM AUTHOR]

Detecting drug-drug interactions (DDI) is important because information on DDIs can help prevent adverse effects from drug combinations. Since there are many new DDI-related papers published in the biomedical domain, manually extracting DDI information from the literature is a laborious task. However, text mining can be used to find DDIs in the biomedical literature. Among the recently developed neural networks, we use a Recursive Neural Network to improve the performance of DDI extraction. Our recursive neural network model uses a position feature, a subtree containment feature, and an ensemble method to improve the performance of DDI extraction. Compared with the state-of-the-art models, the DDI detection and type classifiers of our model performed 4.4% and 2.8% better, respectively, on the DDIExtraction Challenge’13 test data. We also validated our model on the PK DDI corpus that consists of two types of DDIs data: in vivo DDI and in vitro DDI. Compared with the existing model, our detection classifier performed 2.3% and 6.7% better on in vivo and in vitro data respectively. The results of our validation demonstrate that our model can automatically extract DDIs better than existing models. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new automatic hyperparameter selection approach for determining the optimal network configuration (network structure and hyperparameters) for deep neural networks using particle swarm optimization (PSO) in combination with a steepest gradient descent algorithm. In the proposed approach, network configurations were coded as a set of real-number m-dimensional vectors as the individuals of the PSO algorithm in the search procedure. During the search procedure, the PSO algorithm is employed to search for optimal network configurations via the particles moving in a finite search space, and the steepest gradient descent algorithm is used to train the DNN classifier with a few training epochs (to find a local optimal solution) during the population evaluation of PSO. After the optimization scheme, the steepest gradient descent algorithm is performed with more epochs and the final solutions (pbest and gbest) of the PSO algorithm to train a final ensemble model and individual DNN classifiers, respectively. The local search ability of the steepest gradient descent algorithm and the global search capabilities of the PSO algorithm are exploited to determine an optimal solution that is close to the global optimum. We constructed several experiments on hand-written characters and biological activity prediction datasets to show that the DNN classifiers trained by the network configurations expressed by the final solutions of the PSO algorithm, employed to construct an ensemble model and individual classifier, outperform the random approach in terms of the generalization performance. Therefore, the proposed approach can be regarded an alternative tool for automatic network structure and parameter selection for deep neural networks. [ABSTRACT FROM AUTHOR]

In this article, we propose a new data mining algorithm, by which one can both capture the non-linearity in data and also find the best subset model. To produce an enhanced subset of the original variables, a preferred selection method should have the potential of adding a supplementary level of regression analysis that would capture complex relationships in the data via mathematical transformation of the predictors and exploration of synergistic effects of combined variables. The method that we present here has the potential to produce an optimal subset of variables, rendering the overall process of model selection more efficient. This algorithm introduces interpretable parameters by transforming the original inputs and also a faithful fit to the data. The core objective of this paper is to introduce a new estimation technique for the classical least square regression framework. This new automatic variable transformation and model selection method could offer an optimal and stable model that minimizes the mean square error and variability, while combining all possible subset selection methodology with the inclusion variable transformations and interactions. Moreover, this method controls multicollinearity, leading to an optimal set of explanatory variables. [ABSTRACT FROM AUTHOR]

Text documents can be described by a number of abstract concepts such as semantic category, writing style, or sentiment. Machine learning (ML) models have been trained to automatically map documents to these abstract concepts, allowing to annotate very large text collections, more than could be processed by a human in a lifetime. Besides predicting the text’s category very accurately, it is also highly desirable to understand how and why the categorization process takes place. In this paper, we demonstrate that such understanding can be achieved by tracing the classification decision back to individual words using layer-wise relevance propagation (LRP), a recently developed technique for explaining predictions of complex non-linear classifiers. We train two word-based ML models, a convolutional neural network (CNN) and a bag-of-words SVM classifier, on a topic categorization task and adapt the LRP method to decompose the predictions of these models onto words. Resulting scores indicate how much individual words contribute to the overall classification decision. This enables one to distill relevant information from text documents without an explicit semantic information extraction step. We further use the word-wise relevance scores for generating novel vector-based document representations which capture semantic information. Based on these document vectors, we introduce a measure of model explanatory power and show that, although the SVM and CNN models perform similarly in terms of classification accuracy, the latter exhibits a higher level of explainability which makes it more comprehensible for humans and potentially more useful for other applications. [ABSTRACT FROM AUTHOR]

Money laundering enforcement operations in the USA and abroad have developed in the past decade from the simple use of informant information to the sophisticated analysis of voluminous, complex financial transaction arrays. Traditional investigative techniques aimed at uncovering patterns consume numerous man-hours. The volume of these records and the complexity of the relationships call for innovative techniques that can aid financial investigators in generating timely, accurate leads. Data mining techniques are well suited for identifying trends and patterns in large data sets often comprised of hundreds or even thousands of complex hidden relationships. This paper explores the use of innovative data mining methodologies that could enhance law enforcement's ability to detect, reduce, and prevent money laundering activities. This paper provides an overview of the money laundering problem in the USA and overseas and describes the nature and scope of the money laundering problems. It reviews traditional approaches to financial crime investigation and discusses various innovative data mining and artificial-intelligence-based solutions that can assist financial investigators. [ABSTRACT FROM AUTHOR]

The development of satellite image acquisition tools helped improving the extraction of information about natural scenes. In the proposed approach, we try to minimize imperfections accompanying the image interpretation process and to maximize useful information extracted from these images through the use of blind source separation (BSS) and fusion methods. In order to extract maximum information from multi-sensor images, we propose to use three algorithms of BSS that are FAST- ICA2D, JADE2D, and SOBI2D. Then by employing various fusion methods such as the probability, possibility, and evidence methods we can minimize both imprecision and uncertainty. In this paper, we propose a hybrid approach based on five main steps. The first step is to apply the three BSS algorithms to the satellites images; it results in obtaining a set of image sources representing each a facet of the land cover. A second step is to choose the image having the maximum of kurtosis and negentropy. After the BSS evaluation, we proceed to the training step using neural networks. The goal of this step is to provide learning regions which are useful for the fusion step. The next step consists in choosing the best adapted fusion method for the selected source images through a case-based reasoning (CBR) module. If the CBR module does not contain a case similar to the one we are seeking, we proceed to apply the three fusion methods. The evaluation of fusion methods is a necessary step for the learning process of our CBR. [ABSTRACT FROM AUTHOR]

Abstract: This contribution describes the pattern recognition based data analysis of an existing industrial batch dryer, and the comparison of three artificial intelligence techniques suited to perform classification tasks: neural networks trained using the Levenberg–Marquardt and the Levenberg–Marquardt method with Bayesian regularization, the neuro-fuzzy model based on clustering and grid partition, and the Takagi–Sugeno fuzzy models. The classifiers are used to quantify the dryer batch time and its variation during a certain production period, thus the motivation behind the work is genuine. The presented pattern recognition method implements a supervised learning approach and is based on pressure measurement profiles recorded by the plant data management software—the PI System from OSIsoft. It is found that the neural networks trained with the Bayesian regularization have shown the most robust classification performance with respect to separation threshold selection. Furthermore, it is concluded that the application of artificial intelligence techniques in real chemical manufacturing facilities is feasible and provides useful information for process performance monitoring purposes. The pattern recognition findings presented in this paper are not case specific and can be directly used for the monitoring of a large variety of drying processes since the pressure profile features – vacuum check, pressure decrease, vacuum break – do not depend on the chemicals which are dried. Since the development of the artificial intelligent classifiers is presented in detail and step by step, this work may be interesting as a pattern recognition tutorial for chemical engineers. [Copyright &y& Elsevier]

Decisions making process represents a point of interest in various domains as financial, insurance, communication, industry etc. Manager's activities focused on good decisions evolving for increasing their institutions profitability. Mining data with different algorithms to find significant knowledge to help decision makers is a continuous concern. In this paper an application of neural networks, as a data mining technique is presented. A decision problem in banking domain is formulated: credit approval and the solution given is neural network technique as a decision support. [ABSTRACT FROM AUTHOR]

Abstract: Software engineering development has gradually become essential element in different aspects of the daily life and an important factor in numerous critical real-industry applications, such as, nuclear plants, medical monitoring control, real-time military, bioinformatics, oil and gas industry, and air traffic control. This paper proposes a functional network as a novel computational intelligence scheme for tracking and predicting the software reliability. Several applications are presented to illustrate this new intelligent system framework models. To demonstrate the usefulness of functional networks and the existing data mining schemes, we briefly describe the learning algorithm of functional networks associativity model in predicting the software reliability. Comparative studies will be carried out to compare the performance of functional networks with the most popular existing data mining techniques, such as, statistical regression multilayer feed forward neural networks, and support vector machines. The results show that the performance of functional networks is more reliable, stable, accurate, and outperforms other techniques. [Copyright &y& Elsevier]

Abstract: The complexity of information intensive critical infrastructures, like electricity networks, telecommunication networks and public transportation networks is today augmented much more than in the past: such complexity augments the number of possible failures and anomalous working conditions and consequently decreases the survivability of the infrastructures. In this paper, the possibility is investigated to detect early anomalies and failures inside information intensive critical infrastructures by the introduction of anomaly detectors being “self-aware” about the normal working conditions of the infrastructure itself. This approach has the objective to improve the performance of the most popular signature-based algorithms for intrusion detection, and makes use of different classes of time-oriented algorithms based on artificial intelligence paradigm. It has the advantage to work also in presence of unknown and unexpected types of attacks or failures. The tests, to evaluate the performance of the utilised detectors, are executed inside an emulated supervisory control and data acquisition (SCADA) system of an electrical power transmission grid, and a proposal for the future integration inside real SCADA systems is also reported. [Copyright &y& Elsevier]

The paper presents a hybrid soft computing system for mining of complex construction databases. The proposed approach hybridizes soft computing techniques, such as fuzzy logic, artificial neural networks (ANNs), and messy genetic algorithms (mGAs), to form a novel computational method for mining of human understandable knowledge from historical databases. The hybridization combines the merits of explicit knowledge representation of fuzzy logic decision-making systems, learning abilities of ANNs, and global search of mGAs. A hybrid soft computing system (HSCS) is developed for mining complex databases in construction with three characteristics: scarcity, incompleteness, and uncertainty. Real-world construction data repositories are selected to test the capabilities of the proposed HSCS for data-mining under the above-mentioned complex conditions. The testing results show the promising potential of the proposed HSCS for mining of complex databases in construction. [ABSTRACT FROM AUTHOR]

This paper is concerned with a SOM-based data mining strategy for adaptive modelling of a slowly varying process. The aim is to follow the process in a way that makes a representative up-to-date data set of a reasonable size available at any time. The technique developed allows analysis and filtering of redundant data, detection of the need to update the process models and the core-module of the system itself and creation of process models of adaptive, data-dependent complexity. Experimental investigations performed using data from a slowly varying offset lithographic printing process have shown that the tools developed can follow the process and make the necessary adaptations of the data set and the process models. A low-process modelling error has been obtained by employing data-dependent committees for modelling the process. [Copyright &y& Elsevier]

In multilabel learning, each instance in the training set is associated with a set of labels and the task is to output a label set whose size is unknown a priori for each unseen instance. In this paper, this problem is addressed in the way that a neural network algorithm named BP-MLL, i.e., Backpropagation for Multilabel Learning, is proposed. It is derived from the popular Backpropagation algorithm through employing a novel error function capturing the characteristics of multilabel learning, i.e., the labels belonging to an instance should be ranked higher than those not belonging to that instance. Applications to two real-world multilabel learning problems, i.e., functional genomics and text categorization, show that the performance of BP-MLL is superior to that of some well-established multilabel learning algorithms. [ABSTRACT FROM AUTHOR]

In this paper, we propose the multi-learner based recursive supervised training (MLRT) algorithm, which uses the existing framework of recursive task decomposition, by training the entire dataset, picking out the best learnt patterns, and then repeating the process with the remaining patterns. Instead of having a single learner to classify all datasets during each recursion, an appropriate learner is chosen from a set of three learners, based on the subset of data being trained, thereby avoiding the time overhead associated with the genetic algorithm learner utilized in previous approaches. In this way MLRT seeks to identify the inherent characteristics of the dataset, and utilize it to train the data accurately and efficiently. We observed that empirically MLRT performs considerably well as compared with RPHP and other systems on benchmark data with 11% improvement in accuracy on the SPAM dataset and comparable performances on the VOWEL and the TWO-SPIRAL problems. In addition, for most datasets, the time taken by MLRT is considerably lower than that of the other systems with comparable accuracy. Two heuristic versions, MLRT-2 and MLRT-3, are also introduced to improve the efficiency in the system, and to make it more scalable for future updates. The performance in these versions is similar to the original MLRT system. [ABSTRACT FROM AUTHOR]

This paper studies empirically the effect of sampling and threshold-moving in training cost-sensitive neural networks. Both oversampling and undersampling are considered. These techniques modify the distribution of the training data such that the costs of the examples are conveyed explicitly by the appearances of the examples. Threshold-moving tries to move the output threshold toward inexpensive classes such that examples with higher costs become harder to be misclassified. Moreover, hard-ensemble and soft-ensemble, i.e., the combination of above techniques via hard or soft voting schemes; are also tested. Twenty-one UCI data sets with three types of cost matrices and a real-world cost-sensitive data set are used in the empirical study. The results suggest that cost-sensitive learning with multiclass tasks is more difficult than with two-class tasks, and a higher degree of class imbalance may increase the difficulty. It also reveals that almost all the techniques are effective on two-class tasks, while most are ineffective and even may cause negative effect on multiclass tasks. Overall, threshold-moving and soft-ensemble are relatively good choices in training cost-sensitive neural networks. The empirical study also suggests that some methods that have been believed to be effective in addressing the class imbalance problem may, in fact, only be effective on learning with imbalanced two-class data sets. [ABSTRACT FROM AUTHOR]

In today's competitive environment, rapid advances in economic globalization and information technology have forced many organizations to anticipate and respond to increasing volatility and competitive pressures. Numerous researches have focused on developing techniques on the analysis, design and management of whole logistics chains in particular in the domain of data mining, data analysis and data classification. While the data mining techniques and computational intelligence techniques are good independently for solving specific tasks, they can be synergized through the formation of an integrated and unified model, which can take advantage of the goods and offset the flaws of the two techniques. In this paper, an intelligent information infrastructure, which is characterized by its ability to encompass a rich collection of knowledge representation formalisms for dealing with a logistics information flow problem, is presented. Such intelligent information infrastructure includes a two-step strategy that embraces the combination of online analytical processing and neural networks to support knowledge discovery. In addition, extensible markup language is used to support the overall infrastructure, in order to facilitate the seamless data interchange within an enterprise. [ABSTRACT FROM AUTHOR]

In the past, neural networks have been viewed as classification and regression systems whose internal representations were extremely difficult to interpret. It is now becoming apparent that algorithms can be designed which extract understandable representations from trained neural networks, enabling them to be used for data mining, i.e. the discovery and explanation of previously unknown relationships present in data. This paper reviews existing algorithms for extracting comprehensible representations from neural networks and describes research to generalize and extend the capabilities of one of these algorithms. The algorithm has been generalized for application to bioinformatics datasets, including the prediction of splice site junctions in Human DNA sequences. Results generated on this datasets are compared with those generated by a conventional data mining technique (C5) and conclusions drawn. [Copyright &y& Elsevier]

This paper presents a novel approach for complex disease prediction that we have developed, exemplified by a study on risk of coronary artery disease (CAD). This multi-disciplinary approach straddles fields of microarray technology and genetics, neural networks (NN), data mining and machine learning, as well as traditional statistical analysis techniques, namely principal components analysis (PCA) and factor analysis (FA). A description of the biological background of the study is given, followed by a detailed description of how the problem has been modeled for analyses by neural networks and FA. A committee learning approach for NN has been used to improve generalization rates. We show that our NN approach is able to yield promising prediction results despite using only the most fundamental network structures. More interestingly, through the statistical analysis process, genes of similar biological functions have been clustered. In addition, a gene marker involved in breaking down lipids has been found to be the most correlated to CAD. [ABSTRACT FROM AUTHOR]