Search

The combination of Machine Learning (ML) algorithms is a solution for constructing stronger predictors than a single one. However, some approximations suggest that combining unstable algorithms provides better results than combining stable algorithms. For instance, Generative ensembles, based on re-sampling techniques, have demonstrated high performance by fusing the information of unstable base learners. Random Forest and Gradient Boosting are two well-known examples, both combining Decision Trees and providing better predictions than those obtained using a single tree. However, such successful results have not been achieved by assembling stable algorithms. This paper introduces the notion of limited learner and a new ensemble general framework called Minimally Overfitted Ensemble (MOE), a re-sampling-based ensemble approach that constructs slightly overfitted-based learners. The proposed framework works well with stable and unstable base algorithms, thanks to a Weighted RAndom Bootstrap (WRAB) sampling that provides the necessary diversity for the stable base algorithms. A hyperparameter analysis of the proposal is carried out on artificial data. Besides, its performance is evaluated on real datasets against well-known ML methods. The results confirm that the MOE framework works successfully using stable and unstable base algorithms, improving in most cases the predictive ability of single ML models and other ensemble methods. • New ensemble general framework. • A novel concept of ensemble learners is introduced. • Minimal overfitted models and limited learners are defined. • The framework works with any base machine learning training algorithm. • The proposal compares successfully to well known ensemble approaches. [ABSTRACT FROM AUTHOR]

Anomaly detection, as an important branch of machine learning, plays a critical role in fraud detection, health care, intrusion detection, military surveillance, etc. An anomaly detection algorithm based on density estimation (called ADDE algorithm) is one of the widely used algorithms. However, the ADDE algorithm is computationally expensive when processing big data sets. To solve this problem, in this paper, we propose an efficient quantum ADDE algorithm based on amplitude estimation. It is shown that our algorithm achieves exponential speedup on the number of training data points M over its classical counterpart. Besides, the idea of our algorithm can be applied to accelerate the anomaly detection algorithm based on kernel principal component analysis (called ADKPCA algorithm), which also has a wide range of applications. Our algorithm shows exponential speedup on M compared with its classical counterpart. [ABSTRACT FROM AUTHOR]

Clinical lab tests play an important role in disease diagnose and medical treatment, the test results have been utilized widely in predictive modeling tasks in healthcare. However, in most existing works, the loss function implicitly assumes that the value of the sample used to be predicted is the only correct one. This assumption fails to hold for lab test data, which usually are within respective tolerable ranges or imprecision ranges. In addition, the historical lab test data is always organized based on their sequential position, the timestamps between the data are often neglected. In this paper, we study the issue of building robust models while simultaneously taking imprecision and timestamp of the data into account with better generalization. In particular, "IR loss" is proposed in which each data in imprecision range space has a certain probability to be the real value, participating in the loss calculation. The loss is then defined as the integral of the error of each point in the impression range space. The sampling and discretization methods are proposed for loss calculation. A heuristic learning algorithm is developed to learn the model parameters. We further apply IR loss for disease progress prediction while the input data is organized as sequence. We reformulate the prediction task with timestamp based on Long Short-Term Memory (LSTM) network. At the same time, the timestamp is readily combined with the proposed IR loss to avoid the change of predicted result caused by the change of the test values in small time range. We conducted the experiments based on two real world datasets. Experimental results show that the prediction method based on IR loss can provide more accurate prediction result for different kinds of task and diverse learning methods. Our method can also provide more stable and consistent results when test samples are generated from imprecision range and small time range. [ABSTRACT FROM AUTHOR]

• 3 layer neural networks outperform other ML algorithms in automated assessments. • In subjective assignments, deep neural networks can improve score prediction. • Word embeddings and linguistic feature results in more precise text mining models. • Extraction and selection of appropriate features can improve results. Machine learning (ML) has recently gained popularity in a variety of domains for automating tasks. This has also been used in the evaluation of student responses/ assignments. This paper presents a qualitatively enhanced methodology for automated score prediction of subjective assignments. To consider all the major aspects of a human grader, 21 linguistic features related to syntactical, grammatical, sentimental, and readability are analyzed from the assignments. This study makes use of the ASAP competition dataset from Kaggle. The evaluation metric used is Quadratic Weighted Kappa (QWK), which measures the agreement between the human graded score and the predicted score. The effect of appropriate feature selection has been observed using Mutual Information Regression. Four ML algorithms are investigated on identified linguistic features. 3 Layer Neural Network with feature selection performed well among all chosen ML algorithms with average QWK of 0.678. To include the benefits of deep learning, a new hybrid model (LF-BiLSTM-att-FS) is proposed that combines a higher level deep neural network (DNN) with the selected features. Pre-trained Glove embedding is used to include contextual information of the text. The proposed model results demonstrated an improvement in overall accuracy, with an average QWK value of 0.768. [ABSTRACT FROM AUTHOR]

• A dataset is created using velocities of wheels and distance of obstacle from AMR. • The left and right turn of AMR is framed as a two class problem. • The parameters of the proposed algorithm are updated by SGD optimization technique. • The proposed algorithm provides better results with other existing algorithms. In this paper, a novel approach based on Machine Learning (ML) concept, i.e., the Adaptive Stochastic Gradient Descent Linear Regression (ASGDLR) algorithm, is developed to segregate an AMR's directional movement as right and left turn. Moreover, the developed algorithm is employed for path planning and navigational purposes. Here, real-time velocities of the right and left wheel and distance from obstacle data have been acquired by two Infrared (IR) sensors and one Ultrasonic (US) sensor positioned on the AMR. The weights of the proposed ASGDLR model are iteratively updated by applying Stochastic Gradient Descent (SGD) optimization technique by considering the difference between the actual velocity and model output velocity as an error signal. For the performance analysis of the proposed algorithm, three different performance indices, such as Mean Absolute Error (MAE), Mean Square Error (MSE), and Root Mean Square Error (RMSE), have been evaluated. The efficacy of the proposed algorithm is compared with six different AI-based regression algorithms for validation purposes. Moreover, the performance of the proposed algorithm is investigated for both obstacle avoidance and path planning in dense and cluttered environments. The simulation findings indicate that our proposed algorithm can accomplish tasks more efficiently than others while eliminating their shortcomings. [Display omitted] [ABSTRACT FROM AUTHOR]

Reduced computational time and cost, reduced skilled professional resources, and diagnostic accuracy have made medical diagnosis using computer aided systems (CAD) increasingly popular and can be comfortably employed in the diagnosis of many acute and chronic diseases in ophthalmology, cardiology, cancer detection, etc. There seems to be a growing necessity for the computational algorithms to be robust enough, to identify the abnormality in each of the cases, aiding in early diagnosis. In this paper, a glaucoma diagnostic approach based on the wrapper method employing bio-inspired algorithms, and a Kernel-Extreme Learning Machine (KELM) classifier is proposed. The bio-inspired algorithms are deployed to select feature sub-sets, generating three feature sub-sets from the pre-processed fundus images by adopting a correlation-based feature selection (CFS) approach. The selected features are utilized to train the salp-swarm optimization based KELM, which finds the optimal parameters of the KELM classifier network. The proposed methodology is evaluated on the public and private retinal fundus datasets containing 7280 images. The experimental outcome revealed that the system is able to attain a maximum overall accuracy of 99.61% with 99.89% sensitivity and 100% specificity. A 5-fold cross validation showed 98.78% accuracy ensuring a bias-free classification. Further, by experimenting on degraded images (Gaussian, salt-pepper noise images) of the original dataset, the model achieved extreme robustness with 99.3% accuracy. The proposed method is compared with other similar methods, which showed the efficiency of our method. The framework proposed can aid in making clinical decisions for various pathologies like lung infection, diabetic retinopathy, etc. • A predictive framework involving bio-inspired algorithms and Kernel-ELM is presented. • Feature Selection based on Correlation (CFS) is employed to find the optimal feature set. • KELM parameters are optimized by bio-inspired evolutionary algorithm, salp-swarm optimization algorithm (SSA) for improved diagnosis. • The proposed method works well on degraded (gaussian and salt-pepper noise) images showing the robustness of the method.. • Superior performance with statistically significant improvements on varied datasets compared to competitive algorithms. [ABSTRACT FROM AUTHOR]

Clustering ensemble is an unsupervised ensemble learning method that is very important in machine learning, since it integrates multiple weak base clustering results to produce a strong consistency result. This paper proposes the Markov clustering ensemble (MCE) model to solve the weak stability and robustness of soft clustering ensemble. First, the base clustering algorithms are regarded as new features of the original datasets. Then, the results of the base clustering algorithms are the values of these features, which can break through the framework of consensus cluster ensemble. Second, as the base clustering results are discrete data, the maximum information coefficient is applied to measure their similarity. Accordingly, a graph-based cluster ensemble model can be constructed with row vectors as vertices and the similarity between row vectors as edges. Then, the Markov process can be applied to infer the graph-based cluster ensemble model; therefore, the MCE algorithm is designed according to the inference. To test the performance of the MCE algorithm, clustering algorithms and clustering ensemble algorithms are used to conduct comparative experiments on ten datasets. The experimental results show that the MCE algorithm outperforms the other algorithms in terms of accuracy and purity. [ABSTRACT FROM AUTHOR]

In this paper, a self-adaptive classification learning hybrid JAYA and Rao-1 algorithm, which is called EHRJAYA, is proposed for solving large-scale numerical problems and real-world complex engineering optimization problems. JAYA algorithm and Rao-1 algorithm are two kinds of algorithms with simple structure and superior performance, which have the characteristics of no particular controlling parameters. In EHRJAYA the evolution strategies of the two algorithms are selected through a random selection mechanism. Then, a novel self-adaptive classification learning strategy is proposed, which fully utilizes information from different individuals. On this basis, two different adaptive coefficients are introduced to guide the population towards the optimal individual and away from the worst individual. Finally, combining the linear population reduction strategy and the dynamic lens opposition-based learning strategy, the convergence speed and ability to jump out of local optimum of the algorithm are greatly improved. To verify the performance of the proposed EHRJAYA, 59 complex functions from the CEC2014 and CEC2017 competitions are solved by EHRJAYA. Then, EHRJAYA and more than 20 algorithms with superior performance jointly solve ten challenging real-world engineering optimization problems. Experimental results show that the proposed EHRJAYA can obtain optimal results with the least computational resources in most cases. Therefore, in the face of these problems, effective solutions can be provided by EHRJAYA. [ABSTRACT FROM AUTHOR]

In this paper, we use multivariate machine learning-based predictors to replace missing data and propose a mechanism to evaluate and track correctness by estimating its confidence level whenever successive missing data points occur. The proposed solution relies on the idea of confidence attribution, which assigns a value to every measurement, indicating how much it is believed to be accurate based on the difference between measured and predicted data. When data is missing, we perform data imputation using the predicted value and estimate confidence. We estimate confidence based solely on parameters used for confidence attribution and information acquired during the predictor's training. We evaluate the solution with two real datasets, one collected from a solar farm and another from a collection of wind turbines. The results show that the accuracy of multivariate models can decrease significantly when input data goes missing, demonstrating the need for the proposed confidence tracking mechanism. • Imputing successive missing data can impact the precision of multivariate predictors. • Multiple missing variables impact the precision of multivariate predictors. • Tracking the prediction error for each imputation based on the model properties. • A mechanism to evaluate the validity of imputing data based on error thresholds. [ABSTRACT FROM AUTHOR]

Background and objective Understanding the causes of disagreement among experts in clinical decision making has been a challenge for decades. In particular, a high amount of variability exists in diagnosis of retinopathy of prematurity (ROP), which is a disease affecting low birth weight infants and a major cause of childhood blindness. A possible cause of variability, that has been mostly neglected in the literature, is related to discrepancies in the sets of important features considered by different experts. In this paper we propose a methodology which makes use of machine learning techniques to understand the underlying causes of inter-expert variability. Methods The experiments are carried out on a dataset consisting of 34 retinal images, each with diagnoses provided by 22 independent experts. Feature selection techniques are applied to discover the most important features considered by a given expert. Those features selected by each expert are then compared to the features selected by other experts by applying similarity measures. Finally, an automated diagnosis system is built in order to check if this approach can be helpful in solving the problem of understanding high inter-rater variability. Results The experimental results reveal that some features are mostly selected by the feature selection methods regardless the considered expert. Moreover, for pairs of experts with high percentage agreement among them, the feature selection algorithms also select similar features. By using the relevant selected features, the classification performance of the automatic system was improved or maintained. Conclusions The proposed methodology provides a handy framework to identify important features for experts and check whether the selected features reflect the pairwise agreements/disagreements. These findings may lead to improved diagnostic accuracy and standardization among clinicians, and pave the way for the application of this methodology to other problems which present inter-expert variability. [ABSTRACT FROM AUTHOR]

Appropriate Tidal level data is indispensable requirement for planning and safe execution of harbour projects, other coastal structures and also for optimal movement of ships. Tidal currents are also used in deciding the orientation of berthing structures and in identifying the area, dominated by eddy currents. Conventional tidal forecasting using least squares method for the prediction of long-term tidal levels requires large number of parameters for harmonic analysis. Accuracy of results depends on the length of measured samples. This paper presents an Artificial Neural Network (ANN) model for forecasting the tidal-levels using the limited measured data as an alternative to conventional harmonic analysis. ANN model developed for forecasting the tidal data is trained with different learning algorithms, number of neurons in its hidden layer and number of epochs (iterations). The aim was to develop a network, which provides a simulated tide which would be in well agreement with the actual tide. Efforts were made to forecast the time series of tidal levels based on the previous data. Study shows that the Feed-Forward Back Propagation (FFBP) network with Levenberg-Marquardt (LM) algorithm provides good correlations as compared to other algorithms. Results were compared using statistical techniques such as correlation coefficients, RMSE and Nash-Sutcliff efficiency. ANN is found to be efficient technique in forecasting the tidal data time series for a short duration. This method can be used for short term predictions of tidal time series, however model developed would remain site specific. This model was also used for predicting the tide at a far distance tidal station using the tide at the local station. It was observed that prediction of tidal levels at the far distant station is not accurately done by the neural network and shall not be used for such purposes. [ABSTRACT FROM AUTHOR]

Takagi–Sugeno–Kang (TSK) fuzzy systems have been widely applied for solving function approximation and regression-centric problems. Existing dynamic TSK models proposed in the literature can be broadly classified into two classes. Class I TSK models are essentially fuzzy systems that are limited to time-invariant environments. Class II TSK models are generally evolving systems that can learn in time-variant environments. This paper attempts to address the issues of achieving compact, up-to-date fuzzy rule bases and interpretable knowledge bases in TSK models. It proposes a novel rule pruning method which is simple, computationally efficient and biologically plausible. This rule pruning algorithm applies a gradual forgetting approach and adopts the Hebbian learning mechanism behind the long-term potentiation phenomenon in the brain. It also proposes a merging approach which is used to improve the interpretability of the knowledge bases. This approach can prevent derived fuzzy sets from expanding too many times to protect their semantic meanings. These two approaches are incorporated into a generic self-evolving Takagi–Sugeno–Kang fuzzy framework (GSETSK) which adopts an online data-driven incremental-learning-based approach. Extensive experiments were conducted to evaluate the performance of the proposed GSETSK against other established evolving TSK systems. GSETSK has also been tested on real world dataset using the high-way traffic flow density and Dow Jones index time series. The results are encouraging. GSETSK demonstrates its fast learning ability in time-variant environments. In addition, GSETSK derives an up-to-date and better interpretable fuzzy rule base while maintaining a high level of modeling accuracy at the same time. [ABSTRACT FROM AUTHOR]

In this paper we proposed reinforcement learning algorithms with the generalized reward function. In our proposed method we use Q-learning1 and SARSA1 algorithms with generalised reward function to train the reinforcement learning agent. We evaluated the performance of our proposed algorithms on Real Time Strategy (RTS) game called BattleCity. There are two main advantages of having such an approach as compared to other works in RTS. (1) We can ignore the concept of a simulator which is often game specific and is usually hard coded in any type of RTS games (2) our system can learn from interaction with any opponents and quickly change the strategy according to the opponents and do not need any human traces as used in previous works. [ABSTRACT FROM AUTHOR]

Feature selection techniques are attracting more and more attention with the growing number of domains that produce high dimensional data. Due to the absence of class labels, many researchers focus on the unsupervised scenario, attempting to find an optimal feature subset that preserves the original data distribution. However, the existing methods either fail to achieve sparsity or ignore the potential redundancy among features. In this paper, we propose a novel unsupervised feature selection algorithm, which retains the preserving power, and implements high sparsity and low redundancy in a unified manner. On the one hand, to preserve the data structure of the whole feature set, we build the graph Laplacian matrix and learn the pseudo class labels through spectral analysis. By finding a feature weight matrix, we are allowed to map the original data into a low dimensional space based on the pseudo labels. On the other hand, to ensure the sparsity and low redundancy simultaneously, we introduce a novel regularization term into the objective function with the nonnegative constraints imposed, which can be viewed as the combination of the matrix norms | | · | | m 1 and | | · | | m 2 on the weights of features. An iterative multiplicative algorithm is accordingly designed with proved convergence to efficiently solve the constrained optimization problem. Extensive experimental results on different real world data sets demonstrate the promising performance of our proposed method over the state-of-the-arts. [ABSTRACT FROM AUTHOR]

With the advent of high-dimensional stored big data and streaming data, suddenly machine learning on a very large scale has become a critical need. Such machine learning should be extremely fast, should scale up easily with volume and dimension, should be able to learn from streaming data, should automatically perform dimension reduction for high-dimensional data, and should be deployable on hardware. Neural networks are well positioned to address these challenges of large scale machine learning. In this paper, we present a method that can effectively handle large scale, high-dimensional data. It is an online method that can be used for both streaming and large volumes of stored big data. It primarily uses Kohonen nets, although only a few selected neurons (nodes) from multiple Kohonen nets are actually retained in the end; we discard all Kohonen nets after training. We use Kohonen nets both for dimensionality reduction through feature selection and for building an ensemble of classifiers using single Kohonen neurons. The method is meant to exploit massive parallelism and should be easily deployable on hardware that implements Kohonen nets. Some initial computational results are presented. [ABSTRACT FROM AUTHOR]

In this paper, a new sequential learning algorithm is constructed by combining the Online Sequential Extreme Learning Machine (OS-ELM) and Kalman filter regression. The Kalman Online Sequential Extreme Learning Machine (KOSELM) handles the problem of multicollinearity of the OS-ELM, which can generate poor predictions and unstable models. The KOSELM learns the training data one-by-one or chunk-by-chunk by adjusting the variance of the output weights through the Kalman filter. The performance of the proposed algorithm has been validated on benchmark regression datasets, and the results show that KOSELM can achieve a higher learning accuracy than OS-ELM and its related extensions. A statistical validation for the differences of the accuracy for all algorithms is performed, and the results confirm that KOSELM has better stability than ReOS-ELM, TOSELM and LS-IELM. [ABSTRACT FROM AUTHOR]

Structural balance enables a comprehensive understanding of the potential tensions and conflicts of signed networks, and its computation and transformation have attracted increasing attention in recent years. The balance computation aims at evaluating the distance from an unbalanced network to a balanced one, and the balance transformation is to convert an unbalanced network into a balanced one. In this paper, firstly, we model the balance computation of signed networks as the optimization of an energy function. Secondly, we model the balance transformation as the optimization of a more general energy function incorporated with transformation cost. Finally, a multilevel learning based memetic algorithm, which incorporates network-specific knowledge such as the neighborhoods of node, cluster and partition, is proposed to solve the modeled optimization problems. Systematical experiments in real-world social networks demonstrate the superior performance of the proposed algorithm compared with the state-of-the-art algorithms on the computation and transformation of structural balance. The results also show that our method can resolve the potential conflicts of signed networks with the minimum cost. [ABSTRACT FROM AUTHOR]

Feature weighting has been an important topic in classification learning algorithms. In this paper, we propose a new paradigm of assigning weights in classification learning, called value weighting method. While the current weighting methods assign a weight to each feature, we assign a different weight to the values of each feature. The proposed method is implemented in the context of naive Bayesian learning, and optimal weights of feature values are calculated using a gradient approach. The performance of naive Bayes learning with value weighting method is compared with that of other state-of-the-art methods for a number of datasets. The experimental results show that the value weighting method could improve the performance of naive Bayes significantly. [ABSTRACT FROM AUTHOR]

In this paper, we propose an extension of the Extreme Learning Machine algorithm for Single-hidden Layer Feedforward Neural network training that incorporates Dropout and DropConnect regularization in its optimization process. We show that both types of regularization lead to the same solution for the network output weights calculation, which is adopted by the proposed DropELM network. The proposed algorithm is able to exploit Dropout and DropConnect regularization, without computationally intensive iterative weight tuning. We show that the adoption of such a regularization approach can lead to better solutions for the network output weights. We incorporate the proposed regularization approach in several recently proposed ELM algorithms and show that their performance can be enhanced without requiring much additional computational cost. [ABSTRACT FROM AUTHOR]

In the circular restricted three-body problem (CR3BP), transit orbit is a class of orbit which can pass through the bottleneck region of the zero velocity curve and escapes from the vicinity of the primary or the secondary. This kind of orbit plays a very important role in the design of space exploration missions. A kind of low-energy interplanetary transfer, which is called Interplanetary Superhighway (IPS), can be realized by utilizing transit orbits. To use the transit orbit in actual mission design, a key issue is to find an algorithm which can separate the states corresponding to transit orbits from the states corresponding to other types of orbits rapidly. In fact, the distribution of transit orbit in the phase space has been investigated by numerical method, and a Fourier series approximation method has been introduced to describe the boundary of transit orbits. However, the Fourier series approximation method needs several hundred sets of Fourier series. The coefficients of these Fourier series are neither easy to be computed nor convenient to be stored, which makes the method can hardly be used in actual mission design. In this paper, the support vector machine (SVM) is used to classify the trajectories in the CR3BP. Using the Gaussian kernel, the 6-dimensional states in the CR3BP are mapped into an infinite-dimensional space, and the bound of the transit orbits is described by a hyperplane. A training data generation method is introduced, which reduces the size of training data by generating the states near the hyperplane. The numerical results show that the proposed algorithm gives the good correct rate of classification, and its computing speed is much faster than that of the Fourier series approximation method. [ABSTRACT FROM AUTHOR]

This paper elaborates on a problem of one-year ahead estimation of energy demand based on macroeconomic variables. To this end, two different Computational Intelligence approaches are herein evaluated: (1) a modified Harmony Search (HS) optimization algorithm with an exponential prediction model and (2) an Extreme Learning Machine (ELM). In the case of the HS, a feature selection of the best set of features for the prediction is carried out jointly with the optimization of the model’s parameters. On the other hand, the ELM will be tested with and without the feature selection carried out by the HS approach. We describe several modifications on the proposed HS, which include a hybrid encoding with a binary part for the feature selection, and a real part to tune the parameters of the prediction model. Other adaptations focused on the HS operators are also introduced. The performance of both approaches has been assessed in a real application scenario, corresponding to the total energy demand estimation in Spain, in which we have 14 macroeconomic variables with history values for the last 30 years, including the recent crisis period starting in 2008. The performance of the proposed HS and ELM models incorporating feature selection is shown to provide an accurate one-year-ahead forecast at a higher prediction’s accuracy when compared to previous proposals in the literature. Specifically, the HS and ELM approaches are able to improve the results of a previous approach (based on a genetic algorithm), obtaining an improvement over 15% in this problem of energy demand estimation. As a final experimental evaluation of the proposed algorithm, a similar problem of one-year ahead CO 2 emissions estimation from macro-economic variables is also tackled, and also in this case the HS and ELM are able to obtain significant improvements over a previous approach based on evolutionary computation, over 10% of improvement in this problem. [ABSTRACT FROM AUTHOR]

This paper describes the application of a neural model in a speed control loop of an electrical drive with an elastic mechanical coupling. Such mechanical construction makes precise speed control more difficult because of the oscillation tendency of state variables caused by a long shaft. The goal of the presented application was the replacement of a classical speed controller by an on-line trained neurocontroller, based on only one feedback from easily measurable driving motor speed. The proposed controller is based on the feedforwad neural network. Internal coefficients of neural model – weights – are adapted on-line according to the Levenberg–Marquardt algorithm. One of the problematic issues in such implementation is selection of a learning factor of the weight adaptation algorithm. In the proposed solution, a fuzzy model was implemented for calculation of this learning coefficient. The proposed solution was compared to the classical one with a PI speed controller. The designed control structure was tested in simulations and verified in experiments, using dSPACE1103 card. [ABSTRACT FROM AUTHOR]

One in every eight women is susceptible to breast cancer, at some point of time in her life. Early detection and effective treatment is the only rescue to reduce breast cancer mortality. Accurate classification of a breast cancer tumor is an important task in medical diagnosis. Machine learning techniques are gaining importance in medical diagnosis because of their classification capability. In this paper, we propose a new, Genetically Optimized Neural Network (GONN) algorithm, for solving classification problems. We evolve a neural network genetically to optimize its architecture (structure and weight) for classification. We introduce new crossover and mutation operators which differ from standard crossover and mutation operators to reduce the destructive nature of these operators. We use the GONN algorithm to classify breast cancer tumors as benign or malignant. To demonstrate our results, we had taken the WBCD database from UCI Machine Learning repository and compared the classification accuracy, sensitivity, specificity, confusion matrix, ROC curves and AUC under ROC curves of GONN with classical model and classical back propagation model. Our algorithm gives classification accuracy of 98.24%, 99.63% and 100% for 50–50, 60–40, 70–30 training–testing partition respectively and 100% for 10 fold cross validation. The results show that our approach works well with the breast cancer database and can be a good alternative to the well-known machine learning methods. [ABSTRACT FROM AUTHOR]

In the past two decades, the dimensionality of datasets involved in machine learning and data mining applications has increased explosively. Therefore, feature selection has become a necessary step to make the analysis more manageable and to extract useful knowledge about a given domain. A large variety of feature selection techniques are available in literature, and their comparative analysis is a very difficult task. So far, few studies have investigated, from a theoretical and/or experimental point of view, the degree of similarity/dissimilarity among the available techniques, namely the extent to which they tend to produce similar results within specific application contexts. This kind of similarity analysis is of crucial importance when two or more methods are combined in an ensemble fashion: indeed the ensemble paradigm is beneficial only if the involved methods are capable of giving different and complementary representations of the considered domain. This paper gives a contribution in this direction by proposing an empirical approach to evaluate the degree of consistency among the outputs of different selection algorithms in the context of high dimensional classification tasks. Leveraging on a proper similarity index, we systematically compared the feature subsets selected by eight popular selection methods, representatives of different selection approaches, and derived a similarity trend for feature subsets of increasing size. Through an extensive experimentation involving sixteen datasets from three challenging domains (Internet advertisements, text categorization and micro-array data classification), we obtained useful insight into the pattern of agreement of the considered methods. In particular, our results revealed how multivariate selection approaches systematically produce feature subsets that overlap to a small extent with those selected by the other methods. [ABSTRACT FROM AUTHOR]

Applying multi-agent reinforcement learning (MARL) in continuous distributed control system is an attractive issue, because it entitles agents adaptively to construct a cooperative behavior, even if the dynamics of such distributed system is unknown a priori. However the implementation of MARL always suffers from dimension explosion, nonstationary learning, and generalization in continuous systems. This paper presents a continuous coordinated learning algorithm with time-sharing tracking framework (CCL-TT) to deal with these problems, in which the value function is dimension reduced to lighten dimension explosion, the time-sharing tracking framework (TTF) is developed to solve nonstationary learning, and Gaussian regression modeling is applied to realize generalization. With TTF, a macroscopic concurrent learning is set up to meet the requirements of temporal stationary condition in value learning and generalization. Finally the simulation illustrates how CCL-TT realizes cooperative learning without knowledge about the dynamics of the system, even with disturbance. [ABSTRACT FROM AUTHOR]

Extreme learning machine (ELM), a simple single-hidden-layer feed-forward neural network with fast implementation, has been successfully applied in many fields. This paper proposes an ELM with a constructional structure (CS-ELM) for improving the performance of ELM in dealing with regression problems. In the CS-ELM, there are some partial input subnets (PISs). The first step in designing the PISs is to divide the data-attribute-space into several sub-spaces through using an improved extension clustering algorithm (IECA). The input data attributes in the same sub-space can build a PIS and the similar information of the data attributes is stored in the corresponding PIS. Additionally, a double parallel structure is applied in the CS-ELM, in which there is a special channel that directly connects the input layer neurons to the output layer neurons. In this regard, the proposed procedure can be called ELM with a data-attribute-space-oriented double parallel (DASODP) structure (DASODP–ELM). To test the validity of the proposed method, it is applied to 4 regression applications. The experimental results indicate that, compared with ELM, DASODP–ELM with less number of parameters can achieve higher regression precision in the generalization phase. [ABSTRACT FROM AUTHOR]

During the last decade, multi-label learning has attracted the attention of more and more researchers in machine learning field due to wide real-world applications. Existing approaches often predict an unseen example for all labels based on the same feature vector. However, this strategy might be suboptimal since different labels usually depend on different aspects of the feature vector. Furthermore, for each label there is close relationship between positive and negative instances, which is quite informative for classification. In this paper, we propose a new algorithm called ML-DFL, which trains a model for each label with newly constructed discriminative features. In order to form these features, we also propose a spectral clustering algorithm SIA to find the closely located local structures between positive and negative instances, which are assumed to be of more discriminative information, and then transform the original data set by consulting the clustering results in a simple but effective way. Comprehensive experiments are conducted on a collection of benchmark data sets. The results clearly validate the superiority of ML-DFL to various competitors. [ABSTRACT FROM AUTHOR]

SANS has warned about the new variants of SSH dictionary attacks that are very stealthy in comparison with a simple attack. In this paper, we propose a new method to detect simple and stealthy attacks by combining two key innovations. First, on the basis of our assumptions, we employ two criteria: "the existence of a connection protocol" and "the inter-arrival time of an auth-packet and the next". These criteria are not available, though, owing to the confidentiality and flexibility of the SSH protocol. Second, we resolve this problem by identifying "the transition point of each sub-protocol" through flow features and machine learning algorithms. We evaluate the effectiveness through experiments on real network traffic at the edges in campus networks. The experimental results show that our method provides high accuracy with acceptable computational complexity. [ABSTRACT FROM AUTHOR]

Image encryption and decryption are essential for securing images from various types of security attacks. In this paper, we have proposed a new algorithm for RGB image encryption and decryption using diffusion method with a combination of chaotic maps. We have formulated a new algorithm for the entire possible range to choose keys for encrypting and decrypting RGB image. Computer simulation with two standard examples and results are given to analyse the capability of the proposed approach. Several important analysis like key space, key sensitivity, NPCR, UACI, MSE and PSNR analysis are performed. Results of various analysis and computer simulation confirm that the new algorithm offers high security and is suitable for practical image encryption. [ABSTRACT FROM AUTHOR]

IDR/QR, which is an incremental dimension reduction algorithm based on linear discriminant analysis (LDA) and QR decomposition, has been successfully employed for feature extraction and incremental learning. IDR/QR can update the discriminant vectors with light computation when new training samples are inserted into the training data set. However, IDR/QR has two limitations: 1) IDR/QR can only process new samples one instance after another even if a chunk of training samples is available at a time; and 2) the approximate trick is used in IDR/QR. Then there exists a gap in performance between incremental and batch IDR/QR solutions. To address the problems of IDR/QR, in this paper, we propose a new chunk IDR method which is capable of processing multiple data instances at a time and can accurately update the discriminant vectors when new data items are added dynamically. Experiments on some real databases demonstrate the effectiveness of the proposed algorithm over the original one. [ABSTRACT FROM AUTHOR]

Classically, unsupervised machine learning techniques are applied on data sets with fixed number of attributes (variables). However, many problems encountered in the field of informetrics face us with the need to extend these kinds of methods in a way such that they may be computed over a set of nonincreasingly ordered vectors of unequal lengths. Thus, in this paper, some new dissimilarity measures (metrics) are introduced and studied. Owing to that we may use, e.g. hierarchical clustering algorithms in order to determine an input data set's partition consisting of sets of producers that are homogeneous not only with respect to the quality of information resources, but also their quantity. [ABSTRACT FROM AUTHOR]

Bayesian Ying–Yang (BYY) harmony learning system is a powerful tool for statistical learning. Via the BYY harmony leaning of finite mixtures, model selection, i.e., the selection of an appropriate number of components for the mixture, can be made automatically during parameter learning on a given dataset. In this paper, an adaptive gradient BYY harmony learning algorithm is proposed for log-normal mixtures to implement parameter learning with automated model selection. It is demonstrated by the experiments on both synthetic and real-world datasets that the proposed BYY harmony learning algorithm not only has the ability of automated model selection, but also leads to a rather good estimation of the parameters in the original log-normal mixture. [ABSTRACT FROM AUTHOR]

Cancer is curable if it can be detected early. One way to detect cancer is by analyzing the change in expression of genes in the suspected tissue. Serial analysis of gene expression (SAGE) is a sequencing technique used for measuring the expression levels of genes. Cancer detection problem can be posed as binary classification problem like whether a tissue is cancerous or normal. SAGE libraries contain expression levels of thousands of genes which are the features. It is impossible to consider all these features for classification and also the general feature selection algorithms are not efficient with this data. In this paper, closed frequent itemset mining is proposed as a feature selection technique for identifying a small set of features which can distinguish the two classes efficiently. The performance of the proposed technique is evaluated on SAGE data related to breast tissue and a group of 26 genes are selected as best features. Two well known classifiers, extreme learning machine (ELM) and support vector machine (SVM), are used to evaluate the effectiveness of the selected features in classification and found that the proposed method works well with these classifiers. [ABSTRACT FROM AUTHOR]

Community structure is a common and important property of complex networks. The detection of communities has great significance for understanding the function and organization of networks. Generally, community detection can be formulated as a modularity optimization problem. However, traditional modularity optimization based algorithms have the resolution limit that they may fail to find communities which are smaller than a certain size. In this paper, we propose a cellular learning automata based algorithm for detecting communities in complex networks. Our algorithm models the whole network as an irregular cellular learning automata (ICLA) and reveals the optimal community structure through the evolution of the cellular learning automata. By interacting with both the global and local environments, our algorithm effectively solves the resolution limit problem of modularity optimization. The experiments on both synthetic and real-world networks demonstrate that our algorithm is effective and efficient at detecting community structure in complex networks. [ABSTRACT FROM AUTHOR]

A new algorithm for ensemble construction based on adapted restricting a set of weights of examples in training data to avoid overfitting and to reduce a number of iterations is proposed in the paper. The algorithm called IDMBoost (Imprecise Dirichlet Model Boost) applies Walley׳s imprecise Dirichlet model for modifying the restricted sets of weights depending on the number and location of classification errors. Updating of weights within the restricted set (simplex) is carried out by using its extreme points. The proposed algorithm has a double adaptation procedure. The first adaptation is carried out within every restricted simplex like the AdaBoost. The second adaptation reduces and changes the restricted sets of possible weights of examples. Various numerical experiments with real data illustrate the proposed algorithm. [ABSTRACT FROM AUTHOR]

This paper proposes a variant of the bacterial foraging optimization (BFO) algorithm with time-varying chemotaxis step length and comprehensive learning strategy which we call adaptive comprehensive learning bacterial foraging optimization (ALCBFO). An adaptive non-linearly decreasing modulation model is used to keep a well balance between the exploration and exploitation of the proposed algorithm. The comprehensive learning mechanism maintains the diversity of the bacterial population and thus alleviates the premature convergence. Compared with the classical GA, PSO, the original BFO and two improved BFO (BFO-LDC and BFO-NDC) algorithm, the proposed ACLBFO shows significantly better performance in solving multimodal problems. We also assess the performance of the ACLBFO method on vehicle routing problem with time windows (VRPTW). Compared with three other BFO algorithms, the proposed algorithm is superior and confirms its potential to solve vehicle routing problem with time windows (VRPTW). [ABSTRACT FROM AUTHOR]

There has been some improvement in the treatment of preterm infants, which has helped to increase their chance of survival. However, the rate of premature births is still globally increasing. As a result, this group of infants is most at risk of developing severe medical conditions that can affect the respiratory, gastrointestinal, immune, central nervous, auditory and visual systems. There is a strong body of evidence emerging that suggests the analysis of uterine electrical signals, from the abdominal surface (Electrohysterography – EHG), could provide a viable way of diagnosing true labour and even predict preterm deliveries. This paper explores this idea further and presents a new dynamic self-organized network immune algorithm that classifies term and preterm records, using an open dataset containing 300 records (38 preterm and 262 term ). Using the dataset, oversampling and cross validation techniques are evaluated against other similar studies. The proposed approach shows an improvement on existing studies with 89% sensitivity , 91% specificity , 90% positive predicted value , 90% negative predicted value , and an overall accuracy of 90%. [ABSTRACT FROM AUTHOR]

This paper proposes a framework for binary classification of the time series with cyclic characteristics. The framework presents an iterative algorithm for learning the cyclic characteristics by introducing the discriminative frequency bands (DFBs) using the discriminant analysis along with k-means clustering method. The DFBs are employed by a hybrid model for learning dynamic characteristics of the time series within the cycles, using statistical and structural machine learning techniques. The framework offers a systematic procedure for finding the optimal design parameters associated with the hybrid model. The proposed model is optimized to detect the changes of the heart sound recordings (HSRs) related to aortic stenosis. Experimental results show that the proposed framework provides efficient tools for the classification of the HSRs based on the heart murmurs. It is also evidenced that the hybrid model, proposed by the framework, substantially improves the classification performance when it comes to detection of the heart disease. [ABSTRACT FROM AUTHOR]

Motifs are short recurring patterns that are of much interest as they help us understand the mechanism behind regulating gene expression. This paper presents an efficient deterministic algorithm that exhaustively discovers all identical string motifs of all sizes that appears in all or most of the input strings. The input is a set of n different strings each of length N . The problem is to discover all exact instances of any size occurring at least θ times per string in q or more input strings. The quorums θ and q are user set positive integers. The algorithm starts with substrings of size 2, and keeps augmenting them one symbol a time. We encode the substrings, such that it is not possible to have two different substrings with the same encoding. After the sorting of the encoded substring, we can tell where, and which of the substrings have multiple copies. Next, we eliminate all the substrings that appear less than the quorums, and repeat the process. Theoretically the algorithm is competitive and scales extremely well with the size of the alphabet. For strings over the alphabet Σ , it is O ( nN + | Σ |) in space, and O ( nNL − nL 2 ) in time, where L is the length of the longest identical string motif satisfying the constraint. For experimental purposes we applied the algorithm on real biological sequences (DNA and protein), and on randomly generated sequences using alphabet of sizes up to 1000 symbols. These experiments confirm that in practice the algorithm has indeed a linear complexity, both in space and time, making it practical for the important and difficult case of large-alphabet inputs. [ABSTRACT FROM AUTHOR]

The highlight of this paper is to demonstrate the concept of adaptive tests which are an efficient way to produce the desired result as compared to the traditional static tests. The algorithm learns to adapt to the user's knowledge, and thus, calculates the true ability of the user. Our test system also provides the user with his performance review in specific categories, thereby giving him an understanding of his current knowledge for the same. Thus, our model can improve the user‟s efficacy over a prolonged period of time. [ABSTRACT FROM AUTHOR]

In this paper, we propose support vector-based supervised learning algorithms, called multiclass support vector data description with weighted dynamic time warping kernel function (MSVDD-WDTWK) and multiclass support vector machines with weighted dynamic time warping kernel function (MSVM-WDTWK), which provides a flexible and robust kernel function for time series classification between non-aligned time series data resulting in improved accuracy. The proposed WDTW kernel function provides an optimal match between two time series data by not only allowing a non-linear mapping between two data sequences, but also considering relative significance depending on the phase difference between points on time series data. We validate the proposed approaches using extensive numerical experiments on a number of multiclass UCR time series data mining archive, and demonstrate that our proposed methods provide lower classification error rates compared with existing techniques. [ABSTRACT FROM AUTHOR]

Classification is a long standing problem in computing. There are two broad kinds of classifiers, frequency based and geometry based. Frequency based classifiers often ignore the geometry underlying the data. Conversely, geometry based classifiers take into account the frequency only indirectly. This paper presents a classification algorithm which considers explicitly geometric and statistical characteristics of the data and combines them into a class representation. Reported here are initial experiments with this algorithm using two well known data sets, both with and without noise. The results show that the proposed algorithm is less sensitive to the training data set than other classifiers. [ABSTRACT FROM AUTHOR]

The prediction of future drought is an effective mitigation tool for assessing adverse consequences of drought events on vital water resources, agriculture, ecosystems and hydrology. Data-driven model predictions using machine learning algorithms are promising tenets for these purposes as they require less developmental time, minimal inputs and are relatively less complex than the dynamic or physical model. This paper authenticates a computationally simple, fast and efficient non-linear algorithm known as extreme learning machine (ELM) for the prediction of Effective Drought Index (EDI) in eastern Australia using input data trained from 1957–2008 and the monthly EDI predicted over the period 2009–2011. The predictive variables for the ELM model were the rainfall and mean, minimum and maximum air temperatures, supplemented by the large-scale climate mode indices of interest as regression covariates, namely the Southern Oscillation Index, Pacific Decadal Oscillation, Southern Annular Mode and the Indian Ocean Dipole moment. To demonstrate the effectiveness of the proposed data-driven model a performance comparison in terms of the prediction capabilities and learning speeds was conducted between the proposed ELM algorithm and the conventional artificial neural network (ANN) algorithm trained with Levenberg–Marquardt back propagation. The prediction metrics certified an excellent performance of the ELM over the ANN model for the overall test sites, thus yielding Mean Absolute Errors, Root-Mean Square Errors, Coefficients of Determination and Willmott's Indices of Agreement of 0.277, 0.008, 0.892 and 0.93 (for ELM) and 0.602, 0.172, 0.578 and 0.92 (for ANN) models. Moreover, the ELM model was executed with learning speed 32 times faster and training speed 6.1 times faster than the ANN model. An improvement in the prediction capability of the drought duration and severity by the ELM model was achieved. Based on these results we aver that out of the two machine learning algorithms tested, the ELM was the more expeditious tool for prediction of drought and its related properties. [ABSTRACT FROM AUTHOR]

In this paper we describe a method for registration of 3D point clouds that represent objects of interest. A stereovision system is used to capture point clouds of a static environment, robot arm and an unknown object. By moving the robot arm in the environment the proposed system defines known occupied zones and is able to identify the robot arm. In order to identify a complete point cloud presentation of the robot gripper it is rotated in front of a stereovision camera and its geometry is captured from different angles. Iterative closest point algorithm is used to determine a rigid transformation between every new robot pose so the original point cloud can be appended with the transformed one. When the robot is holding a new object the registration procedure is repeated and known elements (environment, robot arm and gripper) are removed so that the object can be identified. [ABSTRACT FROM AUTHOR]

Nowadays, many real applications comprise data-sets where the distribution of the classes is significantly different. These data-sets are commonly known as imbalanced data-sets. Traditional classifiers are not able to deal with these kinds of data-sets because they tend to classify only majority classes, obtaining poor results for minority classes. The approaches that have been proposed to address this problem can be categorized into three types: resampling methods, algorithmic adaptations and cost sensitive techniques. Radial Basis Function Networks (RBFNs), artificial neural networks composed of local models or RBFs, have demonstrated their efficiency in different machine learning areas. Centers, widths and output weights for the RBFs must be determined when designing RBFNs. Taking into account the locally tuned response of RBFs, the objective of this paper is to study the influence of global and local paradigms on the weights training phase, within the RBFNs design methodology, for imbalanced data-sets. Least Mean Square and the Singular Value Decomposition have been chosen as representatives of local and global weights training paradigms respectively. These learning algorithms are inserted into classical RBFN design methods that are run on imbalanced data-sets and also on these data-sets preprocessed with re-balance techniques. After applying statistical tests to the results obtained, some guidelines about the RBFN design methodology for imbalanced data-sets are provided. [ABSTRACT FROM AUTHOR]

We present FI 2 DS a file system, host based anomaly detection system that monitors Basic Security Module (BSM) audit records and determines whether a web server has been compromised by comparing monitored activity generated from the web server to a normal usage profile. Additionally, we propose a set of features extracted from file system specific BSM audit records, as well as an IDS that identifies attacks based on a decision engine that employs one-class classification using a moving window on incoming data. We have used two different machine learning algorithms, Support Vector Machines (SVMs) and Gaussian Mixture Models (GMMs) and our evaluation is performed on real-world datasets collected from three web servers and a honeynet. Results are very promising, since FI 2 DS detection rates range between 91% and 95.9% with corresponding false positive rates ranging between 8.1× 10 −2 % and 9.3× 10 −4 %. Comparison of FI 2 DS to another state-of-the-art filesystem-based IDS, FWRAP, indicates higher effectiveness of the proposed IDS in all three datasets. Within the context of this paper FI 2 DS is evaluated for the web daemon user; nevertheless, it can be directly extended to model any daemon-user for both intrusion detection and postmortem analysis. [ABSTRACT FROM AUTHOR]

Inspired by the matrix-based methods used in feature extraction and selection, one matrix-pattern-oriented classification framework has been designed in our previous work and demonstrated to utilize one matrix pattern itself more effectively to improve the classification performance in practice. However, this matrix-based framework neglects the prior structural information of the whole input space that is made up of all the matrix patterns. This paper aims to overcome such flaw through taking advantage of one structure learning method named Alternative Robust Local Embedding (ARLE). As a result, a new regularization term R gl is designed, expected to simultaneously represent the globality and the locality of the whole data domain, further boosting the existing matrix-based classification method. To our knowledge, it is the first trial to introduce both the globality and the locality of the whole data space into the matrixized classifier design. In order to validate the proposed approach, the designed R gl is applied into the previous work matrix-pattern-oriented Ho-Kashyap classifier (MatMHKS) to construct a new globalized and localized MatMHKS named GLMatMHKS. The experimental results on a broad range of data validate that GLMatMHKS not only inherits the advantages of the matrixized learning, but also uses the prior structural information more reasonably to guide the classification machine design. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel machine learning framework called “Online Transfer Learning” (OTL), which aims to attack an online learning task on a target domain by transferring knowledge from some source domain. We do not assume data in the target domain follows the same distribution as that in the source domain, and the motivation of our work is to enhance a supervised online learning task on a target domain by exploiting the existing knowledge that had been learnt from training data in source domains. OTL is in general very challenging since data in both source and target domains not only can be different in their class distributions, but also can be diverse in their feature representations. As a first attempt to this new research problem, we investigate two different settings of OTL: (i) OTL on homogeneous domains of common feature space, and (ii) OTL across heterogeneous domains of different feature spaces. For each setting, we propose effective OTL algorithms to solve online classification tasks, and show some theoretical bounds of the algorithms. In addition, we also apply the OTL technique to attack the challenging online learning tasks with concept-drifting data streams. Finally, we conduct extensive empirical studies on a comprehensive testbed, in which encouraging results validate the efficacy of our techniques. [ABSTRACT FROM AUTHOR]