Search

In this paper, we study a discrete momentum consensus-based optimization (Momentum-CBO) algorithm which corresponds to a second-order generalization of the discrete first-order CBO [S.-Y. Ha, S. Jin and D. Kim, Convergence of a first-order consensus-based global optimization algorithm, Math. Models Methods Appl. Sci. 30 (2020) 2417–2444]. The proposed algorithm can be understood as the modification of ADAM-CBO, replacing the normalization term by unity. For the proposed Momentum-CBO, we provide a sufficient framework which guarantees the convergence of algorithm toward a global minimum of the objective function. Moreover, we present several experimental results showing that Momentum-CBO has an improved success rate of finding the global minimum compared to vanilla-CBO and show the stability of Momentum-CBO under different initialization schemes. We also show that Momentum-CBO can be used as the alternative of ADAM-CBO which does not have a proper convergence analysis. Finally, we give an application of Momentum-CBO for Lyapunov function approximation using symbolic regression techniques. [ABSTRACT FROM AUTHOR]

The clinical diagnosis of heart disease in most situations is based on a difficult amalgamation of pathological and clinical information. Because of this complication, there is a significant level of curiosity among many diagnostic healthcare professionals and researchers who are keenly interested in the efficient, accurate, and early-stage forecasting of heart disease. Deep Learning Algorithms aid in the prediction of heart disease. The main focus of this paper is to develop a method for predicting heart disease through Modified Rough K means + + (MRK + +) clustering along with the Restricted Boltzmann Machine (RBM). This paper is categorized into two modules: (1) Propose a clustering component based on Modified Rough K-means + + ; (2) disease prediction based on RBM. The input Cleveland dataset is clustered using the stochastic probabilistic rough k-means + + clustering technique in the module for clustering. The clustered data is acquired and used in the RBM, and this hybrid structure is then used in the heart disease forecasting module. Throughout the testing procedure, the most valid result is chosen from the clustered test data, and the RBM classifier that correlates to the nearest cluster in the test data is based on the smallest distance or similar parameters. Furthermore, the output value is used to predict heart disease. There are three different types of experiments that are performed: In the first experiment comprises modifying the rough K-means + + clustering algorithm, the second experiment evaluates the classification result, and the third experiment suggests hybrid model representation. When the Hybrid Modified Rough k-means + + - RBM model is compared with any single model, it provides the highest accuracy. [ABSTRACT FROM AUTHOR]

Tremendous efficient optimization methods have been proposed for strongly convex objectives optimization in modern machine learning. For non-strongly convex objectives, a popular approach is to apply a reduction from non-strongly convex to a strongly convex case via regularization techniques. Reduction on objectives with adaptive decrease on regularization tightens the optimal convergence of algorithms to be independent on logarithm factor. However, the initialization of parameter of regularization has a great impact on the performance of the reduction. In this paper, we propose an aggressive reduction to reduce the complexity of optimization for non-strongly convex objectives, and our reduction eliminates the impact of the initialization of parameter on the convergent performances of algorithms. Aggressive reduction not only adaptively decreases the regularization parameter, but also modifies regularization term as the distance between current point and the approximate minimizer. Our aggressive reduction can also shave off the non-optimal logarithm term theoretically, and make the convergent performance of algorithm more compact practically. Experimental results on logistic regression and image deblurring confirm this success in practice. [ABSTRACT FROM AUTHOR]

Human activity recognition (HAR) has a wide application in daily life. With wearable sensors, people's activity can be monitored, recorded and analysed. However, most existing methods did not make full use of human activity data and their features. In this paper, a new method based on feature selection and clustering algorithm is proposed. We established two-layer classification models, respectively, in the ankle, the chest, the wrist and the mixed accelerometer data. K-means clustering algorithm was first used to obtain a broad classification of the activities and then we conducted two rounds of classification, among which feature selection was performed in each layer. A significance analysis was also carried out in the final experiment and we compared the performance of the final model from the mixed accelerometer data with other algorithms, the results showed that the recognition performance of our model was significantly better, and the average F 1 score was as high as 0.969 in publicly available PAMAP2 dataset. Compared with other methods, our model achieved the highest recognition rate. The method proposed in this paper can greatly improve the recognition rate of human activity and effectively evaluate daily activity. [ABSTRACT FROM AUTHOR]

In this paper, we discuss applications of neural networks to recognizing knots and, in particular, to the unknotting problem. One of the motivations for this study is to understand how neural networks work on the example of a problem for which rigorous mathematical algorithms for its solution are known. We represent knots by rectangular Dynnikov diagrams and apply neural networks to distinguish a given diagram's class from a finite family of topological types. The data presented to the program is generated by applying Dynnikov moves to initial samples. The significance of using these diagrams and moves is that in this context the problem of determining whether a diagram is unknotted is a finite search of a bounded combinatorial space. In this way, this paper provides a foundation for further work where the neural network itself will learn to use the Dynnikov moves for knot recognition. Source code of the programs is available at https://github.com/nerusskikh/deepknots. [ABSTRACT FROM AUTHOR]

GPS receivers have a wide range of applications, but are not always secure. A spoofing attack is one source of conscious errors in which the counterfeit signal overcomes the authentic GPS signal and takes control of the receiver's operation. Recently, GPS spoofing attack detection based on computational algorithms, such as machine learning, classification, wavelet transform and clustering, has been developing. This paper proposes multiple clustering algorithms for accurately clustering the authentic and spoofing signals, called subtractive, FCM and DBSCAN clustering. The spoofing attack is recognized using two distinct features: moving phase detector variance and norms of correlators. Spoofing and authentic signals have different patterns in the proposed features. According to the Dunn and Silhouette indexes, the validation of the results is investigated. The Dunn values for the proposed approaches are 0.8592, 0.5285 and 0.6039 for DBSCAN, FCM and subtractive clustering, respectively. Also, the DBSCAN algorithm is implemented at the RTL level because of its highest value for the Dunn index and algorithm verifiability. Using the Vivado tools, this algorithm is implemented and designed on a Xilinx Virtex 7 xc7vx690tffg1930-3 hardware device for two-dimensional data with 32-bit accuracy and 130 data points. [ABSTRACT FROM AUTHOR]

In the last years, the number of machine learning algorithms and their parameters has increased significantly. On the one hand, this increases the chances of finding better models. On the other hand, it increases the complexity of the task of training a model, as the search space expands significantly. As the size of datasets also grows, traditional approaches based on extensive search start to become prohibitively expensive in terms of computational resources and time, especially in data streaming scenarios. This paper describes an approach based on meta-learning that tackles two main challenges. The first is to predict key performance indicators of machine learning models. The second is to recommend the best algorithm/configuration for training a model for a given machine learning problem. When compared to a state-of-the-art method (AutoML), the proposed approach is up to 130x faster and only 4% worse in terms of average model quality. Hence, it is especially suited for scenarios in which models need to be updated regularly, such as in streaming scenarios with big data, in which some accuracy can be traded for a much shorter model training time. [ABSTRACT FROM AUTHOR]

In this paper, we study kernel ridge-less regression, including the case of interpolating solutions. We prove that maximizing the leave-one-out ( CV loo ) stability minimizes the expected error. Further, we also prove that the minimum norm solution — to which gradient algorithms are known to converge — is the most stable solution. More precisely, we show that the minimum norm interpolating solution minimizes a bound on CV loo stability, which in turn is controlled by the smallest singular value, hence the condition number, of the empirical kernel matrix. These quantities can be characterized in the asymptotic regime where both the dimension (d) and cardinality (n) of the data go to infinity (with n d → γ as d , n → ∞). Our results suggest that the property of CV loo stability of the learning algorithm with respect to perturbations of the training set may provide a more general framework than the classical theory of Empirical Risk Minimization (ERM). While ERM was developed to deal with the classical regime in which the architecture of the learning network is fixed and n → ∞ , the modern regime focuses on interpolating regressors and overparameterized models, when both d and n go to infinity. Since the stability framework is known to be equivalent to the classical theory in the classical regime, our results here suggest that it may be interesting to extend it beyond kernel regression to other overparameterized algorithms such as deep networks. [ABSTRACT FROM AUTHOR]

With the limitation of air traffic and the rapid increase in the number of flights, flight delay is becoming more frequent. Flight delay leads to financial and time losses for passengers and increases operating costs for airlines. Therefore, the establishment of an accurate prediction model for flight delay becomes vital to build an efficient airline transportation system. The air transportation system has a huge amount of data and complex operation modes, which is suitable for analysis by using machine learning methods. This paper discusses the factors that may affect the flight delay, and presents a new flight delay prediction model. The five warning levels are defined based on flight delay database by using K-means clustering algorithm. After extracting the key factors related to flight operation by the grey relational analysis (GRA) algorithm, an improved machine learning algorithm called GRA — Genetic algorithm (GA) — back propagation neural network, GRA-GA-BP, is introduced, which is optimized by GA. The calculation results show that, compared with models before optimization and other two algorithms in previous papers, the proposed prediction model based on GRA-GA-BP algorithm shows a higher prediction accuracy and more stability. In terms of operation efficiency and memory consumption, it also has good performance. The analysis presented in this paper indicates that this model can provide effective early warnings for flight delay, and can help airlines to intervene in flights with abnormal trend in advance. [ABSTRACT FROM AUTHOR]

Metric learning is a critical problem in classification. Most classifiers are based on a metric, the simplest one is the KNN classifier, whose outcome is directly decided by the given metric. This paper will discuss semi-supervised metric learning. Most traditional semi-supervised metric learning algorithms preserve the local structure of all the samples (including labeled and unlabeled) in the input space, when making the same labeled samples together and separating different labeled samples. In most existing methods, the local structure is calculated by the Euclidean distance which uses all the features. As we all know, high dimensional data lies on a low dimension manifold, and not all the features are discriminative. Thus, in this paper, we try to explore the latent structure of the samples and use the more discriminative features to calculate the local structure. The latent structure is learned by clustering random forest and cast into similarity between samples. Based on the hierarchical structure of the trees and the split function, the similarity is obtained from discriminant features. Experimental results on public data sets show our algorithm outperforms the traditional similar related algorithms. [ABSTRACT FROM AUTHOR]

To solve the problem of low accuracy in the traditional FunkSVD recommendation algorithm, an improved FunkSVD algorithm (RM-FS) is proposed. RM-FS is an improvement of the traditional FunkSVD algorithm, using RMSProp, a deep learning optimization algorithm. The RM-FS algorithm can not only solve the problem of reduced accuracy of the traditional FunkSVD algorithm because of iterative oscillations but also alleviate the impact of data sparseness on the accuracy of the algorithm, achieving the effect of improving the accuracy of the traditional algorithm. The experimental results show that the RM-FS algorithm proposed in this paper effectively improves the accuracy of the recommendation algorithm, which is better than the traditional FunkSVD recommendation algorithm and other improved FunkSVD algorithms. [ABSTRACT FROM AUTHOR]

Data mining has become a relatively modern platform for information retrieval. The efficient data mining techniques can increase the reliability and accuracy of internal auditing for the various community even while lowering audit risk. Existing audit data mining approaches lack significant identification of hidden connections and interactions in bid data platforms. Hence, this study extends the literature survey on the signification of audit data mining in multiple applications. This survey identifies the scope of improved association algorithms in audit data mining, a rule-based machine learning approach to determine the exciting relationship among variables in large audit datasets. Therefore, a Conceptual Framework of Improved Association Algorithm (CFiAA) and its application in audit data mining is proposed. This study examines the strengths and weaknesses of the proposed CFiAA in audit mining. The proposed model has been trained using an audit data set and validates with various audit datasets. Finally, this paper presents the comparative analysis of the proposal to show its highest performance related to existing models. Thus, CFiAA scores the performance ratio of 94.5%, accuracy ratio of 92.4%, an efficiency ratio of 92.5%, F1 measure of 91.8%, error rate 32.5%, prediction ratio of 93.7%, and the precision ratio of 92.5% compared to existing models. [ABSTRACT FROM AUTHOR]

There are various applications of clustering in the fields of machine learning, data mining, data compression along with pattern recognition. The existent techniques like the Llyods algorithm (sometimes called k-means) were affected by the issue of the algorithm which converges to a local optimum along with no approximation guarantee. For overcoming these shortcomings, an efficient k-means clustering approach is offered by this paper for stream data mining. Coreset is a popular and fundamental concept for k-means clustering in stream data. In each step, reduction determines a coreset of inputs, and represents the error, where P represents number of input points according to nested property of coreset. Hence, a bit reduction in error of final coreset gets n times more accurate. Therefore, this motivated the author to propose a new coreset-reduction algorithm. The proposed algorithm executed on the Covertype dataset, Spambase dataset, Census 1990 dataset, Bigcross dataset, and Tower dataset. Our algorithm outperforms with competitive algorithms like Streamkm++, BICO (BIRCH meets Coresets for k-means clustering), and BIRCH (Balance Iterative Reducing and Clustering using Hierarchies. [ABSTRACT FROM AUTHOR]

Generally, the blades of wind turbine installed in cold regions often encounter the situation of icing on their surface in winter, which affects the performance of wind turbine greatly and reduces power generation. Blade icing of wind turbine is a typical alarm related to multivariates, such as environment temperature, humidity, blade rotation and so on. To successfully achieve a supervised classification, it is necessary to extract the correlated features with blade icing. This paper proposes an extraction algorithm using reward extremum which can extract a group of variables correlated to blade icing from many monitoring variables of wind turbine, and they affect each other. This algorithm, using reward function which can measure the reward or value of alarm problem, can effectively distinguish the variables correlated to blade icing from the uncorrelated ones without knowing the mean of variables and the relations between them. Whether the blade of wind turbine is iced or not is a binary logic, so logistic regression algorithm is better choice to detect blade icing. Hence, this paper proposes a hybrid model using logistic regression algorithm and extraction algorithm using reward extremum, which can be built by machine learning with historic monitoring data. This model can real-time detect blade icing by real-time monitoring data. A correlation variable set can be obtained by the application of extraction algorithm using reward extremum on the historic monitoring data; and then the weights of the correlation variables in the set can be gained from the historic data by using logistic regression algorithm, so a blade icing detection pattern can be found using logistic regression algorithm; finally, whether the blade of wind turbine is iced in delay time or not can be real-time detected with the help of the hybrid model by real-time monitoring data. It is proved by wind turbine data that this hybrid model can work well not only to get correlation variable set but also to improve the generalization performance of logistic regression and achieve better prediction results. The performance of the model on blade icing alarm of wind turbine provides a new way to find out multivariate correlation to alarm from the mass historic monitoring data. [ABSTRACT FROM AUTHOR]

With the continuous improvement of science and technology, the substation remote control system has been constantly improved, which provides the possibility for the complete realization of intelligent and unmanned substation. However, due to the special substation environment, it is easy to cause interference, coupled with the low accuracy of today's video image processing algorithm, which leads to the frequent occurrence of false alarms and missing alarms. Manual intervention is needed to deal with this, which inhibits the display of automatic intelligent substation processing functions. Therefore, in this paper, the most rapidly developed machine learning algorithm — deep learning is applied to the substation instrument equipment identification processing, in order to improve the accuracy and efficiency of instrument equipment identification, and make due contributions to the full realization of unattended substation. [ABSTRACT FROM AUTHOR]

This paper describes a novel call recognizer system based on the machine learning approach. Current trends, intelligence, emotional recognition and other factors are important challenges in the real world. The proposed system provides robustness with high accuracy and adequate response time for the human–computer interaction. Intelligence and emotion recognition from the speech of human–computer interfaces are simulated via multiple classifier systems (MCSs). At a higher-level stage, the acoustic stream phase extracts certain acoustic features based on the pitch and energy of the signal. Here, the feature space is labeled with various emotional types in the training phase. Emotional categories are trained in the acoustic feature space. The semantic stream process converts speech into text in the input speech signal. Text classification algorithms are applied subsequently. The clustering and classification process is performed via a K -means algorithm. The detection of the Tone of Voice of call recognition system is achieved with the XGBoost model for feature extraction and detection of a particular phrase in the client call phase. Speech expressions are used for understanding the human emotion. The algorithms are tested and demonstrate good performance in the simulation environment. [ABSTRACT FROM AUTHOR]

Transactions in web data are huge amounts of data, often consisting of fuzzy and quantitative values. Mining fuzzy association rules can help discover interesting relationships between web data. The quality of these rules depends on membership functions, and thus, it is essential to find the suitable number and position of membership functions. The time spent by users on each web page, which shows their level of interest in those web pages, can be considered as a trapezoidal membership function (TMF). In this paper, the optimization problem was finding the appropriate number and position of TMFs for each web page. To solve this optimization problem, a learning automata-based algorithm was proposed to optimize the number and position of TMFs (LA-ONPTMF). Experiments conducted on two real datasets confirmed that the proposed algorithm enhances the efficiency of mining fuzzy association rules by extracting the optimized TMFs. [ABSTRACT FROM AUTHOR]

Due to the inability of the accuracy-driven methods to address the challenging problem of learning from imbalanced data, several alternative measures have been proposed in the literature, like the Area Under the ROC Curve (AUC), the Average Precision (AP), the F-measure, the G-Mean, etc. However, these latter measures are neither smooth, convex nor separable, making their direct optimization hard in practice. In this paper, we tackle the challenging problem of imbalanced learning from a nearest-neighbor (NN) classification perspective, where the minority examples typically belong to the class of interest. Based on simple geometrical ideas, we introduce an algorithm that rescales the distance between a query sample and any positive training example. This leads to a modification of the Voronoi regions and thus of the decision boundaries of the NN classifier. We provide a theoretical justification about this scaling scheme which inherently aims at reducing the False Negative rate while controlling the number of False Positives. We further formally establish a link between the proposed method and cost-sensitive learning. An extensive experimental study is conducted on many public imbalanced datasets showing that our method is very effective with respect to popular Nearest-Neighbor algorithms, comparable to state-of-the-art sampling methods and even yields the best performance when combined with them. [ABSTRACT FROM AUTHOR]

As short video social apps develop rapidly, feed has become the main approach or algorithm to present recommendation content to users in such apps. There are big differences in the way that video apps make use of feed flow based on artificial intelligence algorithm. Two kinds of short video social apps including DouYin and KuaiShou are studied with a user experiment in this paper. Several indicators are established to quantify the user experience differences of these two apps. The results are analyzed with correlation analysis to find out the relationship between user experience performance and content presentation mode of feed flow. The differences found from the results are explained from the perspectives of user cognition and behavior. [ABSTRACT FROM AUTHOR]

Correntropy-based learning has achieved great success in practice during the last decades. It is originated from information-theoretic learning and provides an alternative to classical least squares method in the presence of non-Gaussian noise. In this paper, we investigate the theoretical properties of learning algorithms generated by Tikhonov regularization schemes associated with Gaussian kernels and correntropy loss. By choosing an appropriate scale parameter of Gaussian kernel, we show the polynomial decay of approximation error under a Sobolev smoothness condition. In addition, we employ a tight upper bound for the uniform covering number of Gaussian RKHS in order to improve the estimate of sample error. Based on these two results, we show that the proposed algorithm using varying Gaussian kernel achieves the minimax rate of convergence (up to a logarithmic factor) without knowing the smoothness level of the regression function. [ABSTRACT FROM AUTHOR]

Accurate identification of Golgi protein types can provide useful clues to reveal the correlation between GA dysfunction and disease pathology and improve the ability to develop more effective treatments for the diseases. This paper introduces an effective and robust method to classify Golgi protein type with traditional machine learning algorithms. In which various features such as n-GDip, DCCA, psePSSM were used as training features and SVM with linear kernel was employed as a classifier. To solve the imbalance problem of the benchmark datasets, the oversampling technique SMOTE was adopted. To deal with the huge amount of features, the PCA algorithm and Fisher feature selection method were adopted to reduce feature dimensions and remove redundant features. The experimental results show that the proposed method had a further improvement compared with other traditional machine learning methods in 10-fold cross-validation, Jackknife cross-validation and independent testing, which means a further step for the clinical application of computational methods to predict the Golgi protein types. [ABSTRACT FROM AUTHOR]

The 3D measurement system based on line-structured light uses a camera to capture laser stripes due to changing in the shape of an object, and uses the acquired pixel coordinates for 3D reconstruction. System calibration is an important step in 3D measurement. The current camera calibration algorithm research mainly focuses on improving the algorithm itself, and there is less research on the influence of external factors. This paper proposes a coplanar hybrid calibration algorithm based on the error screening model by combining the error screening model, mathematical model and neural network model. It is mainly divided into two steps. The first step is to use the radial array constraint calibration algorithm based on the error screening model to solve the camera's internal and external parameters. The second step uses the camera internal and external parameters obtained in the first step to convert the pixel coordinates into real three-dimensional coordinates, and compares the calculated three-dimensional coordinates with the actual coordinates. Using machine learning to establish a compensation network, get a compensation function, and use the resulting 3D world coordinates to perform point cloud stitching. Experiments show that compared with the traditional calibration algorithm, the calibration algorithm has a small error and reduces the calibration error by about 6.5%. [ABSTRACT FROM AUTHOR]

In this paper, a new auto-scoring algorithm that automatically evaluates the Berg balance scale (BBS) tasks is proposed. The BBS can be used as an indicator for patients to analyze their rehabilitation status by themselves. In the proposed method, the patient must use a wearable inertial measurement unit (IMU) sensor, and the result of the patient's BBS task execution would be scored automatically by the evaluation algorithm. The proposed evaluation algorithm involves only few computations and has high scoring accuracy. Nonlinear kernel principal component analysis and a small number of linear features were combined to reduce the features from each sensor, and the algorithm model was implemented using a support vector machine (SVM), a machine learning technique with low computational complexity. The effectiveness of the algorithm was evaluated through clinical evaluation of 53 subjects with up to eight IMU sensors. The average accuracy of the proposed algorithm using eight sensors was 93.2%, and that using five sensors was 91.5%. There was a 12.6% and 10.9% increase in accuracy, respectively, compared to a previous study. The training and testing times of the proposed SVM model were over 38 times faster than the multi-layer perceptron model used in a previous study. [ABSTRACT FROM AUTHOR]

It is known that one aim of semi-supervised learning is to improve the prediction performance using a few labeled data with a large set of unlabeled data. Recently, a Laplacian regularized semi-supervised learning gradient (LapRSSLG) algorithm associated with data adjacency graph edge weights is proposed in the literature. The algorithm receives success in applications, but there is no theory on the performance analysis. In this paper, an explicit learning rate estimate for the algorithm is provided, which shows that the convergence is indeed controlled by the unlabeled data. [ABSTRACT FROM AUTHOR]

In recent years, how to use renewable energy to reduce the energy cost of internet data center (IDC) has been an urgent problem to be solved. More and more solutions are beginning to consider machine learning, but many of the existing methods need to take advantage of some future information, which is difficult to obtain in the actual operation process. In this paper, we focus on reducing the energy cost of IDC by controlling the energy flow of renewable energy without any future information. we propose an efficient energy dynamic control algorithm based on the theory of reinforcement learning, which approximates the optimal solution by learning the feedback of historical control decisions. For the purpose of avoiding overestimation, improving the convergence ability of the algorithm, we use the double Q -method to further optimize. The extensive experimental results show that our algorithm can on average save the energy cost by 18.3% and reduce the rate of grid intervention by 26.2% compared with other algorithms, and thus has good application prospects. [ABSTRACT FROM AUTHOR]

Protein structure prediction is the prediction of the 3D structure of a protein based on its amino acid sequence. It is a key component in disciplines such as medicine, biology, and biochemistry. The prediction of the protein secondary structure of Homo sapiens is one of the more important domains. Many methods have been used to feed forward neural networks or SVMs combined with a sliding window. This method's mechanisms are too complex to be able to extract clear and straightforward physical meanings from it. This paper explores population-based incremental learning (PBIL), which is a method that combines the mechanisms of a generational genetic algorithm with simple competitive learning. The result shows that its accuracies are particularly associated with the Homo species. This new perspective reveals a number of different possibilities for the purposes of performance improvements. [ABSTRACT FROM AUTHOR]

This paper intends to propose an integrated hybrid algorithm for training radial basis function neural network (RBFNN) learning. The proposed integrated of particle swarm and genetic algorithm based optimization (IPGO) algorithm is composed of two approaches based on particle swarm optimization (PSO) and genetic algorithm (GA) for gathering both their virtues to improve the learning performance of RBFNN. The diversity of individuals results in higher chance to search in the direction of global optimal instead of being confined to local optimal particularly in problem with higher complexity. The IPGO algorithm with PSO-based and GA-based approaches has shown promising results in some benchmark problems with three continuous test functions. After proposing the algorithm for these problems with result providing its outperforming performance, this paper supplements a practical application case for the papaya milk sales forecasting to expound the superiority of the IPGO algorithm. In addition, model evaluation results of the case have showed that the IPGO algorithm outperforms other algorithms and auto-regressive moving average (ARMA) models in terms of forecasting accuracy and execution time. [ABSTRACT FROM AUTHOR]

In recent years, the theory of sparse representation (SR) has been widely exploited in sparse subspace learning (SSL). Among all these methods, SR is a parameter-free global algorithm in nature which is mostly utilized to construct the correlations between samples to avoid some negative effects incurred by k-nearest neighbor (KNN) or some other methods. However, these SSL algorithms always lack obvious discrimination because of the ignorance of samples distribution. Meanwhile, some incorrect correlations are taken into consideration owing to the global feature of SR. To solve these two problems, a new SSL algorithm called locality structured sparsity preserving embedding (LSPE) is proposed in this paper. We add the local structured information to SR and construct correlations between samples. However, LSPE is an unsupervised method which wastes all label information. Therefore, LSPE is extended to semi-supervised LSPE (SLSPE) in this paper. SLSPE not only makes good use of the label information but also enhances the discriminative power of LSPE. Extensive experiments have been performed on three image datasets (CMU, COIL20, ORL) and two UCI datasets (Glass, Segment) to prove the efficiency of the LSPE and SLSPE. [ABSTRACT FROM AUTHOR]

In this paper, an existing rule-based intrusion detection system (IDS) is made more intelligent through the application of machine learning. Snort was chosen as it is an open source software and though it was performing well, it showed false positives (FPs). To find the best performing machine learning algorithms (MLAs) to use with Snort so as to improve its detection, we tested some algorithms on three available datasets. Support vector machine (SVM) was chosen along with fuzzy logic and decision tree based on their accuracy. Combined versions of algorithms through ensemble SVM along with other variants were tried on the generated traffic of normal and malicious packets at 10Gbps. Optimized versions of the SVM along with firefly and ant colony optimization (ACO) were also tried, and the accuracy improved remarkably. Thus, the application of combined and optimized MLAs to Snort at 10Gbps worked quite well. [ABSTRACT FROM AUTHOR]

In reinforcement learning (RL), the exploration/exploitation (E/E) dilemma is a very crucial issue, which can be described as searching between the exploration of the environment to find more profitable actions, and the exploitation of the best empirical actions for the current state. We focus on the single trajectory RL problem where an agent is interacting with a partially unknown MDP over single trajectories, and try to deal with the E/E in this setting. Given the reward function, we try to find a good E/E strategy to address the MDPs under some MDP distribution. This is achieved by selecting the best strategy in mean over a potential MDP distribution from a large set of candidate strategies, which is done by exploiting single trajectories drawn from plenty of MDPs. In this paper, we mainly make the following contributions: (1) We discuss the strategy-selector algorithm based on formula set and polynomial function. (2) We provide the theoretical and experimental regret analysis of the learned strategy under an given MDP distribution. (3) We compare these methods with the 'state-of-the-art' Bayesian RL method experimentally. [ABSTRACT FROM AUTHOR]

In the past decades, graphs that are determined by their spectrum have received much more and more attention, since they have been applied to several fields, such as randomized algorithms, combinatorial optimization problems and machine learning. An important part of spectral graph theory is devoted to determining whether given graphs or classes of graphs are determined by their spectra or not. So, finding and introducing any class of graphs which are determined by their spectra can be an interesting and important problem. The main aim of this study is to characterize two classes of multicone graphs which are determined by their adjacency, Laplacian and signless Laplacian spectra. A multicone graph is defined to be the join of a clique and a regular graph. Let denote a complete graph on vertices. In the paper, we show that multicone graphs and are determined by both their adjacency spectra and their Laplacian spectra, where and denote the Local Higman-Sims graph and the Local graph, respectively. In addition, we prove that these multicone graphs are determined by their signless Laplacian spectra. [ABSTRACT FROM AUTHOR]

Multi-view learning aims to effectively learn from data represented by multiple independent sets of attributes, where each set is taken as one view of the original data. In real-world application, each view should be acquired in unequal cost. Taking web-page classification for example, it is cheaper to get the words on itself (view one) than to get the words contained in anchor texts of inbound hyper-links (view two). However, almost all the existing multi-view learning does not consider the cost of acquiring the views or the cost of evaluating them. In this paper, we support that different views should adopt different representations and lead to different acquisition cost. Thus we develop a new view-dependent cost different from the existing both class-dependent cost and example-dependent cost. To this end, we generalize the framework of multi-view learning with the cost-sensitive technique and further propose a Cost-sensitive Multi-View Learning Machine named CMVLM for short. In implementation, we take into account and measure both the acquisition cost and the discriminant scatter of each view. Then through eliminating the useless views with a predefined threshold, we use the reserved views to train the final classifier. The experimental results on a broad range of data sets including the benchmark UCI, image, and bioinformatics data sets validate that the proposed algorithm can effectively reduce the total cost and have a competitive even better classification performance. The contributions of this paper are that: (1) first proposing a view-dependent cost; (2) establishing a cost-sensitive multi-view learning framework; (3) developing a wrapper technique that is universal to most multiple kernel based classifier. [ABSTRACT FROM AUTHOR]

Extracting knowledge from huge data in a reasonable time is still a challenging problem. Most real data (structured or not) can be mapped to an equivalent binary context, with or without using a scaling method, as for extracting associations between words in a text, or in machine learning systems. In this paper, our objective is to find a minimal coverage of a relation with formal concepts. The problem is known to be NP-complete.1 In this paper, we exploit a particular difunctional relation embedded in any binary relation , the fringe of , to find an approximate conceptual coverage of . We use formal properties of fringes to find better algorithms calculating the minimal rectangular coverage of binary relation. Here, a formal context is considered as a binary relation. By exploiting some background on relational algebra in the present work, we merge some results of Belohlavek and Vichodyl,2 using formal concept analysis with previous results obtained by Kcherif et al.3 using relational algebra. We finally propose decomposition algorithms based on the relational formalization and fringe relations. [ABSTRACT FROM AUTHOR]

Graphical avatars have gained popularity in many application domains such as three-dimensional (3D) animation movies and animated simulations for product design. However, the methods to edit avatars' behaviors in the 3D graphical environment remained to be a challenging research topic. Since the hand-crafted methods are time-consuming and inefficient, the automatic actions of the avatars are required. To achieve the autonomous behaviors of the avatars, artificial intelligence should be used in this research area. In this paper, we present a novel approach to construct a system of automatic avatars in the 3D graphical environments based on the machine learning techniques. Specific framework is created for controlling the behaviors of avatars, such as classifying the difference among the environments and using hierarchical structure to describe these actions. Because of the requirement of simulating the interactions between avatars and environments after the classification of the environment, Reinforcement Learning is used to compute the policy to control the avatar intelligently in the 3D environment for the solution of the problem of different situations. Thus, our approach has solved problems such as where the levels of the missions will be defined and how the learning algorithm will be used to control the avatars. In this paper, our method to achieve these goals will be presented. The main contributions of this paper are presenting a hierarchical structure to control avatars automatically, developing a method for avatars to recognize environment and presenting an approach for making the policy of avatars' actions intelligently. [ABSTRACT FROM AUTHOR]

This paper presents a new approach of face recognition based on wavelet network using 2D fast wavelet transform and multiresolution analysis. This approach is divided in two stages: the training stage and the recognition stage. The first consists to approximate every training face image by a wavelet network. The second consists in recognition of a new test image by comparing it to all the training faces, the distances between this test face and all images from the training set are calculated in order to identify the searched person. The usual training algorithms presents some disadvantages when the weights of the wavelet network are computed by applying the back-propagation algorithm or by direct solution which requires computing an inversion of matrix, this computation may be intensive when the learning data is too large. We present in this paper our solutions to overcome these limitations. We propose a novel learning algorithm based on the 2D Fast Wavelet Transform. Furthermore, we have increased the performances of our algorithm by introducing the Levenberg-Marquardt method to optimize the learning functions and using the Beta wavelet which has at both an analytical expression and wavelet filter bank. Extensive empirical experiments are performed to compare the proposed method with other approaches as PCA, LDA, EBGM and RBF neural network using the ORL and FERET benchmarks. [ABSTRACT FROM AUTHOR]

In the paper a special incremental identification procedure for learning a Neural Singleton (NS) Model, as a simplification of the RBFNN model is proposed. At each epoch of this incremental procedure a special integrated learning algorithm for the neuron centers and then a supervised learning for the singletons are performed. As a result, the approximation error of the NS model is gradually decreased with the increment of the neurons at each learning epoch. The newly proposed integrated unsupervised and Error-Driven learning algorithm is the main part of the incremental identification procedure. It is a modified version of the Neural-Gas learning algorithm, which uses the normalised evaluation error as feedback information. Thus the integrated learning algorithm is able to drive the neurons to the "more interesting" areas in the input space, where bigger evaluation errors exist, instead of just locating them in the areas of higher data density. The final result is an incremental growing type of NS model that gradually improves its approximation accuracy at each learning epoch. A detailed test example is shown in the paper in order to evaluate the performance and to show the important features of the whole incremental identification procedure, including the integrated learning algorithm. Comparison results with the conventional (one-epoch and non-incremental) model are also given. [ABSTRACT FROM AUTHOR]

Ontology, a model of knowledge representation and storage, has had extensive applications in pharmaceutics, social science, chemistry and biology. In the age of 'big data', the constructed concepts are often represented as higher-dimensional data by scholars, and thus the sparse learning techniques are introduced into ontology algorithms. In this paper, based on the alternating direction augmented Lagrangian method, we present an ontology optimization algorithm for ontological sparse vector learning, and a fast version of such ontology technologies. The optimal sparse vector is obtained by an iterative procedure, and the ontology function is then obtained from the sparse vector. Four simulation experiments show that our ontological sparse vector learning model has a higher precision ratio on plant ontology, humanoid robotics ontology, biology ontology and physics education ontology data for similarity measuring and ontology mapping applications. [ABSTRACT FROM AUTHOR]

In this paper, we employ multiple Single-hidden Layer Feedforward Neural Networks for multi-view action recognition. We propose an extension of the Extreme Learning Machine algorithm that is able to exploit multiple action representations and scatter information in the corresponding ELM spaces for the calculation of the networks' parameters and the determination of optimized network combination weights. The proposed algorithm is evaluated by using two state-of-the-art action video representation approaches on five publicly available action recognition databases designed for different application scenarios. Experimental comparison of the proposed approach with three commonly used video representation combination approaches and relating classification schemes illustrates that ELM networks employing a supervised view combination scheme generally outperform those exploiting unsupervised combination approaches, as well as that the exploitation of scatter information in ELM-based neural network training enhances the network's performance. [ABSTRACT FROM AUTHOR]

A framework for constructing robust classification models is proposed in the paper. An assumption about importance of one of the classes in comparison with other classes is incorporated into the models. It often takes place in the real application, for example, in reliability, in medical diagnostic, etc. A main idea underlying the models is to consider a set of probability distributions on training examples produced by the imprecise probability models such as linear-vacuous mixture and constant odd-ratio contaminated models. Extreme points of the sets of probability distributions are a main tool for constructing the robust classifiers. It is shown that algorithms for computing optimal classification parameters are reduced to a finite number of weighted support vector machines with weights determined by the extreme points. Experimental results with synthetic and real data illustrate the proposed models. [ABSTRACT FROM AUTHOR]

Extreme Learning Machine (ELM) is an emergent technique for training Single-hidden Layer Feedforward Networks (SLFNs). It attracts significant interest during the recent years, but the randomly assigned network parameters might cause high learning risks. This fact motivates our idea in this paper to propose an evolving ELM paradigm for classification problems. In this paradigm, a Differential Evolution (DE) variant, which can online select the appropriate operator for offspring generation and adaptively adjust the corresponding control parameters, is proposed for optimizing the network. In addition, a 5-fold cross validation is adopted in the fitness assignment procedure, for improving the generalization capability. Empirical studies on several real-world classification data sets have demonstrated that the evolving ELM paradigm can generally outperform the original ELM as well as several recent classification algorithms. [ABSTRACT FROM AUTHOR]

Recently an Online Sequential Fuzzy Extreme Learning (OS-Fuzzy-ELM) algorithm has been developed by Rong et al. for the RBF-like fuzzy neural systems where a fuzzy inference system is equivalent to a RBF network under some conditions. In the paper the learning ability of the batch version of OS-Fuzzy-ELM, called as Fuzzy-ELM is further evaluated to train a class of fuzzy inference systems which can not be represented by the RBF networks. The equivalence between the output of the fuzzy system and that of a generalized Single-Hidden Layer Feedforward Network as presented in Huang et al. is shown first, which is then used to prove the validity of the Fuzzy-ELM algorithm. In Fuzzy-ELM, the parameters of the fuzzy membership functions are randomly assigned and then the corresponding consequent parameters are determined analytically. Besides an input variable selection method based on the correlation measure is proposed to select the relevant inputs as the inputs of the fuzzy system. This can avoid the exponential increase of number of fuzzy rules with the increase of dimension of input variables while maintaining the testing performance and reducing the computation burden. Performance comparison of Fuzzy-ELM with other existing algorithms is presented using some real-world regression benchmark problems. The results show that the proposed Fuzzy-ELM produces similar or better accuracies with a significantly lower training time. [ABSTRACT FROM AUTHOR]

Extreme learning machine (ELM) is an efficient and practical learning algorithm used for training single hidden layer feed-forward neural networks (SLFNs). ELM can provide good generalization performance at extremely fast learning speed. However, ELM suffers from instability and over-fitting, especially on relatively large datasets. Based on probabilistic SLFNs, an approach of fusion of extreme learning machine (F-ELM) with fuzzy integral is proposed in this paper. The proposed algorithm consists of three stages. Firstly, the bootstrap technique is employed to generate several subsets of original dataset. Secondly, probabilistic SLFNs are trained with ELM algorithm on each subset. Finally, the trained probabilistic SLFNs are fused with fuzzy integral. The experimental results show that the proposed approach can alleviate to some extent the problems mentioned above, and can increase the prediction accuracy. [ABSTRACT FROM AUTHOR]

A strong assumption to derive generalization guarantees in the standard PAC framework is that training (or source) data and test (or target) data are drawn according to the same distribution. Because of the presence of possibly outdated data in the training set, or the use of biased collections, this assumption is often violated in real-world applications leading to different source and target distributions. To go around this problem, a new research area known as Domain Adaptation (DA) has recently been introduced giving rise to many adaptation algorithms and theoretical results in the form of generalization bounds. This paper deals with self-labeling DA whose goal is to iteratively incorporate semi-labeled target data in the learning set to progressively adapt the classifier from the source to the target domain. The contribution of this work is three-fold: First, we provide the minimum and necessary theoretical conditions for a self-labeling DA algorithm to perform an actual domain adaptation. Second, following these theoretical recommendations, we design a new iterative DA algorithm, called GESIDA, able to deal with structured data. This algorithm makes use of the new theory of learning with (ε,γ,τ)-good similarity functions introduced by Balcan et al., which does not require the use of a valid kernel to learn well and allows us to induce sparse models. Finally, we apply our algorithm on a structured image classification task and show that self-labeling domain adaptation is a new original way to deal with scaling and rotation problems. [ABSTRACT FROM AUTHOR]

The problem of training classifiers only with target data arises in many applications where nontarget data are too costly, difficult to obtain, or not available at all. Several one-class classification methods have been presented to solve this problem, but most of the methods are highly sensitive to the presence of outliers in the target class. Ensemble methods have therefore been proposed as a powerful way to improve the classification performance of binary/multi-class learning algorithms by introducing diversity into classifiers. However, their application to one-class classification has been rather limited. In this paper, we present a new ensemble method based on a nonparametric weighted bagging strategy for one-class classification, to improve accuracy in the presence of outliers. While the standard bagging strategy assumes a uniform data distribution, the method we propose here estimates a probability density based on a forest structure of the data. This assumption allows the estimation of data distribution from the computation of simple univariate and bivariate kernel densities. Experiments using original and noisy versions of 20 different datasets show that bagging ensemble methods applied to different one-class classifiers outperform base one-class classification methods. Moreover, we show that, in noisy versions of the datasets, the nonparametric weighted bagging strategy we propose outperforms the classical bagging strategy in a statistically significant way. [ABSTRACT FROM AUTHOR]

Due to the infeasibility of randomized controlled experiments, the existence of unobserved variables and the fact that equivalent direct acyclic graphs obtained generally can not be distinguished, it is difficult to learn the true causal relations of original graph. This paper presents an algorithm called BSPC based on adjacent nodes to learn the structure of Causal Bayesian Networks with unobserved variables by using observational data. It does not have to adjust the structure as the existing algorithms FCI and MBCS*, while it can guarantee to obtain the true adjacent nodes. More important is that algorithm BSPC reduces computational complexity and improves reliability of conditional independence tests. Theoretical results show that the new algorithm is correct. In addition, the advantages of BSPC in terms of the number of conditional independence tests and the number of orientation errors are illustrated with simulation experiments from which we can see that it is more suitable in order to learn the structure of Causal Bayesian Networks with latent variables. Moreover a better latent structure representation is returned. [ABSTRACT FROM AUTHOR]

Sparse representation based classification has led to interesting image recognition results, while the dictionary used for sparse coding plays a key role in it. This paper presents a novel su-pervised structure dictionary learning (SSDL) algorithm to learn a discriminative and block structure dictionary. We associate label information with each dictionary item and make each class-specific sub-dictionary in the whole structured dictionary have good representation ability to the training samples from the associated class. More specifically, we learn a structured dictionary and a multiclass classifier simultaneously. Adding an inhomogeneous representation term to the objective function and considering the independence of the class-specific sub-dic-tionaries improve the discrimination capabilities of the sparse coordinates. An iteratively op-timization method be proposed to solving the new formulation. Experimental results on four face databases demonstrate that our algorithm outperforms recently proposed competing sparse coding methods. [ABSTRACT FROM AUTHOR]

Current existing representative works to semi-supervised incremental learning prefer to select unlabeled instances predicted with high confidence for model retraining. However, this strategy may degrade the classification performance rather than improve it, because relying on high confidence for data selection can lead to an erroneous estimate to the true distribution, espe-cially when the confidence annotator is highly correlated with the confidence annotator. In this paper, a new semi-supervised incremental learning algorithm was proposed, which selected the high confidence unlabeled instances with symmetrical distribution from unlabeled data, it can reduce the bias in the estimation in some degree. In detail, expectation maximization algorithm was used to estimate the confidence of each instance, and Gaussian function was used to calculate the data distribution, then the selected unlabeled data was used for retraining model with classifier algorithm. The experimental results based on a large number of UCI data sets show that our algorithm can effectively exploit unlabeled data to enhance the learning performance. [ABSTRACT FROM AUTHOR]

Most dimensionality reduction methods depend significantly on the distance measure used to compute distances between different examples. Therefore, a good distance metric is essential to many dimensionality reduction algorithms. In this paper, we present a new dimensionality reduction method for data visualization, called Distance-ratio Preserving Embedding (DrPE), which preserves the ratio between the pairwise distances. It is achieved by minimizing the mismatch between the distance ratios derived from input and output space. The proposed method can preserve the relational structures among points of the input space. Extensive visualization experiments compared with existing dimensionality reduction algorithms demonstrate the effectiveness of our proposed method. [ABSTRACT FROM AUTHOR]

A new multi-kernel regression learning algorithm is studied in this paper. In our setting, the hypothesis space is generated by two Mercer kernels, thus it has stronger approximation ability than the single kernel case. We provide the mathematical foundation for this regularized learning algorithm. We obtain satisfying capacity-dependent error bounds and learning rates by the covering number method. [ABSTRACT FROM AUTHOR]