Search

Breast cancer (BC) is the leading cause of death among women worldwide. It affects in general women older than 40 years old. Medical images analysis is one of the most promising research areas since it provides facilities for diagnosis and decision-making of several diseases such as BC. This paper conducts a Structured Literature Review (SLR) of the use of Machine Learning (ML) and Image Processing (IP) techniques to deal with BC imaging. A set of 530 papers published between 2000 and August 2019 were selected and analyzed according to ten criteria: year and publication channel, empirical type, research type, medical task, machine learning techniques, datasets used, validation methods, performance measures and image processing techniques which include image pre-processing, segmentation, feature extraction and feature selection. Results showed that diagnosis was the most used medical task and that Deep Learning techniques (DL) were largely used to perform classification. Furthermore, we found out that classification was the most ML objective investigated followed by prediction and clustering. Most of the selected studies used Mammograms as imaging modalities rather than Ultrasound or Magnetic Resonance Imaging with the use of public or private datasets with MIAS as the most frequently investigated public dataset. As for image processing techniques, the majority of the selected studies pre-process their input images by reducing the noise and normalizing the colors, and some of them use segmentation to extract the region of interest with the thresholding method. For feature extraction, we note that researchers extracted the relevant features using classical feature extraction techniques (e.g. Texture features, Shape features, etc.) or DL techniques (e. g. VGG16, VGG19, ResNet, etc.), and finally few papers used feature selection techniques in particular the filter methods. [ABSTRACT FROM AUTHOR]

Collecting and analyzing data from all devices to improve the efficiency of business processes is an important task of Industrial Internet of Things (IIoT). In the age of data explosion, extensive text data generated by the IIoT have given birth to a variety of text representation methods. The task of text representation is to convert the natural language to a form that computer can understand with retaining the original semantics. However, these methods are difficult to effectively extract the semantic features among words and distinguish polysemy in natural language. Combining the advantages of convolutional neural network (CNN) and variational autoencoder (VAE), this paper proposes an intelligent CNN-VAE text representation algorithm as an advanced learning method for social big data within next-generation IIoT, which help users identify the information collected by sensors and perform further processing. This method employs the convolution layer to capture the local features of the context and uses the variational technique to reconstruct feature space to make it conform to the normal distribution. In addition, the improved word2vec model based on topical word embedding (TWE) is utilized to add topical information to word vectors to distinguish polysemy. This paper takes the social big data as an example to illustrate the way of the proposed algorithm applied in the next-generation IIoT and utilizes Cnews dataset to verify the performance of proposed method with four evaluating metrics (i.e., recall, accuracy, precision, and F1-score). Experimental results indicate that the proposed method outperforms word2vec-avg and CNN-AE in K-nearest neighbor (KNN), random forest (RF), and support vector machine (SVM) classifiers and distinguishes polysemy effectively. [ABSTRACT FROM AUTHOR]

Twin support vector regression (TSVR) has been widely applied in regression problems. TSVR seeks a pair of ε -insensitive proximal planes by solving two support vector machine type problems. TSVR assumes that the matrices appearing in the dual formulation are positive definite. However, in real-world scenarios, such an assumption may not be fulfilled and, hence, leads to suboptimal performance. ε -Twin support vector regression (ε -TSVR) improved the TSVR by introducing the regularisation term to avoid the singularity issues. Most of the twin support vector machine models involve the computation of matrix inverses. Also, TSVR implements the empirical risk minimization principle. In this paper, we propose an improved twin support vector regression (ITSVR) for brain age estimation by introducing different Lagrangian functions for the primal problems of the TSVR. The proposed ITSVR implements the structural risk minimization principle and avoids the computation of the matrix inverses. To solve the optimization problem more efficiently, we used successive overrelaxation (SOR) technique. We evaluated the proposed ITSVR on cognitively healthy subjects, mild cognitive impairment subjects and Alzheimer's disease subjects. The experimental results demonstrate that the proposed ITSVR has superior performance compared to the baseline models for brain age estimation. [ABSTRACT FROM AUTHOR]

This paper is proposing an machine learning based expert system for preliminary prediction of the insulation fire resisting performance of shallow floor systems when subject to exposure to the ISO 834 Standard Fire Curve. The proposed system is a digital tool which incorporates a machine learning algorithm trained on the outcomes of pre-run two-dimensional finite element heat transfer analyses of shallow floor system details. The algorithm predicts the insulation performance of similar details with a measurable accuracy of 96% (i.e the insulation rating band was predicted correctly in 96% of the cases) without the requirement for an explicit deterministic analysis. The paper presents the challenges and prerequisites to build such an expert system and acts as a proof of concept for the use of machine learning for the assessment of the insulation performance of shallow floor system under the guidance of BS EN 1363-1:2012. A Support Vector Machine machine learning algorithm is adopted in this work. The required processes that were needed for the development of the expert system include the stages of data acquisition, exploratory data analysis, choice of machine learning algorithm, model training, tuning, and validation. This expert system is useful for practitioners to rapidly assess the feasibility of different construction details at early stages of the design process. [ABSTRACT FROM AUTHOR]

This study utilizes classification models to provide a robust algorithm for imbalanced data where the minority class is of the interest, that is, in the context of default payments. In developing an integrated predictive accuracy algorithm, this study proposes machine learning classifiers and applies DNN, SVM, KNN, and ANN. The proposed algorithm utilizes a 30,000 imbalanced dataset to improve the accuracy of the prediction of default payments by implementing oversampling and undersampling strategies, such as synthetic minority oversampling technique (SMOTE), SVM SMOTE, random undersampling, and ALL-KNN. The results indicate that the SVM under the ALL-KNN sampling technique is able to achieve an accuracy of 98.6%, with the lowest cross entropy loss measurement of 0.028. Through the accurate implementation of the neural networks and neurons used in the proposed algorithm, this paper presents better insights into the functioning of the neural networks when used in conjunction with the resampling techniques. Using the methodology and algorithm presented in this study, credit risk assessments can be more accurately predicted in practical applications where most of the clients are categorized as non-default payments. [ABSTRACT FROM AUTHOR]

To avoid the considerable challenges and losses caused by stuck drilling to normal drilling operations, this article analyses the mechanism of stuck drilling, then combines the artificial fish swarm algorithm (AFSA) and support vector machine (SVM), and finally proposes an early warning model for the stuck-in medical drilling process based on the AFSA and SVM. The model realizes real-time sticking risk warnings by using the four parameters of riser pressure, torque, speed and hook load collected in real time and promotes real-time drilling parameter monitoring for the real-time dynamic warning of sticking risk. By comparing the AFSA-SVM sticking prediction model with the particle swarm optimization model and the traditional cross-validation optimization model, it is found that the AFSA-SVM stuck prediction model accuracy can reach 97.561% and that the training and testing times are 5.874 s and 0.76 s, respectively. Its accuracy and computational efficiency are higher than those of the particle swarm optimization model and traditional cross-validation optimization model. In comparison with the existing technology, the four-parameter early sticking warning model based on AFSA-SVM presented in this paper shows powerful comprehensive performance and field application value. [ABSTRACT FROM AUTHOR]

The aim of this paper is to introduce a new two-step inertial method for approximating a solution to a generalized split common fixed point problem, which is a unique solution to a variational inequality problem. We establish a strong convergence theorem for the sequence generated by the algorithm. We explore various special cases related to fundamental problems, including the split feasibility problem, the split common null point problem, and the constrained convex minimization problem. To demonstrate the efficacy and performance of our proposed algorithm, we apply it to a practical scenario involving support vector machines for binary classification. The algorithm is employed on diverse datasets sourced from the UC Irvine Machine Learning Repository, serving as the training set. [ABSTRACT FROM AUTHOR]

The psychological health of a person plays an important role in their daily life activities. The paper addresses depression issues with the machine learning model using facial expressions of the patient. Some research has already been done on visual based on depression detection methods, but those are illumination variant. The paper uses feature extraction using LBP (Local Binary Pattern) descriptor, which is illumination invariant. The Viola-Jones algorithm is used for face detection and SVM (support vector machine) is considered for classification along with the LBP descriptor to make a complete model for depression level detection. The proposed method captures frontal face from the videos of subjects and their facial features are extracted from each frame. Subsequently, the facial features are analyzed to detect depression levels with the post-processing model. The performance of the proposed system is evaluated using machine learning algorithms in MATLAB. For the real-time system design, it is necessary to test it on the hardware platform. The LBP descriptor has been implemented on FPGA using Xilinx VIVADO 16.4. The results of the proposed method show satisfactory performance and accuracy for depression detection comparison with similar previous work. [ABSTRACT FROM AUTHOR]

Over the past two decades, support vector machines (SVMs) have become a popular supervised machine learning model, and plenty of distinct algorithms are designed separately based on different KKT conditions of the SVM model for classification/regression with different losses, including convex and or nonconvex loss. In this paper, we propose an algorithm that can train different SVM models in a unified scheme. First, we introduce a definition of the least squares type of difference of convex loss (LS-DC) and show that the most commonly used losses in the SVM community are LS-DC loss or can be approximated by LS-DC loss. Based on the difference of convex algorithm (DCA), we then propose a unified algorithm called UniSVM which can solve the SVM model with any convex or nonconvex LS-DC loss, wherein only a vector is computed by the specifically chosen loss. UniSVM has a dominant advantage over all existing algorithms for training robust SVM models with nonconvex losses because it has a closed-form solution per iteration, while the existing algorithms always need to solve an L1SVM/L2SVM per iteration. Furthermore, by the low-rank approximation of the kernel matrix, UniSVM can solve large-scale nonlinear problems efficiently. To verify the efficacy and feasibility of the proposed algorithm, we perform many experiments on small artificial problems and large benchmark tasks both with and without outliers for classification and regression for comparison with state-of-the-art algorithms. The experimental results demonstrate that UniSVM can achieve comparable performance in less training time. The foremost advantage of UniSVM is that its core code in Matlab is less than 10 lines; hence, it can be easily grasped by users or researchers. [ABSTRACT FROM AUTHOR]

Principles of typical motion planning algorithms are investigated and analyzed in this paper. These algorithms include traditional planning algorithms, classical machine learning algorithms, optimal value reinforcement learning, and policy gradient reinforcement learning. Traditional planning algorithms investigated include graph search algorithms, sampling-based algorithms, interpolating curve algorithms, and reaction-based algorithms. Classical machine learning algorithms include multiclass support vector machine, long short-term memory, Monte-Carlo tree search and convolutional neural network. Optimal value reinforcement learning algorithms include Q learning, deep Q-learning network, double deep Q-learning network, dueling deep Q-learning network. Policy gradient algorithms include policy gradient method, actor-critic algorithm, asynchronous advantage actor-critic, advantage actor-critic, deterministic policy gradient, deep deterministic policy gradient, trust region policy optimization and proximal policy optimization. New general criteria are also introduced to evaluate the performance and application of motion planning algorithms by analytical comparisons. The convergence speed and stability of optimal value and policy gradient algorithms are specially analyzed. Future directions are presented analytically according to principles and analytical comparisons of motion planning algorithms. This paper provides researchers with a clear and comprehensive understanding about advantages, disadvantages, relationships, and future of motion planning algorithms in robots, and paves ways for better motion planning algorithms in academia, engineering, and manufacturing. [ABSTRACT FROM AUTHOR]

In this paper, we present difference of convex algorithms for solving bilevel programs in which the upper level objective functions are difference of convex functions, and the lower level programs are fully convex. This nontrivial class of bilevel programs provides a powerful modelling framework for dealing with applications arising from hyperparameter selection in machine learning. Thanks to the full convexity of the lower level program, the value function of the lower level program turns out to be convex and hence the bilevel program can be reformulated as a difference of convex bilevel program. We propose two algorithms for solving the reformulated difference of convex program and show their convergence to stationary points under very mild assumptions. Finally we conduct numerical experiments to a bilevel model of support vector machine classification. [ABSTRACT FROM AUTHOR]

Recently, healthcare data analysis has become an attractive research topic. Data gathering is the first step in data analysis and processing. During the collection of the data, some errors may occur due to human mistakes, devices' errors, or the transmission process noise. The correct treatment of the missed data and outliers conserve the data size and improve the model's performance. This paper provides two enhanced algorithms to handle missing values and outliers in big datasets. The main idea is dividing the dataset into its different classes, or clustering it by using k-means++, then calculate the average value of each part, finally replace the missed data and outliers with its corresponding part mean value. The projected imputation and outliers' data handling algorithms are tested on a dataset called Pima Indian diabetic, which contains 2768 patients dividing into 952 diabetic and 1816 controls. Four classifiers (Random Forest, Decision Tree, Support Vector Machine, and Naïve Bayes) are used to evaluate the effect of the proposed algorithms. The results show that the proposed algorithms improve classification accuracy by 8% and decrease the RMSE by 17% over Deep Learning (DL). DL is the most powerful algorithms used in repairing the missed data. [ABSTRACT FROM AUTHOR]

Predicting stock market price is considered as a challenging task of financial time series analysis, which is of great interest to stock investors, stock traders and applied researchers. Many machine learning techniques have been used in this area to predict the stock market price, including regression algorithms which can be useful tools to provide good performance of financial time series prediction. Support Vector Regression is one of the most powerful algorithms in machine learning. There have been countless successes in utilizing SVR algorithm for stock market prediction. In this paper, we propose a novel hybrid approach based on machine learning and filtering techniques. Our proposed approach combines Support Vector Regression and Hodrick-Prescott filter in order to optimize the prediction of stock price. To assess the performance of this proposed approach, we have conducted several experiments using real world datasets. The principle objective of this paper is to demonstrate the improvement in predictive performance of stock market and verify the works of our proposed model in comparison with other optimized models. The experimental results confirm that the proposed algorithm constitutes a powerful model for predicting stock market prices. [ABSTRACT FROM AUTHOR]

This paper utilizes the cognitive radio (CR) spectrum to the fullest extent for extended applications requiring discretion. The CR technology provides various supports for cognitive radio networks (CRNs). The latter has CR nodes that sense free channels. Then, the CRN allocates the unused channels to secondary users (SUs) or unlicensed users. This allocation is termed the spectrum handoff. In this paper, by considering the identical channels in CR networks, a novel machine learning algorithm (the support vector machine—SVM) is employed. In addition, the queuing model of the preemptive resume priority M/M/1 is used. The proposed spectrum handoff algorithm selects the best possible CR network channel. The spectrum handoff algorithm uses the stated SVM algorithm scheme, which covers the transmitted and received power, the minimum service time, the data rate and the maximum vacancy time for the SU, to attain the maximum throughput. However, in multi-user greedy channel selection (GCS), only two parameters are considered. The proposed spectrum handoff algorithm based on the SVM scheme enhances the performance, and the SU throughput is improved to 68.7%. This approach is better than the GCS channel selection scheme. Additionally, this approach decreases the number of spectrum handoffs. As a result, the training accuracy of the SVM method is 97.6%, and it outperforms conventional methods. [ABSTRACT FROM AUTHOR]

Machine learning-based predictive modeling is to develop machine learning-based or data-driven models to predict failures before they occur and estimate the remaining useful life or time to failure (TTF) accurately. Accurate TTF estimation plays a vital role in predictive maintenance or PHM (Prognostic and Health Management). Despite the availability of large amounts of data and a variety of powerful data analysis methods, predictive models developed for PHM still fail to provide accurate and precise TTF estimations. This paper addresses this problem by integrating machine learning algorithms such as classification, regression and clustering. A classification system is used to determine the likelihood of component failures such that rough indications of TTF are provided. Clustering and SVM-based local regression are then introduced to refine the time to failure estimations provided by the classification system. The paper illustrates the applicability of the proposed approach through a real world aerospace application with details on data pre-processing requirements. The results demonstrate that the proposed method can reduce uncertainty in estimating time to failure, which in turn helps augment the usefulness of predictive maintenance. [ABSTRACT FROM AUTHOR]

Fuzzy support vector machine and their variants are prominent classification techniques that reduce the adverse effects of noises and outliers as compare to classical support vector machine. However, in these methods, only the distance between the training pattern and the class center is considered, and hence, the edge support vectors cannot be distinguished from noises and outliers properly. These limitations are overcome by intuitionistic fuzzy-based support vector machine which allocate two parameters (membership and non-membership degrees) to each pattern of a dataset and hence define score number based on the importance of a pattern. In this paper, an intuitionistic fuzzy proximal support vector machine for multi-category classification problems is proposed. The method significantly reduces the impacts of noises and outliers present in the dataset by assigning the intuitionistic fuzzy score function to each training point based on its location and surroundings. Moreover, the method is computationally efficient as the robust classifiers are obtained by solving the system of linear equations instead of large size quadratic programming problems. In the proposed method, using polynomial and Gaussian kernels, the hyperplanes are also developed in the feature space. The geometrical advantages of the suggested method over the existing techniques are ascertained using the simulated two-dimensional artificial dataset having three target classes. Further, extensive experimental studies on ten UCI benchmark datasets have been performed which demonstrate that the proposed algorithm predicts more precisely about future data as comparison to some well-established algorithms. Figures are also illustrated by varying different parameters involved in the model which confirms the performance of the method over the existing algorithms. Moreover, the analysis of the predictive behavior of the proposed approach is done using the Friedman test, a nonparametric alternative to the analysis of variance test at 5 % significance level. Further, the proposed method has also been applied to image classification and gesture-phase segmentation problems which confirms the efficiency and handling capabilities of the proposed algorithm in practical applications. [ABSTRACT FROM AUTHOR]

Knowledge in the source domain can be used in transfer learning to help train and classification tasks within the target domain with fewer available data sets. Therefore, given the situation where the target domain contains only a small number of available unlabeled data sets and multi-source domains contain a large number of labeled data sets, a new Multi-source Fast Transfer Learning algorithm based on support vector machine(MultiFTLSVM) is proposed in this paper. Given the idea of multi-source transfer learning, more source domain knowledge is taken to train the target domain learning task to improve classification effect. At the same time, the representative data set of the source domain is taken to speed up the algorithm training process to improve the efficiency of the algorithm. Experimental results on several real data sets show the effectiveness of MultiFTLSVM, and it also has certain advantages compared with the benchmark algorithm. [ABSTRACT FROM AUTHOR]

Recently, Bayesian Optimization (BO) provides an efficient technique for selecting the hyperparameters of machine learning models. The BO strategy maintains a surrogate model and an acquisition function to efficiently optimize the computation-intensive functions with a few iterations. In this paper, we demonstrate the utility of the BO to fine-tune the hyperparameters of a Random Forest (RF) model for a problem related to the recognition of splice-junction genetic sequences. Locating these splice-junctions prompts further understanding of the DNA splicing process. Specifically, the BO algorithm optimizes four RF hyperparameters: number of trees, number of splitting features, splitting criterion, and leaf size. The optimized RF model automatically selects the most predictive features of the training data. The dataset is obtained from the UCI machine learning repository where half of the records represent two different types of splice-junctions and the other half does not represent any splice-junction. Experimental results proved the advantage of the BO-RF with 99.96% and 97.34% training and test classification accuracies respectively. The results also demonstrated the ability of the RF model to select the most important features, ensuring the best possible results using Support Vector Machine (SVM), K-Nearest Neighbor (KNN), and decision tree (DT) models. Some practical procedures in model development and evaluation such as out-of-bag error and cross-validation approaches are also referred to. [ABSTRACT FROM AUTHOR]

Learning from categorical data is a critical yet challenging task. Current research focuses on either leveraging the complex interaction between and within categorical values to generate a numerical representation, or designing a model that can tackle this types of data directly. However, both of these paradigms overlook the relation between the data characteristics and learning model hypothesis. In this paper, we propose a model-aware categorical data embedding framework that jointly reveals the intrinsic categorical data characteristics and optimizes the fitness of the representation for the follow-up learning model. An ELM-aware and a SVM-aware representation methods have been instantiated under this framework. Extensive experiments of classification with the embedded representation on 17 data sets demonstrate that the proposed framework can significantly improve the categorical data representation performance compared with state-of-the-art competitors. [ABSTRACT FROM AUTHOR]

As recent development of technology, it enables patients to get treatment remotely from doctors through audio conversation. The fourth highest number of death every year is caused by diabetics. Almost 50% to 80% of patients can avoid diabetics if the cause is found at the early stage. In this paper, we propose a new methodology to detect Diabetes at an early stage and recommend few attributes in which the patient needs to be careful in order to avoid diabetics. The proposed methodology makes use of fuzzy logic and kNN classifier to find out the caution attributes and recommends them as soon as possible. The proposed algorithm detects the audio signals from patients or clinical labs to process the data. We implemented our proposed methodology on Pima Indian dataset and compared with existing algorithms and the result shows that our algorithm outperforms existing algorithms. [ABSTRACT FROM AUTHOR]

Among the extensions of twin support vector machine (TSVM), some scholars have utilized K-nearest neighbor (KNN) graph to enhance TSVM's classification accuracy. However, these KNN-based TSVM classifiers have two major issues such as high computational cost and overfitting. In order to address these issues, this paper presents an enhanced regularized K-nearest neighbor-based twin support vector machine (RKNN-TSVM). It has three additional advantages: (1) Weight is given to each sample by considering the distance from its nearest neighbors. This further reduces the effect of noise and outliers on the output model. (2) An extra stabilizer term was added to each objective function. As a result, the learning rules of the proposed method are stable. (3) To reduce the computational cost of finding KNNs for all the samples, location difference of multiple distances-based K-nearest neighbors algorithm (LDMDBA) was embedded into the learning process of the proposed method. The extensive experimental results on several synthetic and benchmark datasets show the effectiveness of our proposed RKNN-TSVM in both classification accuracy and computational time. Moreover, the largest speedup in the proposed method reaches to 14 times. [ABSTRACT FROM AUTHOR]

Support vector machine (SVM) is a widely used and reliable machine learning algorithm. It has been successfully applied to many real-world problems, with remarkable results. However, it has also been observed that SVM computational complexity increases with the increase in data size. Although many SVM speed optimization techniques have been proposed in the literature, there is still need for further improvement on the performance speed and accuracy of this algorithm. In this paper, a boundary detection algorithm for SVM speed optimization called ant colony optimization instance selection algorithm (ACOISA) is proposed. ACOISA is inspired by edge selection in ant colony optimization (ACO) algorithm, and it performs two primary functions: boundary detection and boundary instance selection. In the algorithm, ACO is used for boundary detection and k-nearest neighbor algorithm is used for boundary instance selection. Different sets of experiments are carried out to validate the efficiency of the proposed technique. All the experiments were performed on 35 datasets containing three well-known e-fraud types (credit card fraud, email spam and phishing email) and 31 other datasets available at UCI data repository. The experimental results showed that the proposed technique efficiently improved SVM training speed in 100% of the datasets used for evaluation, without significantly affecting SVM classification quality. Furthermore, the Freidman's and Holm's post hoc tests were conducted to statistically validate the credibility of the results. The statistical test results revealed that the proposed technique is significantly faster, compared to the standard SVM and some existing instance selection techniques, in all cases. [ABSTRACT FROM AUTHOR]

Several code smell detection tools have been developed providing different results, because smells can be subjectively interpreted, and hence detected, in different ways. In this paper, we perform the largest experiment of applying machine learning algorithms to code smells to the best of our knowledge. We experiment 16 different machine-learning algorithms on four code smells (Data Class, Large Class, Feature Envy, Long Method) and 74 software systems, with 1986 manually validated code smell samples. We found that all algorithms achieved high performances in the cross-validation data set, yet the highest performances were obtained by J48 and Random Forest, while the worst performance were achieved by support vector machines. However, the lower prevalence of code smells, i.e., imbalanced data, in the entire data set caused varying performances that need to be addressed in the future studies. We conclude that the application of machine learning to the detection of these code smells can provide high accuracy (>96 %), and only a hundred training examples are needed to reach at least 95 % accuracy. [ABSTRACT FROM AUTHOR]

Objective: This study aimed to evaluate the usability of morphometric features obtained from mandibular panoramic radiographs in gender determination using machine learning algorithms. Materials and methods: High-resolution radiographs of 200 patients aged 20–77 (41.0 ± 12.7) were included in the study. Twelve different morphometric measurements were extracted from each digital panoramic radiography included in the study. These measurements were used as features in the machine learning phase in which six different machine learning algorithms were used (k-nearest neighbor, decision trees, support vector machines, naive Bayes, linear discrimination analysis, and neural networks). To evaluate the reliability, we have performed tenfold cross-validation and we repeated this 10 times for every classification process. This process enhances the reliability of the results for other datasets. Results: When all 12 features are used together, the accuracy rate is found to be 82.6 ± 0.5%. The classification accuracies are also compared using each feature alone. Three features that give the highest accuracy are coronoid height (80.9 ± 0.9%), condyle height (78.2 ± 0.5%), and ramus height (77.2 ± 0.4%), respectively. When compared to the classification algorithms, the highest accuracy was obtained with the naive Bayes algorithm with a rate of 84.0 ± 0.4%. Conclusion: Machine learning techniques can accurately determine gender by analyzing mandibular morphometric structures from digital panoramic radiographs. The most precise results are achieved by evaluating the structures in combination, using attributes obtained from applying the MRMR algorithm to all features. [ABSTRACT FROM AUTHOR]

Movie trailers are usually extracted from the most exciting, interesting, or other noteworthy parts of the movies in order to attract the audience and persuade them to see the film. At present, hand-crafted movie trailers currently occupy almost all the filming market, which is costly and time-consuming. In this paper, we propose an embedded learning algorithm to generate movie trailers automatically without human interventions. Firstly, we use CNN to extract features of candidate frames from the film by a rank-tracing technique. Secondly, SURF algorithm is utilized to match the frames of the movie with the corresponding trailer, thus the labeled and unlabeled dataset are prepared. Thirdly, the mutual information theory is introduced into the embedded machine learning to formulate a new embedded classification algorithm and hence characterize similar key elements of the trailers. Finally, semi-supervised support vector machine is applied as the classifier to obtain the satisfactory key frames to produce the predicted trailers. By treating several famous movies and their manual handling trailers as the ground-truth, series of experiments are carried out, which indicate that our method is feasible and competitive, providing a good potential for promoting the rapid development of the film industry in terms of publicity, as well as providing users with possible solutions for filtering large amounts of Internet videos. [ABSTRACT FROM AUTHOR]

In many real-world applications in the areas of data mining, the distributions of testing data are different from that of training data. And on the other hand, many data are often represented by multiple views which are of importance to learning. However, little work has been done for it. In this paper, we explored to leverage the multi-view information across different domains for knowledge transfer. We proposed a novel transfer learning model which integrates the domain distance and view consistency into a 2-view support vector machine framework, namely DV2S. The objective of DV2S is to find the optimal feature mapping such that under the projections the classification margin is maximized, while both the domain distance and the disagreement between multiple views are minimized simultaneously. Experiments showed that DV2S outperforms a variety of state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Choosing multiple hyperparameters for support vector machines has gained wide attention from researchers and this also provides a strategy for automatically selecting scaling factors of features. This paper proposes an efficient feature scaling method for support vector machines with a quadratic kernel. The proposed method alternately performs the standard SVM algorithm and the eigen-decomposition until some criteria are met. It is interesting to note that scaling factors of features can be analytically obtained for fixed support vectors. The experiments on a toy example, UCI data sets, and face images are carried out to demonstrate the feasibility and effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Relevance ranking has been a popular and interesting topic over the years, which has a large variety of applications. A number of machine learning techniques were successfully applied as the learning algorithms for relevance ranking, including neural network, regularized least square, support vector machine and so on. From machine learning point of view, extreme learning machine actually provides a unified framework where the aforementioned algorithms can be considered as special cases. In this paper, pointwise ELM and pairwise ELM are proposed to learn relevance ranking problems for the first time. In particular, ELM type of linear random node is newly proposed together with kernel version of ELM to be linear as well. The famous publicly available dataset collection LETOR is tested to compare ELM-based ranking algorithms with state-of-art linear ranking algorithms. [ABSTRACT FROM AUTHOR]

Twin support vector machine (TWSVM) has gained increasing interest from various research fields recently. In this paper, we aim to report the current state of the theoretical research and practical advances on TWSVM. We first give the basic thought and theory of TWSVM, including the theory of proximal support vector machine, generalized eigenvalue proximal support vector machine, and TWSVM. Then, we focus on the various improvements made to TWSVM, mainly including least squares twin support vector machine, smooth twin support vector machine, regularized twin support vector machine, projection twin support vector machine, and modified TWSVM on the model selection problem. These newly emerging algorithms greatly expand the applications of TWSVM. In recent years, there is a lot of research on application of TWSVM. Next, we list some research on application of TWSVM in detail. Finally, we try to provide a comprehensive view of these advances in TWSVM and discuss the direction of future research. [ABSTRACT FROM AUTHOR]

Most state-of-the-art speaker recognition systems are based on discriminative learning approaches. On the other hand, generative Gaussian mixture models (GMM) have been widely used in speaker recognition during the last decades. In an earlier work, we proposed an algorithm for discriminative training of GMM with diagonal covariances under a large margin criterion. In this paper, we propose an improvement of this algorithm, which has the major advantage of being computationally highly efficient, thus well suited to handle large-scale databases. We also develop a new strategy to detect and handle the outliers that occur in the training data. To evaluate the performances of our new algorithm, we carry out full NIST speaker identification and verification tasks using NIST-SRE'2006 data, in a Symmetrical Factor Analysis compensation scheme. The results show that our system significantly outperforms the traditional discriminative support vector machines (SVM)-based system of SVM-GMM supervectors, in the two speaker recognition tasks. [ABSTRACT FROM AUTHOR]

The Support Vector Machine (SVM) is an interesting classifier with excellent power of generalization. In this paper, we consider applying the SVM to semi-supervised learning. We propose using an additional criterion with the standard formulation of the semi-supervised SVM ( S VM) to reinforce classifier regularization. Since, we deal with nonconvex and combinatorial problem, we use a genetic algorithm to optimize the objective function. Furthermore, we design the specific genetic operators and certain heuristics in order to improve the optimization task. We tested our algorithm on both artificial and real data and found that it gives promising results in comparison with classical optimization techniques proposed in literature. [ABSTRACT FROM AUTHOR]

Recently, learning to rank technology is attracting increasing attention from both academia and industry in the areas of machine learning and information retrieval. A number of algorithms have been proposed to rank documents according to the user-given query using a human-labeled training dataset. A basic assumption behind general learning to rank algorithms is that the training and test data are drawn from the same data distribution. However, this assumption does not always hold true in real world applications. For example, it can be violated when the labeled training data become outdated or originally come from another domain different from its counterpart of test data. Such situations bring a new problem, which we define as cross domain learning to rank. In this paper, we aim at improving the learning of a ranking model in target domain by leveraging knowledge from the outdated or out-of-domain data (both are referred to as source domain data). We first give a formal definition of the cross domain learning to rank problem. Following this, two novel methods are proposed to conduct knowledge transfer at feature level and instance level, respectively. These two methods both utilize Ranking SVM as the basic learner. In the experiments, we evaluate these two methods using data from benchmark datasets for document retrieval. The results show that the feature-level transfer method performs better with steady improvements over baseline approaches across different datasets, while the instance-level transfer method comes out with varying performance depending on the dataset used. [ABSTRACT FROM AUTHOR]

We propose a computationally efficient method for cross-validation of the Support Vector Regression (SVR) by generalizing the decremental algorithm of SVR. Incremental and decremental algorithm of Support Vector Machines (SVM)) efficiently update the trained SVM model when a single data point is added to or removed from the training set. The computational cost of leave-one-out cross-validation can be reduced using the decremental algorithm. However, when we perform leave-m-out cross-validation (m>1), we have to repeatedly apply the decremental algorithm for each data point. In this paper, we extend the decremental algorithm of SVR ) in such a way that several data points can be removed more efficiently. Experimental results indicate that the proposed approach can reduce the computational cost. In particular, we observed that the number of breakpoints, which is the main computational cost of the involved path-following, were reduced from (O(m) to O(√m). [ABSTRACT FROM AUTHOR]

In recent years, computational diagnostic tools and artificial intelligence techniques provide automated procedures for objective judgments by making use of quantitative measures and machine learning techniques. In this paper we propose a Support Vector Machines (SVMs) based classifier in comparison with Bayesian classifiers and Artificial Neural Networks for the prognosis and diagnosis of breast cancer disease. The paper provides the implementation details along with the corresponding results for all the assessed classifiers. Several comparative studies have been carried out concerning both the prognosis and diagnosis problem demonstrating the superiority of the proposed SVM algorithm in terms of sensitivity, specificity and accuracy. [ABSTRACT FROM AUTHOR]

Convex optimization methods are used for many machine learning models such as support vector machine. However, the requirement of a convex formulation can place limitations on machine learning models. In recent years, a number of machine learning methods not requiring convexity have emerged. In this paper, we study non-convex optimization problems on the Stiefel manifold in which the feasible set consists of a set of rectangular matrices with orthonormal column vectors. We present examples of non-convex optimization problems in machine learning and apply three nonlinear optimization methods for finding a local optimal solution; geometric gradient descent method, augmented Lagrangian method of multipliers, and alternating direction method of multipliers. Although the geometric gradient method is often used to solve non-convex optimization problems on the Stiefel manifold, we show that the alternating direction method of multipliers generally produces higher quality numerical solutions within a reasonable computation time. [ABSTRACT FROM AUTHOR]

Background: Previous studies have developed the Migraine Aura Complexity Score (MACS) system. MACS shows great potential in studying the complexity of migraine with aura (MwA) pathophysiology especially when implemented in neuroimaging studies. The use of sophisticated machine learning (ML) algorithms, together with deep profiling of MwA, could bring new knowledge in this field. We aimed to test several ML algorithms to study the potential of structural cortical features for predicting the MACS and therefore gain a better insight into MwA pathophysiology. Methods: The data set used in this research consists of 340 MRI features collected from 40 MwA patients. Average MACS score was obtained for each subject. Feature selection for ML models was performed using several approaches, including a correlation test and a wrapper feature selection methodology. Regression was performed with the Support Vector Machine (SVM), Linear Regression, and Radial Basis Function network. Results: SVM achieved a 0.89 coefficient of determination score with a wrapper feature selection. The results suggest a set of cortical features, located mostly in the parietal and temporal lobes, that show changes in MwA patients depending on aura complexity. Conclusions: The SVM algorithm demonstrated the best potential in average MACS prediction when using a wrapper feature selection methodology. The proposed method achieved promising results in determining MwA complexity, which can provide a basis for future MwA studies and the development of MwA diagnosis and treatment. [ABSTRACT FROM AUTHOR]

Multiple instance learning (MIL) is a binary classification problem with loosely supervised data where a class label is assigned only to a bag of instances indicating presence/absence of positive instances. In this paper we introduce a novel MIL algorithm using Gaussian processes (GP). The bag labeling protocol of the MIL can be effectively modeled by the sigmoid likelihood through the max function over GP latent variables. As the non-continuous max function makes exact GP inference and learning infeasible, we propose two approximations: the soft-max approximation and the introduction of witness indicator variables. Compared to the state-of-the-art MIL approaches, especially those based on the Support Vector Machine, our model enjoys two most crucial benefits: (i) the kernel parameters can be learned in a principled manner, thus avoiding grid search and being able to exploit a variety of kernel families with complex forms, and (ii) the efficient gradient search for kernel parameter learning effectively leads to feature selection to extract most relevant features while discarding noise. We demonstrate that our approaches attain superior or comparable performance to existing methods on several real-world MIL datasets including large-scale content-based image retrieval problems. [ABSTRACT FROM AUTHOR]

Motivated by the support vector data description, a classical one-class support vector machine, and the twin support vector machine classifier, this paper formulates a twin support vector hypersphere (TSVH) classifier, a novel binary support vector machine (SVM) classifier that determines a pair of hyperspheres by solving two related SVM-type quadratic programming problems, each of which is smaller than that of a conventional SVM, which means that this TSVH is more efficient than the classical SVM. In addition, the TSVH successfully avoids matrix inversion compared with the twin support vector machine, which indicates learning algorithms of the SVM can be easily extended to this TSVH. Computational results on several synthetic as well as benchmark data sets indicate that the proposed TSVH is not only faster, but also obtains better generalization. [ABSTRACT FROM AUTHOR]

In this paper, a simple and linearly convergent Lagrangian support vector machine algorithm for the dual of the twin support vector regression (TSVR) is proposed. Though at the outset the algorithm requires inverse of matrices, it has been shown that they would be obtained by performing matrix subtraction of the identity matrix by a scalar multiple of inverse of a positive semi-definite matrix that arises in the original formulation of TSVR. The algorithm can be easily implemented and does not need any optimization packages. To demonstrate its effectiveness, experiments were performed on well-known synthetic and real-world datasets. Similar or better generalization performance of the proposed method in less training time in comparison with the standard and twin support vector regression methods clearly exhibits its suitability and applicability. [ABSTRACT FROM AUTHOR]

In this paper, extreme learning machine (ELM) for ε-insensitive error loss function-based regression problem formulated in 2-norm as an unconstrained optimization problem in primal variables is proposed. Since the objective function of this unconstrained optimization problem is not twice differentiable, the popular generalized Hessian matrix and smoothing approaches are considered which lead to optimization problems whose solutions are determined using fast Newton-Armijo algorithm. The main advantage of the algorithm is that at each iteration, a system of linear equations is solved. By performing numerical experiments on a number of interesting synthetic and real-world datasets, the results of the proposed method are compared with that of ELM using additive and radial basis function hidden nodes and of support vector regression (SVR) using Gaussian kernel. Similar or better generalization performance of the proposed method on the test data in comparable computational time over ELM and SVR clearly illustrates its efficiency and applicability. [ABSTRACT FROM AUTHOR]

This paper presents a new classification method utilizing distance-based decision surface with nearest neighbor projection approach, called DDC. Kernel type of DDC has been extended to take into account the effective nonlinear structure of the data. DDC has some properties: (1) does not need conventional learning procedure (as k-NN algorithm), (2) does not need searching time to locate the k-nearest neighbors, and (3) does not need optimization process unlike some classification methods such as Support Vector Machine (SVM). In DDC, we compute the weighted average of distances to all the training samples. Unclassified sample will be classified as belonging to a class that has the minimum obtained distance. As a result, by such a rule we can derive a formula that can be used as the decision surface. DDC is tested on both synthetic and real-world data sets from the UCI repository, and the results were compared with k-NN, RBF Network, and SVM. The experimental results indicate DDC outperforms k-NN in the most experiments and the results are comparable to or better than SVM with some data sets. [ABSTRACT FROM AUTHOR]

Twin support vector machine (TSVM) is a new machine learning algorithm, which aims at finding two nonparallel planes for each class. In order to do so, one needs to resolve a pair of smaller-sized quadratic programming problems (QPPs) rather than a single large one. However, when constructing the classification plane for one class, a large number of samples of this class are considered in the objective function, but only fewer samples in the other class are considered, which easily results in over-fitting problem. In addition, the same penalties are given to each misclassified samples in the TSVM. In fact, the misclassified samples have different effects on the decision of the hyper-plane. In order to overcome these two disadvantages, by introducing the rough set theory into ν-TSVM, we propose a rough margin-based ν-TSVM in this paper. In the proposed algorithm, the different points in the different positions are proposed to have different effects on the separating hyper-plane. We firstly construct rough lower margin, rough upper margin, and rough boundary in the ν-TSVM and then give the different penalties to the different misclassified samples according to their positions. The new classifier can avoid the over-fitting problem to a certain extent. Numerical experiments on one artificial dataset and six benchmark datasets demonstrate the feasibility and validity of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Audio information hiding has attracted more attentions recently. Spread spectrum (SS) technique has developed rapidly in this area due to the advantages of good robustness and immunity to noise attack. Accordingly detecting the SS hiding effectively and verifying the presence of the secrete message are important issues. In this paper we present two steganalysis algorithms for SS hiding. Both the two methods are based on machine learning theory and discrete wavelet transform (DWT). In the algorithm I, we introduce Gaussian mixture model (GMM) and generalize Gaussian distribution (GGD) to character the probability distribution of wavelet sub-band. Then the absolute probability distribution function (PDF) moment is extracted as feature vectors. In the algorithm II, we propose distance metric between GMM and GGD of wavelet sub-band to distinguish cover and stego audio. Four distance metrics (Kullback-Leibler Distance, Bhattacharyya Distance, Earth Mover's Distance, L2 Distance) are calculated as feature vectors. The support vector machine (SVM) classifier is utilized for classification. The experiment results of both two proposed algorithms can achieve better detecting performance. Even when embedding strength gets 0.0005, the correct detection rate can reach up to 90%. Its simplicity and extensibility indicate further application in other audio steganalysis. [ABSTRACT FROM AUTHOR]

The 'Brainstorming' approach presented in this paper is a weighted voting method that can improve the quality of predictions generated by several machine learning (ML) methods. First, an ensemble of heterogeneous ML algorithms is trained on available experimental data, then all solutions are gathered and a consensus is built between them. The final prediction is performed using a voting procedure, whereby the vote of each method is weighted according to a quality coefficient calculated using multivariable linear regression (MLR). The MLR optimization procedure is very fast, therefore no additional computational cost is introduced by using this jury approach. Here, brainstorming is applied to selecting actives from large collections of compounds relating to five diverse biological targets of medicinal interest, namely HIV-reverse transcriptase, cyclooxygenase-2, dihydrofolate reductase, estrogen receptor, and thrombin. The MDL Drug Data Report (MDDR) database was used for selecting known inhibitors for these protein targets, and experimental data was then used to train a set of machine learning methods. The benchmark dataset (available at ) can be used for further testing of various clustering and machine learning methods when predicting the biological activity of compounds. Depending on the protein target, the overall recall value is raised by at least 20% in comparison to any single machine learning method (including ensemble methods like random forest) and unweighted simple majority voting procedures. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we propose a sparse semi-supervised learning method, which combines the large margin approach and the constraint. The main contribution of the paper is to develop an efficient optimization algorithm. The objective function to be minimized in a large margin semi-supervised learning method is non-convex and non-differentiable, and hence special optimization algorithms are required. For this purpose, we develop an optimization algorithm, which is a hybrid of the CCCP and the gradient LASSO algorithm. The advantage of the proposed method over existing semi-supervised learning methods is that it can identify a small number of relevant input variables while keeping the prediction accuracy high. Also, the proposed algorithm is simple enough that it can be applied to various real problems without being much hampered by computational limitations. To confirm these advantages, we compare the proposed method with the standard semi-supervised method by analyzing simulated as well as real data sets. [ABSTRACT FROM AUTHOR]

This paper presents a novel classification approach that integrates fuzzy class association rules and support vector machines. A fuzzy discretization technique based on fuzzy c-means clustering algorithm is employed to transform the training set, particularly quantitative attributes, to a format appropriate for association rule mining. A hill-climbing procedure is adapted for automatic thresholds adjustment and fuzzy class association rules are mined accordingly. The compatibility between the generated rules and fuzzy patterns is considered to construct a set of feature vectors, which are used to generate a classifier. The reported test results show that compatibility rule-based feature vectors present a highly- qualified source of discrimination knowledge that can substantially impact the prediction power of the final classifier. In order to evaluate the applicability of the proposed method to a variety of domains, it is also utilized for the popular task of gene expression classification. Further, we show how this method provide biologists with an accurate and more understandable classifier model compared to other machine learning techniques. [ABSTRACT FROM AUTHOR]

Ordinal regression is one of the most important tasks of relation learning, and several techniques based on support vector machines (SVMs) have also been proposed for tackling it, but the scalability aspect of these approaches to handle large datasets still needs much of exploration. In this paper, we will extend the recent proposed algorithm Core Vector Machine (CVM) to the ordinal-class data, and propose a new algorithm named as Ordinal-Class Core Vector Machine (OCVM). Similar with CVM, its asymptotic time complexity is linear with the number of training samples, while the space complexity is independent with the number of training samples. We also give some analysis for OCVM, which mainly includes two parts, the first one shows that OCVM can guarantee that the biases are unique and properly ordered under some situation; the second one illustrates the approximate convergence of the solution from the viewpoints of objective function and KKT conditions. Experiments on several synthetic and real world datasets demonstrate that OCVM scales well with the size of the dataset and can achieve comparable generalization performance with existing SVM implementations. [ABSTRACT FROM AUTHOR]

This paper presents a new method for fault classification in series-compensated transmission line using multiclass support vector machine (MCSVM) and multi class extreme learning machine (MCELM). These methods use the information obtained from the wavelet decomposition of fault current signals for fault classification. Using MATLAB simulink, data set has been generated with different types of fault and system variables, which include the fault resistance, fault distance, load angle and fault inception angle. The proposed method has been tested on a 400-kV, 300-km transmission line under variety of fault conditions. The performance of MCSVM and MCELM is compared in terms of training time and classification accuracy. The comparisons have been made for both One-Versus-One and One-Versus-Rest methods of SVMs and ELMs. Results show that MCELMs need less training time compared to MCSVMs, and the classification accuracy of MCELMs is more or less similar to MCSVMs. The feasibility of the proposed methods is also tested on a practical 220-kV series-compensated transmission line, and the results obtained are quite promising. [ABSTRACT FROM AUTHOR]

Abstract  Aiming at the common support vector machine’s biased disadvantage and computational complexity, an unbiased least squares support vector machine (LSSVM) model is proposed in this paper. The model eliminates the bias item of LSSVM by improving the form of structure risk, then the unbiased least squares support vector classifier and the unbiased least squares support vector regression are deduced. Based on this model, we design a new learning algorithm using Cholesky factorization according to the characteristic of kernel function matrix, in this way the calculation of Lagrangian multipliers is greatly simplified. Several experiments on diffenert datasets are carried out, including the common datasets classification, synthetic aperture radar image automatic target recognition and chaotic time series prediction. The experimental results of correct recognition rate and the fitting precision testify that the unbiased LSSVM model has good universal ability and fitting accuracy, better generalization capability and stability, and have a great improvement in learning speed. [ABSTRACT FROM AUTHOR]

Anomaly detection in time-series data is a relevant problem in many fields such as stochastic data analysis, quality assurance, and predictive modeling. Markov models are an effective tool for time-series data analysis. Previous approaches utilizing Markov models incorporate transition matrices (TMs) at varying dimensionalities and resolutions. Other analysis methods treat TMs as vectors for comparison using search algorithms such as the nearest neighbors comparison algorithm, or use TMs to calculate the probability of discrete subsets of time-series data. We propose an analysis method that treats the elements of a TM as random variables, parameterizing them hierarchically. This approach creates a metric for determining the 'normalcy' of a TM generated from a subset of time-series data. The advantages of this novel approach are discussed in terms of computational efficiency, accuracy of anomaly detection, and robustness when analyzing sparse data. Unlike previous approaches, this algorithm is developed with the expectation of sparse TMs. Accounting for this sparseness significantly improves the detection accuracy of the proposed method. Detection rates in a variety of time-series data types range from (97 % TPR, 2.1 % FPR) to (100 % TPR, <0.1 % FPR) with very small sample sizes (20-40 samples) in data with sparse transition probability matrices. [ABSTRACT FROM AUTHOR]