Search

• Two-phase flow regime identification using ML with enhanced interpretability. • Evaluation of various dimensionality reduction techniques to optimize feature representation. • Kernel Fisher discriminant analysis is the best performing dimensionality reduction technique. • AutoML can impartially select the most effective ML classifier. • Our workflow can develop a virtual flow regime map to automate flow regime recognition. Accurate identification of flow regimes is paramount in several industries, especially in chemical and hydrocarbon sectors. This paper describes a comprehensive data-driven workflow for flow regime identification. The workflow encompasses: i) the collection of dynamic pressure signals using an experimentally verified numerical two-phase flow model for three different flow regimes: stratified, slug and annular flow, ii) feature extraction from pressure signals using Discrete Wavelet Transformation (DWT), iii) Evaluation and testing of 12 different Dimensionality Reduction (DR) techniques, iv) the application of an AutoML framework for automated Machine Learning classifier selection among K-Nearest Neighbors, Artificial Neural Networks, Support Vector Machines, Gradient Boosting, Random Forest, and Logistic Regression, with hyper-parameter tuning. Kernel Fisher Discriminant Analysis (KFDA) is the best DR technique, exhibiting superior goodness of clustering, while KNN proved to be the top classifier with an accuracy of 92.5 % and excellent repeatability. The combination of DWT, KFDA and KNN was used to produce a virtual flow regime map. The proposed workflow represents a significant step forward in automating flow regime identification and enhancing the interpretability of ML classifiers, allowing its application to opaque pipes fitted with pressure sensors for achieving flow assurance and automatic monitoring of two-phase flow in various process industries. [ABSTRACT FROM AUTHOR]

Purpose: The purpose of this paper is to achieve high accuracy in forecasting generation reliability by accurately evaluating the reliability of power systems. This study uses the RTS-79 reliability test system to measure the method's effectiveness, using mean absolute percentage error as the performance metrics. Accurate reliability predictions can inform critical decisions related to system design, expansion and maintenance, making this study relevant to power system planning and management. Design/methodology/approach: This paper proposes a novel approach that uses a radial basis kernel function-based support vector regression method to accurately evaluate the reliability of power systems. The approach selects relevant system features and computes loss of load expectation (LOLE) and expected energy not supplied (EENS) using the analytical unit additional algorithm. The proposed method is evaluated under two scenarios, with changes applied to the load demand side or both the generation system and load profile. Findings: The proposed method predicts LOLE and EENS with high accuracy, especially in the first scenario. The results demonstrate the method's effectiveness in forecasting generation reliability. Accurate reliability predictions can inform critical decisions related to system design, expansion and maintenance. Therefore, the findings of this study have significant implications for power system planning and management. Originality/value: What sets this approach apart is the extraction of several features from both the generation and load sides of the power system, representing a unique contribution to the field. [ABSTRACT FROM AUTHOR]

The temporal and spatial scale dependent relation of Convective Available Potential Energy (CAPE) and precipitation is investigated. Using the COSMO-REA6 data set, we ask which of the standard machine learning algorithms: perceptron, support vector machine, decision tree, random forest, k-nearest neighbor and a simple kept deep neural network algorithm can best relate these two variables. Then, we concentrate on decision trees and evaluate the relation of CAPE and precipitation across different scales. We investigate temporal resolutions of 1 hour to 24 hours and horizontal resolutions of 6 km up to 768 km. Regarding ten CAPE and two precipitation classes we find accuracy scores mostly of about 0.7 across all scales. Taking the Dynamic State Index (DSI) as additional predictor into account leads to an overall increase of the scores. We further introduce a theoretical relation of CAPE and precipitation based on the works of Hans Ertel (1933), which will be analyzed in future studies. Today it is natural to tackle complex atmospheric processes using machine learning methods. These data based methods are suggested as additional tool to complement the results gained by the governing equations of atmospheric motion. [ABSTRACT FROM AUTHOR]

Purpose: The bank is termed as an imperative part of the marketing economy. The failure or success of an institution relies on the ability of industries to compute the credit risk. The loan eligibility prediction model utilizes analysis method that adapts past and current information of credit user to make prediction. However, precise loan prediction with risk and assessment analysis is a major challenge in loan eligibility prediction. Design/methodology/approach: This aim of the research technique is to present a new method, namely Social Border Collie Optimization (SBCO)-based deep neuro fuzzy network for loan eligibility prediction. In this method, box cox transformation is employed on input loan data to create the data apt for further processing. The transformed data utilize the wrapper-based feature selection to choose suitable features to boost the performance of loan eligibility calculation. Once the features are chosen, the naive Bayes (NB) is adapted for feature fusion. In NB training, the classifier builds probability index table with the help of input data features and groups values. Here, the testing of NB classifier is done using posterior probability ratio considering conditional probability of normalization constant with class evidence. Finally, the loan eligibility prediction is achieved by deep neuro fuzzy network, which is trained with designed SBCO. Here, the SBCO is devised by combining the social ski driver (SSD) algorithm and Border Collie Optimization (BCO) to produce the most precise result. Findings: The analysis is achieved by accuracy, sensitivity and specificity parameter by. The designed method performs with the highest accuracy of 95%, sensitivity and specificity of 95.4 and 97.3%, when compared to the existing methods, such as fuzzy neural network (Fuzzy NN), multiple partial least squares regression model (Multi_PLS), instance-based entropy fuzzy support vector machine (IEFSVM), deep recurrent neural network (Deep RNN), whale social optimization algorithm-based deep RNN (WSOA-based Deep RNN). Originality/value: This paper devises SBCO-based deep neuro fuzzy network for predicting loan eligibility. Here, the deep neuro fuzzy network is trained with proposed SBCO, which is devised by combining the SSD and BCO to produce most precise result for loan eligibility prediction. [ABSTRACT FROM AUTHOR]

Long-distance high-speed train localization based on distributed optical fiber sensors (DOFS) has been a challenging issue due to the large-scale heterogeneous sensor nodes. It requires a competent localization algorithm to be capable of strong generalization and quick response. This paper proposes a cooperative multi-classifier network (CMCN) for locating HSTs based on heterogeneous DOFS signals by adaptive modeling of the local characteristics. The proposed CMCN is composed of adaptive feature extraction, lightweight base classifiers and spatial boostrap aggregating (SBA). First, the heterogeneous signals are adaptively transformed to an optimal intrinsic mode function for extracting the statistical features of base classifiers. The base classifiers are constructed based on dynamic soft-margin support vector machine to model local characteristics without computationally burdensome kernel functions by introducing a dynamic penalty factor. The factor is automatically initialized by evaluating the regional consistency before training. Furthermore, the SBA estimates the location of HSTs based on the local states of nodes. It can cooperate with base classifiers for enhanced accuracy by searching for the interval with maximum regional consistency. Finally, a trial is conducted in a high-speed railway in China in long-term running of 92 days. The results prove feasibility and accuracy of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Quantum algorithms can enhance machine learning in different aspects. The quantum support vector machine was proposed to improve the performance, in which the Swap Test plays a crucial role in realizing the classification. However, as the Swap Test is destructive, the quantum support vector machine must be repeated in preparing qubits and manipulating operations. This paper proposes a quantum support vector machine based on the amplitude estimation (AE-QSVM) which gets rid of the constraint of repetitive process and saves the quantum resources. At first, a generalized quantum amplitude estimation is introduced in which the initial state can be arbitrary instead of being |0〉. Then, AE-QSVM is trained by the quantum singular value decomposition and a query sample is classified by the generalized quantum amplitude estimation. In AE-QSVM, a high accuracy can be achieved by adding auxiliary qubits instead of repeating the algorithm. The time and space complexity of AE-QSVM are reduced compared with other algorithms. Finally, we ran experiments on the IBM's quantum computer and experimental results demonstrate that classification with a 95% probability of success only uses 12 qubits. • The generalized amplitude estimation can take any quantum state as the initial state. • The complexity of AE-QSVM is reduced compared with other algorithms. • Quantum support vector machine gets rid of the constraint of repetitive process. • Classification is realized on the IBM quantum computer using 12 qubits. [ABSTRACT FROM AUTHOR]

In this paper, we present a new classifier called elastic net twin support vector machine (ETSVM). It resolves two smaller-sized quadratic programming problems (QPPs) similarly to the twin support vector machine (TSVM). The key difference between them is that ETSVM uses elastic net penalty for slack variables, which enhances classification performance. The dual QPPs of the ETSVM do not involve matrix inversion, in contrast to conventional TSVM. As a result, ETSVM can avoid the ill-conditioning case. We theoretically discuss its properties, including exploring the violation tolerance upper bound for the two QPPs of ETSVM. In order to increase computing efficiency, we derive a sequence of safe screening rules using variational inequalities to quicken the ETSVM parameter tuning process (SSR-ETSVM). We compare the proposed algorithm with SVM, nonparallel hyperplane SVM, elastic net SVM, and elastic net nonparallel hyperplane SVM. Numerical experiments confirm the rationality and effectiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

In this study, a conductive hollow rotating conic object (H-RCO) is developed for convection control and thermal management in a 3D partially heated enclosure under uniform magnetic field with nanofluid considering two phase mixture formulation. Analysis is conducted for different parameters of interest as: Rayleigh number (Ra between 10 4 and 10 6), angular rotational speed of the H-RCO (Ω between −60 and 60), Hartmann number (Ha between 0 and 50), expansion ratio (r1 between 1.1 and 2.5) and conductivity ratio (KR between 0.01 and 50). The rotational speed and expansion ratio of the object contributes significantly to the overall performance improvements. At the highest speed of the H-RCO, the average Nusselt number (Nu) rises up to 38% when compared to cases of non-rotating object. When object with highest expansion ratio is used at rotational speed of Ω = − 40 , the average Nu rises by about 36%. The impacts of using magnetic field on the reduction of convective effects are stronger when rotations are active while up to 69% reduction of average Nu is seen at the highest strength. Thermal conductivity of the object at higher speeds contributes slightly to the overall heat transfer. Support vector machine based regression model is used for thermal performance predictions while model with third order polynomial kernel gives the best results as compared to high fidelity 3D computational results. [ABSTRACT FROM AUTHOR]

In the present research, the least squares support vector machine technique is employed to develop a relationship to predict the total useful exergy yield of a hybrid renewable energy system consisting of a building integrated photovoltaic thermal (PVT) system and an earth air heat exchanger (EAHX) system. This system is designed to bring the temperature of the ambient air to the desired temperature and also to supply the required electricity of the building. In the cold months of the year, the ambient air is preheated by passing through the PVT and EAHX systems. In the hot months of the year, the ambient air is pre-cooled by passing through the EAHX system, while the air leaving the building is also used to cool the photovoltaic panels. The electricity produced by photovoltaic panels is used throughout the year to meet the electricity needs of the building. The dimensions of different parts of the PVT and EAHE systems along with air flow rate are considered as the main variables and the annual exergy yield of the hybrid system is considered as the dependent variable. The LSSVM model in terms of (R = 0.9670, RMSE=1,152,165.21, and MAPE=29,264.80) resulted in promising outcomes to simulate the useful exergy. [ABSTRACT FROM AUTHOR]

In today's fuzzy community, the blending of fuzzy model and deep learning has become one hot topic for the development of more sophisticated and high-powered fuzzy systems. In this study, a rule-based deep fuzzy system with nonlinear fuzzy feature transform for data classification named by NFFT-DFSC is proposed, borrowing the "hierarchically stacked thought" originated from deep learning. Firstly, two fuzzy models with nonlinear fuzzy feature transform function and with decision-making function, and their neuro-fuzzy network implementations are proposed. Subsequently, we stack multiple fuzzy models with nonlinear fuzzy feature transform function and the fuzzy model with decision-making function together in a cascade way, which results in the formation of NFFT-DFSC. Here the stacked fuzzy models with nonlinear fuzzy feature transform function snoops and transforms features of raw data continuously to finally obtain the high-level latent fuzzy features, while the fuzzy model with decision-making function completes classification by exploring the multiple prototypes used to represent the high-level latent fuzzy features of each class. Besides, batch normalization operation and mini-batch gradient descent optimization rooted from the training of deep neural networks are also used to enhance the generalization of NFFT-DFSC and learn its parameters in end-to-end manner, respectively. Experiments between NFFT-DFSCs with three nonlinear mapping functions and 15 benchmark classifiers involving gaussian kernel support vector machine (GSVM), Takagi-Sugeno-Kang classifier (TSK), deep neural network (DNN) and state-of-the-art deep TSK ones on synthetic datasets and nineteen diverse real-world datasets demonstrate that the proposed NFFT-DFSC is a more competitive classifier on classification accuracy. [ABSTRACT FROM AUTHOR]

Compared with single-task learning, multi-tasks can obtain better classifiers by the information provided by each task. In the process of multi-task data collection, we always focus on the target task data in the training process, and ignore the non-target task data and unlabeled data that may be contained in the target task. In response to this issue, this paper introduces auxiliary or Universum into semi-supervised multi-task problem, and proposes a multi-task support vector machine (SU-MTLSVM) method based on semi-supervised learning to handle the case where each task contains the labeled, unlabeled, and Universum samples in the training set. This method introduces Universum as prior knowledge and provides high-dimensional information for semi-supervised learning, and builds a unique classifier from a large amount of unlabeled data. We then use KKT conditions and Lagrangian method to optimize the formulation of the model, and get the model parameters. Finally, we collect different data sets in the experiment part, and compare the performance of multiple baselines with the proposed method. Experiments prove that the method proposed in this paper is more effective for multi-task applications. [ABSTRACT FROM AUTHOR]

• A novel hesitant fuzzy set with SVM modeling scheme is proposed for FTSF. • Length-based discretization (LBD) method is proposed to compute number of interval. • Two set of UODs are used to handle the non-determinism in time series data. • FLRs are established using normalized crisp value with its mean of aggregated HFEs. • FLRs are modeled using SVM. Over the last three decades, several researchers have been putting their efforts into developing non-deterministic fuzzy time series (FTS) models using the traditional fuzzy set. However, considering a set of membership values to each element of the time series, the hesitant fuzzy set glorifies the chances to capture the fuzziness and uncertainty due to randomness better than the traditional fuzzy set. Motivated by this, the present study proposes a novel hesitant FTS forecasting model employing a support vector machine (HFTSF-SVM). In this model, to diminish the computational complexity and increase the forecasting accuracy, we have addressed three key issues by proposing three new methods such as (a) a length-based discretization technique is developed to compute the number of intervals, (b) two sets of universe of discourse are determined for the same time series to compute the non-membership and membership value for each observation (c) each observation along with its mean aggregated membership values are adopted to construct the fuzzy logical relationships (FLR). To assess the competency and reliability of the proposed HFTSF-SVM model, six promising FTS models from the recent literature and four promising time series forecasting (TSF) models are considered for comparison using sixteen time series datasets. Extensive statistical analyses of obtained results confirm the supremacy of the proposed HFTSF-SVM model. [ABSTRACT FROM AUTHOR]

Deep neural mapping support vector machine (DNMSVM) has achieved good results in numerous tasks by mapping the input from a low-dimensional space to a high-dimensional space and then using support vector machine for classification. However, it did not consider the connection of different spaces and increased the model parameters. To improve the classification performance while reducing the number of model parameters, we propose a deep Convolutional Cross-connected Kernel Mapping Support Vector Machine framework based on SelectDropout (CCKMSVM-SD). It consists of a feature extraction module and a classification module. The feature extraction module maps the data from low-dimensional to high-dimensional space by fusing the representations of different dimensional spaces through convolutional layers with cross-connections. For some convolutional layers, we use the depthwise separable convolution to replace the original convolution to reduce the number of parameters. Besides, we use SelectDropout to improve its generalization capability. The classification module uses a soft margin support vector machine for classification. The results on three tasks with ten different datasets indicate that CCKMSVM-SD obtains higher classification accuracy than other models with fewer parameters, demonstrating its effectiveness. [ABSTRACT FROM AUTHOR]

• A novel non-parallel bounded support matrix machine (NPBSMM) is proposed. • A constraint norm group (CNG) is constructed, which can suppress negative influence of outliers and enhance robustness. • The dual problem of NPBSMM avoids the calculation of matrix inversion. • Multi-rank left and right projection matrices are employed to realize a better ability of data fitting. At present, the excellent performance of support vector machine (SVM) has made it successfully applied in many fields. However, when SVM is used for two-dimensional matrix data classification, vectorization of these data easily leads to dimension curse and the loss of structural information. Moreover, SVM is sensitive to outliers, which causes the hyperplane to move towards outliers. Therefore, this paper proposes a novel classification method for data in matrix-form, named non-parallel bounded support matrix machine (NPBSMM). In NPBSMM, a constraint norm group (CNG) is constructed and applied to objective function, which can not only suppress the negative impact of outliers on the model, but also make NPBSMM has better sparsity. By constructing CNG, the operation of matrix inversion in dual problem of traditional classification methods is avoided, so NPBSMM is more suitable for solving large-scale data problems. Further, to extract structure information of matrix for modeling, multi-rank left and right projection matrices are employed to establish objective function, which makes NPBSMM has a better ability of data fitting. Experiments performed on three roller bearing fault datasets show that the proposed NPBSMM method has powerful performance and robustness as compared with other typical classification methods. [ABSTRACT FROM AUTHOR]

Feature selection wrapper methods are powerful mechanisms for reducing the complexity of prediction models while preserving and even improving their precision. Meta-heuristic methods, such as multi-objective evolutionary algorithms, are commonly used as search strategies in feature selection wrapper methods since they allow minimizing the cardinality of the attribute subset and simultaneously maximizing the predictive capacity of the model. However, in high-dimensional problems, multi-objective evolutionary algorithms for wrapper-type feature selection may require excessive computational time, sometimes impractical, especially when the learning algorithm has a high computational cost, such as deep learning. To address this drawback, in this paper we propose a multi-surrogate assisted multi-objective evolutionary algorithm for feature selection, specially designed to improve generalization error. The proposed method has been compared with conventional feature selection wrapper methods that use random forest, support vector machine and long short-term memory learning algorithms to evaluate subsets of attributes. The experiments have been carried out with regression and classification problems with time series data for air quality forecasting in the south-east of Spain and for indoor temperature forecasting in a domotic house. The results demonstrate the superiority of the proposed multi-surrogate assisted method over conventional wrapper methods using the same run times. [ABSTRACT FROM AUTHOR]

In this study, we propose an oversampling method called the minority-predictive-probability-based synthetic minority oversampling technique (MPP-SMOTE) for imbalanced learning. First, MPP-SMOTE removes noisy samples from minority classes. Subsequently, it divides minority samples into two types (hard-to-learn and easy-to-learn) by predicting the probability of samples belonging to the minority class. For both sample types, we adopt a divide-and-conquer strategy. We separately calculate the probability of each sample being selected to generate a new synthetic sample. The relative density of a sample in both the majority and minority classes is considered in the method for calculating the selection probability of hard-to-learn samples, and the relative density of a sample in only the minority class is considered in that of easy-to-learn samples. Finally, according to the types and selection probabilities, MPP-SMOTE separately selects samples and generates synthetic samples based on them by using different sample-generation schemes. Experimental results reveal that the proposed method outperforms other oversampling methods in terms of three imbalanced-learning metrics for three common classifiers. According to the results, when a support vector machine classifier is applied, the area under the curve performance of the MPP-SMOTE improves by a factor of 1.44%. [ABSTRACT FROM AUTHOR]

One of the most important hydraulic structures that has been overlooked is culverts. Regular maintenance and inspections are required to ensure that these structures are used effectively and safely. Culvert inspection, on the other hand, takes a significant amount of time and resources, and agencies cannot afford to do it on a regular basis. To overcome physical inspection challenges, this study proposed employing machine learning algorithms to predict the condition of Utah's culverts based on historical data. This study assessed multiclass classification algorithms that had not been used before. The classification machine learning algorithms employed in this study are Random Forest, Decision Tree, Support Vector Machine, k-Nearest Neighbor, and Artificial Neural Network. Also, a combined dataset from four states was used to address Utah's limited culvert inventory data. The final dataset contained 2555 culvert records. Out of the developed prediction models, Random Forest had the highest accuracy of 82%. The findings demonstrated that Random Forests could be used to improve culvert network quality while reducing culvert maintenance expenses for UDOT. The proposed model may be applied to similarly existing culvert inventories located around the nation. [ABSTRACT FROM AUTHOR]

This paper focuses on the use of differential evolution to improve the approximation properties of function approximation models based on fuzzy partitions. Two cases are considered: Fuzzy transform and Fuzzy projection, and the design of hybrid evolutionary fuzzy systems, is studied. Even though function approximation techniques based on fuzzy partitions have been well studied, few papers consider the problem of centroid selection of the basis functions. Thus, in most cases uniform fuzzy partitions are considered. By using an evolutionary algorithm a systematic approach on the selection of the partition, is provided. The optimisation problem involves the determination of the model parameters, which in our case are the fuzzy partition's membership functions' locations. The proposed method is tested on the scattered data approximation problem, in a regression sense, that is given a set of sparse data the latent function is approximated. Numerical studies on one and two-dimensional test functions demonstrate that the evolutionary algorithm based fuzzy projection displays high performance in terms of approximation error. Moreover, the proposed approach shows high approximation capabilities with a small number of basis functions. Comparison results, with uniform fuzzy partition models, neural networks and support vector machines, are provided. [ABSTRACT FROM AUTHOR]

In this paper, we propose a multi-class classification method using kernel supports and a dynamical system under differential privacy. For small datasets, kernel methods, such as kernel support vector machines (SVMs), show good generalization performance with high-dimensional feature mapping. However, kernel SVMs have a fundamental weakness in achieving differential privacy because they construct decision functions based on a subset of the training data called support vectors. Furthermore, multi-class SVMs must decompose the training data into a binary class, which requires multiple accesses to the same training data. To address these limitations, we develop a two-phase classification algorithm based on support vector data description (SVDD). We first generate and prove a differentially private SVDD (DP-SVDD) by perturbing the sphere center in a high-dimensional feature space. Next, we partition the input space using a dynamical system and classify the partitioned regions using a noisy count. The proposed method results in robust, fast, and user-friendly multi-class classification, even on small-sized datasets, where differential privacy performs poorly. [ABSTRACT FROM AUTHOR]

• A novel approach is proposed to evaluate health based on body area networks. • A multi-sign parameter fusion health assessment model is developed based on BPNN. • The effect of the number of nodes and activation function on the model is explored. • The optimized model is validated by comparing it with other machine learning methods. The widespread application of technologies such as the Internet of Things (IoT) and wireless sensors has promoted the development of body area networks (BAN) in the area of intelligent monitoring. However, current health assessment methods based on BAN still have problems such as a high false alarm rate and low efficiency in identifying signs and states, which not only increase the psychological burden of the ward but also bring unnecessary troubles to the medical staff. In response to this problem, this paper proposes a multi-sign parameter fusion health assessment model based on BP neural network (BPNN). Firstly, the blood pressure, heart rate, pulmonary hypertension, respiration rate, blood oxygen, and body temperature are obtained by sensors in real-time, and then these six parameters are fused by the BPNN. In addition, aiming at the problems of slow convergence speed and easy falling into a local minimum in BPNN, the structure of this model is optimized, and the influence of the number of neurons and activation function of the hidden layer on the performance of the model is explored. Results show that when the number of neurons in the hidden layer is 13 and the activation function is Logsit, the performance of the model is optimal. Among them, the recognition accuracy of the model is 95 %, and the running time is 2.798 s. Finally, comparing the recognition results of this model with support vector machines (SVM), genetic BP neural networks (GA-BPNN), and fuzzy neural networks (FNN), it is found that the accuracy of these three methods is 70 %, 70 % and 80 % respectively, which verifies the validity of the model proposed in this paper. [ABSTRACT FROM AUTHOR]

Support vector machine (SVM) is a particularly powerful and flexible supervised learning model that analyzes data for both classification and regression, whose usual algorithm complexity scales polynomially with the dimension of data space and the number of data points. To tackle the big data challenge, a quantum SVM algorithm was proposed, which is claimed to achieve exponential speedup for least squares SVM (LS-SVM). Here, inspired by the quantum SVM algorithm, we present a quantum-inspired classical algorithm for LS-SVM. In our approach, an improved fast sampling technique, namely indirect sampling, is proposed for sampling the kernel matrix and classifying. We first consider the LS-SVM with a linear kernel, and then discuss the generalization of our method to nonlinear kernels. Theoretical analysis shows our algorithm can make classification with arbitrary success probability in logarithmic runtime of both the dimension of data space and the number of data points for low rank, low condition number, and high dimensional data matrix, matching the runtime of the quantum SVM. [ABSTRACT FROM AUTHOR]

Twin support vector machine (TWSVM), which constructs two nonparallel classifying hyperplanes, is widely applied to various fields. However, TWSVM solves two quadratic programming problems (QPPs) separately such that the final classifiers lack consistency and enough prediction accuracy. Moreover, by reason of only considering the 1-norm penalty for slack variables, TWSVM is not well defined in the geometrical view. In this article, we propose a novel elastic net nonparallel hyperplane support vector machine (ENNHSVM), which adopts elastic net penalty for slack variables and constructs two nonparallel separating hyperplanes simultaneously. We further discuss the properties of ENNHSVM theoretically and derive the violation tolerance upper bound to better demonstrate the relative violations of training samples in the same class. In particular, we design a safe screening rule for ENNHSVM to speed up the calculations. We finally compare the performance of ENNHSVM on both synthetic datasets and benchmark datasets with the Lagrangian SVM, the twin parametric-margin SVM, the elastic net SVM, the TWSVM, and the nonparallel hyperplane SVM. [ABSTRACT FROM AUTHOR]

One-shot semantic segmentation poses the challenging task of segmenting object regions from unseen categories with only one annotated example as guidance. Thus, how to effectively construct robust feature representations from the guidance image is crucial to the success of one-shot semantic segmentation. To this end, we propose in this article a simple, yet effective approach named rich embedding features (REFs). Given a reference image accompanied with its annotated mask, our REF constructs rich embedding features of the support object from three perspectives: 1) global embedding to capture the general characteristics; 2) peak embedding to capture the most discriminative information; 3) adaptive embedding to capture the internal long-range dependencies. By combining these informative features, we can easily harvest sufficient and rich guidance even from a single reference image. In addition to REF, we further propose a simple depth-priority context module to obtain useful contextual cues from the query image. This successfully raises the performance of one-shot semantic segmentation to a new level. We conduct experiments on pattern analysis, statical modeling and computational learning (Pascal) visual object classes (VOC) 2012 and common object in context (COCO) to demonstrate the effectiveness of our approach. [ABSTRACT FROM AUTHOR]

The brain–computer interface (BCI) P300 speller analyzes the P300 signals from the brain to achieve direct communication between humans and machines, which can assist patients with severe disabilities to control external machines or robots to complete expected tasks. Therefore, the classification method of P300 signals plays an important role in the development of BCI systems and technologies. In this article, a novel ensemble support vector recurrent neural network (E-SVRNN) framework is proposed and developed to acquire more accurate and efficient electroencephalogram (EEG) signal classification results. First, we construct a support vector machine (SVM) to formulate EEG signals recognizing model. Second, the SVM formulation is transformed into a standard convex quadratic programming (QP) problem. Third, the convex QP problem is solved by combining a varying parameter recurrent neural network (VPRNN) with a penalty function. Experimental results on BCI competition II and BCI competition III datasets demonstrate that the proposed E-SVRNN framework can achieve accuracy rates as high as 100% and 99%, respectively. In addition, the results of comparison experiments verify that the proposed E-SVRNN possesses the best recognition accuracy and information transfer rate (ITR) compared with most of the state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

The feature selection process plays an important role in different fields, particularly in bioinformatics and microarray gene expression data analysis, for choosing discriminative genes from high-dimensional datasets and selecting a subset of highly relevant features with low redundancy that may lead to build improved prediction models. Consequently, this study proposes a new feature selection method that integrates Preordonnances theory in terms of new Relevance and Complementarity criteria introduced here and also connectivity in undirected Weighted Graphs (PCRWG). The method can handle high-dimensional data. PCRWG retains the relevant and complementary features to select effective features in large scale gene datasets. The proposed algorithm operates in two phases: filtering and wrapping. The strength of the first phase is that it is preceded by a step that further reduces the number of predictors by removing those in disagreement with the target based on the new proposed relevance criterion. Then, the proposed heuristic uses the relevance-complementarity ratio between preordonnances to automatically update the compromise rule between relevance and complementarity. In the wrapping phase, the suggested graph-based approach using maximal clique is based on a powerful relevance-complementarity matrix to consolidate edges, two connected interdependent features are complementary to each other, and it is possible to have high discriminative power when they serve as a group. We highlight the fact that existing graph-based feature selection algorithms do not consider relevance and complementarity simultaneously. The experiments were carried out on three simulated scenarios and the thirteen most popular cancer microarray gene datasets. Formally, they are eight binary and five multi-class microarray data. A 10-fold cross validation was used to evaluate the Support Vector Machine (SVM), Naive Bayes (NB) and artificial Neural Network (NN) classifiers. The empirical results demonstrate the high performance of the proposed hybrid approach when compared to the most recently published articles. [ABSTRACT FROM AUTHOR]

Several techniques have long been proposed to enhance text classification performance, such as: classifier ensembles, feature selection, the integration of similarity measures with classifiers, and meta -heuristic algorithms. The integration of similarity measures with machine learning models (ML), however, has not yet received thorough analysis for text classification. As a result, in an effort to thoroughly investigate the impact of similarity measures integration with ML models, this work makes three major contributions: (1) proposing newly-integrated models and presenting benchmarking studies for integration methodology over balanced/imbalanced datasets; (2) offering detailed analysis for dozens of integrated models that are established, and experimentally proven, to significantly outperform state-of-the-art performance. The models' construction used fourteen similarity measures, three knowledge representations (BoW, TFIDF, and Word embedding), and five models (Support Vector Machine, N -Centroid-based Classifier, Multinomial Naïve Bayesian, Convolutional Neural Network, and Artificial Neural Network); and (3) introducing significantly-effective and highly-efficient variations of these five models. The evaluation study has been conducted internally for integrated models against their baselines, and externally against the state-of-the-art models. While the internal evaluation constantly showed a total enhancement rate of 49.3% and 59% over the balanced and imbalanced datasets, respectively, the external evaluation attested to the superiority of the integrated models. [ABSTRACT FROM AUTHOR]

• Propose an integrated method for product ranking through online reviews. • A complete framework to solve the ranking problem is developed. • A data-driven method of applying ER theory to sentiment analysis is proposed. • An MCDM-based method of ranking products with SD and PROMETHEE-II is proposed. • Propose an SMAA-PROMETHEE method for sensitivity analysis of the parameters. Online reviews play an important role in consumers' purchasing decisions. However, many online reviews confuse consumers when they wish to make a purchase but lack experience. To solve the problem of product ranking based on online reviews, two important issues must be addressed: sentiment analysis and product ranking based on multi-criteria decision-making (MCDM) methods. Therefore, this paper proposes an integrated MCDM method for product ranking through online reviews based on evidential reasoning (ER) theory and stochastic dominance (SD) rules. First, online reviews are preprocessed to obtain product attributes and weight values. Then, we use naive Bayes (NB), logistic regression (LR), and support vector machines (SVM) for the sentiment analysis of online reviews, and the results of the three classifiers are aggregated using ER theory. In addition, according to the confidence distribution matrix of sentiment orientations, SD rules are used to determine the stochastic dominance relations between pairwise alternatives for each attribute. Furthermore, we use the stochastic multi-criteria acceptability analysis (SMAA)-PROMETHEE method to obtain the final product ranking results and conduct sensitivity analysis. Finally, a case study on ranking computer products from JD Mall through online reviews is provided to illustrate the validity of the proposed method. [ABSTRACT FROM AUTHOR]

Learning with label proportions (LLP) deals with the problem that the training data are provided as bags, where the label proportions of training bags rather than the labels of individual training instances are accessible. Existing LLP studies assume that the label proportions of all training bags are accessible. However, in many applications, it is time-consuming to mark all training bags with label proportions, which leads to the problem of learning with both marked and unmarked bags, namely, semisupervised LLP (SLLP). In this work, we propose semisupervised proportional support vector machine (SS- $\propto $ SVM), which extends the proportional SVM ($\propto $ SVM) model to its semisupervised version. To the best of our knowledge, SS- $\propto $ SVM is the first attempt to cope with the SLLP problem. Two realizations, alter-SS- $\propto $ SVM and conv-SS- $\propto $ SVM, which are based on alternating optimization and convex relaxation, respectively, are developed to solve the proposed SS- $\propto $ SVM model. Moreover, we design a cutting plane (CP) method to optimize conv-SS- $\propto $ SVM with a guaranteed convergence rate and present a fast accelerated proximal gradient method to solve the multiple kernel learning subproblem in conv-SS- $\propto $ SVM efficiently. Empirical experiments not only justify the superiority of SS- $\propto $ SVM over its supervised counterpart in classification accuracy but also demonstrate the high competitive computational efficiency of the CP optimization of conv-SS- $\propto $ SVM. [ABSTRACT FROM AUTHOR]

Multi-instance learning (MIL) has garnered considerable attention in recent years due to its favorable performance in various scenarios. Nonetheless, most previous studies have implicitly expressed the correlation between instances and bags. Moreover, the importance of negative instances has been largely overlooked. Hence, we seek to present an explicit and intuitively understandable method that can compensate for these deficiencies. In this article, we creatively introduce a metric-based multiconcept MIL approach based on two aspects. First, the triplet-based bag embedding method identifies instance categories and builds attention weights for every instance explicitly. Accordingly, bag embedding is accomplished under the limitation of weak supervision. Second, the developed instance correlation metric approach in the superbag considers the multiconcept issue to boost the model generalization performance. We have designed a rich variety of experiments to demonstrate the performance of our algorithm. The artificial data experiment reveals the interpretability of the proposed network. The results of the comparison experiment confirm that our method shows favorable performance in multiple tasks. Finally, we illustrate the motivation of the presented method by the ablation experiments. [ABSTRACT FROM AUTHOR]

In classification, the use of 0–1 loss is preferable since the minimizer of 0–1 risk leads to the Bayes optimal classifier. However, due to the nonconvexity of 0–1 loss, this optimization problem is NP-hard. Therefore, many convex surrogate loss functions have been adopted. Previous works have shown that if a Bayes-risk consistent loss function is used as a surrogate, the minimizer of the empirical surrogate risk can achieve the Bayes optimal classifier as the sample size tends to infinity. Nevertheless, the comparison of convergence rates of minimizers of different empirical surrogate risks to the Bayes optimal classifier has rarely been studied. Which characterization of the surrogate loss determines its convergence rate to the Bayes optimal classifier? Can we modify the loss function to achieve a faster convergence rate? In this article, we study the convergence rates of empirical surrogate minimizers to the Bayes optimal classifier. Specifically, we introduce the notions of consistency intensity and conductivity to characterize a surrogate loss function and exploit this notion to obtain the rate of convergence from an empirical surrogate risk minimizer to the Bayes optimal classifier, enabling fair comparisons of the excess risks of different surrogate risk minimizers. The main result of this article has practical implications including: 1) showing that hinge loss (SVM) is superior to logistic loss (Logistic regression) and exponential loss (Adaboost) in the sense that its empirical minimizer converges faster to the Bayes optimal classifier and 2) guiding the design of new loss functions to speed up the convergence rate to the Bayes optimal classifier with a data-dependent loss correction method inspired by our theorems. [ABSTRACT FROM AUTHOR]

In this article, anomaly detection is considered for hyperspectral imagery in the Gaussian background with an unknown covariance matrix. The anomaly to be detected occupies multiple pixels with an unknown pattern. Two adaptive detectors are proposed based on the generalized likelihood ratio test design procedure and ad hoc modification of it. Surprisingly, it turns out that the two proposed detectors are equivalent. Analytical expressions are derived for the probability of false alarm of the proposed detector, which exhibits a constant false alarm rate against the noise covariance matrix. Numerical examples using simulated data reveal how some system parameters (e.g., the background data size and pixel number) affect the performance of the proposed detector. Experiments are conducted on five real hyperspectral data sets, demonstrating that the proposed detector achieves better detection performance than its counterparts. [ABSTRACT FROM AUTHOR]

Data stream classification methods exploiting cohesion in a single data stream have demonstrated promising performance. However, scenarios involving multiple data streams are indeed common in practice. They involve several correlated data streams and can be viewed as multi-task data streams. Instead of considering them separately as individual data streams, it is beneficial to leverage the correlations among the multi-task data streams in data stream modeling. In this regard, a novel classification method called double-coupling support vector machines (DC-SVM) is proposed to classify multiple data streams simultaneously, where both the external correlations between multiple data streams and the internal relationships within each individual data stream are considered. Experimental results on synthetic and real-world multi-task data streams show that the proposed method outperforms traditional data stream classification methods. [ABSTRACT FROM AUTHOR]

This work proposes a decision tree (DT)-based method for initializing a dendritic neuron model (DNM). Neural networks become larger and larger, thus consuming more and more computing resources. This calls for a strong need to prune neurons that do not contribute much to their network’s output. Pruning those with low contribution may lead to a loss of accuracy of DNM. Our proposed method is novel because 1) it can reduce the number of dendrites in DNM while improving training efficiency without affecting accuracy and 2) it can select proper initialization weight and threshold of neurons. The Adam algorithm is used to train DNM after its initialization with our proposed DT-based method. To verify its effectiveness, we apply it to seven benchmark datasets. The results show that decision-tree-initialized DNM is significantly better than the original DNM, k-nearest neighbor, support vector machine, back-propagation neural network, and DT classification methods. It exhibits the lowest model complexity and highest training speed without losing any accuracy. The interactions among attributes can also be observed in its dendritic neurons. [ABSTRACT FROM AUTHOR]

Feature selection aims to select strongly relevant features and discard the rest. Recently, embedded feature selection methods, which incorporate feature weights learning into the training process of a classifier, have attracted much attention. However, traditional embedded methods merely focus on the combinatorial optimality of all selected features. They sometimes select the weakly relevant features with satisfactory combination abilities and leave out some strongly relevant features, thereby degrading the generalization performance. To address this issue, we propose a novel embedded framework for feature selection, termed feature selection boosted by unselected features (FSBUF). Specifically, we introduce an extra classifier for unselected features into the traditional embedded model and jointly learn the feature weights to maximize the classification loss of unselected features. As a result, the extra classifier recycles the unselected strongly relevant features to replace the weakly relevant features in the selected feature subset. Our final objective can be formulated as a minimax optimization problem, and we design an effective gradient-based algorithm to solve it. Furthermore, we theoretically prove that the proposed FSBUF is able to improve the generalization ability of traditional embedded feature selection methods. Extensive experiments on synthetic and real-world data sets exhibit the comprehensibility and superior performance of FSBUF. [ABSTRACT FROM AUTHOR]

Purpose: Based on climate issues and carbon emissions, this study aims to promote low-carbon consumption and compel consumers to actively shift to energy-saving appliances. In this big data era, online reviews in social and electronic commerce (e-commerce) websites contain valuable product information, which can facilitate firm business strategies and consumer comparison shopping. This study is designed to advance existing research on energy-saving refrigerators by incorporating machine learning models in the analysis of online reviews to provide valuable suggestions to e-commerce platform managers and manufacturers to effectively understand the psychological cognition of consumers. Design/methodology/approach: This study proposes an online e-commerce review mining and management strategy model based on "data acquisition and cleaning, data mining and analysis and strategy formation" through multiple machine learning methods, namely, Bayes networks, support vector machine (SVM), latent Dirichlet allocation (LDA) and importance–performance analysis (IPA), to help managers. Findings: Based on a case study of one of the largest e-commerce platforms in China, this study linguistically analyzes 29,216 online reviews of energy-saving refrigerators. Results indicate that the energy-saving refrigerator features that consumers are generally satisfied with are, in sequential order, logistics, function, price, outlook, after-sales service, brand, quality and space. This study also identifies ten topics with 100 keywords by analyzing 18 different refrigerator models. Finally, based on the IPA, this study allocates different priorities to the features and provides suggestions from the perspective of consumers, the government and manufacturers. Research limitations/implications: In terms of limitations, future research may focus on the following points. First, the topics identified in this study derive from specific points in time and reviews; thus, the topics may change with the text data. A machine learning-based online review analysis platform could be developed in the future to dynamically improve consumer satisfaction. Moreover, given that consumers' needs may change over time, e-commerce platform types and consumer characteristics, such as user profiles, can be incorporated into the model to effectively analyze trends in consumers' perceived dimensions. Originality/value: This study fills the gap in previous research in this field, which uses small-sample data for qualitative analysis, while integrating management ideas and proposes an online e-commerce review mining and management strategy model based on machine learning methods. Moreover, this study considers how consumers' emotional and thematic preferences for products affect their purchase decision-making from the perspective of their psychological perception and linguistically analyzes online reviews of energy-saving refrigerators using the proposed mining model. Through the improved IPA model, this study provides optimizing strategies to help e-commerce platform managers and manufacturers. [ABSTRACT FROM AUTHOR]

Universum, as third class that does not belong to the positive class and negative class, allows to incorporate the prior knowledge into the learning process. A lot of reaserchers confirmed that Universum is helpful in the supervised and semi-supervised learning. Moreover, Universum has already been introduced into the support vector machine (SVM) to enhance the generalisation performance. The twin support vector machine (TSVM) serves as an updated classification algorithm based on SVM, having a fast calculation speed. So we introduce Universum into TWSVM to improve the generalisation performance in the same way. Furthermore, in order to make the generalisation performance better in complex environment, an adaptive robust Adaboost-based twin support vector machine with universum learning (ARABUTWSVM) is put up in this paper. First of all, universum learning is used in TSVM to settle a matter of universum data. In order to make our method more robust, we use various loss functions selected by adaptive parameter θ and an update metric induced by correntropy method for distance measure. Further, so as to make further improvement of the learning effect of our method, the Adaboost method is embeded to ARABUTWSVM. The comprehensive experimentation has been performed on various datasets compared with previous universum learning methods, which show that ARABUTWSVM is an adaptive and robust method. [ABSTRACT FROM AUTHOR]

• Implied-ratings systems are able to capture timely information from the market to provide early warnings on Credit Rating Agency ratings movements • Employing 2-dimensional market features, bond+CDS, is able to improve the quality of the implied rating systems • Using more sophisticated tools, such Xgboost and Support Vector Machines, is prone to overfitting and does not provide significantly better results than a simple linear-boundary optimizer • The 2-dimensional implied rating system that we propose embodies both simplicity and efficiency, and allows to bypass the debate about which of the two markets should be relied on in order to predict eventual rating migrations. Market-implied ratings gained importance as efficient early warnings of official credit rating migrations. We build a two-dimensional implied rating system based on bonds and CDS spreads, and test its forecast performances on a set of worldwide sovereign issuers. We show that the method efficiently uses the information from both markets, and is able to outperform implied ratings based on recent machine learning techniques. [ABSTRACT FROM AUTHOR]

• Two closely related nature inspired algorithms are evaluated extensively on a function for which nothing can be proved. • Results show that both of them are similar. • The case of randomized hypercomputation is debunked. • Three theses related to nature inspired algorithms and hypercomputation are proposed. • Antithesis to belief, these theses are meant to be proved (or disproved) and not only meant as theses. The intention of this paper is to empirically compare two closely related (and considered novel metaphor based) nature inspired algorithms and then to use these comparisons and other ideas to shed some light on randomized hypercomputation. The Bat and the Novel Bat algorithms are two of the recently created nature inspired algorithms with the claim of novelty in both of them as both of them uses different randomization, but how novel or similar are they in reality remains hazy especially from the practical perspective. Therefore, here I compare both of them on a data dependent, unexplainable real world classification function. Particularly, I create two classification machines, one derived by BA and another derived by the NBA, using weighted liner loss twin support vector machine and compare it with other trailblazing classifiers on the real world UCI machine learning data sets. These machines are, especially compared with each other also. The results show that the formulated machines perform better with respect to other classifiers on more than 80% of comparisons but are extraordinarily similar between themselves, as they agree with each other up to even four decimal places in almost 100% of cases. Moreover, these optimization algorithms form a perfect example of partially random machines which are claimed to be hypercomputational. But, here it can be shown that unless one has an access to the uncomputable input and uses it intelligently one cannot hypercompute. [ABSTRACT FROM AUTHOR]

Margin is an important concept in machine learning; theoretical analyses further reveal that the distribution of margin plays a more critical role than the minimum margin in generalization power. Recently, several approaches have achieved performance breakthroughs by optimizing the margin distribution, but their computational cost, which is usually higher than before, still hinders them to be widely applied. In this article, we propose margin distribution analysis (MDA), which optimizes the margin distribution more simply by maximizing the margin mean and minimizing the margin variance simultaneously. MDA is efficient and resistive to class-imbalance naturally, since its objective distinguishes the margin means of different classes and can be broken up into two linear equations. In practice, it can also cooperate with other frameworks such as reweight-minimization when facing complex circumstances with noise and outliers. Empirical studies validate the superiority of MDA in real-world data sets, and demonstrate that simple approaches can also perform competitively by optimizing margin distribution. [ABSTRACT FROM AUTHOR]

Categorizing aerial photographs with varied weather/lighting conditions and sophisticated geomorphic factors is a key module in autonomous navigation, environmental evaluation, and so on. Previous image recognizers cannot fulfill this task due to three challenges: 1) localizing visually/semantically salient regions within each aerial photograph in a weakly annotated context due to the unaffordable human resources required for pixel-level annotation; 2) aerial photographs are generally with multiple informative attributes (e.g., clarity and reflectivity), and we have to encode them for better aerial photograph modeling; and 3) designing a cross-domain knowledge transferal module to enhance aerial photograph perception since multiresolution aerial photographs are taken asynchronistically and are mutually complementary. To handle the above problems, we propose to optimize aerial photograph’s feature learning by leveraging the low-resolution spatial composition to enhance the deep learning of perceptual features with a high resolution. More specifically, we first extract many BING-based object patches (Cheng et al., 2014) from each aerial photograph. A weakly supervised ranking algorithm selects a few semantically salient ones by seamlessly incorporating multiple aerial photograph attributes. Toward an interpretable aerial photograph recognizer indicative to human visual perception, we construct a gaze shifting path (GSP) by linking the top-ranking object patches and, subsequently, derive the deep GSP feature. Finally, a cross-domain multilabel SVM is formulated to categorize each aerial photograph. It leverages the global feature from low-resolution counterparts to optimize the deep GSP feature from a high-resolution aerial photograph. Comparative results on our compiled million-scale aerial photograph set have demonstrated the competitiveness of our approach. Besides, the eye-tracking experiment has shown that our ranking-based GSPs are over 92% consistent with the real human gaze shifting sequences. [ABSTRACT FROM AUTHOR]

Image ordinal estimation is to predict the ordinal label of a given image, which can be categorized as an ordinal regression (OR) problem. Recent methods formulate an OR problem as a series of binary classification problems. Such methods cannot ensure that the global ordinal relationship is preserved since the relationships among different binary classifiers are neglected. We propose a novel OR approach, termed convolutional OR forest (CORF), for image ordinal estimation, which can integrate OR and differentiable decision trees with a convolutional neural network for obtaining precise and stable global ordinal relationships. The advantages of the proposed CORF are twofold. First, instead of learning a series of binary classifiers independently, the proposed method aims at learning an ordinal distribution for OR by optimizing those binary classifiers simultaneously. Second, the differentiable decision trees in the proposed CORF can be trained together with the ordinal distribution in an end-to-end manner. The effectiveness of the proposed CORF is verified on two image ordinal estimation tasks, i.e., facial age estimation and image esthetic assessment, showing significant improvements and better stability over the state-of-the-art OR methods. [ABSTRACT FROM AUTHOR]

Accurate electroencephalogram (EEG) pattern decoding for specific mental tasks is one of the key steps for the development of brain–computer interface (BCI), which is quite challenging due to the considerably low signal-to-noise ratio of EEG collected at the brain scalp. Machine learning provides a promising technique to optimize EEG patterns toward better decoding accuracy. However, existing algorithms do not effectively explore the underlying data structure capturing the true EEG sample distribution and, hence, can only yield a suboptimal decoding accuracy. To uncover the intrinsic distribution structure of EEG data, we propose a clustering-based multitask feature learning algorithm for improved EEG pattern decoding. Specifically, we perform affinity propagation-based clustering to explore the subclasses (i.e., clusters) in each of the original classes and then assign each subclass a unique label based on a one-versus-all encoding strategy. With the encoded label matrix, we devise a novel multitask learning algorithm by exploiting the subclass relationship to jointly optimize the EEG pattern features from the uncovered subclasses. We then train a linear support vector machine with the optimized features for EEG pattern decoding. Extensive experimental studies are conducted on three EEG data sets to validate the effectiveness of our algorithm in comparison with other state-of-the-art approaches. The improved experimental results demonstrate the outstanding superiority of our algorithm, suggesting its prominent performance for EEG pattern decoding in BCI applications. [ABSTRACT FROM AUTHOR]

Distance metric learning (DML) aims to learn a distance metric to process the data distribution. However, most of the existing methods are $k$ NN DML methods and employ the $k$ NN model to classify the test instances. The drawback of $k$ NN DML is that all training instances need to be accessed and stored to classify the test instances, and the classification performance is influenced by the setting of the nearest neighbor number $k$. To solve these problems, there are several DML methods that employ the SVM model to classify the test instances. However, all of them are nonconvex and the convex support vector DML method has not been explicitly proposed. In this article, we propose a convex model for support vector DML (CSV-DML), which is capable of replacing the $k$ NN model of DML with the SVM model. To make CSV-DML can use the most kernel functions of the existing SVM methods, a nonlinear mapping is used to map the original instances into a feature space. Since the explicit form of nonlinear mapped instances is unknown, the original instances are further transformed into the kernel form, which can be calculated explicitly. CSV-DML is constructed to work directly on the kernel-transformed instances. Specifically, we learn a specific Mahalanobis distance metric from the kernel-transformed training instances and train a DML-based separating hyperplane based on it. An iterated approach is formulated to optimize CSV-DML, which is based on generalized block coordinate descent and can converge to the global optimum. In CSV-DML, since the dimension of kernel-transformed instances is only related to the number of original training instances, we develop a novel parameter reduction scheme for reducing the feature dimension. Extensive experiments show that the proposed CSV-DML method outperforms the previous methods. [ABSTRACT FROM AUTHOR]

Learning classifiers with imbalanced data can be strongly biased toward the majority class. To address this issue, several methods have been proposed using generative adversarial networks (GANs). Existing GAN-based methods, however, do not effectively utilize the relationship between a classifier and a generator. This article proposes a novel three-player structure consisting of a discriminator, a generator, and a classifier, along with decision boundary regularization. Our method is distinctive in which the generator is trained in cooperation with the classifier to provide minority samples that gradually expand the minority decision region, improving performance for imbalanced data classification. The proposed method outperforms the existing methods on real data sets as well as synthetic imbalanced data sets. [ABSTRACT FROM AUTHOR]

Truncated Newton (TN) methods have been a useful technique for large-scale optimization. Instead of obtaining the full Newton direction, a truncated method approximately solves the Newton equation with an inner conjugate gradient (CG) procedure (TNCG for the whole method). These methods have been employed to efficiently solve linear classification problems. However, even in this deeply studied field, various theoretical and numerical aspects were not completely explored. The first contribution of this work is to comprehensively study the global and local convergence when TNCG is applied to linear classification. Because of the lack of twice differentiability under some losses, many past works cannot be applied here. We prove various missing pieces of theory from scratch and clarify many proper references. The second contribution is to study the termination of the CG method. For the first time when TNCG is applied to linear classification, we show that the inner stopping condition strongly affects the convergence speed. We propose using a quadratic stopping criterion to achieve both robustness and efficiency. The third contribution is that of combining the study on inner stopping criteria with that of preconditioning. We discuss how convergence theory is affected by preconditioning and finally propose an effective preconditioned TNCG. [ABSTRACT FROM AUTHOR]

Mitigating label noise is a crucial problem in classification. Noise filtering is an effective method of dealing with label noise which does not need to estimate the noise rate or rely on any loss function. However, most filtering methods focus mainly on binary classification, leaving the more difficult counterpart problem of multiclass classification relatively unexplored. To remedy this deficit, we present a definition for label noise in a multiclass setting and propose a general framework for a novel label noise filtering learning method for multiclass classification. Two examples of noise filtering methods for multiclass classification, multiclass complete random forest (mCRF) and multiclass relative density, are derived from their binary counterparts using our proposed framework. In addition, to optimize the NI_threshold hyperparameter in mCRF, we propose two new optimization methods: a new voting cross-validation method and an adaptive method that employs a 2-means clustering algorithm. Furthermore, we incorporate SMOTE into our label noise filtering learning framework to handle the ubiquitous problem of imbalanced data in multiclass classification. We report experiments on both synthetic data sets and UCI benchmarks to demonstrate our proposed methods are highly robust to label noise in comparison with state-of-the-art baselines. All code and data results are available at https://github.com/syxiaa/Multiclass-Label-Noise-Filtering-Learning. [ABSTRACT FROM AUTHOR]

Both regularized multi-task learning (RMTL) and direct multi-task twin support vector machine (DMTSVM) have shown good performances in dealing with multi-task problems. They all use hyperplanes to realize classification. However, the hyperplane cannot reflect the distribution of data well. Therefore, in this paper, we propose a novel multi-task twin hypersphere support vector machine (MTTHSVM) to solve multi-task classification problems. It will generate two hyperspheres rather than hyperplanes for each task. So, the proposed method could better describe the distribution information of all training samples compared with the existing RMTL and DMTSVM. Based on Hierarchical Bayes theory, MTTHSVM divides the center of each hypersphere into task-specific and task-common parts to better measure the commonality and individuality of tasks. Then the shared information contained in multiple related tasks could be adaptively mined well. Therefore, the prediction accuracy will be improved to some extent. Besides, MTTHSVM is superior to RMTL and DMTSVM in terms of computational efficiency. This is because our MTTHSVM just solves two smaller-sized quadratic programming problems without any matrix inverse operations. Experimental results on one artificial data set, thirty-five benchmark data sets and a real image data set Cifar100 have verified the effectiveness of our method. [ABSTRACT FROM AUTHOR]