Search

In recent years, the performance of semisupervised learning (SSL) has been theoretically investigated. However, most of this theoretical development has focused on binary classification problems. In this paper, we take it a step further by extending the work of Castelli and Cover to the multiclass paradigm. In particular, we consider the key problem in SSL of classifying an unseen instance x into one of $K$ different classes, using a training data set sampled from a mixture density distribution and composed of $l$ labeled records and $u$ unlabeled examples. Even under the assumption of identifiability of the mixture and having infinite unlabeled examples, labeled records are needed to determine the $K$ decision regions. Therefore, in this paper, we first investigate the minimum number of labeled examples needed to accomplish that task. Then, we propose an optimal multiclass learning algorithm, which is a generalization of the optimal procedure proposed in the literature for binary problems. Finally, we make use of this generalization to study the probability of error when the binary class constraint is relaxed. [ABSTRACT FROM PUBLISHER]

Most feature selection methods first compute a similarity matrix by assigning a fixed value to pairs of objects in the whole data or to pairs of objects in a class or by computing the similarity between two objects from the original data. The similarity matrix is fixed as a constant in the subsequent feature selection process. However, the similarities computed from the original data may be unreliable, because they are affected by noise features. Moreover, the local structure within classes cannot be recovered if the similarities between the pairs of objects in a class are equal. In this paper, we propose a novel local adaptive projection (LAP) framework. Instead of computing fixed similarities before performing feature selection, LAP simultaneously learns an adaptive similarity matrix $\mathbf{S}$ and a projection matrix $\mathbf{W}$ with an iterative method. In each iteration, $\mathbf{S}$ is computed from the projected distance with the learned $\mathbf{W}$ and W is computed with the learned $\mathbf{S}$. Therefore, LAP can learn better projection matrix $\mathbf{W}$ by weakening the effect of noise features with the adaptive similarity matrix. A supervised feature selection with LAP (SLAP) method and an unsupervised feature selection with LAP (ULAP) method are proposed. Experimental results on eight data sets show the superiority of SLAP compared with seven supervised feature selection methods and the superiority of ULAP compared with five unsupervised feature selection methods. [ABSTRACT FROM AUTHOR]

As an emerging research topic, online class imbalance learning often combines the challenges of both class imbalance and concept drift. It deals with data streams having very skewed class distributions, where concept drift may occur. It has recently received increased research attention; however, very little work addresses the combined problem where both class imbalance and concept drift coexist. As the first systematic study of handling concept drift in class-imbalanced data streams, this paper first provides a comprehensive review of current research progress in this field, including current research focuses and open challenges. Then, an in-depth experimental study is performed, with the goal of understanding how to best overcome concept drift in online learning with class imbalance. [ABSTRACT FROM AUTHOR]

This paper presents the development of a motivated learning (ML) agent with symbolic I/O. Our earlier work on the ML agent was enhanced, giving it autonomy for interaction with other agents. Specifically, we equipped the agent with drives and pains that establish its motivations to learn how to respond to desired and undesired events and create related abstract goals. The purpose of this paper is to explore the autonomous development of motivations and memory in agents within a simulated environment. The ML agent has been implemented in a virtual environment created within the NeoAxis game engine. Additionally, to illustrate the benefits of an ML-based agent, we compared the performance of our algorithm against various reinforcement learning (RL) algorithms in a dynamic test scenario, and demonstrated that our ML agent learns better than any of the tested RL agents. [ABSTRACT FROM PUBLISHER]

Although hash function learning algorithms have achieved great success in recent years, most existing hash models are off-line, which are not suitable for processing sequential or online data. To address this problem, this paper proposes an online hash model to accommodate data coming in stream for online learning. Specifically, a new loss function is proposed to measure the similarity loss between a pair of data samples in hamming space. Then, a structured hash model is derived and optimized in a passive-aggressive way. Theoretical analysis on the upper bound of the cumulative loss for the proposed online hash model is provided. Furthermore, we extend our online hashing (OH) from a single model to a multimodel OH that trains multiple models so as to retain diverse OH models in order to avoid biased update. The competitive efficiency and effectiveness of the proposed online hash models are verified through extensive experiments on several large-scale data sets as compared with related hashing methods. [ABSTRACT FROM AUTHOR]

Network representation learning (NRL) has far-reaching effects on data mining research, showing its importance in many real-world applications. NRL, also known as network embedding, aims at preserving graph structures in a low-dimensional space. These learned representations can be used for subsequent machine learning tasks, such as vertex classification, link prediction, and data visualization. Recently, graph convolutional network (GCN)-based models, e.g., GraphSAGE, have drawn a lot of attention for their success in inductive NRL. When conducting unsupervised learning on large-scale graphs, some of these models employ negative sampling (NS) for optimization, which encourages a target vertex to be close to its neighbors while being far from its negative samples. However, NS draws negative vertices through a random pattern or based on the degrees of vertices. Thus, the generated samples could be either highly relevant or completely unrelated to the target vertex. Moreover, as the training goes, the gradient of NS objective calculated with the inner product of the unrelated negative samples and the target vertex may become zero, which will lead to learning inferior representations. To address these problems, we propose an adversarial training method tailored for unsupervised inductive NRL on large networks. For efficiently keeping track of high-quality negative samples, we design a caching scheme with sampling and updating strategies that has a wide exploration of vertex proximity while considering training costs. Besides, the proposed method is adaptive to various existing GCN-based models without significantly complicating their optimization process. Extensive experiments show that our proposed method can achieve better performance compared with the state-of-the-art models. [ABSTRACT FROM AUTHOR]

Feature selection (FS), which identifies the relevant features in a data set to facilitate subsequent data analysis, is a fundamental problem in machine learning and has been widely studied in recent years. Most FS methods rank the features in order of their scores based on a specific criterion and then select the k top-ranked features, where k is the number of desired features. However, these features are usually not the top-k features and may present a suboptimal choice. To address this issue, we propose a novel FS framework in this article to select the exact top-k features in the unsupervised, semisupervised, and supervised scenarios. The new framework utilizes the ℓ0,2-norm as the matrix sparsity constraint rather than its relaxations, such as the ℓ1,2 -norm. Since the ℓ0,2-norm constrained problem is difficult to solve, we transform the discrete ℓ0,2-norm-based constraint into an equivalent 0–1 integer constraint and replace the 0–1 integer constraint with two continuous constraints. The obtained top-k FS framework with two continuous constraints is theoretically equivalent to the ℓ0,2-norm constrained problem and can be optimized by the alternating direction method of multipliers (ADMM). Unsupervised and semisupervised FS methods are developed based on the proposed framework, and extensive experiments on real-world data sets are conducted to demonstrate the effectiveness of the proposed FS framework. [ABSTRACT FROM AUTHOR]

Low-rank matrix completion aims to recover matrices with missing entries and has attracted considerable attention from machine learning researchers. Most of the existing methods, such as weighted nuclear-norm-minimization-based methods and Qatar Riyal (QR)-decomposition-based methods, cannot provide both convergence accuracy and convergence speed. To investigate a fast and accurate completion method, an iterative QR-decomposition-based method is proposed for computing an approximate singular value decomposition. This method can compute the largest $r (r>0)$ singular values of a matrix by iterative QR decomposition. Then, under the framework of matrix trifactorization, a method for computing an approximate SVD based on QR decomposition (CSVD-QR)-based $L_{2,1}$ -norm minimization method (LNM-QR) is proposed for fast matrix completion. Theoretical analysis shows that this QR-decomposition-based method can obtain the same optimal solution as a nuclear norm minimization method, i.e., the $L_{2,1}$ -norm of a submatrix can converge to its nuclear norm. Consequently, an LNM-QR-based iteratively reweighted $L_{2,1}$ -norm minimization method (IRLNM-QR) is proposed to improve the accuracy of LNM-QR. Theoretical analysis shows that IRLNM-QR is as accurate as an iteratively reweighted nuclear norm minimization method, which is much more accurate than the traditional QR-decomposition-based matrix completion methods. Experimental results obtained on both synthetic and real-world visual data sets show that our methods are much faster and more accurate than the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

With the emergence of online social networks, the social network-based recommendation approach is popularly used. The major benefit of this approach is the ability of dealing with the problems with cold-start users. In addition to social networks, user trust information also plays an important role to obtain reliable recommendations. Although matrix factorization (MF) becomes dominant in recommender systems, the recommendation largely relies on the initialization of the user and item latent feature vectors. Aiming at addressing these challenges, we develop a novel trust-based approach for recommendation in social networks. In particular, we attempt to leverage deep learning to determinate the initialization in MF for trust-aware social recommendations and to differentiate the community effect in user’s trusted friendships. A two-phase recommendation process is proposed to utilize deep learning in initialization and to synthesize the users’ interests and their trusted friends’ interests together with the impact of community effect for recommendations. We perform extensive experiments on real-world social network data to demonstrate the accuracy and effectiveness of our proposed approach in comparison with other state-of-the-art methods. [ABSTRACT FROM AUTHOR]