Search

• Jointly consider the task dependency and edge cloud environment. • Combine deep reinforcement learning and genetic algorithm. • Generate the initial population of genetic algorithm by DQN. Edge computing can greatly decrease the delay between users and cloud servers, which can significantly improve system service performance. However, it remains challenging for more efficient scheduling and allocation of users' application demands with dependence constraints to edge cloud servers. Due to the randomness of the initial population, traditional intelligent optimization algorithms have poor convergence speed in addressing resource scheduling. Therefore, to minimize the execution time of the application, this paper proposes a hybrid algorithm to solve the resource scheduling problem with parallelism and subtask dependency. To improve the convergence speed of the algorithm, this paper makes full use of the features of deep Q networks (DQN) and genetic algorithms (GA). The initial population of GA is generated using DQN. Finally, to evaluate the effectiveness of our proposed algorithm, this paper selects three real scientific workflows for experiments. The experimental results show that the hybrid algorithm can converge quickly and improve the optimization effect in a short time. [ABSTRACT FROM AUTHOR]

• Using machine learning to recommend appropriate parameters and operators for differential evolution algorithm. • The fitness landscape features are used as a basis for recommending parameters for the optimization problems. • A mutation operator selector based on AdaBoost & decision tree, and a parameter selector based on BP neural network are established for the differential evolution algorithm. Due to the problems of low accuracy and increasing control parameters in the existing parameter adaptive methods of differential evolution (DE) algorithm, in this paper a mutation operator selector and a parameter selector are proposed through Fitness Landscape (FL) analysing. At first, the performance differences of the two categories of mutation operators named DE / b e s t / 1 and DE / c u r r e n t - t o - r a n d / 1 were analyzed on many test problems. Secondly, the relationship between the FL and mutation operator is founded by using ensemble learning and decision tree, and achieved a classifier named mutation operator selector. Thirdly, the relationship between the FL and algorithm parameters is founded by using a neural network, and then a classifier named parameter selector is achieved. Finally, the improved DE algorithm equip with the two selectors is tested on the CEC2017 benchmark set. The results show that the proposed improved DE algorithm is outperforms both the basis DE algorithm and other three state-of-the-arts algorithms. [ABSTRACT FROM AUTHOR]

Automatic clustering and dimension reduction are two of the most intriguing topics in the field of clustering. Affinity propagation (AP) is a representative graph-based clustering algorithm in unsupervised learning. However, extracting features from high-dimensional data and providing satisfactory clustering results is a serious challenge for the AP algorithm. Besides, the clustering performance of the AP algorithm is sensitive to preference. In this paper, an improved affinity propagation based on optimization of preference (APBOP) is proposed for automatic clustering on high-dimensional data. This method is optimized to solve the difficult problem of determining the preference of affinity propagation and the poor clustering effect for non-convex data distribution. First, t-distributed stochastic neighbor embedding is introduced to reduce the dimensionality of the original data to solve the redundancy problem caused by excessively high dimensionality. Second, an improved hybrid equilibrium optimizer based on the crisscross strategy (HEOC) is proposed to optimize preference selection. HEOC introduces the crisscross strategy to enhance local search and convergence efficiency. The benchmark function experiments indicate that the HEOC algorithm has better accuracy and convergence rate than other swarm intelligence algorithms. Simulation experiments on high-dimensional and real-world datasets show that APBOP has better effectiveness. [ABSTRACT FROM AUTHOR]

Various learning problems can be represented as certain canonical forms of orthogonal matrix nearness problems under the unitarily invariant norm. Since varying unitarily invariant norms favor different structured learning of input data, it is crucial to construct a unified scheme to learn encouraging distributions via any unitarily invariant norm. In this paper, we find that the orthogonal matrix nearness problem can be generalized to a common architecture of the unitarily invariant norm minimization, thus further exploit iterative and closed-form solutions. Firstly, we start with several special circumstances of orthogonal matrix nearness problems, where the unitarily invariant norm resorts to Frobenius norm. Secondly, a general scheme for orthogonal matrix nearness problems under any unitarily invariant norm is addressed and the corresponding iterative algorithm is proposed. Thirdly, feature representation problems are formulated as certain forms of orthogonal matrix nearness, indicating a joint framework for learning low-dimensional features with closed-form solutions. Finally, comprehensive experiments on real-world data sets demonstrate the effectiveness of the proposed method against compared state-of-the-art feature representation approaches. In particular, the proposed method gains about 15% improvements of clustering accuracy on BASEHOCK, CNAE and RCV1 data sets than benchmarks, and achieves at least 3% improvements on other data sets. [ABSTRACT FROM AUTHOR]

• Explainable framework for meta-learning. • Efficiency and high causality. • Intervention and counterfactual. With the growing convergence of artificial intelligence and daily life scenarios, the application scenarios for intelligent decision methods are becoming increasingly complex. The development of various machine learning algorithms has benefited all disciplines of study, but choosing which algorithm is most suitable for a certain problem among a large number of algorithms is a challenge that every field must overcome. Another challenge at the practical application level is that machine learning algorithms currently trained with large amounts of data are primarily black-box and uninterpretable. This indicates that these methods pose potential risks and are difficult to rely on, thus hindering their application in sensitive fields such as finance and healthcare. The first challenge can be overcome by using meta-learning to combine data and prior knowledge to efficiently and automatically select the machine learning models. The second challenge remains to be addressed due to the lack of interpretability of traditional meta-learning techniques and deficiencies in transparency and fairness. Achieving the interpretability of meta-learning in autonomous algorithm selection for classification is crucial to balance the need for high accuracy and transparency of machine learning models in practical application scenarios. This paper proposes EFFECT , an interpretable meta-learning framework that can explain the recommendation results of meta-learning algorithm selection and provide a more complete and accurate explanation of the recommendation algorithm's performance on specific datasets combined with business scenarios. Extensive experiments have demonstrated the validity and correctness of this framework. [ABSTRACT FROM AUTHOR]

• Explainable framework for meta-learning. • Efficiency and high causality. • Intervention and counterfactual. With the growing convergence of artificial intelligence and daily life scenarios, the application scenarios for intelligent decision methods are becoming increasingly complex. The development of various machine learning algorithms has benefited all disciplines of study, but choosing which algorithm is most suitable for a certain problem among a large number of algorithms is a challenge that every field must overcome. Another challenge at the practical application level is that machine learning algorithms currently trained with large amounts of data are primarily black-box and uninterpretable. This indicates that these methods pose potential risks and are difficult to rely on, thus hindering their application in sensitive fields such as finance and healthcare. The first challenge can be overcome by using meta-learning to combine data and prior knowledge to efficiently and automatically select the machine learning models. The second challenge remains to be addressed due to the lack of interpretability of traditional meta-learning techniques and deficiencies in transparency and fairness. Achieving the interpretability of meta-learning in autonomous algorithm selection for classification is crucial to balance the need for high accuracy and transparency of machine learning models in practical application scenarios. This paper proposes EFFECT , an interpretable meta-learning framework that can explain the recommendation results of meta-learning algorithm selection and provide a more complete and accurate explanation of the recommendation algorithm's performance on specific datasets combined with business scenarios. Extensive experiments have demonstrated the validity and correctness of this framework. [ABSTRACT FROM AUTHOR]

Supervised Learning (SL) celebrates a lot of research topics in machine learning (ML) and provides a large number of applications in multimedia. The effectiveness and performance of supervised learning are highly dependent on the quality of labeled samples, and poor or mislabeled labels always damage pre-knowledge and produce disastrous results. One of the notable works is the development of a model for detecting noisy labels in large-scale data sets for supervised classification. This paper introduces a new method for noisy label detection that combines the support vector data description (SVDD) model with manifold regularization. The proposed method is able to detect the noisy label in the testing set. In addition, we adopt K nearest neighbors to obtain the K -nearest neighbor graph, and the k -core algorithm is used to select noise nodes by selecting the influential nodes with larger k values in the K -NN graph. Experimental results over UCI data sets show that the manifold regularized SVDD can effectively detect the noise labels data, and presents better performance over most methods in terms of the outlier detection accuracy, and the average accuracy is mostly above 50 % , even 100 %. Moreover, manifold regularized SVDD has good performance in detecting the most influential top- k nodes compared with other methods. [ABSTRACT FROM AUTHOR]

This paper develops a novel insight and procedure that includes a variety of algorithms for finding the best solution in a structured multi-agent system with internal communications and a global purpose. In other words, it finds the optimal communication structure among agents and the optimal policy in this structure. First, a unique reinforcement learning algorithm is proposed to find the optimal policy of each agent in a fixed structure with non-linear function approximators like artificial neural networks (ANN) and with eligibility traces. Secondly, a mechanism is presented to perform self-organization based on the information of the learned policy. Finally, an algorithm that can discover an appropriate inter-structure mapping and then can transfer the previous knowledge to the new structure is developed, which increases the speed of the learning in this new environment after self-organization. This paper is one of the first works that analyzes the problem fully theoretically and devises some algorithms to find the best solution. We use a simplified version of the distributed task allocation problem (DTAP) as our case study. The experimental results verify the stability of our approach and show the high speed of finding the optimal solution as a result of using transfer learning. [ABSTRACT FROM AUTHOR]

• The existing methods may lead to a contradictory in the transitivity. • A novel neighborhood geometric similarity was firstly defined. • Four new kinds of neighborhood similarities were proposed. • A new VPNRS model is constructed, and a novel decision tree algorithm is proposed. The decision tree algorithm has been widely used in data mining and machine learning due to its high accuracy, low computational cost and high interpretability. However, when dealing with the continuous data, the classical decision tree algorithm needs to replace continuous attributes with discretized attributes by the strategy of discretization. Discretization may cause a loss of information structure, which will affect the performance of classification. To tackle this problem, many researchers have proposed different decision tree methods based on variable precision neighborhood rough sets. However, these methods do not consider the geometric structure of neighborhood systems, which may lead to a contradiction in the transitivity of the equivalence relation. In this paper, we first define a novel neighborhood geometric similarity in a neighborhood system from the perspective of geometry. Second, by combining the neighborhood geometric similarity and the neighborhood algebraic similarity, we propose four new kinds of neighborhood similarities, which can solve the contradictory transitivity of the equivalence relation. Third, a variable precision neighborhood rough set model is constructed using the new similarities, and a novel decision tree algorithm is proposed based on this model, where the degree of attribute dependence is used as the partition measure. Experimental results on 14 selected datasets from the UCI Machine Learning Repository show that our algorithm is effective. The average accuracy of our algorithm is over 90%, which is 10% higher than the classical decision tree algorithms, and the number of leaf nodes increases slightly. [ABSTRACT FROM AUTHOR]

In large-scale distributed learning, the direct application of traditional inference is often not feasible, because it may contain multiple themes, such as communication costs, privacy issues, and Byzantine failures. Nowadays, the internet is vulnerable to attacks, and Byzantine failures frequently occur. For copying with Byzantine failures, the paper develops two Byzantine-robust distributed learning algorithms under a framework of communication-efficient surrogate likelihood. In our algorithms, we adopt the δ -approximate compressors, including sign-based operator and top k sparsification, to improve communication efficiency, and an unsophisticated thresholding of local gradient norms to guard against Byzantine failures. For accelerating convergence and achieving an optimal statistical error rate, error feedback is exploited in the second algorithm. The two algorithms are robust to arbitrary adversaries, although Byzantine workers don't adhere to the mandated compression mechanism. We explicitly establish statistical error rates, which imply that our algorithms don't sacrifice the quality of learning, and attain the order-optimal under some settings. In addition, we provide a trade-off between compression and adversary in the presence of Byzantine worker machines. Extensive numerical experiments validate our theoretical results and demonstrate a good performance of our algorithms. [ABSTRACT FROM AUTHOR]

Graph mining has been widely studied to analyze real-world graph properties and applied to various applications. In particular, graph subspace clustering performance, defined as partitioning high-dimensional graph data into several clusters by finding minimum weights for the edges, has been consistently improved by exploiting deep learning algorithms with Euclidean features extracted from Euclidean domains (image datasets). Most subspace clustering algorithms tend to extract features from the Euclidean domain to identify graph characteristics and structures, and hence are limited for real-world data applications in non-Euclidean domains. This paper proposes a self-supervised deep geometric subspace clustering algorithm optimized for non-Euclidean high-dimensional graph data by emphasizing spatial features and geometric structures while simultaneously reducing redundant nodes and edges. Quantitative and qualitative experimental results verified the proposed approach is effective for graph clustering compared with previous state-of-the-art algorithms on public datasets. [ABSTRACT FROM AUTHOR]

Graph mining has been widely studied to analyze real-world graph properties and applied to various applications. In particular, graph subspace clustering performance, defined as partitioning high-dimensional graph data into several clusters by finding minimum weights for the edges, has been consistently improved by exploiting deep learning algorithms with Euclidean features extracted from Euclidean domains (image datasets). Most subspace clustering algorithms tend to extract features from the Euclidean domain to identify graph characteristics and structures, and hence are limited for real-world data applications in non-Euclidean domains. This paper proposes a self-supervised deep geometric subspace clustering algorithm optimized for non-Euclidean high-dimensional graph data by emphasizing spatial features and geometric structures while simultaneously reducing redundant nodes and edges. Quantitative and qualitative experimental results verified the proposed approach is effective for graph clustering compared with previous state-of-the-art algorithms on public datasets. [ABSTRACT FROM AUTHOR]

• Two closely related nature inspired algorithms are evaluated extensively on a function for which nothing can be proved. • Results show that both of them are similar. • The case of randomized hypercomputation is debunked. • Three theses related to nature inspired algorithms and hypercomputation are proposed. • Antithesis to belief, these theses are meant to be proved (or disproved) and not only meant as theses. The intention of this paper is to empirically compare two closely related (and considered novel metaphor based) nature inspired algorithms and then to use these comparisons and other ideas to shed some light on randomized hypercomputation. The Bat and the Novel Bat algorithms are two of the recently created nature inspired algorithms with the claim of novelty in both of them as both of them uses different randomization, but how novel or similar are they in reality remains hazy especially from the practical perspective. Therefore, here I compare both of them on a data dependent, unexplainable real world classification function. Particularly, I create two classification machines, one derived by BA and another derived by the NBA, using weighted liner loss twin support vector machine and compare it with other trailblazing classifiers on the real world UCI machine learning data sets. These machines are, especially compared with each other also. The results show that the formulated machines perform better with respect to other classifiers on more than 80% of comparisons but are extraordinarily similar between themselves, as they agree with each other up to even four decimal places in almost 100% of cases. Moreover, these optimization algorithms form a perfect example of partially random machines which are claimed to be hypercomputational. But, here it can be shown that unless one has an access to the uncomputable input and uses it intelligently one cannot hypercompute. [ABSTRACT FROM AUTHOR]

Multi-label classification is a machine learning task to construct a model for assigning an entity in the dataset to two or more class labels. In order to improve the performance of multi-label classification, a multi-objective feature selection algorithm has been proposed in this paper. Feature selection as a preprocessing task for Multi-label classification problems aims to choose a subset of relevant features. Selecting a small number of high-quality features decreases the computational cost and at the same time maximizes the classification performance. However extreme decreasing the number of features causes the failure of classification. As a result, feature selection has two conflicting objectives, namely, minimizing the classification error and minimizing the number of selected features. This paper proposes a multi-objective optimization algorithm to tackle the multi-label feature selection. The task is to find a set of solutions (a subset of features) in a sophisticated large-scale search space using a reference-based multi-objective optimization method. The proposed algorithm utilizes an opposition-based binary operator to generate more diverse solutions. Injection of extreme point of the Pareto-front is another component of the algorithm which aims to find feature subsets with less classification error. The proposed method is compared with two other existing methods on eight multi-label benchmark datasets. The experimental results show that the proposed method outperforms existing algorithms in terms of various multi-objective evaluation measures, such as Hyper-volume indicator, Pure diversity, Two-set coverage, and Pareto-front proportional contribution. The proposed method leads to get a set of well-distributed trade-off solutions which reach less classification error in comparing with competitors, even with the fewer number of features. [ABSTRACT FROM AUTHOR]

• An end-to-end framework which naturally incorporates KG into the recommender system. • The first KG-based recommender system combining path-based method and propagation-based method. • A method that can automatically discover the most representative user preference is proposed. • Both High-order relationship and local domain features of users/items are learned and aggregated. • The proposed algorithm achieves competitive results on three real-world datasets. Because a knowledge graph's huge amount of item information can help recommender systems develop user/item representations, it has become the most important source of side information. Regardless of the numerous types of user/item representation approaches used in knowledge graph-based recommendation scenarios, they all have problems. In this paper, we propose a knowledge graph-based multi-context-aware recommendation algorithm for learning user/item representations that combines the advantages of path-based and propagation-based methods. A new concept (i.e., rule) is proposed first, which can be a useful way to characterize the user's preferences. Next, based on user-item interactions, an automatic rule discovery algorithm is proposed that can automatically select the most representative user preferences templates in a given recommendation scenario based on the knowledge graph and user behaviors. Then, the learning of high-order connectivity between long-distance user-item pairs is realized according to these templates. After that, a feature representation method of the local neighborhood characteristics of users and items is introduced to compensate for the defect that the path-based method can only catch the high-order connectivity. The experimental results demonstrate MANN's superiority over eight state-of-the-art baselines. [ABSTRACT FROM AUTHOR]

Clustering, as an essential technique in unsupervised learning, plays a pivotal role in the fields of data mining and machine learning. However, the classic K -means clustering algorithm has intrinsic drawbacks such as sensitivity to initial cluster centers, susceptibility to a local optimal solution, and challenges in handling data uncertainty. To address these problems, this paper proposes an artificial hummingbird algorithm (AHA)-based three-way K -means clustering algorithm, called AHA-3WKM. First, AHA is introduced to address the problems of sensitivity to initial cluster centers and local optima. Second, a fitness function of AHA is specifically constructed to find the best initial clustering centers so that the hummingbirds can search for high-quality food sources, i.e., the global optimum cluster centers. Third, a three-way clustering approach is utilized to capture information about data uncertainty. In this way, the results of clustering are divided into three distinct regions based on the relationship between objects and clusters. The experimental results demonstrate that AHA-3WKM has good performance, and enhances the stability and the accuracy of clustering results. • AHA is introduced to address the problems of the sensitivity to initial cluster centers and the proneness to local optima. Hummingbirds are treated as data points, which dynamically update their strategies and effectively find the optimal cluster centers during multiple iterations. • A fitness function is designed based on the clustering principle of "birds of a feather flock together", with the aim of simplifying calculations, which enhances the specificity and practicality of K-means algorithm. • An AHA-based three-way K-means clustering algorithm (i.e., AHA-3WKM) is proposed. The clustering process is initialized with cluster centers optimized by AHA, and the results are represented in three regions, which can capture the uncertainty within the datasets. [ABSTRACT FROM AUTHOR]

In a modern distributed learning framework, the speeds of intra-worker calculation and inter-worker communication may differ by 1,000 times. It is advisable to perform expensive calculations on worker machines and communicate as few rounds as possible. In this paper, we propose three novel distributed sparse learning algorithms (EDSL-ET, EDSL-IBCD, and EDSL-IBCD-ET) in high dimensions, which are efficient in both computation and communication. Algorithm EDSL-ET utilizes the Top k sparse technique to reduce communication costs and adopts error feedback to guarantee convergence. Algorithm EDSL-IBCD greatly reduces communication costs by introducing an independent block coordinate descent method. Algorithm EDSL-IBCD-ET takes advantage of EDSL-ET and EDSL-IBCD algorithms for extra-high dimensional feature learning, which is the best one whether in communication, computation, and stability. We give theoretical guarantees of the three algorithms under mild conditions, which match the performance of the centralized algorithm. Extensive experiments on simulated and real data validate our theoretical analysis and demonstrate that the proposed algorithms perform well with just a few rounds of communications. [ABSTRACT FROM AUTHOR]

K-nearest neighbors search (K-NNS) is a fundamental problem in many areas of machine learning and data mining. In an attempt to solve NNS problems by locality-sensitive hashing (LSH)-based algorithms and avoid the F1-trap, in this paper, we theoretically prove the necessity of maintaining close edges of each dimension, the independence of the partition location and the precision of retrieval. Based on the theories, a new method is proposed for constructing the hash function family. Then, a volume-balancing locality-sensitive hashing (VBLSH) algorithm is implemented. In this algorithm, the problem of constructing the hash function family is transformed into an optimization problem under a fixed number of hypercubes. By solving the optimization problem, different hyperplane intervals are set for each dimension to ensure that the hash functions can evenly segment the data space, where each segmented hypercube has a close volume and close side lengths. Finally, the analysis of the algorithm shows that the VBLSH algorithm has a high index coding efficiency and hash code calculation efficiency. The experimental results show that the VBLSH algorithm can simultaneously take into account the efficiency, precision and recall rate, and avoid the F1-trap; it shows excellent effectiveness on 4 datasets. [ABSTRACT FROM AUTHOR]

• We consider an incremental learning mechanism based on progressive fuzzy three-way concept. • We design a novel classification mechanism and develop its corresponding algorithm for individual recognition. • For the changing individual, a dynamic update mechanism is studied for progressive concepts. • Effectiveness of the proposed mechanism is verified by comparing with classification mechanism. Fuzzy concept has been an important methodology for data analysis, especially in the classification research. Particularly, fuzzy concept could directly process the continuous data through contrasting the numerical data into the membership degree of object to attribute. However, the classical fuzzy concept only focuses on the positive information, that is, the information about membership degree, while ignoring non-membership degree. Meanwhile, since the limitations of individual cognition and cognitive environment, the concept learning is progressive. Inspired by these thoughts, we design an incremental learning mechanism based on progressive fuzzy three-way concept for object classification in dynamic environment. In this paper, the object and attribute learning operators are first defined to obtain fuzzy three-way concept. Then, a progressive fuzzy three-way concept and its corresponding concept space are learned considering the progressive process of concept learning. Moreover, the object classify mechanism and dynamic update mechanism based on the progressive concept space are proposed, and their effectiveness is verified by numerical experiments. Finally, an incremental learning mechanism is further designed for dynamic increased data and compared with other fuzzy classify methods. All the experimental results carried on ten datasets from UCI and KEEL illustrate the proposed learning mechanism is an excellent object classify algorithm. [ABSTRACT FROM AUTHOR]

The data collected from the real world are diverse and include categorical data, numerical data, incomplete data and noisy data. In addition, many real data sets may dynamically vary, and dynamic data display characteristics with multi-dimensional variations. However, for mixed incomplete data systems, most of the existing incremental methods only work well with single-dimensional dynamic data sets and are not suitable for processing specific multi-dimensional variations of objects and attributes. In this paper, we focus on researching dynamic approaches to efficiently update three-way regions based on the simultaneous variations of the object set and the attribute set in an incomplete neighborhood decision system (INDS). First, considering the complexity of data, we utilize matrix approaches to calculate three-way regions of the INDS based on a proposed neighborhood tolerance relation. Then, under the simultaneous addition of the object set and the attribute set in the INDS, we research incremental mechanisms based on the matrix to obtain three-way regions from previous knowledge. Subsequently, an incremental algorithm for updating three-way regions is proposed when the object set and the attribute set are simultaneously added to the INDS. Finally, the results of a series of experiments and comparisons based on UCI data sets show that the performance of the proposed incremental algorithm is much better than that of the traditional static algorithm, the integrated single-dimensional incremental algorithm and the single-level combined incremental algorithm. [ABSTRACT FROM AUTHOR]

In this paper, the optimal output synchronization of heterogeneous multi-agent systems is studied. To overcome the shortcoming of previous methods that require system model, a data-based optimal control policy is proposed for the output synchronization problem. Besides, the intrinsic equivalence relationship between model-based optimal solution and the proposed data-based optimal solution is proved. To solve the proposed control policy, a reinforcement learning algorithm based on measured input-output data is presented, which is different from existing model-free algorithms based on internal state. According to the algorithm, the optimal output synchronization problem of heterogeneous multi-agent systems can be solved when the full-state vector is unavailable. Finally, the effectiveness of proposed algorithm is verified by simulation examples. [ABSTRACT FROM AUTHOR]

Multi-label learning algorithms have significant challenges due to high-dimensional feature space and noises in multi-label datasets. Feature selection methods are effective techniques to deal with these problems. ParetoCluster is an effective multi-label feature selection algorithm based on Pareto dominance and cluster analysis concepts which considers each label an objective function. This algorithm loses its effectiveness to differentiate features when dealing with high labeled datasets and makes most features incomparable (e.g., when most features fall into the first layer). Thus, a cluster analysis criterion in ParetoCluster will play a decisive role in determining the most relevant features. Bearing this in mind, in this paper, we have modeled the multi-label feature selection problem into a bi-objective optimization problem regarding the relevancy and redundancy degree of the features. We then handle it using Pareto dominance in this bi-objective domain. To illustrate the optimality and efficiency of the proposed method, we have compared our approach against some similar techniques. [ABSTRACT FROM AUTHOR]

Often, machine learning applications have to cope with dynamic environments where data are collected in the form of continuous data streams with potentially infinite length and transient behavior. Compared to traditional (batch) data mining, stream processing algorithms have additional requirements regarding computational resources and adaptability to data evolution. They must process instances incrementally because the data's continuous flow prohibits storing data for multiple passes. Ensemble learning achieved remarkable predictive performance in this scenario. Implemented as a set of (several) individual classifiers, ensembles are naturally amendable for task parallelism. However, the incremental learning and dynamic data structures used to capture the concept drift increase the cache misses and hinder the benefit of parallelism. This paper proposes a mini-batching strategy that can improve memory access locality and performance of several ensemble algorithms for stream mining in multi-core environments. With the aid of a formal framework, we demonstrate that mini-batching can significantly decrease the reuse distance (and the number of cache misses). Experiments on six different state-of-the-art ensemble algorithms applying four benchmark datasets with varied characteristics show speedups of up to 5X on 8-core processors. These benefits come at the expense of a small reduction in predictive performance. [ABSTRACT FROM AUTHOR]

[Display omitted] • We propose the TWD-SFNN model to optimize the hidden layer nodes of neural network. • TWD-SFNN adopts three-way decisions to determine the number of hidden layer nodes. • We analyze the space and time complexities of TWD-SFNN. • TWD-SFNN has a more compact network structure than SFNN using empirical formulas. • TWD-SFNN has a better generalization performance than other classification models. Neural networks have a strong self-learning ability and a wide range of applications. The current neural network models mainly determine the number of hidden layer nodes using empirical formulas, which lack theoretical guidance and can easily lead to poor learning performance. To improve the performance of the neural network model, inspired by the three-way decisions method, this paper proposes a model called three-way decisions with a single hidden layer feedforward neural network (TWD-SFNN). TWD-SFNN adopts three-way decisions to find the number of hidden layer nodes for a neural network in a dynamic way. TWD-SFNN has three key issues: discretizing the datasets, adjusting the learning process of the network, and evaluating the learning results of the network. TWD-SFNN adopts the k-means++ algorithm to discretize the datasets, employs the Adam algorithm to adjust the learning process of the network, and uses a confusion matrix to evaluate the learning results of the network. Therefore, the topological structure of the neural network is obtained. The experimental results verify that the network structure of TWD-SFNN is more compact than those of the SFNN models that use empirical formulas to determine the number of hidden layer nodes, and the generalization ability of TWD-SFNN is better than the state-of-the-art classification models. [ABSTRACT FROM AUTHOR]

• Fragile watermarking method for (Deep) Neural Networks for integrity protection. • The method is secure by design. • The performances of the watermarked DNN are not degraded. • Very fast verification of the integrity of the network compatible with real time applications. The last decade has witnessed a massive deployment of Machine Learning tools in everyday life automated tasks. Neural Networks are nowadays in use in a growing number of application areas because of their excellent performances. Unfortunately, it has been shown by many researchers that they can be attacked and fooled in several different ways, and this can dangerously impair their ability to correctly perform their tasks. In this paper we describe a watermarking algorithm that can protect and verify the integrity of (Deep) Neural Networks when deployed in safety critical systems, such as autonomous driving systems or monitoring and surveillance systems. [ABSTRACT FROM AUTHOR]

• A surrogate model based on IRL is suggested, which applies the max-margin method to approximate the reward function. • Under different dynamic environments, IRL learns the reward function and helps accelerate the convergence of the algorithm. • On the basis of learned reward function, IRL-DMOEA generates promising non-dominated solutions during the evolutionary process. Due to the widespread interest in dynamic multiobjective optimization in real-world applications, more and more approaches exploiting machine learning are deployed to tackle this type of problems. Unfortunately, recent works do not make full use of the data obtained during the optimization process, which could be benefit for model training thereby mining the dynamic characteristics of the underlying problem. To address this issue, this paper proposes a dynamic multiobjective evolutionary algorithm driven by inverse reinforcement learning to solve the dynamic multiobjective optimization problems. IRL is widely used to recover the unknown reward function, making it possible to perform at an expert level. The notable features of the proposed algorithm mainly consist of data-driven evolutionary technique, which uses inverse reinforcement learning as a surrogate-assisted model for model training. This design makes full use of the surrogate management strategy based on inverse reinforcement learning to optimize the reward function within a reinforcement learning framework. At the same time, the algorithm can generate a promising policy based on limited training data during the optimization process to achieve better algorithm evolution and guide the search. The experimental results on the benchmark problems validate that the proposed algorithm is effective in dealing with dynamic multiobjective optimization. [ABSTRACT FROM AUTHOR]

For many-objective problems, how to maintain the diversity and convergence of the distribution of the solution set over Pareto front (PF) has always been the research emphasis. In the iteration process, the state of population is critical to improve the level of evolution. Therefore, this paper will use two convergence and diversity indicators to further strengthen the usage of evolutionary state information and propose a dynamic learning strategy. In addition, a dynamic learning strategy based many-objective evolutionary algorithm (MaOEA) is proposed, called dynamic learning evolution algorithm (DLEA), which continuously changes the direction of learning: convergence and diversity in the iteration process. The purpose is to make the algorithm prefer to convergence in the early iteration and prefer to diversity when it is close to PF in the late iteration, so that the convergence and diversity of the final solution set can be well maintained. And then, the performance of DLEA is measured by two indicators. Meanwhile, DLEA will be compared with four state-of-the-art algorithms on the DTLZ and MaF, and its performance will be verified on a many-objective combinatorial problem. And the experimental results and Friedman test show that DLEA has great advantages. [ABSTRACT FROM AUTHOR]

Matrix completion from very limited information is an important machine learning topic, and has received extensive attention in various scientific applications. Matrix completion aims at finding a low-rank matrix to approximate the incomplete data matrix. However, noise in the data matrix may degrade the performance of the existing matrix completion algorithms, especially if there are different types of noise. In this paper, we proposed a robust matrix completion method with column outliers and sparse noise. The incomplete matrix is iteratively divided into low-rank and sparse parts. The ℓ 2 , 1 -norm based objective function makes the recovered matrix keeps a low-rank structure and lets the algorithm robust to column outliers, while the regularization term based on ℓ 1 -norm can alleviate the influence of sparse noise. Besides, a vector completion algorithm has been proposed to help us estimate the missing entries of the out-of-sample vectors. Moreover, the proposed model can be optimized by an efficient iterative re-weighted method, without introducing any additional parameters, while the adaptive weights obtained in the optimization process can help us detect column outliers. Both theoretical analysis and experiments based on synthetic datasets and real world datasets are implemented to validate the performance of the proposed method. [ABSTRACT FROM AUTHOR]

Unsupervised attribute reduction becomes very challenging due to a lack of decision information, which is to select a subset of attributes that can maintain learning ability without decision information. However, most of the existing unsupervised attribute reduction methods are proposed for numerical or nominal attributes, and little research has been done on unsupervised mixed attribute reduction methods. In view of this, this paper proposes a generalized unsupervised mixed attribute reduction model based on fuzzy rough sets. First, based on all single attribute subsets, the significance is defined to indicate the importance of a candidate attribute. Then, a specific fuzzy rough-based unsupervised attribute reduction (FRUAR) algorithm is designed. Finally, the proposed algorithm is compared with the existing algorithms by using thirty public data sets. Experimental results show that the algorithm FRUAR can select fewer attributes to maintain or improve the performance of learning algorithms, and it is suitable for mixed attribute data. [ABSTRACT FROM AUTHOR]

The foremost requirement for a decision forest to achieve better ensemble accuracy is building a set of accurate and diverse individual decision trees as base classifiers. Existing decision forest algorithms mainly differ from each other on how they induce diversity among the decision trees. At the same time, most of the drawbacks of existing algorithms originate from their induction processes of diversity. In this paper, we propose a new decision forest algorithm that is more balanced through effective synchronization between different sources of diversity. The proposed algorithm is balanced theoretically and empirically. We carried out experiments on 25 well-known data sets that are publicly available from the UCI Machine Learning Repository, to perform an extensive empirical evaluation. The experimental results indicate that the proposed algorithm has the best average ensemble accuracy rank of 1.8 compared to its closest competitor at 3.5. Using the Friedman and Bonferroni-Dunn tests, we also show that such an improvement is indeed statistically significant. In addition, the proposed algorithm is found to be competitive in terms of complexity and other relevant parameters. [ABSTRACT FROM AUTHOR]

• Micro-MetaStream selects the best learning algorithm for each example of data streams. • This method relates data characteristics and the performance of regression models over time. • It combines characteristics from training data and raw features from the predictive attributes of each test example. • Predictive performance was improved when compared with two other meta-learning methods. Data stream mining needs to deal with scenarios where data distribution can change over time. As a result, different learning algorithms can be more suitable in different time periods. This paper proposes micro-MetaStream, a meta-learning based method to recommend the most suitable learning algorithm for each new example arriving in a data stream. It is an evolution of MetaStream, which recommends learning algorithms for batches of examples. By using a unitary granularity, micro-MetaStream is able to respond more efficiently to changes in data distribution than its predecessor. The meta-data combines meta-features, characteristics describing recent data, with base-level features, the original variables of the new example. In experiments on real-world regression data streams, micro-metaStream outperformed MetaStream and a baseline method at the meta-level and frequently improved the predictive performance at the base-level. [ABSTRACT FROM AUTHOR]

Class imbalance and class overlapping problems can exist simultaneously in the imbalanced learning. However, most of the existing algorithms mainly focus on the former. Although some recent algorithms focus on the class overlapping problem, they do not effectively identify the overlapping region, resulting in a loss of sample information, and they are always applied to the original samples with low quality. To address these problems, this paper proposes an imbalanced ensemble learning algorithm based on weighted projection clustering grouping and consistent fuzzy sample transformation (PCGDST-IE). Firstly, a weighted projection clustering combination framework (WPCC) guided by Davies-Bouldin clustering effectiveness index (DBI) is designed to obtain high-quality clusters and the clusters are combined to form cross-complete subsets (CCS) with low overlapping. Secondly, a stage-wise hybrid sampling algorithm is designed to realize the de-overlapping and balancing of subsets. Finally, a local–global structure consistency mechanism (LGSCM) is constructed by fuzzy clustering and domain adaption, thereby reducing class overlapping and improving the quality of samples in subsets. Weak classifiers are trained on the balanced subsets, and fused. More than 30 public datasets and over ten representative algorithms are chosen to verify the proposed method. The experimental results show that the PCGDST-IE is significantly better in terms of anti-overlapping, Recall, F1-M, G-M, AUC, and diversity. The major originality of the paper is: (a) proposing the WPCC to realize weighted projection clustering for subsets generation; (b) proposing the SHS to balance class imbalance and overlapping better;(c) proposing the LGSCM for sample transformation to address the quality of subsets; and (d) forming an imbalanced algorithm to better solve the class imbalance and class overlapping problems simultaneously. [ABSTRACT FROM AUTHOR]

Algorithm selection technology, aimed at choosing the most suitable algorithm for a given machine learning task, has achieved significant success in the domain of 2D vision. However, few studies have explored its application to the 3D point cloud domain. The rapid proliferation and development of 3D point cloud classification algorithms underscore the urgent need for exploration in selecting these algorithms. In this paper, we propose a novel meta-learning-based 3D classification approach, termed 3D meta-classification , to address this gap. The approach operates at both the base-level and meta-level phases. At the base level, candidate 3D classification algorithms constantly classify various 3D datasets, recording their classification performance as empirical knowledge. As for the meta-level phase, it specifically tailors the 3D meta-knowledge generator, 3D meta-feature extractor, and 3D meta-database constructor to establish the 3D meta-database, capturing the relationship between the 3D meta-features and empirical knowledge. Leveraging the 3D meta-database, the 3D classification meta-learner trains the meta-model and predicts suitable algorithms for new incoming 3D datasets. Extensive experiments are conducted to select the best-performing classification algorithm for specific 3D datasets from ModelNet40. The results demonstrate the effectiveness of the proposed 3D meta-classification model; the accuracies of one-from-two and one-from-three algorithm selection tasks reach over 98% and 80%, respectively. [ABSTRACT FROM AUTHOR]

Optimal granularity selection is a key issue in rough set, by which decision rules can be generated to assign corresponding decision labels for new samples. The current evaluation criteria of optimal granularity mainly focus on the experimental effect of prediction algorithms, thus ignoring the theoretical relationship between granularity and the generalization ability of algorithms. In this paper, we study the issue overlooked above and propose a novel framework for optimal granularity selection with theoretical guarantee. In our framework, a rough set-based prediction algorithm that incorporates a global confidence as scoring function is introduced, which is characterized by the granularity-based loss function. On the bias, the relationship between granularity and the generalization ability of algorithm is studied by introducing the stability theory in machine learning. The generalization error bound of algorithm is derived as a theoretical guarantee for optimal granularity selection. Finally, a novel optimal granularity selection strategy with theoretical guarantee is proposed, which is combined with existing attribute reduction methods to design the attribute re-reduction algorithms for generate optimal granularity by reducing the relatively unimportant attributes. Numerical experiments verify the rationality of optimal granularity selection framework and prove the effectiveness of the optimal granularity generated by the attribute re-reduction algorithms. [ABSTRACT FROM AUTHOR]

This paper presents an interpretable neural system—termed Evolving Long-term Cognitive Network—for pattern classification. The proposed model was inspired by Fuzzy Cognitive Maps, which are interpretable recurrent neural networks for modeling and simulation. The network architecture is comprised of two neural blocks: a recurrent input layer and an output layer. The input layer is a Long-term Cognitive Network that gets unfolded in the same way as other recurrent neural networks, thus producing a sort of abstract hidden layers. In our model, we can attach meaningful linguistic labels to each neuron since the input neurons correspond to features in a given classification problem and the output neurons correspond to class labels. Moreover, we propose a variant of the backpropagation learning algorithm to compute the required parameters. This algorithm includes two new regularization components that are aimed at obtaining more interpretable knowledge representations. The numerical simulations using 58 datasets show that our model achieves higher prediction rates when compared with traditional white boxes while remaining competitive with the black boxes. Finally, we elaborate on the interpretability of our neural system using a proof of concept. [ABSTRACT FROM AUTHOR]

Ensemble learning is an effective approach for improving the generalization ability of base classifiers. To generate a set of accurate and diverse base classifiers, different data perturbation schemes have been proposed. For instance, Bagging perturbs the training data via bootstrap sampling. However, when a stable learning algorithm (e.g., KNN, Naive Bayes) is used to train base classifiers, the sole perturbation on the training data may not produce diverse base classifiers. In this paper, by using the attribute reduction technology in rough sets, a multi-modal perturbation-based algorithm (called 'E _ EARBS') is proposed for the ensemble of base classifiers. E _ EARBS simultaneously perturbs the feature space, training data and learning parameters, where the relative decision entropy(RDE)-based approximate reducts are used to perturb the feature space, and bootstrap sampling is used to perturb the training data. Experimental results show that E _ EARBS can provide competitive solutions for ensemble learning. [ABSTRACT FROM AUTHOR]

• We devise a reinforcement learning-based framework to relocate the POS more adaptively. • Based on different severity degree of environmental changes, three change response methods are integrated to predict the POS. • The proposed algorithm learns the dynamic environment and determines the appropriate prediction-based strategy. Dynamic Multi-objective Optimization Problem (DMOP) is emerging in recent years as a major real-world optimization problem receiving considerable attention. Tracking the movement of Pareto front efficiently and effectively over time has been a central issue in solving DMOPs. In this paper, a reinforcement learning-based dynamic multi-objective evolutionary algorithm, called RL-DMOEA, which seamlessly integrates reinforcement learning framework and three change response mechanisms, is proposed for solving DMOPs. The proposed algorithm relocates the individuals based on the severity degree of environmental changes, which is estimated through the corresponding changes in the objective space of their decision variables. When identifying different severity degree of environmental changes, the proposed RL-DMOEA approach can learn better evolutionary behaviors from environment information, based on which apply the appropriate response mechanisms. Specifically, these change response mechanisms including the knee-based prediction, center-based prediction and indicator-based local search, are devised to promote both convergence and diversity of the algorithm under different severity of environmental changes. To verify this idea, the proposed RL-DMOEA is evaluated on CEC 2015 test problems involving various problem characteristics. Empirical studies on chosen state-of-the-art designs validate that the proposed RL-DMOEA is effective in addressing the DMOPs. [ABSTRACT FROM AUTHOR]

Hypergraph is popularly used for describing multi-relationships among objects in a unified manner, and spectral clustering is regarded as one of the most effective algorithms for partitioning those objects (vertices) into different communities. However, the traditional spectral clustering for hypergraph (HC) incurs expensive costs in terms of both time and space. In this paper, we propose a framework called GraphLSHC to tackle the scalability problem faced by the large scale hypergraph spectral clustering. In our solution, the hypergraph used in GraphLSHC is expanded into a general format to capture complicated higher-order relationships. Moreover, GraphLSHC is capable to simultaneously partition both vertices and hyperedges according to the "eigen-trick", which provides an approach for reducing the computational complexity of the clustering. To improve the performance further, several hyperedge-based sampling techniques are proposed, which can supplement the sampled matrix with the whole graph information. We also give a theoretical guarantee for the error boundary of the supplement. Several experiments show the superiority of the proposed framework over the state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

The firefly algorithm (FA) is a nature-inspired heuristic optimization algorithm based on the luminescence and attraction behavior of fireflies. Although the FA can effectively solve complex optimization problems, it suffers from premature convergence because of its simple full attraction model. However, in nature, fireflies exhibit luminous behavior to attract mates. The attractiveness between fireflies of opposite sexes depends not only on the light intensity they emit, but also on their individual size, location, and other factors. In the original FA, all fireflies are assumed to be the same, i.e., they have no gender-based difference, which is not true biologically. Therefore, in this paper, we proposed a novel courtship learning (CL) framework to enhance the performance of the FA. In the proposed framework, the population is divided into female and male subpopulations. The female archiving mechanism is adopted to select excellent fireflies, which are assumed to be female fireflies. When the selected male firefly emits light that is less bright than that of the current firefly, a female individual will be selected from the female archive to guide the movement of the selected male firefly. Comprehensive experiments are conducted on the CEC 2013 benchmark set and the proposed CL framework is integrated with other advanced FA variants to verify its effects. Experimental results confirm that the proposed framework significantly enhances the performance of the original FA and advanced FA variants. [ABSTRACT FROM AUTHOR]

• Tackle dynamic incomplete hybrid data based on three-way decision theory. • A Three-Way Neighborhood Decision Model (TWNDM) is proposed. • Matrix-based approaches for computing three-way regions in TWNDM are presented. • Incremental methods for updating three-way regions are proposed when objects vary. • Experiments are conducted to verify the effectiveness of the proposed approaches. In practical applications, there generally exist incomplete hybrid data with heterogeneous and missing features. The complex structures and the fast update of incomplete hybrid data bring a series of challenges for decision making in dynamic data environments. Three-way decisions, as an important cognitive method for analyzing uncertain problems, have been extensively applied into various fields. However, the existing studies rarely focus on exploring three-way decisions with incomplete hybrid information. To tackle this issue, we propose a Three-Way Neighborhood Decision Model (TWNDM) based on the data-driven neighborhood relation in terms of two pseudo-distance functions only satisfying the reflexivity. Considering that the addition and deletion of objects will result in the variation of information granules and decision structures, this paper presents a matrix-based dynamic framework for updating three-way regions (positive, boundary and negative regions) in TWNDM. A novel relation matrix is first constructed by using a pair of values to replace single value in the classical relation matrix. Then, the matrix-based approach for computing the three-way regions is established in the light of the new relation matrix, the decision matrix and the related induced matrices. Moreover, the matrix-based incremental mechanisms and algorithms for the maintenance of the three-way regions are presented when adding and removing objects, respectively. The results of comparative experiments demonstrate that the proposed incremental algorithms can improve the computational performance for maintaining three-way regions in TWNDM compared with the static algorithm. [ABSTRACT FROM AUTHOR]

Quantum machine learning has recently attracted much attention from the community of quantum computing. In this paper, we explore the ability of generative adversarial networks (GANs) based on quantum computing. More specifically, we propose a quantum GAN for generating classical discrete distribution, which has a classical-quantum hybrid architecture and is composed of a parameterized quantum circuit as the generator and a classical neural network as the discriminator. The parameterized quantum circuit only consists of simple one-qubit rotation gates and two-qubit controlled-phase gates that are available in current quantum devices. Our scheme has the following characteristics and potential advantages: (i) It is intrinsically capable of generating discrete data (e.g., text data), while classical GANs are clumsy for this task due to the vanishing gradient problem. (ii) Our scheme avoids the input/output bottlenecks embarrassing most of the existing quantum learning algorithms that either require to encode the classical input data into quantum states, or output a quantum state corresponding to the solution instead of giving the solution itself, which inevitably compromises the speedup of the quantum algorithm. (iii) The probability distribution implicitly given by data samples can be loaded into a quantum state, which may be useful for some further applications. [ABSTRACT FROM AUTHOR]

Reinforcement learning has shown to be profoundly successful at learning optimal policies for simulated environments using distributed training with extensive compute capacity. Model-free reinforcement learning uses the notion of trial and error, where the error is a vital part of learning the agent to behave optimally. In mission-critical, real-world environments, there is little tolerance for failure and can cause damaging effects on humans and equipment. In these environments, current state-of-the-art reinforcement learning approaches are not sufficient to learn optimal control policies safely. On the other hand, model-based reinforcement learning tries to encode environment transition dynamics into a predictive model. The transition dynamics describes the mapping from one state to another, conditioned on an action. If this model is accurate enough, the predictive model is sufficient to train agents for optimal behavior in real environments. This paper presents the Dreaming Variational Autoencoder (DVAE) for safely learning good policies with a significantly lower risk of catastrophes occurring during training. The algorithm combines variational autoencoders, risk-directed exploration, and curiosity to train deep-q networks inside "dream" states. We introduce a novel environment, ASRS-Lab, for research in the safe learning of autonomous vehicles in grid-based warehousing. The work shows that the proposed algorithm has better sample efficiency with similar performance to novel model-free deep reinforcement learning algorithms while maintaining safety during training. [ABSTRACT FROM AUTHOR]

Agglomerative clustering is a mainstream clustering method that can produce an informative hierarchical structure of clusters. Existing similarities in agglomerative clustering are typically based on the pairwise distance. Although this type of similarity captures the local structure of data well, it is sensitive to noise and outliers because it considers only the distance between data points. In this paper, we propose a new similarity called RCPD by combining the reconstruction coefficient, which is robust to noise and outliers, with the pairwise distance for agglomerative clustering. Our new similarity takes advantage of both the distance between data points and the linear representation among data points. Thus, RCPD not only captures the local structure of data well but is also robust to noise and outliers. The experimental results on 11 real-world benchmark datasets show that our new clustering method consistently outperforms many state-of-the-art clustering approaches. [ABSTRACT FROM AUTHOR]

Rule learning is a special type of machine learning approaches, and its key advantage is the generation of interpretable models, which provides a transparent process of showing how an input is mapped to an output. Traditional rule learning algorithms are typically based on Boolean logic for inducing rule antecedents, which are very effective for training models on data sets that involve discrete attributes only. When continuous attributes are present in a data set, traditional rule learning approaches need to employ crisp intervals. However, in reality, problems usually show shades of grey, which motivated the development of fuzzy rule learning approaches by employing fuzzy intervals for handling continuous attributes. While a data set contains a large portion of discrete attributes or even no continuous attributes, fuzzy approaches cannot be used to learn rules effectively, leading to a drop in the performance. In this paper, a multi-stage approach of mixed rule learning is proposed, which involves strategic combination of both traditional and fuzzy approaches to handle effectively various types of attributes. We compare our proposed approach with existing algorithms of rule learning. Our experimental results show that our proposed approach leads to significant advances in the performance compared with the existing algorithms. [ABSTRACT FROM AUTHOR]

Abstract In this paper, we propose a weighted multiple classifier framework based on random projections. Similar to the mechanism of other homogeneous ensemble methods, the base classifiers in our approach are obtained by a learning algorithm on different training sets generated by projecting the original up-space training set to lower dimensional down-spaces. We then apply a Least SquarE−based method to weigh the outputs of the base classifiers so that the contribution of each classifier to the final combined prediction is different. We choose Decision Tree as the learning algorithm in the proposed framework and conduct experiments on a number of real and synthetic datasets. The experimental results indicate that our framework is better than many of the benchmark algorithms, including three homogeneous ensemble methods (Bagging, RotBoost, and Random Subspace), several well-known algorithms (Decision Tree, Random Neural Network, Linear Discriminative Analysis, K Nearest Neighbor, L2-loss Linear Support Vector Machine, and Discriminative Restricted Boltzmann Machine), and random projection-based ensembles with fixed combining rules with regard to both classification error rates and F1 scores. [ABSTRACT FROM AUTHOR]

Highlights • Orthogonal projection is proposed to mitigate the domain shift problem. • Semantic feature representation is included to alleviate visual category ambiguity. • Deep model is applied to improve the performance. • Extensive experiments show the superiority of our algorithm. Abstract To mitigate the problems of visual ambiguity and domain shift in conventional zero-shot learning (ZSL), in this paper, we propose a novel method, namely, dual-verification network (DVN), which accepts features and attributes in a pairwise manner as input and verifies the result in both the attribute and feature spaces. First, the DVN projects a feature onto an orthogonal space, where the projected feature has maximum correlation with its corresponding attribute and is orthogonal to all the other attributes. Second, we adopt the concept of semantic feature representation, which computes the relationship between the semantic feature and class labels. Based on this concept, we project the attributes onto the feature space by extending the attributes and labels from the class level to instance level. In addition, we employ a deep architecture and utilize the cross entropy loss to train an end-to-end network for dual verification. Extensive experiments in ZSL and generalized ZSL are performed on four well-known datasets, and the results show that the proposed DVN exhibits a competitive performance relative to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

As an important preprocessing step for data mining, attribute reduction has become a hot research topic in rough set theory. In practice, many real data may vary dynamically with time, therefore, reduct will change dynamically under the variation of objects and attributes in decision systems. The classical attribute reduction methods need to recompute from scratch, which are ineffective to deal with dynamic decision systems. How to implement updating reducts by utilizing previous results is vital for improving the efficiency of attribute reduction approaches. In the paper, we firstly introduce incremental mechanisms of computing reduct when objects and attributes of the decision system change dynamically. Then, incremental methods are developed to update reduct when attributes and objects increase simultaneously. Finally, a series of experiments are conducted to validate the proposed incremental attribute reduction methods. Experimental results show that they are effective to update reduct with change of attributes and objects in the decision systems. [ABSTRACT FROM AUTHOR]

In multi-objective evolutionary algorithms, one of the focal points is finding a balance between diversity and convergence. In decomposition-based algorithms, the role of weight vectors is crucial. Despite numerous studies dedicated to these aspects, there is a scarcity of utilizing transfer learning algorithms for dual-operator feature fusion and employing neural networks for accurate partitioning of the objective space population. To address the aforementioned issues, this paper proposes the following improvements: (1) Implementing a Balanced Distribution Adaptation (BDA) transfer learning algorithm to achieve dual-operator feature fusion, resulting in a transfer population guiding the adaptive adjustment of weight vectors. (2) Integrating the BDA algorithm with multi-objective algorithms requires labeling the data, a challenge in the multi-objective evolutionary algorithm. To tackle this issue, non-dominated sorting is introduced as a bridge connecting the BDA and multi-objective evolutionary algorithms. This serves as a method to combine the advantages of decomposition-based and Pareto-dominance principle-based multi-objective algorithms. (3) To overcome the impact of traditional Euclidean distance on population sparsity, a neural network is employed to determine the population's distribution in the objective space accurately. This ensures the precise identification of individuals to be removed from the current population and the areas where additions are needed. In order to fully validate the effectiveness of the proposed algorithm, four different sets of experiments are conducted in the experimental section, where three sets of benchmarking problems are compared to a variety of algorithms that have received much attention in recent years, as well as ablation experiments. [ABSTRACT FROM AUTHOR]

As heavy-tailed distributions are ubiquitous in many real applications, robust regression has been extensively applied in machine learning and exhibits the superiority in deal with heavy-tailed distribution. Current existing robust methods for regression are based on independence assumption of features and ignore interaction between them, which may lead to poor generalization due to most datasets unsatisfying the assumption. Actually, interaction features formed by the composition of two or more features are particularly helpful to obtain the high-order information in many applications. In this paper, we propose a novel adaptive robust feature interaction selection model for regression with heavy-tailed distributions, termed Adaptive Robust Feature Interaction Selection (ARFIS). Firstly, we consider pairwise feature interaction by augmenting a feature vector with product of features for regression with heavy tailed distribution. Secondly, we propose feature interaction selection models based on quantile loss with different regularizers to learn parameters. The consistency of the proposed model ARFIS is theoretically proven, and an efficient algorithm is presented for solving proposed model. Finally, experimental results on simulation data, UCI datasets and a real-world dataset validate good accuracy, interpretability and robustness of our proposed models. • Robust models are proposed, which considering features interaction and ignoring the independence assumption of features. • Theoretical analysis guarantees the generalization of the model ARFIS. • Experimental results validate good accuracy, interpretability and robustness of the model on UCI and real-world datasets. [ABSTRACT FROM AUTHOR]

Over the last two decades, there has been a surge of interest in the research of image quality assessment due to its wide applicability to many domains. In general, the aim of image quality assessment algorithms is to evaluate the perceptual quality of an image using an objective index which should be highly consistent with the human subjective index. The objective image quality assessment algorithms can be classified into three main classes: full-reference, reduced-reference, and no-reference. While full-reference and reduced-reference algorithms require full information or partial information of the reference image respectively, no reference information is required for no-reference algorithms. Consequently, a no-reference (or blind) image quality assessment algorithm is highly preferred in cases where the availability of any reference information is implausible. In this paper, a survey of the recent no-reference image quality algorithms, specifically for non-distortion-specific cases, is provided in the first half of this paper. Two major approaches in designing the non-distortion-specific no-reference algorithms, namely natural scene statistics-based and learning-based, are studied. In the second half of this paper, their performance and limitations are discussed before current research trends addressing the limitations are presented. Finally, possible future research directions are proposed towards the end of this paper. [ABSTRACT FROM AUTHOR]