Search

Condition-based maintenance (CBM) policy can avoid premature or late maintenance and reduce system failures and maintenance costs. Most existing CBM studies cannot solve the dimensional disaster problem in multi-component complex systems. Only some studies consider the constraint of maintenance resources when searching for the optimal maintenance policy, which is hard to apply to practical maintenance. This paper studies the joint optimization of the CBM policy and spare components inventory for the multi-component system in large state and action spaces. We use Markov Decision Process to model it and propose an improved deep reinforcement learning algorithm based on the stochastic policy and actor-critic framework. In this algorithm, factorization decomposes the system action into the linear combination of each component's action. The experimental results show that the algorithm proposed in this paper has better time performance and lower system cost compared with other benchmark algorithms. The training time of the former is only 28.5% and 9.12% of that of PPO and DQN algorithms, and the corresponding system cost is decreased by 17.39% and 27.95%, respectively. At the same time, our algorithm has good scalability and is suitable for solving Markov decision-making problems in large-scale state and action space. • Considering minor and major repair, we model the joint optimization of CBM and spare ordering for large multi-component systems based on MDP. • An improved DRL algorithm is presented to deal with the MDP model in large-scale discrete state and action space. • We validate our DRL algorithm has good time performance and optimal decision-making series solution via comparisons with DQN and PPO algorithm. [ABSTRACT FROM AUTHOR]

Forgery of a signature with the aim of deception is a serious crime. Machine learning is often employed to detect real and forged signatures. In this study, we present results which argue that robotic arms and generative models can overcome these systems and mount false-acceptance attacks. Convolutional neural networks and data augmentation strategies are tuned, producing a model of 87.12% accuracy for the verification of 2,640 human signatures. Two approaches are used to successfully attack the model with false-acceptance of forgeries. Robotic arms (Line-us and iDraw) physically copy real signatures on paper, and a conditional Generative Adversarial Network (GAN) is trained to generate signatures based on the binary class of 'genuine' and 'forged'. The 87.12% error margin is overcome by all approaches; prevalence of successful attacks is 32% for iDraw 2.0, 24% for Line-us, and 40% for the GAN. Fine-tuning with examples show that false-acceptance is preventable. We find attack success reduced by 24% for iDraw, 12% for Line-us, and 36% for the GAN. Results show exclusive behaviours between human and robotic forgers, suggesting training wholly on human forgeries can be attacked by robots, thus we argue in favour of fine-tuning systems with robotic forgeries to reduce their prevalence. • Development of a computer vision-based system for signature spoofing attack detection. • A Conditional GAN can generate "real" and "fake" signatures. • Two robots can physically replicate human signatures with pen and paper. • The GAN and both robots can fool the model and mount false-acceptance attacks. • Verification model can be defended by fine-tuning on generative and robotic forgeries. [ABSTRACT FROM AUTHOR]

Stochastic configuration networks (SCNs) have universal approximation capability and fast modeling properties, which have been successfully employed in large-scale data analytics. Based on SCNs, Stochastic configuration networks with block increments (BSC) use the node block increments mechanism to improve training speed but increase the complexity of the model. This paper presents a parallel configuration method (PCM), develops an extension of the original BSC with chaos theory and proposes stochastic configuration networks with chaotic maps (SCNCM), and establishes a hierarchical learning strategy (HLS) to enhance the compactness and construction speed of the model. Firstly, PCM randomly assigns the input weights w and biases b of hidden layer nodes by using uniform and normal distributions. In PCM, an iterative learning algorithm is intended to generate the scope control set and improve configuration efficiency. Secondly, the paper presents two kinds of stochastic configuration networks with chaotic maps, which are SCNCM-I and SCNCM-II. SCNCM-I adjusts block size by using multiple error values and chaotic maps to improve the training speed. Based on SCNCM-I, SCNCM-II utilizes node removal mechanism to enhance the compactness. Finally, HLS integrates with SCNCM-I, SCNCM-II, and the Harris-hawks optimization algorithm (HHO). The purpose of training is to enhance the training speed and compactness for three algorithms. The experiments are conducted on four benchmark data sets and an industrial application shows its effectiveness. [ABSTRACT FROM AUTHOR]

• Algorithmic trading using self-attention based recurrent reinforcement learning is developed. • Self-attention layer reallocates temporal weights in the sequence of temporal embedding. • Hybrid loss feature is incorporated to have predictive and reconstructive power. Algorithmic trading based on machine learning has the advantage of using intrinsic features and embedded causality in complex stock price time series. We propose a novel algorithmic trading model based on recurrent reinforcement learning, optimized for making consecutive trading signals. This paper elaborates on how temporal features from complex observation are optimally extracted to maximize the expected rewards of the reinforcement learning model. Our model incorporates the hybrid learning loss to allow sequences of hidden features for reinforcement learning to contain the original state's characteristics fully. The self-attention mechanism is also introduced to our model for learning the temporal importance of the hidden representation series, which helps the reinforcement learning model to be aware of temporal dependence for its decision-making. In this paper, we verify the effectiveness of proposed model using some major market indices and the representative stocks in each sector of S&P500. The augmented structure that we propose has a significant dominance on trading performance. Our proposed model, self-attention based deep direct recurrent reinforcement learning with hybrid loss (SA-DDR-HL), shows superior performance over well-known baseline benchmark models, including machine learning and time series models. [ABSTRACT FROM AUTHOR]

Multimodal emotion recognition, that is, emotion recognition uses machine learning to generate multi-modal features on the basis of videos which has become a research hotspot in the field of artificial intelligence. Traditional multi-modal emotion recognition method only simply connects multiple modalities, and the interactive utilization rate of modal information is low, and it cannot reflect the real emotion under the conflict of modal features well. This article first proves that effective weighting can improve the discrimination between modalities. Therefore, this paper takes into account the importance differences between multiple modalities, and assigns weights to them through the importance attention network. At the same time, considering that there is a certain complementary relationship between the modalities, this paper constructs an attention network with complementary modalities. Finally, the reconstructed features are fused to obtain a multi-modal feature with good interaction. The method proposed in this paper is compared with traditional methods in public datasets. The test results show that our method is accurate in It performs well in both the rate and confusion matrix metrics. [ABSTRACT FROM AUTHOR]

The optimal balance and dispatch of power plants in a smart grid require an accurate short-term forecast of photovoltaic (PV) power generation. The climatic condition may have an impact on the PV output, but it is difficult to be used in forecasting due to untimely sampling of meteorological data. To this end, this paper presents an incremental learning using privileged information (LUPI) paradigm for PV power forecasting by using stochastic configuration network. This novel algorithm can employ the meteorological data as privileged information for building PV power forecasting model in the training stage. Additionally, the model performance has been fully discussed in this paper. Finally, experimental results indicate that the proposed model indeed performs favorably in PV power forecasting. [ABSTRACT FROM AUTHOR]

• This paper presents MASK, a multilevel algorithm for approximate similarity search • MASK can distribute the index over as many computing nodes as we can afford. • Experimental results show the applicability of this novel indexing method • MASK achieves superior performance with high-dimensional and high-sparsity datasets. Similarity search based on a distance function in metric spaces is a fundamental problem for many applications. Queries for similar objects lead to the well-known machine learning task of nearest-neighbours identification. Many data indexing strategies, collectively known as Metric Access Methods (MAM), have been proposed to speed up these queries. Moreover, since exact approaches to solving similarity queries can be complex and time-consuming, alternative options have emerged to reduce query execution time, such as returning approximate results or resorting to distributed computing platforms. In this paper, we introduce MASK (Multilevel Approximate Similarity search with k -means), an unconventional application of the k -means algorithm as the foundation of a multilevel index structure for approximate similarity search suitable for metric spaces. We show that this method leverages inherent properties of k -means for this purpose, like representing high-density data areas with fewer prototypes. An implementation of this new indexing procedure is evaluated using a synthetic dataset and two real-world datasets in high-dimensional and high-sparsity spaces. Experimental tests show that MASK performs better than alternative algorithms for approximate similarity search. Results are promising and underpin the applicability of this novel indexing method in multiple domains. [ABSTRACT FROM AUTHOR]

• Many ML algorithms are used to identify the main determinants of a bank default. • We use of a graph neural network that has never been used in a financial context. • We obtain a balanced dataset by customizing the heuristic oversampling method. • Like previous literature, we show that neural network outperforms other approaches. • We include, for the first time, competition among the possible default determinants. The financial sector is very interested in Artificial Intelligence due to the opportunities that it offers, especially those related to methods of machine-learning. The aim of this paper is to employ a variety of machine-learning algorithms to identify the main determinants of bank default and to understand the impact of each variable on it. Bank default is one of the most studied topics in financial literature because of the severity of its consequences on the whole economic system. However, little attention has been paid to the identification of the major determinants of bank failures via machine-learning approaches. This paper employs several machine-learning algorithms, including a graph neural network that has never been used in a financial context. Another novelty is the implementation of a balanced dataset by customising the heuristic oversampling method based on k-means and synthetic minority over-sampling technique. This paper also deals with the inclusion of competition among the possible default determinants. The dataset consists of all the banks in the Euro Area in the period 2018–2020. The results obtained are useful from both micro- and macro-economic points of view. [ABSTRACT FROM AUTHOR]

This paper introduces the Ulimisana Optimisation Algorithm enabled Population Based Training (PBT-UOA) framework which allows for hyperparameters to be fine-tuned using a population based meta-heuristic algorithm at the same time as parameters are being optimised. Models are trained until near-convergence on the updated hyperparameters and the parameters of the best performing model are shared to warm start the other models in the next hyperparameter tuning iteration. In the PBT-UOA, all models are trained using the same dataset. This framework performed better than the Bayesian Optimisation algorithm. This paper also introduces the Ulimisana Optimisation Algorithm enabled Federated Learning (FL-UOA) framework which is an extension of the PBT-UOA. This framework is introduced to address the challenges of scattered datasets and privacy that is presented by the increase in connected end-devices. The FL-UOA learns on local data in scattered end-devices without sending datasets to a central server. The training datasets in local end-devices are used to evaluate models trained in other end-devices. The performance metrics are used to update the Social Trust Network (STN) of the FL-UOA framework. The FL-UOA outperformed the classic Federated Learning framework. This STN updating technique was tested in Machine Learning (ML) Unfairness to see how well it functioned as a regularisation term. This was achieved by training different models on subsets that contained datasets representing only specific sensitive groups. Results showed that by updating the hyperparameters while learning the parameters on the dataset scattered across different devices, the FL-UOA, takes advantage of diversified learning and reduces the ML Unfairness for models trained on group specific datasets. [ABSTRACT FROM AUTHOR]

In real applications, the label information on many data is inaccurate, or a completely reliable label needs to be obtained at a high cost. The previous supervised hashing algorithms consider only the label information in the mapping process from Euclidean space to Hamming space when learning hash codes. However, there is no doubt that these algorithms are suboptimal in maintaining the relationships between high-dimensional data spaces. To overcome this problem, this paper advances a new weakly supervised discrete discriminant hashing (WDDH) to ensure a more effective representation of data and better retrieval of information. First, we consider the nearest neighbour relationship between samples, and new neighbourhood graphs are constructed to describe the geometric relationship between samples. Second, the algorithm embeds the learning of the hash function into the model and optimises the hash codes by a one-step iterative updating algorithm. Finally, it is compared with the existing classical unsupervised hashing algorithm and supervised hashing algorithm on different databases. The results and discussion of the experiments clearly show that the proposed WDDH algorithm in this paper is more robust for data representation in learning low-quality label data, coarse-grained label data and noisy data. [ABSTRACT FROM AUTHOR]

• A parallel EA with primary-auxiliary knowledge is proposed, using our improved cuckoo search algorithm as the primary knowledge and integrating several other EA as the auxiliary knowledge, which fundamentally enables the hybrid of different algorithms and greatly enhances algorithmic diversity. • The way of acting of the evolutionary strategy change from the traditional greedy selection to the balanced primary-auxiliary approach in this paper, which allows each strategy to work in iteration. • A novel knowledge migration strategy is proposed, which allows the primary knowledge to learn the excellent knowledge from the auxiliary knowledge through a novel topology designed for knowledge migration. The development of hybrid algorithms or the application of multiple strategies is one of the focal points for research on improving evolutionary algorithms. However, since most of the evolutionary equations of different algorithms can be transformed into each other, it is difficult to change the established properties of the given algorithm through a hybrid algorithm based on a mixture of evolutionary equations. In addition, multi-strategy methods tend to adopt the best strategy for the current local domain through greedy strategies in the solution process, which does not ensure validity in the global domain. Recently, Federated Learning has achieved remarkable results in machine learning, where the idea of model independence, parallelism and data sharing can essentially compensate for the weaknesses of hybrid and multi-strategy algorithms. Inspired by the idea of Federated Learning, this paper proposes an evolutionary algorithm named as parallel based Evolutionary Algorithm with primary-auxiliary knowledge. Specifically, a Spark-based primary-auxiliary knowledge model is developed, with different evolutionary algorithms used on each parallel sub-model. Then, an effective topological knowledge (individual) migration method is devised, which enables the primary knowledge model to learn the best knowledge from different auxiliary knowledge models through a topological structure. In this way, the best knowledge on the auxiliary knowledge models can be transferred to the primary knowledge model. Through a test conducted on the CEC2013 test set, it can be found out that the proposed algorithm clearly outperforms the 10 algorithms compared, which demonstrates the excellent performance of our proposed parallel based evolutionary algorithm with primary-auxiliary knowledge. [ABSTRACT FROM AUTHOR]

• A parallel EA with primary-auxiliary knowledge is proposed, using our improved cuckoo search algorithm as the primary knowledge and integrating several other EA as the auxiliary knowledge, which fundamentally enables the hybrid of different algorithms and greatly enhances algorithmic diversity. • The way of acting of the evolutionary strategy change from the traditional greedy selection to the balanced primary-auxiliary approach in this paper, which allows each strategy to work in iteration. • A novel knowledge migration strategy is proposed, which allows the primary knowledge to learn the excellent knowledge from the auxiliary knowledge through a novel topology designed for knowledge migration. The development of hybrid algorithms or the application of multiple strategies is one of the focal points for research on improving evolutionary algorithms. However, since most of the evolutionary equations of different algorithms can be transformed into each other, it is difficult to change the established properties of the given algorithm through a hybrid algorithm based on a mixture of evolutionary equations. In addition, multi-strategy methods tend to adopt the best strategy for the current local domain through greedy strategies in the solution process, which does not ensure validity in the global domain. Recently, Federated Learning has achieved remarkable results in machine learning, where the idea of model independence, parallelism and data sharing can essentially compensate for the weaknesses of hybrid and multi-strategy algorithms. Inspired by the idea of Federated Learning, this paper proposes an evolutionary algorithm named as parallel based Evolutionary Algorithm with primary-auxiliary knowledge. Specifically, a Spark-based primary-auxiliary knowledge model is developed, with different evolutionary algorithms used on each parallel sub-model. Then, an effective topological knowledge (individual) migration method is devised, which enables the primary knowledge model to learn the best knowledge from different auxiliary knowledge models through a topological structure. In this way, the best knowledge on the auxiliary knowledge models can be transferred to the primary knowledge model. Through a test conducted on the CEC2013 test set, it can be found out that the proposed algorithm clearly outperforms the 10 algorithms compared, which demonstrates the excellent performance of our proposed parallel based evolutionary algorithm with primary-auxiliary knowledge. [ABSTRACT FROM AUTHOR]

Proximal policy optimization (PPO) algorithm is a deep reinforcement learning algorithm with outstanding performance, especially in continuous control tasks. But the performance of this method is still affected by its exploration ability. Based on continuous control tasks, this paper analyzes the original Gaussian action exploration mechanism in PPO algorithm, and clarifies the influence of exploration ability on performance. Afterward, aiming at the problem of exploration, an exploration enhancement mechanism based on uncertainty estimation is designed in this paper. Then, we apply exploration enhancement theory to PPO algorithm and propose the proximal policy optimization algorithm with intrinsic exploration module (IEM-PPO). In the experimental parts, we evaluate our method on multiple tasks in MuJoCo phsysical simulator, and compare IEM-PPO algorithm with PPO and PPO with intrinsic curiosity module (ICM-PPO). The experimental results demonstrate that IEM-PPO algorithm performs better in terms of sample efficiency and cumulative reward, and has stability and robustness. [ABSTRACT FROM AUTHOR]

• Jointly consider the task dependency and edge cloud environment. • Combine deep reinforcement learning and genetic algorithm. • Generate the initial population of genetic algorithm by DQN. Edge computing can greatly decrease the delay between users and cloud servers, which can significantly improve system service performance. However, it remains challenging for more efficient scheduling and allocation of users' application demands with dependence constraints to edge cloud servers. Due to the randomness of the initial population, traditional intelligent optimization algorithms have poor convergence speed in addressing resource scheduling. Therefore, to minimize the execution time of the application, this paper proposes a hybrid algorithm to solve the resource scheduling problem with parallelism and subtask dependency. To improve the convergence speed of the algorithm, this paper makes full use of the features of deep Q networks (DQN) and genetic algorithms (GA). The initial population of GA is generated using DQN. Finally, to evaluate the effectiveness of our proposed algorithm, this paper selects three real scientific workflows for experiments. The experimental results show that the hybrid algorithm can converge quickly and improve the optimization effect in a short time. [ABSTRACT FROM AUTHOR]

• The paper proposes two weighted c-medoid batch SOM for dissimilarity data. • The paper extends the batch SOM with C-medoids and multi-medoids algorithms. • The training of the new batch SOM algorithms are based on suitable cost functions. • The paper gives the cluster representatives and relevance weights of the c-medoids. • The paper provides the optimal assignment of the objects to the clusters. • The paper gives a meaningful evaluation of the proposed methods. This paper proposes two new batch SOM algorithms for dissimilarity data, namely RBSOM-CWMdd and RBSOM-ACWMdd, both designed to give a crisp partition aiming to preserve the topological properties of the data on the map. RBSOM-CWMdd is a batch SOM algorithm for dissimilarity data where each cluster representative is a set of weighted objects whose cardinality is fixed, being the same for all clusters. These weights are computed according to each object relevance to the referred cluster. Likewise, RBSOM-ACWMdd is a batch SOM algorithm for dissimilarity data where each cluster representative is a vector of weighted objects selected according to its relevance to the referred cluster. Therefore, the dimensionality of the cluster representatives self adapt to the particular dataset analysed, change at each algorithm iteration and can differ from one cluster to another. Experiments with 12 datasets from the UCI machine learning repository regarding the metrics of Normalized Mutual Information, Topological error, and Silhouette Coefficient showed that the proposed methods improved, respectively, the traditional set-medoids and multi-medoids SOM methods with a competitive temporal complexity. In addition, it was performed an application study on Ecoli dataset where the proposed RBSOM-ACWMdd algorithms produced a better mapping from a clustering point of view. [ABSTRACT FROM AUTHOR]

Difficult and expensive financing has always been a problem for domestic and foreign enterprises, and how to effectively improve financing efficiency and improve the financing environment is a key issue to be studied. LightGBM is an advanced machine learning algorithm, which uses histogram algorithm and Leaf-wise strategy with depth limitation to improve the accuracy of the model. However, there are almost no cases of applying this method to corporate financing risk prediction. Therefore, the paper establishes the LightGBM model to predict the financing risk profile of 186 enterprises. In order to compare the prediction performance of LightGBM for enterprise financing risk, the paper conducted comparison experiments using k-nearest-neighbors algorithm, decision tree algorithm, and random forest algorithm on the same data set. The experiments show that LightGBM has better prediction results than the other three algorithms for several metrics in corporate financing risk prediction. Therefore, we believe that the LightGBM algorithm can be used as an effective tool to predict the financing risk of enterprises. [ABSTRACT FROM AUTHOR]

Selecting discriminative features to build effective learning models is a significant research work in machine learning. In practical applications, the data distribution characteristics are diverse, and the uncertainties pose challenges for building learning models with robustness and generalization capabilities. Since one-hot encoding is good at representing independent labels, the label matrix of regression-based feature selection (FS) methods is usually encoded with one-hot encoding. However, it's not well adapted to the different data distributions. This paper proposes a sparse orthogonal supervised FS model with global redundancy minimization, label scaling, and robustness (GRMLSRSOFS) to address the above problems. This model uses the label scaling technique proposed in this paper to better adapt to different data distributions. An iterative optimization method is given, and its convergence is demonstrated theoretically and experimentally. Further, experimental results on 12 public datasets show that 1) The GRMLSRSOFS can achieve higher classification accuracy with fewer features in most cases than several state-of-the-art FS methods. For example, the GRMLSRSOFS achieves 100% classification accuracy using only 20 features on the warpPIE10P dataset and obtains nearly 6% improvement over other methods on the Yale dataset. 2) The convergence speed of the GRMLSRSOFS will be faster after label scaling. [ABSTRACT FROM AUTHOR]

• Based on the Euclidean distance between the input layer and the fuzzification layer, two sets of indicators are incorporated to assess the effectiveness and redundancy of current fuzzy rules. Rules determined as redundant are deleted. Meanwhile, new fuzzy rules are added when the current fuzzy rules become insufficient. Furthermore, a significance index is set to judge the essentiality of each fuzzy rule and remove unimportant rules. So that the algorithm can delete unimportant rules while deleting redundant rules. • The antecedent parameters of fuzzy rules are optimized by the AdaBound algorithm. By limiting the learning rate under a boundary value based on the Adam algorithm, this method has a higher training speed than the traditional gradient algorithm and simultaneously avoids the non-convergence caused by the extreme learning rate. • The adaptive gradient method is used to learn the consequences of fuzzy rules. By adjusting its learning rate with RMSE, this method can speed up the training speed while ensuring generalization capacity. • The proposed SIT2FWNN model is used to predict short-term traffic flow, Mackey-Glass chaotic time series, and the opening index of the Shanghai stock index and compared with similar studies. The simulation results demonstrate the superior network performance of the proposed SIT2FWNN in this paper. This paper presents a self-organizing interval type-2 fuzzy wavelet neural network (SIT2FWNN) model for predicting and identifying nonlinear systems. Based on traditional TSK interval type-2 fuzzy neural network (IT2FNN), the proposed SIT2FWNN utilizes the wavelet basis function with the locating abilities of time-domain and frequency-domain as the consequent of fuzzy rules, combining the capacity of IT2FNN to handle the uncertainty and the great learning potentiality of wavelet neural network (WNN). For the structural adjustment of SIT2FWNN, fuzzy rules are added and deleted according to the Euclidean distance between the input layer and the fuzzification layer. The self-organizing algorithm can delete redundant and unimportant fuzzy rules, thus optimizing the structure of SIT2FWNN and simplifying the calculation. In parameter learning, the AdaBound algorithm and self-adaptive gradient learning algorithm are used to find optimal values of unknown parameters of the SIT2FWNN model. Finally, the designed SIT2FWNN model is applied in the predictions of short-term traffic flow, Mackey-Glass time series, and the opening index of the Shanghai stock index. The evaluation comparison between the proposed model and similar studies proves that SIT2FWNN has higher prediction accuracy and speed. [ABSTRACT FROM AUTHOR]

• An adaptive forecast combination of highly diversified models is proposed. • The hyperparameter tuning is conducted through Bayesian optimization. • Our model significantly outperforms the classic and latest methods on extensive datasets. Among various time series (TS) forecasting methods, ensemble forecast is extensively acknowledged as a promising ensemble approach achieving great success in research and industry. Due to the high diversification of individual model assumptions, heterogeneous information fusion contributes to generating effective and robust forecasts for Economics, Meteorology, and Transportation. This paper proposes a Bayesian optimization-based dynamic ensemble (BODE) that overcomes the single model-based methods limitation and provides a dynamic ensemble forecast combination for TS with time-varying underlying patterns. The proposed BODE method combines ten disparate model candidates, including statistical methods, machine learning (ML)-based models, and the latest deep neural networks (DNN). We take into consideration their prediction performance for the recent past to adjust their weights for combination and apply the model-based Bayesian optimization algorithm (BOA) for the combination hyperparameter (HP) tuning to endow our method with higher adaptability and better generalization performance. Besides, the frequency impact of TS data on the ensemble forecast methods is under-researched in the current literature. Therefore, four groups of distinct seasonal TS datasets are investigated in this paper. The empirical result demonstrates that our method performs robustly better performance with the main reasons analyzed in a detailed ablation study. [ABSTRACT FROM AUTHOR]

In numerous binary classification tasks, the two groups of instances are not equally represented, which often implies that the training data lack sufficient information to model the minority class correctly. Furthermore, many traditional classification models make arbitrarily overconfident predictions outside the range of the training data. These issues severely impact the deployment and usefulness of these models in real life. In this paper, we propose the boundary regularizing out-of-distribution (BROOD) sampler, which adds artificial data points on the edge of the training data. By exploiting these artificial samples, we are able to regularize the decision surface of discriminative machine learning models and make more prudent predictions. Next, it is crucial to correctly classify many positive instances in a limited pool of instances that can be investigated with the available resources. By smartly assigning predetermined nonuniform class probabilities outside the training data, we can emphasize certain data regions and improve classifier performance on various material classification metrics. The good performance of the proposed methodology is illustrated in a case study that consists of both benchmark balanced and imbalanced classification data sets. [ABSTRACT FROM AUTHOR]

Ensemble classifier is a well-known method that has been used to solve several machine learning problems. To have reliable results, one should ensure the build of a good ensemble. In order to do so, researchers have proposed some heuristics like Random Subspace Ensemble (RSM), Rough set bas. The drawback of these mentioned approaches is their disability to handle uncertain data especially when uncertainty is represented by the evidence theory. The aim of this paper is to adapt both RSM and Rough set based ensemble in order to let them working in the context of evidential data. Three ensemble classifier approaches based on the rough set theory have been proposed and have been compared with each other. For the comparison purpose, we have relied on Ensemble Enhanced Evidential k Nearest Neighbor (EE k -NN) classifier, real world datasets from the UCI repository as well as synthetic databases. [ABSTRACT FROM AUTHOR]

Classification with incomplete data is practically challenging since missing data hinders classification. The two-stage strategy of filling missing values first and then classifying complete data degenerates performance due to the separate optimization of classification and imputation. While efforts have been made to classify incomplete data directly without imputation, the benefits of imputation on classification have been sacrificed. To this end, this paper proposes a Category-Aware Optimal Transport Neural Network (CAOT-NN) for incomplete data classification. Specifically, we design a category-aware optimal transport method conditioned on intra-category data distribution, which explicitly employs the category information of each sample and fills in discriminative values for the downstream classification module. Moreover, we reconstruct the observed values during the imputation process when performing classification learning, thus implicitly utilizing the category information of the classifier for missing values estimation. Extensive experiments conducted on 16 real-world and 30 synthetic datasets from the UCI repository demonstrate the superiority of CAOT-NN against existing state-of-the-art methods. Also, the visualization analysis reveals that CAOT-NN enables a tighter intra-category distribution of the imputed data, implying that the missing mechanism may be utilized as a data augmentation strategy to boost accuracy. • CAOT-NN can make the imputed values more favorable for classification. • A category-aware imputation loss function is defined based on optimal transport. • Good imputation performance doesn't improve incomplete data classification results. [ABSTRACT FROM AUTHOR]

To further enhance the reliability of Machine Learning (ML) systems, considerable efforts have been dedicated to developing privacy protection techniques. Recently, membership privacy has gained increasing attention, with a focus on determining whether a specific data point is present in the confidential training set of an ML model. However, most current attacks only prioritize attack accuracy and fail to extend their range to the evidence that contributes to the member/non-member classification. This limitation greatly reduces the practicality of Membership Inference Attacks (MIA), as real-world data typically includes multiple features, making it challenging to identify which features are involved in the sensitive training set. Therefore, this paper targets one of the fundamental challenges in membership inference attack: measuring the distance between an attack sample and a member sample. Specifically, we propose a novel threat model called Membership Reconstruction Attack (MRA), which aims to reconstruct the exact distribution of the target training set. MRA achieves this by marking each input dimension (e.g., pixels) according to its similarity to the target dataset in feature space. Our attack demonstrates its effectiveness across various settings, including different major datasets (MNIST, CIFAR-10, CIFAR-100) and different model architectures (AlexNet, ResNet, DenseNet, and generative models). Additionally, we evaluate MRA from the defenders' perspective and test several defense approaches against our attack. [ABSTRACT FROM AUTHOR]

The training effect of the neural network model is directly influenced by optimization algorithms. The Barzilai-Borwein(BB) method is used in the stochastic gradient descent (SGD) and other deep learning optimization algorithms because of its outstanding performance in terms of convergence speed. In order to improve the stability of BB step size and the output quality of network model in deep learning, this paper presents two improved optimization algorithms based on the BB method: BBbound and AdaBBbound. BBbound reduces the floating BB step size by generating the upper bound of the step in the current iteration. It avoids the occasional occurrence of long steps; AdaBBbound is an adaptive gradient method based on BB method with dynamic bound. The algorithm has a fast convergence effect in the early stage, then calculates learning rate and smoothly transitions to SGD by setting the better conditions of the BB method. We analyzed the performance of the two algorithms based on the initial conditions and the learning rate change curve. Meanwhile, we tested our algorithms on popular network models such as ResNet and DenseNet. The results showed that the new optimization algorithms have achieved high stability and significantly improved the performance of the network model. [ABSTRACT FROM AUTHOR]

In this paper, we improve the robustness of Machine Learning (ML) classifiers against training-time attacks by linking the risk of training data being tampered with to the redundancy in the ML model's design needed to prevent it. Our defense mechanism is directly applicable to classifiers' training data, without any knowledge of the specific ML model to be hardened. First, we compute the training data proximity to class separation surfaces, identified via a reference linear model. Each data point is associated with a risk index, which is used to partition the training set by an unsupervised technique. Then, we train a learner for each partition and combine the learners' output in an ensemble. Our method treats the protected ML classifier as a black box and is inherently robust to transfer attacks. Experiments show that, for data poisoning rates between 6 and 25 percent of the training set, our method is more robust compared to benchmarks and to a monolithic version of the model trained on the whole training set. Our results make a convincing case for adopting training set partitioning and ensemble generation as a stage of ML models' development and deployment lifecycle. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of secure event-triggered control is to guarantee the stability and security of a general class of industrial cyber-physical systems (CPSs) under limited resource budget and deception attacks. Note that the coupled data and the complicated structure of the CPS make it hard for the traditional control algorithms to fulfill the stability and security requirements. To this end, a neural network learning-based approximation algorithm is first proposed to separate the coupling influence, and then estimate the lumped uncertain nonlinearities. This is done by using the separated data as the input of the neural networks. Second, Nussbaum-type functions are presented to settle the unknown sign of the time-varying attack injection signal. Third, an event triggering mechanism is developed to reduce the communication load. Under the developed secure control laws, all the signals of the resulted closed-loop CPS are bounded. Finally, to validate the effectiveness of the proposed secure event-triggered control method, a benchmark control example for the one-link manipulator actuated by a DC motor is exploited. • Secure event-triggered control is proposed to ensure the security of industrial CPSs. • Neural network learning algorithm is used to estimate the uncertain nonlinearities. • Nussbaum-type functions are presented to handle the unknown sign of the time-varying attack injection signal. • Event triggering mechanism is developed to reduce the communication. [ABSTRACT FROM AUTHOR]

How to derive an investor's preference is vital for investment advisors and online lending platforms for targeted marketing strategies, e.g., market segmentation and financial product recommendation. However, investor preference analysis usually depends on judgments from human investment experts, which are inherently subjective and costly. Intelligent investment advisors (or Robo-advisors), supported by cutting-edge technologies such as machine learning and artificial intelligence, are established to relieve these pending issues. This paper employs an online optimization framework to obtain investors' preferences for further financial product recommendations. This proposed method allows us to update the investor's preference for newly-arriving data sets and tackle the situation where plenty of missing values in investors' records are present. Unlike the black-box-like machine learning approach, our method can provide more managerial implications regarding why one financial product/service is preferred. Real-world data set from an online financial platform is used to compare the existing approaches and shows the stronger and more stable performance of our method when facing different data-missing types and situations with different missing degrees, followed by a recommendation system application. [ABSTRACT FROM AUTHOR]

The multivariate spatio-temporal data contains complex spatio-temporal background and channel coupling information. Effective extraction of these features is crucial for data generation. In this paper, a separated multiple attention network is proposed, which can capture the correlation of multiple types of variables in the same space-time, different spaces at the same time, and different times in the same space. Meanwhile, a new multiscale loss processing method based on homoscedasticity uncertainty and the assumption of gaussian loss distribution is proposed to balance the numerical scale of each channel loss in training. The experiment shows that our method has better performance and robustness than the classical machine learning model and higher computational efficiency than the physical model. It can generate multivariate intermittent spatial-temporal fields with a maximum lead time of 3 days and multivariate continuous spatial-temporal fields with a maximum lead time of 7 days. • Novel spatio-temporal data generation model is designed based on separated attention. • New method for balance the learning progress of multivariate data generation. • The lead time for spatio-temporal data generation has been extended to 7 days. [ABSTRACT FROM AUTHOR]

Equation discovery, also known as symbolic regression, is a machine learning task of inducing closed-form equations from data and background knowledge. The latter takes various forms. Domain-specific knowledge can constrain the space of candidate equations to those that make sense in the scientific or engineering domain of use. Cross-domain knowledge, on the other hand, imposes general rules for model acceptability, such as parsimony, understandability, or consistency of the equations with the dimensional units of the variables. In this paper, we propose using attribute grammars to ensure the induced equations' dimensional consistency. Attribute grammars are flexible enough to combine cross-domain knowledge on dimensional consistency with domain-specific knowledge expressed as a probabilistic context-free grammar. At the same time, we show that attribute grammars can be efficiently transformed into probabilistic context-free grammars for equation discovery with existing algorithms. Finally, we provide empirical evidence that attribute grammars ensuring dimensional consistency of equations can significantly improve the performance of equation discovery on the standard set of a hundred Feynman benchmarks. • Attribute grammars elegantly express dimensionally consistent expressions. • Efficient combining of dimensional analysis with other background knowledge. • Generate equations by transforming into probabilistic context-free grammar. • Discovered 60% more benchmark equations by accounting for dimensional consistency. [ABSTRACT FROM AUTHOR]

Explainable artificial intelligence (XAI) is gradually becoming a key component of many artificial intelligence systems. However, such pursuit of transparency may bring potential privacy threats to the model confidentially, as the adversary may obtain more critical information about the model. In this paper, we systematically study how model decision explanations impact model extraction attacks, which aim at stealing the functionalities of a black-box model. Based on the threat models we formulated, an XAI-aware model extraction attack (XaMEA), a novel attack framework that exploits spatial knowledge from decision explanations is proposed. XaMEA is designed to be model-agnostic: it achieves considerable extraction fidelity on arbitrary machine learning (ML) models. Moreover, we proved that this attack is inexorable, even if the target model does not proactively provide model explanations. Various empirical results have also verified the effectiveness of XaMEA and disclosed privacy leakages caused by decision explanations. We hope this work would highlight the need for techniques that better trade off the transparency and privacy of ML models. • We propose XaMEA, three XAI-aware model extraction attack architectures. • We further carry out XAI-aware model extraction attacks against non-explanation target models. • We evaluate the attack effectiveness of XaMEA with an exhaustive set of experiments. [ABSTRACT FROM AUTHOR]

Uncovering the reward function of optimal controllers is crucial to determine the desired performance that an expert wants to inject to a certain dynamical system. In this paper, a reward inference algorithm of discrete-time expert's controllers is proposed. The approach is inspired by the complementary mechanisms of the striatum, neocortex, and hippocampus for decision making and experience transference. These systems work together to infer the reward function associated to expert's controller using the complementary merits of data-driven and online learning methods. The proposed approach models the neocortex system as two independent learning algorithms given by a Q-learning algorithm and a gradient identification rule. The hippocampus is modelled by a least-squares update rule that extracts the relation from the states and control inputs of the expert's data. The striatum is modelled by an inverse optimal control algorithm which iteratively finds the hidden reward function. Lyapunov stability theory is used to show the stability and convergence of the proposed approach. Simulation studies are given to demonstrate the effectiveness of the proposed complementary learning algorithm. [ABSTRACT FROM AUTHOR]

The existing single dictionary learning algorithms are applied to face recognition and achieve satisfactory results. However, their performance is poor when dealing with noisy images and images involving complex variations such as large pose variations and occlusions. In this paper, a novel noise-related face image recognition method based on double dictionary transform learning (DDTL) is proposed. On the one hand, DDTL introduces the L 2 , p -norm to remove the redundant information in the dictionary and the noise involved in the training images, which makes the learned dictionary more discriminative. On the other hand, DDTL introduces the analysis dictionary and performs double dictionary transform learning with the synthetic dictionary. This can better reveal the relationship between the samples and the representation coefficients, and improve the accuracy of the learned dictionary and representation coefficients. Besides, DDTL introduces a linear regression term in the model learning process, which can distinguish and expand the differences between classes. Experimental results on six databases show that DDTL is superior to existing methods. [ABSTRACT FROM AUTHOR]

This paper presents a method called Interval Incremental Learning (IIL) to capture spatial and temporal patterns in uncertain data streams. The patterns are represented by information granules and a granular rule base with the purpose of developing explainable human-centered computational models of virtual and physical systems. Fundamentally, interval data are either included into wider and more meaningful information granules recursively, or used for structural adaptation of the rule base. An Uncertainty-Weighted Recursive-Least-Squares (UW-RLS) method is proposed to update affine local functions associated with the rules. Online recursive procedures that build interval-based models from scratch and guarantee balanced information granularity are described. The procedures assure stable and understandable rule-based modeling. In general, the model can play the role of a predictor, a controller, or a classifier, with online sample-per-sample structural adaptation and parameter estimation done concurrently. The IIL method is aligned with issues and needs of the Internet of Things, Big Data processing, and eXplainable Artificial Intelligence. An application example concerning real-time land-vehicle localization and tracking in an uncertain environment illustrates the usefulness of the method. We also provide the Driving Through Manhattan interval dataset to foster future investigation. [ABSTRACT FROM AUTHOR]

For the consumer lending industry, credit risk assessment is a crucial task for evaluating the default probability of loan applications given the potential loss caused by default. In particular, loan default prediction is a challenging classification problem due to the highly imbalanced class distribution and a model with both strong default identification ability and overall classification accuracy is exceedingly hard to achieve. Furthermore, most existing studies analyze loans with distinct credit ratings as a whole, ignoring the important relationship between class imbalance and specific credit rating categories. Meanwhile, considering loan companies' different risk preferences when reviewing loan applications, this paper proposes a rating-specific multi-objective ensemble learning framework. Specifically, a credit rating-specific modeling strategy is used to construct candidate models for subpopulations of loans having similar default risk. A multi-objective ensemble learning method is developed using an effective imbalanced classification method, One-Class SVM, as the base learning algorithm. A multi-objective evolutionary process is employed for ensemble construction and an adaptive classification boundary adjustment process is proposed to improve the imbalanced classification performance. To validate the proposed methodology, we conduct comprehensive experimental studies and compare the proposed method with several benchmark algorithms on an industry provided data set. Our experimental results demonstrate the advantage of a rating-specific strategy and also demonstrate the superiority of the proposed multi-objective ensemble learning method. Important managerial implications in the consumer lending industry are discussed. • We propose a multi-objective ensemble learning method for imbalanced loan default prediction. • We employ a rating-specific modeling strategy to improve prediction performance. • Our OC-SVM ensemble models are adaptively constructed for data with distinct imbalance ratios. • Different risk preferences can be satisfied with our multi-objective learning scheme. • Our method can achieve both strong default identification ability and overall prediction accuracy. [ABSTRACT FROM AUTHOR]

Inconsistency in classification and regression problems occurs when instances that relate in a certain way on the condition attributes, do not follow the same relation on the decision attribute. It typically appears as a result of perturbation in data caused by incomplete knowledge (missing attributes) or by random effects that occur during data generation (instability in the assessment of decision attribute values). Inconsistencies with respect to a crisp preorder relation (expressing either dominance or indiscernibility between instances) can be handled with set-theoretic approaches like rough sets and by using statistical/machine learning approaches that involve optimization methods. In particular, the Kotłowski-Słowiński (KS) approach relabels the objects from a dataset such that inconsistencies are removed, and such that the new class labels are as close as possible to the original ones in terms of a given loss function. In this paper, we generalize the KS approach to handle inconsistency determined by a fuzzy preorder relation rather than a crisp one. The method produces a consistent fuzzy relabeling of the instances and may be used as a preprocessing tool with algorithms for binary classification and regression. As the obtained fuzzy sets can be represented as unions of meaningful simple fuzzy sets or granules, we call them granular approximations. We provide statistical foundations for our method, develop appropriate optimization procedures, provide didactic examples, and prove several important properties. • Proposing the definition of granular approximations; a novel way of integrating machine learning, fuzzy sets, and rough sets. • Inconsistencies in data, defined according to the rough set framework, are removed with optimization methods. • The new method generalizes Kotłowski and Słowiński's approach defined for crisp relations. • The fuzzy rough set lower and upper approximations are extreme cases of the granular approximation. [ABSTRACT FROM AUTHOR]

For many classification tasks, particularly in the bioinformatics field, only experimentally validated positive samples are available, and experimentally validated negative samples are not recorded. The lack of negative samples poses a challenge for using machine learning to perform such tasks. To address this problem, we propose a novel deep reinforcement learning-based model to screen reliable negative samples from unlabeled samples, named SURE. The model has two modules: a sample selector and a sample inspector. The sample selector screens reliable negative samples from unlabeled samples by two reinforcement strategies (learn to identify positive samples and reduce sample noise) and feeds the screened samples into the sample inspector. The sample inspector classifier provides rewards to the sample selector. The two modules are trained together to optimize the sample selector and sample inspector strategies. In this paper, we focus on one popular issue in the bioinformatics field: the ncRNA-protein interaction (NPI) prediction task, which lacks reliable negative samples. Thirty datasets for NPI prediction are used to test the screening effect of SURE. The experimental results show that our model has a robust negative sample screening capability and is superior to all outstanding sample screening methods used in the NPI prediction task. In addition, we refine 5 NPI datasets containing reliable negative samples screened by SURE, and a web server (ww.csbg-jlu.info/sure) is available offering the NPI prediction refined by SURE. • Screening reliable negative samples from unlabeled samples based on deep reinforcement learning. • Improving the performance of ncRNA-protein interaction prediction models based on screened negative samples. • Providing five complete datasets containing positive and negative samples of ncRNA-protein interactions. • Developing a web server for ncRNA-protein interaction prediction. [ABSTRACT FROM AUTHOR]

This paper develops a novel insight and procedure that includes a variety of algorithms for finding the best solution in a structured multi-agent system with internal communications and a global purpose. In other words, it finds the optimal communication structure among agents and the optimal policy in this structure. First, a unique reinforcement learning algorithm is proposed to find the optimal policy of each agent in a fixed structure with non-linear function approximators like artificial neural networks (ANN) and with eligibility traces. Secondly, a mechanism is presented to perform self-organization based on the information of the learned policy. Finally, an algorithm that can discover an appropriate inter-structure mapping and then can transfer the previous knowledge to the new structure is developed, which increases the speed of the learning in this new environment after self-organization. This paper is one of the first works that analyzes the problem fully theoretically and devises some algorithms to find the best solution. We use a simplified version of the distributed task allocation problem (DTAP) as our case study. The experimental results verify the stability of our approach and show the high speed of finding the optimal solution as a result of using transfer learning. [ABSTRACT FROM AUTHOR]

Many black-box adversarial attack algorithms perform attacks on machine learning models based on the transferability of adversarial examples, and the input transformation-based attack is one of the most effective methods. However, existing input transformation methods ignore that each pixel contributes differently to the output of the model, and the focus regions of different models on the same images are similar. Therefore, this paper proposes a targeted data augmentation-based adversarial attack algorithm named MixCam, which augments the input data based on the contribution of each pixel to the prediction result. This is done to enhance the transferability of the crafted adversarial example from the perspective of shifting the regions to which the model pays most of its attention. In addition, this paper proposes further boosting the transferability of the crafted adversarial examples by fusing the class activation maps of multiple models for the input image. Furthermore, the MixCam can integrate other input transformation methods to further boost the transferability of crafted adversarial examples. Extensive experiments on ImageNet demonstrate that MixCam outperforms other state-of-the-art methods in black-box attacks against considered adversarially trained models, with an average increase of 11.7% and 10.7% in attack success rates for single and ensemble attack settings, respectively. • A targeted data augmentation-based black-box adversarial attack method. • Performing adversarial attacks from the perspective of moving the focus region of DNN. • Boosting the transferability of the crafted adversarial examples. • Extensive experiments have demonstrated the effectiveness of our method. [ABSTRACT FROM AUTHOR]

• This paper proposes PNAS, a novel privacy preserving training framework for MLaaS scenarios, supports the optimization for both network parameter and model architecture. • We design a double encryption scheme for PNAS. Our scheme eliminates privacy leakage during remote training. Samples and their corresponding labels, as well as intermediate feature maps are well protected from an untrusted server. • We implement a prototype PNAS and evaluate its performance. Experimental result shows that PNAS is able to deliver models with higher accuracy. The success of deep neural networks has contributed to many fields, such as finance, medic and speech recognition. Machine learning models adopted in these fields are always trained with a massive amount of distributed and highly personalized data harvested directly from users. Concerns for data privacy and the demand for better data exploitation have prompted the design of several secure schemes that allow an untrusted server to train ML models for one or multiple parties. However, these existing schemes only focus on network parameter, and hardly extend their optimization range to model architecture scope. Sine the performance of a neural network is closely related to both parameter and its architecture, service providers are difficult to deliver customized and flexible neural networks to each client. To this end, in this paper we propose PNAS, a novel MLaaS framework that enables a server to jointly optimize network parameter and architecture while ensuring the privacy of training sets. A double-encryption scheme is derived to prevent privacy leakage from sample itself, as well as intermediate feature maps during training. Specifically, we adopt functional encryption and feature transformation to secure forward and back propagation. Extensive experiments have demonstrated the superiority of our proposal. [ABSTRACT FROM AUTHOR]

One-class classification is a machine learning problem, where training data has only one class. The objective is to determine if the input data is seen class or unseen class. Traditional deep learning algorithms are not suitable for this task since the algorithm can predict only class in training data. In this paper, the one-class classification algorithm using construction error of image transformation network (OCITN) is proposed. In particular, image transformation network (ITN) is introduced as a subtask, which transforms input image into one image, namely goal image. Moreover, the error of ITN, namely construction error (CE), is computed as a distance metric between the goal image and model output. ITN model is trained using only one-class images and is applied for testing images. Since the model is trained with only one-class images, the CE for one-class is small relative to other classes. Thus, one-class classification is made determining CE is large or small. The proposed method is experimented with using MNIST, Fashion MNIST, CIFAR10, CIFAR100, and Cat-vs-Dog datasets. OCITN shows good results where the goal image has high entropy. Additionally, the extension of OCITN, namely OCITNE, is implemented. This method shows the state of the art performance in MNIST (98.0), Fashion MNIST(95.6), and acceptable performance in CIFAR10(78.4). Furthermore, these methods provide high-speed processing, OCITN process 5291 images, and OCITNE 1261 images per second, 137 times and 33 times faster than state of the art. The source code used in this paper can be downloaded from: https://github.com/ToshiHayashi/OCITN. [ABSTRACT FROM AUTHOR]

Decentralized federated learning (DFL) is an emerging privacy-preserving machine learning framework, where multiple data owners cooperate to train a global model without any aggregation server. In addition to protecting data privacy, it also avoids a single point of failure, and reduces the communication traffic congestion due to the central server. However, there are still many inherent privacy and security issues, e.g., training samples could be revealed by inferring gradients and malicious participants tend not to execute federated learning tasks as intended. Even worse, existing works rarely simultaneously consider a fundamental issue that data owners may join in and drop out of a DFL task at any time. In this paper, we propose PTDFL, a privacy-enhanced and trustworthy decentralized federated learning scheme. Specifically, we firstly design an efficient gradient encryption algorithm to protect data privacy, and then devise a succinct proof without trapdoors to ensure the correctness of the gradients. Meanwhile, we design a novel local aggregation strategy without trusted third party to ensure that the aggregated result is trustworthy. Moreover, our PTDFL is also supportive of data owners joining in and dropping out during the whole DFL task. Finally, we provide privacy and security analysis, implement a prototype of the PTDFL, and conduct extensive experiments on a real dataset. The results show that our PTDFL achieves more efficiency than prior works. • We propose a decentralized federated learning scheme that enhances both privacy protection and trustworthiness. • Our scheme is suitable for the condition where nodes join in or drop out at any time. • Our scheme leverages a local proof based on zk-SNARK to enhance the trustworthiness of the gradients. • We make a theoretical security and privacy analysis of our scheme, implement a proof-of-concept prototype and evaluate it. [ABSTRACT FROM AUTHOR]

Machine learning models, especially deep neural network (DNN) models, have widespread and valuable applications in business activities. Training a deep learning model for commercial use requires plenty of private data, expert knowledge, and computing resources. The huge commercial value of such trained models has attracted the attention of attackers. Attackers can construct a dataset by repeatedly querying the target model for the output of the requested samples and then train a substitute model on this dataset that functions similarly to the target model. In this paper, we propose a defense scheme based on physical unclonable function (PUF) against such black-box model extraction attacks. We deploy a PUF on the user side and the corresponding PUF model on the service provider side to ensure that only legitimate users can obtain the correct model predictions. Our experimental results show that by choosing a suitable fuzzy extractor threshold d , legitimate users can recover more than 99.5% of the prediction results with a little additional computational overhead to the service provider. We perform a model extraction attack in the most favorable case for the attacker, and the prediction accuracy of the obtained substitute model is only about 10%, which demonstrates the effectiveness of our proposed scheme. Compared to existing defenses, our scheme not only effectively prevents black-box model extraction attacks but also ensures that the accuracy of the prediction service for legitimate users is not affected. • Defense based on PUF obfuscation against black-box model extraction attacks. • A PUF and the corresponding PUF model ensure that legitimate users can obtain correct model predictions. • Legitimate users can correctly recover more than 99.5% of the predictions with a little additional computational overhead. • The prediction accuracy of the model extracted by the attacker is only about 10%. [ABSTRACT FROM AUTHOR]

An accurate estimation of tourism demand is of great significance to tourism management. The seasonal and non-stationary features present a significant challenge in developing tourism demand estimation. An effective tourism demand forecasting model is important to address this problem. This paper proposes a novel model for tourism demand forecasting based on developed mechanism-guided attention. The model consists of three sections: the first section defines the degree of stationarity of the demand time series, the second section develops a guided attention mechanism for improving the feature recognition of neural networks; the third section generates forecasting results of tourism demand, in the meanwhile, the developed fully connected recurrent neural network is adopted to identify complex features of tourism demand time series. The proposed model is helpful in identifying the features of seasonality and non-linearity in tourism demand data, the model maintains good forecasting accuracy with the appropriate training process. Case studies show that the proposed method can forecast the daily tourism demand of Macau, China accurately compared with other traditional forecasting methods. • A novel model basing a guided attention mechanism is proposed to forecast tourism demand. • Identify non-stationary feature in tourism demand time series using Fully connected recurrent neural network. • Use adaptive evolutionary algorithms to evaluate the tourism demand forecasting results in different scenarios. • Adopt NSGA-II to optimize the mapping matrix of guided attention mechanism. • Validate the proposed model using open data sets with complex characteristics. [ABSTRACT FROM AUTHOR]

Generation of massive data in the digital age leads to possible violations of individual privacy. The search for personal data becomes an increasingly recurrent exposure today. The present work corresponds to the area of differential privacy, which guarantees data confidentiality and robustness against invasive identification attacks. This area stands out in the literature for its rigorous mathematical basis capable of quantifying the loss of privacy. A differentially private method based on regression models was developed to prevent inversion attacks while retaining model efficacy. In this paper, we propose a novel approach to improve the data privacy based on regression models under heteroscedasticity, a common aspect, but not studied, in practical situations of differential privacy. The influence of privacy restriction on the statistical performance of the estimators of model parameters is evaluated using Monte Carlo simulations, including a study of performance associated with test rejection rates for the proposed approach. The results of the numerical evaluation show high inferential distortion for stricter privacy restrictions. Empirical illustrations with real-world data are presented to show potential applications. [ABSTRACT FROM AUTHOR]

Smoothing rugged and rough fitness landscapes by machine learning (ML) models is a feasible approach to improve the global optimization performance of evolutionary algorithms (EA). However, systematic studies on the smoothing performance of ML models are rare. Moreover, existing model management methods have a relatively low efficiency and can cause the error propagation problem. In this paper, we empirically evaluate and compare the performance of common ML models in smoothing high-dimensional fitness landscapes. Based on the experimental results, we design a novel framework called data-driven multi-task optimization (DDMTO) to enhance EAs' search abilities in complex solution spaces with ML smoothing models. The proposed framework firstly smooths the fitness landscape by a ML model. Then, it models the original fitness landscape optimization and the smoothed fitness landscape optimization as a two-task optimization problem and solves them with an evolutionary multi-task optimizer (EMTO). Coordinated by EMTO, the easy task can help optimize the difficult task through the proposed knowledge transfer operator, and the error propagation problem can be solved by the proposed knowledge transfer control scheme. To prove the effectiveness of the proposed methodology, we design several example algorithms by embedding different EAs and ML models into the DDMTO framework and compare their optimization performance on high-dimensional continuous benchmark functions with rugged and rough fitness landscapes and a real-world optimization problem. Experimental results show that, by embedding an EA into the DDMTO framework with an appropriate smoothing model, its exploration ability and global optimization performance in complex solution spaces can be significantly enhanced without increasing the total computational cost. [ABSTRACT FROM AUTHOR]

Hybrid recommender systems utilize advanced algorithms capable of learning heterogeneous sources of data and generating personalized recommendations for users. The data can range from user preferences (e.g., ratings or reviews) to item content (e.g., description or category). Prior studies in the field of recommender systems have primarily relied on "ratings" as the user feedback, when building user profiles or evaluating the quality of the recommendation. While ratings are informative, they may still fail to represent a comprehensive picture of actual user preferences. In contrast, there are other types of feedback data that differently or complementarily represent users and their preferences, including the reviews and the sentiments encapsulated within them. Such data can reveal important parts of a user's profile that are not necessarily correlated with user ratings, and hence, they potentially reflect a different side of the user's profile. In this paper, we propose a novel form of hybrid recommender system, capable of analyzing the reviews and extracting their sentiments that are incorporated into the recommendation process. We used advanced algorithms to generate recommendations for users capable of incorporating additional data, such as the review sentiment. We conducted analyses and showed that sentiments of user reviews are not always highly correlated with the ratings (e.g., in music domain). This might mean that sentiment can be indicative of a different aspect of user preferences and can be used as an alternative signal of user feedback. Hence, we have used both ratings and sentiments of reviews when evaluating our proposed hybrid recommender system. We selected two common datasets for the evaluation, Amazon Digital Music and Amazon Video Games, and showed the superior performance of the proposed hybrid recommender system compared to different baselines. The comparison were made in two evaluation scenarios, namely, when the ratings were considered the user feedback and when sentiments of the review were considered the user feedback. [ABSTRACT FROM AUTHOR]

Clustering is an essential task in machine learning, and learning clustering-friendly representation is crucial for clustering performance. Recently, methods that combine contrastive learning with feature decorrelation have demonstrated their efficacy in learning representation for clustering. However, the feature decorrelation is usually performed in an additional feature space, making the optimization more complex. Moreover, the dimension of this extra feature space is linked to the batch size, but this essential correlation that affects feature decorrelation has been less studied. This paper explores the connection between batch size and feature decorrelation in contrastive learning, then proposes to utilize the linear independence created by contrastive learning with small batch sizes to promote feature decorrelation indirectly. Further, this work introduces bi-vector-based contrastive learning to enable contrastive learning with small batch sizes, which is also of interest to researchers with limited resources. Besides, spectral clustering is utilized to explain the advantage of using linear independence for feature decorrelation and clarify its condition. Experiment results on five benchmark datasets demonstrate the efficacy of the proposed method against state-of-the-art methods. As for the CIFAR-10 and CIFAR-100, our method achieves the clustering accuracy of 88.9% and 52.1%, which are 7.4% and 9.6% higher than the non-prior SOTA method. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel ensemble and robust anomaly detection method based on collaborative representation-based detector. The focused pixels used to estimate the background data are randomly sampled from the image. To soften the outliers' contributions among the selected pixels, we assign low weights to the outliers by adopting a robust norm regression. Consequently, the estimation result is less sensitive to the presence of outliers, as the experiment results attest. However, the algorithm performance is unstable due to the randomness of pixel sampling. To eliminate the instability and boost the detection performance, an ensemble learning method is employed. We repeat modeling background based on random pixel selection, and the detection result is an ensemble of all batches. We show that in most datasets, the proposed method outperforms the traditional algorithms. Moreover, batch processes for detection boosting secure future advances in performance utilization with parallel computing applied. [ABSTRACT FROM AUTHOR]

Multi-label learning deals with a kind of problem that the given samples areassociated with multiple labels simultaneously. Recently, multi-label learning has become a populartopic in the literatures of machine learning and has attracted lots of researches. In this paper, we propose a new multi-view multi-label learning method by considering the label correlation, which is called ELSMML. Based on the high-order strategy, we construct a crafted label correlation matrix to describe the relationships among labels. We further utilize multi-view learning and dimension reduction to exploit the high-level latent semantic label information and the latent feature information, so as to build a classifier in the low dimensional space. In addition, we apply manifold regularization terms to make the data samples in the low dimensional space have the same intrinsic structure as the original data. After that, we put forward the accelerated proximal gradient method to optimize the ELSMML model and obtain thepredictive classifier. Besides, we conduct convergence analysis and computational complexity analysis for ELSMML method. In the experiments, the ELSMML method can achieve better performance on the evaluation metrics compared with other baselines. [ABSTRACT FROM AUTHOR]

• Using machine learning to recommend appropriate parameters and operators for differential evolution algorithm. • The fitness landscape features are used as a basis for recommending parameters for the optimization problems. • A mutation operator selector based on AdaBoost & decision tree, and a parameter selector based on BP neural network are established for the differential evolution algorithm. Due to the problems of low accuracy and increasing control parameters in the existing parameter adaptive methods of differential evolution (DE) algorithm, in this paper a mutation operator selector and a parameter selector are proposed through Fitness Landscape (FL) analysing. At first, the performance differences of the two categories of mutation operators named DE / b e s t / 1 and DE / c u r r e n t - t o - r a n d / 1 were analyzed on many test problems. Secondly, the relationship between the FL and mutation operator is founded by using ensemble learning and decision tree, and achieved a classifier named mutation operator selector. Thirdly, the relationship between the FL and algorithm parameters is founded by using a neural network, and then a classifier named parameter selector is achieved. Finally, the improved DE algorithm equip with the two selectors is tested on the CEC2017 benchmark set. The results show that the proposed improved DE algorithm is outperforms both the basis DE algorithm and other three state-of-the-arts algorithms. [ABSTRACT FROM AUTHOR]