Search

• This paper proposes a novel multivariate dictionary learning framework. • A coherence enhancement term to improve the coherent representing of the coding coefficients under different resolutions. • A multivariate dictionary optimization method to solve dictionaries involving the calculation of fractional norm. • The proposed method achieves the state-of-the-art performance on several benchmark datasets. Off-the-shelf dictionary learning algorithms have achieved satisfactory results in small sample face recognition applications. However, the achieved results depend on the facial images obtained at a single resolution. In practice, the resolution of the images captured on the same target is different because of the different shooting equipment and different shooting distances. These images of the same category at different resolutions will pose a great challenge to these algorithms. In this paper, we propose a Joint Coupled Representation and Homogeneous Reconstruction (JCRHR) for multi-resolution small sample face recognition. In JCRHR, an analysis dictionary is introduced and combined with the synthetic dictionary for coupled representation learning, which better reveals the relationship between coding coefficients and samples. In addition, a coherence enhancement term is proposed to improve the coherent representation of the coding coefficients at different resolutions, which facilitates the reconstruction of the sample by its homogeneous atoms. Moreover, each sample at different resolutions is assigned a different coding coefficient in the multi-dictionary learning process, so that the learned dictionary is more in line with the actual situation. Furthermore, a regularization term based on the fractional norm is drawn into the dictionary coupled learning to remove the redundant information in the dictionary, which can reduce the negative impacts of the redundant information. Comprehensive results demonstrate that the proposed JCRHR method achieves better results than the state-of-the-art methods, on several small sample face databases. [ABSTRACT FROM AUTHOR]

Federated learning is a novel distribute machine learning paradigm to support cooperative model training among multiple participant clients, where each client keeps its private data locally to protect its data privacy. However, in practical application domains, Federated learning still meets several heterogeneous challenges such data heterogeneity, model heterogeneity, and computation heterogeneity, significantly decreasing its global model performance. To the best of our knowledge, existing solutions only focus on one or two challenges in their heterogeneous settings. In this paper, to address the above challenges simultaneously, we present a novel solution called Full Heterogeneous Federated Learning (FHFL). Firstly, we propose a synthetic data generation approach to mitigate the Non-IID data heterogeneity problem. Secondly, we use knowledge distillation to learn from heterogeneous models of participant clients for model aggregation in the central server. Finally, we produce an opportunistic computation schedule strategy to exploit the idle computation resources for fast-computing clients. Experiment results on different datasets show that our FHFL method can achieve an excellent model training performance. We believe it will serve as a pioneer work for distributed model training among heterogeneous clients in Federated learning. [ABSTRACT FROM AUTHOR]

The optimize control of the ultra supercritical (USC) unit has been a major concern in power industry. The intermediate point temperature process is a multi-variable system with strong nonlinearity, large scale and great delay, which greatly affects the safety and economy of the USC unit. Generally, it is difficult to realize effective control by using conventional methods. This paper presents a nonlinear generalized predictive control based on a composite weighted human learning optimization network (CWHLO-GPC) to improve the control performance of intermediate point temperature. Based on the characteristics of the onsite measurement data, the heuristic information is incorporated into the CWHLO network, and expressed by different local linear models. Then, global controller is elaborately constituted based on a scheduling program inferred from the network. Compared with classical generalized predictive control (GPC), the non-convex problem is effectively solved by introducing CWHLO models into the convex quadratic program (QP) routine of local linear GPC. Finally, detailed analysis on set point tracking and interference resisting via simulation is addressed to illustrate the efficiency of the proposed strategy. • A network with self-learning and self-organizing abilities is delicately constructed. Through reasonable partition, the nonlinearity of the object system is diluted to an acceptable range. • Real-coded human learning optimization algorithm is first proposed and applied to local linear modeling, by which more accurate models are obtained. • An improved generalized predictive control algorithm is proposed based on local models. And by the mutual cooperation of local controllers, the problem of non-convex in quadratic programming is effectively solved. [ABSTRACT FROM AUTHOR]

Accurate estimation of State of Health (SOH) is crucial to ensure optimal performance and safe operation of lithium-ion battery. This paper proposes a Stacking ensemble learning paradigm for SOH estimation. The Stacking ensemble learning increases adaptability to different features by using base learners with different structures, reducing the risk of overfitting. The model utilizes random vector functional link (RVFL) and active state tracking long-short-term memory network (AST-LSTM) as base learners, where AST-LSTM actively tracks long-term information of lithium-ion battery, and RVFL acts as the meta-learner for stacking. The random vector functional link network helps to avoid the problem of gradient vanishing that is commonly encountered in neural networks due to the gradient descent principle. To further improve estimation accuracy, Singer initialization method and dimension learning method are employed to enhance the Heap-based optimization (HBO) algorithm. In this study, the IHBO algorithm is used to optimize the hyperparameters of the model. Comparing with other methods, the hybrid model proposed in this paper demonstrates superior estimation performance under different operating conditions: at a temperature of 24 °C with a discharge current of 1 A, at a temperature of 4 °C with a discharge current of 1 A, and at a temperature of 4 °C with a discharge current of 2 A. The highest RMSE of the proposed method for the three working conditions are 0.006, 0.01, and 0.017, respectively. Therefore, the proposed Stacking ensemble learning is feasible for SOH estimation of lithium-ion battery and can better adapt to lithium-ion battery data under different operating conditions. • A Stacking ensmeble learning strategy is proposed for SOH estimation. • The improved HBO algorithm is introduced to optimize the Stacking model. • SOH for lithium batteries under three different operating conditions is estimated. • Six benchmark models are used to verify the performance of the proposed model. [ABSTRACT FROM AUTHOR]

Metric learning has attracted significant attention due to its high effectiveness and efficiency for pattern recognition task. Traditional supervised metric learning algorithms attempt to seek a global distance metric with labeled samples. When data are represented with multimodal and only limited supervision information is available, these approaches are insufficient to obtain satisfactory results. In this paper, we develop a robust semi-supervised multi-metric learning method (RSMM) to improve classification performance. The proposed RSMM learns multiple local metrics and a background metric instead of a single global metric. Specifically, we divide the metric space into influential regions and background region, and then regulate the effectiveness of each local metric to be within the related regions. Simultaneously, a geometrically interpretable, symmetric distance is defined with local metrics and background metric. Based on the resultant learning bounds, we obtain the regularization term to improve the classifier's generalization ability. Moreover, the manifold regularization term is introduced to preserve the supervision information as well as geometry structure. The substantial unlabeled samples may cause potential threats and large uncertainties, so the logarithmic loss function is utilized to enhance the robustness. An efficient gradient descent algorithm is exploited to solve the non-convex challenging problem. To further understand the proposed algorithm, we theoretically derive its robustness and generalization error bounds. Finally, numerical experiments on UCI datasets and image datasets demonstrate the feasibility and validity of the RSMM. [ABSTRACT FROM AUTHOR]

Class imbalance distribution widely exists in real-world engineering. However, the mainstream optimization algorithms that seek to minimize error will trap the deep learning model in sub-optimums when facing extreme class imbalance. It seriously harms the classification precision, especially in the minor classes. The essential reason is that the gradients of the classifier weights are imbalanced among the components from different classes. In this paper, we propose Attraction–Repulsion-Balanced Loss (ARB-Loss) to balance the different components of the gradients. We perform experiments on large-scale classification and segmentation datasets, and our ARB-Loss can achieve state-of-the-art performance via only one-stage training instead of 2-stage learning like nowadays SOTA works. [ABSTRACT FROM AUTHOR]

Cost-sensitive learning (CSL), which has gained widespread attention in class imbalance learning (CIL), can be implemented either by tuning penalty parameters or by designing new loss functions. In this paper, we propose a cost-sensitive learning method with a modified Stein loss function (CSMS) and a robust CSMS (RCSMS). Specifically, CSMS is flexible, as it realizes CSL from above two aspects simultaneously. In contrast, RCSMS merely achieves CSL by tuning penalty parameters, but the adopted loss function makes it insensitive to noise. To our best knowledge, it is the first time for Stein loss function derived from statistics to be applied in machine learning, which not only offers two alternative class imbalance solutions but also provides a novel idea for the design of loss functions in CIL. The mini-batch stochastic sub-gradient descent (MBGD) approach is employed to optimize CSMS and RCSMS. Meanwhile, the Rademacher complexity is used to analyze their generalization error bounds. Extensive experiments profoundly confirm the superiority of both models over benchmarks. [ABSTRACT FROM AUTHOR]

Federated Learning is a novel framework that allows multiple devices or institutions to train a machine learning model collaboratively while preserving their data private. This decentralized approach is prone to suffer the consequences of data statistical heterogeneity, both across the different entities and over time, which may lead to a lack of convergence. To avoid such issues, different methods have been proposed in the past few years. However, data may be heterogeneous in lots of different ways, and current proposals do not always determine the kind of heterogeneity they are considering. In this work, we formally classify data statistical heterogeneity and review the most remarkable learning Federated Learning strategies that are able to face it. At the same time, we introduce approaches from other machine learning frameworks. In particular, Continual Learning strategies are worthy of special attention, since they are able to handle habitual kinds of data heterogeneity. Throughout this paper, we present many methods that could be easily adapted to the Federated Learning settings to improve its performance. Apart from theoretically discussing the negative impact of data heterogeneity, we examine it and show some empirical results using different types of non-IID data. • We review the advances in Federated Learning, particularly on heterogeneous data. • We consider time-evolving situations in Federated Learning. • We establish a classification of the literature according to their data heterogeneity. • We settle restrictions inherent to some of the problem scenarios. • We examine the open challenges and future directions for Federated Learning. [ABSTRACT FROM AUTHOR]

Lifelong learning, the ability to continually learn new concepts throughout our life, is a hallmark of human intelligence. Generally, humans learn a new concept by knowing what it looks like and what makes it different from the others , which are correlated. Those two ways can be characterized by generation and classification in machine learning respectively. In this paper, we carefully design a dynamically growing GAN called Introspective GAN (IntroGAN) that can perform incremental generation and classification simultaneously with the guidance of prototypes, inspired by their roles of efficient information organization in human visual learning and excellent performance in other fields like zero-shot/few-shot/incremental learning. Specifically, we incorporate prototype-based classification which is robust to feature change in incremental learning and GAN as a generative memory to alleviate forgetting into a unified end-to-end framework. A comprehensive benchmark on the joint incremental generation and classification task is proposed and our method demonstrates promising results. Additionally, we conduct comprehensive analyses over the properties of IntroGAN and verify that generation and classification can be mutually beneficial in incremental scenarios, which is an inspiring area to be further exploited. The code is available at https://github.com/TonyPod/IntroGAN. • Propose a benchmark to evaluate joint incremental generation and classification. • Propose an IntroGAN to address the joint incremental and classification task. • Introduce a fine-tuning technique that can accelerate incremental GAN training. • Superior performance on proposed benchmarks and more challenging scenarios. [ABSTRACT FROM AUTHOR]

This paper investigates the end-to-end human-Mars entry, powered-descent, and landing (EDL) guidance problem by developing an on-board learning-based optimal control method (L-OCM) to achieve the goal of precise and fuel-efficient planetary landing. First, the end-to-end EDL guidance problem is formulated as a multi-phase optimal control problem with hybrid dynamics and constraints. Then a customized alternating direction method of multipliers is applied to solve the end-to-end EDL guidance problem with varying initial states off-line. After that, the L-OCM is developed to generate real-time optimal guidance commands. To be specific, supported by the optimal control theory, the necessary conditions of optimality for optimal control of the entry phase and powered-descent phase are derived, respectively, which leads to two two-point-boundary-value-problems (TPBVPs). Then, critical parameters are identified to approximate the complete solutions of the TPBVPs. To find the implicit relationship between the initial states and these critical parameters, deep neural networks are constructed to learn the values of these critical parameters in real-time with training data obtained from the off-line solutions. Furthermore, when random disturbances are considered during the EDL process, the single stage L-OCM is extended to the multi-stage L-OCM to regenerate real-time optimal guidance commands. Finally, the proposed L-OCM is implemented in extensive simulation cases to verify the effectiveness and efficiency of the new method. • We consider the multi-phase guidance problem integrating the entry and powered descent phases. • The learning method only needs to learn identified critical parameters to reduce computation load. • The proposed approach can handle multi-type disturbances during the EDL mission. • The proposed method can achieve precise and fuel-efficient planetary landing in real-time. [ABSTRACT FROM AUTHOR]

The architecture of AND/OR fuzzy neural networks exhibits outstanding learning abilities and significant interpretation capabilities. However, AND/OR networks suffer from structure-related problems namely low efficiency and slow convergence of learning due to several reasons such as high dimensionality and gradient-based learning algorithms which lead to a visible computing overhead. In this paper, we present a two-phase fuzzy logic-oriented network design that is composed of AND/OR neurons. This design takes advantages of Randomized Neural Network (RNN) to achieve higher convergence while exhibiting good nonlinear approximation capabilities. A gradient–based learning algorithm is implemented in the second phase of the design to further reduce values of performance index. The quality of the proposed design and resulting architecture is quantified through the use of numeric data along with fuzzy sets (information granules). Experimental results meet the research's objectives and the proposed design methodology opens up new future directions for proceeding with more improvements. [ABSTRACT FROM AUTHOR]

Continual lifelong learning is an machine learning framework inspired by human learning, where learners are trained to continuously acquire new knowledge in a sequential manner. However, the non-stationary nature of streaming training data poses a significant challenge known as catastrophic forgetting, which refers to the rapid forgetting of previously learned knowledge when new tasks are introduced. While some approaches, such as experience replay (ER), have been proposed to mitigate this issue, their performance remains limited, particularly in the class-incremental scenario which is considered natural and highly challenging. In this paper, we present a novel algorithm, called adaptive-experience replay (AdaER), to address the challenge of continual lifelong learning. AdaER consists of two stages: memory replay and memory update. In the memory replay stage, AdaER introduces a contextually-cued memory recall (C-CMR) strategy, which selectively replays memories that are most conflicting with the current input data in terms of both data and task. Additionally, AdaER incorporates an entropy-balanced reservoir sampling (E-BRS) strategy to enhance the performance of the memory buffer by maximizing information entropy. To evaluate the effectiveness of AdaER, we conduct experiments on established supervised continual lifelong learning benchmarks, specifically focusing on class-incremental learning scenarios. The results demonstrate that AdaER outperforms existing continual lifelong learning baselines, highlighting its efficacy in mitigating catastrophic forgetting and improving learning performance. [ABSTRACT FROM AUTHOR]

• The paper explores the construction method for compact heterogeneous ensemble models. • The paper adopts the exhaustive search method for optimal model identification. • The paper proves the necessity for optimal model identification. • The paper analyzes the benefits and limitations of the exhaustive search method. Heterogeneous ensemble learning has received increasing attention in building energy prediction research because it provides more stable and accurate predictions than the conventional single model-based methods. Although existing studies have proved the feasibility of adopting heterogeneous ensemble learning in building energy prediction, they have not fully explored the method of constructing an optimal heterogeneous ensemble learning model based on limited base model resources. This paper attempts to identify the optimal heterogeneous ensemble learning model for building energy prediction by using the exhaustive search method. Six machine learning methods are used as the alternative base model for constructing heterogeneous ensemble learning models. All 57 heterogeneous ensemble learning models that can be built from the six base models are compared and the optimal model is identified. The hourly building energy prediction experiment is performed on an institutional building located at the University of Florida to verify the proposed exhaustive search method. Three modules representing different semesters of the test building are developed and tested to examine the predictive performance of the proposed heterogeneous ensemble learning models. The research findings demonstrate the feasibility of the proposed exhaustive search method in identifying the optimal heterogeneous ensemble learning model. The optimal heterogeneous ensemble learning model found in this study can reduce the MAPE by 4.9% − 6.6% compared with its most accurate base model in the model testing stage. The experimental results and research findings provide critical guidance for applying heterogeneous ensemble learning to data-driven building energy prediction. [ABSTRACT FROM AUTHOR]

When performing learning tasks in pervasive environments, the main challenge arises from the need of combining federated and continual settings. The former comes from the massive distribution of devices with privacy-regulated data. The latter is required by the low resources of the participating devices, which may retain data for short periods of time. In this paper, we propose a setup for learning with Echo State Networks (ESNs) in pervasive environments. Our proposal focuses on the use of Intrinsic Plasticity (IP), a gradient-based method for adapting the reservoir's non-linearity. First, we extend the objective function of IP to include the uncertainty arising from the distribution of the data over space and time. Then, we propose Federated Intrinsic Plasticity (FedIP), which is intended for client–server federated topologies with stationary data, and adapts the learning scheme provided by Federated Averaging (FedAvg) to include the learning rule of IP. Finally, we further extend this algorithm for learning to Federated Continual Intrinsic Plasticity (FedCLIP) to equip clients with CL strategies for dealing with continuous data streams. We evaluate our approach on an incremental setup built upon real-world datasets from human monitoring, where we tune the complexity of the scenario in terms of the distribution of the data over space and time. Results show that both our algorithms improve the representation capabilities and the performance of the ESN, while being robust to catastrophic forgetting. • Reservoir computing is very effective for continual and distributed learning. • Adding continual and federated learning to echo state networks. • Unsupervised federated and continual adaptation of reservoir. • Federated and continual learning for human action recognition and stress prediction. [ABSTRACT FROM AUTHOR]

Human Learning Optimization (HLO) is a simple yet powerful meta-heuristic developed based on a simplified human learning model. Many cognitive activities of humans contain an element of reasoning, and with reasoning, humans can gain deeper information on problems to boost learning performance. Inspired by this fact, this paper proposes a novel human learning optimization algorithm with reasoning learning (HLORL), in which a social reasoning learning operator (SRLO) is developed by using multiple social information sources to improve the global search ability of the algorithm. A parameter study is performed to give the recommended values of the control parameters. It also analyzes and discusses the role and function of the social reasoning learning operator. Finally, the proposed HLORL is applied to solve the CEC14 benchmark functions and 0-1 knapsack problems. The performance of HLORL is compared with the previous HLO variants and other state-of-art meta-heuristics. The experimental results demonstrate that the proposed HLORL has significant advantages over the compared algorithms. • This paper proposes a novel human learning optimization algorithm with reasoning learning (HLORL). • A social reasoning learning operator is developed by using multiple social information sources. • The role and function of the social reasoning learning operator is deeply analyzed and discussed. • The results demonstrate that the proposed HLORL has significant advantages over the previous HLO variants. [ABSTRACT FROM AUTHOR]

• Several critical issues in the data-driven model update raised wide concerns, e.g., concept drift, and catastrophic forgetting. • A comprehensive investigation on three conventional model update methods and five emerging continual learning methods is carried out on a large open-source building dataset. • The model performances under different concept drifts are further investigated via clustering analysis. Data-driven models have been increasingly employed in smart building energy management. To avoid performance degradation over time, data-driven models need to be continually updated to adapt to the changes in building operations. However, several critical issues in the model update process raised wide concerns, especially the concept drift and catastrophic forgetting issues. The concept drift issue happens when the statistical properties of target variable change over time in unforeseen ways. The catastrophic forgetting issue refers to the process that the previously learnt knowledge or patterns may be diluted and eventually lost in model update. Although a few model update methods were proposed, there is a lack of comprehensive comparison of the methods for adaptive data-driven building energy prediction. This paper conducted a comprehensive investigation on the performance of three conventional model update methods and five emerging continual learning methods using 2-year data of 100 buildings extracted from an open-source dataset. The results show that continual learning methods are more effective in ensuring long-term accuracy while cutting down on computation time and data storage expenses. The CV-RMSE of Elastic weight consolidation and Gradient episodic memory decreased by around 14% and 8% on average compared with static model and accumulative learning. The comparison results are valuable to the development of adaptive data-driven building energy prediction models which are more reliable over time and robust against changing operation conditions, thus more practically applicable in smart building energy management. [ABSTRACT FROM AUTHOR]

• An effective approach for class incremental learning. • Simulate the human brain's ability to acquire new knowledge while integrating old knowledge. • Isolate the learning environments of new knowledge and old knowledge to mitigate catastrophic forgetting. • It outperforms state-of-the-art methods on avoiding catastrophic forgetting. Humans are capable of acquiring new knowledge on a constant basis, while integrating and optimizing old knowledge without forgetting them. This is mainly attributed to the human brain's ability of partitioned learning and memory replay. In this paper, we simulate this ability and propose an incremental learning network of Sustainable Regional Isolation and Integration (SRII). SRII consists of two phases, regional isolation and regional integration, which are iterated to achieve continuous incremental class learning. Regional isolation isolates new learning processes to avoid interfering with existing knowledge, while regional integration uses knowledge distillation and a margin loss regularization term. The knowledge distillation transfers replay knowledge for alleviating catastrophic forgetting, while the margin loss regularization term is to clarify the boundaries of new and old knowledge for alleviating recency bias. Experimental results on the CIFAR100 and miniImageNet datasets demonstrate that SRII outperforms the state-of-the-arts on avoiding catastrophic forgetting. In all 5-stage and 10-stage incremental settings, SRII outperforms the baseline and achieves at least 5.09%+ average accuracy improvement. Our source code is available at https://github.com/Wuziyi123/SRII. [Display omitted] [ABSTRACT FROM AUTHOR]

Ultra supercritical power plant (USC) is a complex system associated with nonlinearity, uncertainties and multivariable couplings. Generally, it is difficult to build an accurate model to approximate the dynamic behavior of USC. This paper presents a novel composite weighted human learning optimization network (CWHLO) to tackle the above-mentioned problem. Firstly, by fully using of the statistic characteristic of the history operating data, K-means clustering algorithm is applied to partition the raw date, which extremely reduces the operating nonlinearity. Then, an improved real-coded human learning optimization (HLO) is adopted to built linear models in local regions. Different from conventional methods, the advantage of the proposed CWHLO is that the nonlinear model of the object is effectively replaced by a real-time dynamic linear model, which is more suitable for other control methods. Finally, the CWHLO model is compared with the traditional recursive least square method (RLS), and four other meta-heuristic algorithms, to show the advantages in approximating the dynamic behavior of USC. • By partitioning, the nonlinearity of the target is diluted to an acceptable range. • By using a compensation mechanism, the linearity of the obtained model is ensured. • Through iterative learning of HLO, reliable local model parameters are obtained. • By fuzzy fusion of local models, an accurate model of the whole system is obtained. [ABSTRACT FROM AUTHOR]

Machine learning (ML) and deep learning (DL) are both branches of AI. As a form of AI, ML automatically adapts to changing datasets with minimal human interference. Deep learning is a subset of ML that uses artificial neural networks to imitate the learning process of the human brain. The 'black box' nature of ML and DL makes their inner workings difficult to understand and interpret. Deploying explainable artificial intelligence (XAI) can help explain why and how the output of ML and DL models are generated. As a result, understanding a model's functioning, behavior, and outputs can be garnered, reducing bias and error and improving confidence in decision-making. Despite providing an improved understanding of model outputs, XAI has received limited attention in construction. This paper presents a narrative review of XAI and a taxonomy of precepts and models to raise awareness about its potential opportunities for use in construction. It is envisaged that the opportunities suggested can stimulate new lines of inquiry to help alleviate the prevailing skepticism and hesitancy toward AI adoption and integration in construction. [ABSTRACT FROM AUTHOR]

Humans perform Bayesian inference in a wide variety of tasks, which can help people make selection decisions effectively and therefore enhances learning efficiency and accuracy. Inspired by this fact, this paper presents a novel human learning optimization algorithm with Bayesian inference learning (HLOBIL), in which a Bayesian inference learning operator (BILO) is developed to utilize the inference strategy for enhancing learning efficiency. The in-depth analysis shows that the proposed BILO can efficiently improve the exploitation ability of the algorithm as it can achieve the optimal values and retrieve the optimal information with the accumulated search information. Besides, the exploration ability of HLOBIL is also strengthened by the inborn characteristics of Bayesian inference. The experimental results demonstrate that the developed HLOBIL is superior to previous HLO variants and other state-of-art algorithms with its improved exploitation and exploration abilities. [ABSTRACT FROM AUTHOR]

Teaching plays a fundamental role in human learning. Typically, a human teaching strategy involves assessing a student's knowledge progress for tailoring the teaching materials to enhance the learning progress. A human teacher can achieve this by tracing a student's knowledge over essential learning concepts in a task. Albeit, such a teaching strategy is not well exploited yet in machine learning as current machine teaching methods tend to directly assess the progress of individual training samples without paying attention to the underlying learning concepts in a learning task. In this paper, we propose a novel method, called Knowledge Augmented Data Teaching (KADT), which can optimize a data teaching strategy for a student model by tracing its knowledge progress over multiple learning concepts in a learning task. Specifically, the KADT method incorporates a knowledge tracing model to dynamically capture the knowledge progress of a student model in terms of latent learning concepts. We further develop an attention-pooling mechanism to distill knowledge representations of a student model with respect to class labels, which enables to develop a data teaching strategy on critical training samples. We have evaluated the performance of the KADT method on four different machine learning tasks, including knowledge tracing, sentiment analysis, movie recommendation, and image classification. The KADT method consistently outperforms the state-of-the-art methods on all these tasks. [ABSTRACT FROM AUTHOR]

Modern power system demonstrates good performance to withstand a single failure. However, the recent progress has shown that power system is increasingly threatened by the sequential attacks. In this paper, we aim at evaluating the robustness of power system under the sequential attacks with incomplete information, taking into account both the electrical properties and the cascading failure process. In light of this, we first formulate the sequential attacks as a partial observable Markov decision process, and use Deep Q-network algorithm to identify the optimal attack sequence. The influences of network structures, attacks methods and protection measures are demonstrated in complex networks and IEEE 118-Bus system. Experimental results show that our proposed algorithm can effectively identify the optimal attack sequence under different attack methods and protection measures. Moreover, a larger redundant parameter and homogeneous network can improve the robustness of power system. Our findings can provide practical insights for building a robust power system. • A partial observable Markov decision process is used to depict the strategy learning process of sequential attacks. • Deep Q-network is adopted to adaptively identify the optimal attack sequence. • Various critical factors are analyzed to evaluate the robustness of power system under sequential attacks. [ABSTRACT FROM AUTHOR]

Reference governors (RGs) provide an effective method for ensuring safety via constraint enforcement in closed-loop nonlinear control systems. When the system parameters are uncertain but constant, robust formulations of RGs that consider only the worst-case effect may be overly conservative and exhibit poor performance. This paper proposes a parameter-adaptive reference governor (PARG) architecture that is capable of generating safe trajectories in spite of parameter uncertainties, without being as conservative as robust RGs. The proposed approach employs machine learning on a combination of off-line simulations and on-line measurements to estimate parameter-robust constraint-admissible sets (PRCASs) that can be leveraged by the PARG. We illustrate the robust set learning and constraint enforcement qualities of the PARG using a two-dimensional electromagnetic actuator example, and further demonstrate the potential of the PARG on a vehicle case study for preventing rollover despite aggressive maneuvering. [ABSTRACT FROM AUTHOR]

The gradient-based meta learning and its approximation algorithms have been widely used in the few-shot scenarios. In practice, it is common for the trained meta-model to employ uniform settings for gradient descent steps across different tasks. However, the meta-model may be biased toward some tasks. The convergence issue occurs that some tasks may see convergence in a few steps while others fail to approach the optimum in the whole inner loop. The bias problem may cause the trained meta-model works well in some tasks but has unexpected bad performance in other tasks, which hurts the generality of the meta-model. To address this issue, in this paper, we formally establish the approximation between the metric-based strategy and gradient descent in meta-test. By directly calculating similarity to classify data, the trained meta-model avoids the convergence issue. We point out that the metric-based methods can closely approximate the gradient descent in meta-test if the representation capability of the derived features and the convergence of the inner loop during meta-training are guaranteed. Based on such observation, we propose a new meta-learning model GMT2 (Gradient-based Meta-Train with Metric-based meta-Test) by combining gradient descent in meta-training with metric-based methods in meta-test. GMT2 employs a new first-order approximation scheme using the adversarial update strategy which not only enhances the feature representation of inner layers, but also allows enough inner gradient steps without calculating second-order derivatives. Experiments show that GMT2 achieves better efficiency and competitive accuracy comparing with popular meta-learning models. [ABSTRACT FROM AUTHOR]

• We design a three layer deep learning framework for outlier rejection. • We propose a novel joint representation attention mechanism. • We design an innovative weight function to improve the generalization ability. • Experimental results show the proposed network is superior to state of the art networks. In this paper, we propose a Joint Representation Attention Network (JRA-Net), an end-to-end network, to establish reliable correspondences for image pairs. The initial correspondences generated by the local feature descriptor usually suffer from heavy outliers, which makes the network unable to learn a powerful enough representation for distinguishing inliers and outliers. To this end, we design a novel attention mechanism. The proposed attention mechanism not only takes into account the correlations between global context and geometric information, but also introduces the joint representation of different scales to suppress trivial correspondences and highlight crucial correspondences. In addition, to improve the generalization ability of attention mechanism, we present an innovative weight function, to effectively adjust the importance of the attention mechanism in a learning manner. Finally, by combining the above components, the proposed JRA-Net is able to effectively infer the probabilities of correspondences being inliers. Empirical experiments on challenging datasets demonstrate the effectiveness and generalization of JRA-Net. We achieve remarkable improvements compared with the current state-of-the-art approaches on outlier rejection and relative pose estimation. [ABSTRACT FROM AUTHOR]

Federated learning (FL) guaranteeing data privacy is of great interest in decentralized fault diagnosis. However, limited research attention has been paid to the dynamic domain-shift issue due to varying working conditions. This paper proposes an active federated transfer algorithm based on broad learning to address the domain shift issue in FL. First, a central server dispatches a global model to the source clients for collaborative modeling. Subsequently, the global model is initialized with a federated averaging strategy. Next, the initialized global model is used to annotate emerging signals from the target clients based on an active sampling strategy proposed. Finally, an asynchronous update scheme is designed to adapt the global model to the target domain. The performance of the AFTBL algorithm is validated with three datasets, including 24 centralized- and decentralized-modeling tasks. The computational results indicate that the proposed algorithm is more accurate and efficient than the prevalent algorithms. • The cross-domain incremental federated learning problem is investigated. • An active federated broad transfer algorithm is proposed for fault diagnosis. • Automated data annotation and selection are proposed for incremental model update. [ABSTRACT FROM AUTHOR]

The most widely accepted view of memory in the brain holds that synapses are the storage sites of memory, and that memories are formed through associative modification of synapses. This view has been challenged on conceptual and empirical grounds. As an alternative, it has been proposed that molecules within the cell body are the storage sites of memory, and that memories are formed through biochemical operations on these molecules. This paper proposes a synthesis of these two views, grounded in a computational model of memory. Synapses are conceived as storage sites for the parameters of an approximate posterior probability distribution over latent causes. Intracellular molecules are conceived as storage sites for the parameters of a generative model. The model stipulates how these two components work together as part of an integrated algorithm for learning and inference. [ABSTRACT FROM AUTHOR]