Search

The traditional method for transformer moisture diagnosis is to establish empirical equations between feature parameters extracted from frequency domain spectroscopy (FDS) and the transformer’s moisture content. However, the established empirical equation may not be applicable to a novel testing environment, resulting in an unreliable evaluation result. In this regard, it is acknowledged that FDS combined with machine learning is more suitable for estimating moisture content in a variety of test environments. Nonetheless, the accuracy of the estimation results obtained using the existing method is limited by the algorithm’s inability to generalize. To address this issue, we propose an AdaBoost algorithm-enhanced back-propagation neural network (BP_AdaBoost). This study creates a database by extracting feature parameters from the FDS that characterize the insulation states of the prepared samples. Then, using the BP_AdaBoost algorithm and the newly constructed database, the moisture estimation models are trained. Finally, the results of the estimation are discussed in terms of laboratory and field transformers. By comparing the proposed BP_AdaBoost algorithm to other intelligence algorithms, it is demonstrated that it not only performs better in generalization, but also maintains a high level of accuracy. [ABSTRACT FROM AUTHOR]

This paper investigates the stochastic optimization problem focusing on developing scalable parallel algorithms for deep learning tasks. Our solution involves a reformation of the objective function for stochastic optimization in neural network models, along with a novel parallel computing strategy, coined the weighted aggregating stochastic gradient descent (WASGD). Following a theoretical analysis on the characteristics of the new objective function, WASGD introduces a decentralized weighted aggregating scheme based on the performance of local workers. Without any center variable, the new method automatically gauges the importance of local workers and accepts them by their contributions. Furthermore, we have developed an enhanced version of the method, WASGD+, by (1) implementing a designed sample order and (2) upgrading the weight evaluation function. To validate the new method, we benchmark our pipeline against several popular algorithms including the state-of-the-art deep neural network classifier training techniques (e.g., elastic averaging SGD). Comprehensive validation studies have been conducted on four classic datasets: CIFAR-100, CIFAR-10, Fashion-MNIST, and MNIST. Subsequent results have firmly validated the superiority of the WASGD scheme in accelerating the training of deep architecture. Better still, the enhanced version, WASGD+, is shown to be a significant improvement over its prototype. [ABSTRACT FROM AUTHOR]

This paper focuses on similarity caching systems, in which a user request for an object $o$ that is not in the cache can be (partially) satisfied by a similar stored object $o'$ , at the cost of a loss of user utility. Similarity caching systems can be effectively employed in several application areas, like multimedia retrieval, recommender systems, genome study, and machine learning training/serving. However, despite their relevance, the behavior of such systems is far from being well understood. In this paper, we provide a first comprehensive analysis of similarity caching in the offline, adversarial, and stochastic settings. We show that similarity caching raises significant new challenges, for which we propose the first dynamic policies with some optimality guarantees. We evaluate the performance of our schemes under both synthetic and real request traces. [ABSTRACT FROM AUTHOR]

The Area under the ROC curve (AUC) is a well-known ranking metric for problems such as imbalanced learning and recommender systems. The vast majority of existing AUC-optimization-based machine learning methods only focus on binary-class cases, while leaving the multiclass cases unconsidered. In this paper, we start an early trial to consider the problem of learning multiclass scoring functions via optimizing multiclass AUC metrics. Our foundation is based on the M metric, which is a well-known multiclass extension of AUC. We first pay a revisit to this metric, showing that it could eliminate the imbalance issue from the minority class pairs. Motivated by this, we propose an empirical surrogate risk minimization framework to approximately optimize the M metric. Theoretically, we show that: (i) optimizing most of the popular differentiable surrogate losses suffices to reach the Bayes optimal scoring function asymptotically; (ii) the training framework enjoys an imbalance-aware generalization error bound, which pays more attention to the bottleneck samples of minority classes compared with the traditional $O(\sqrt{1/N})$ O (1 / N) result. Practically, to deal with the low scalability of the computational operations, we propose acceleration methods for three popular surrogate loss functions, including the exponential loss, squared loss, and hinge loss, to speed up loss and gradient evaluations. Finally, experimental results on 11 real-world datasets demonstrate the effectiveness of our proposed framework. The code is now available at https://github.com/joshuaas/Learning-with-Multiclass-AUC-Theory-and-Algorithms. [ABSTRACT FROM AUTHOR]

In real-world applications, it is important for machine learning algorithms to be robust against data outliers or corruptions. In this paper, we focus on improving the robustness of a large class of learning algorithms that are formulated as low-rank semi-definite programming (SDP) problems. Traditional formulations use the square loss, which is notorious for being sensitive to outliers. We propose to replace this with more robust noise models, including the $\ell _1$ ℓ 1 -loss and other nonconvex losses. However, the resultant optimization problem becomes difficult as the objective is no longer convex or smooth. To alleviate this problem, we design an efficient algorithm based on majorization-minimization. The crux is on constructing a good optimization surrogate, and we show that this surrogate can be efficiently obtained by the alternating direction method of multipliers (ADMM). By properly monitoring ADMM's convergence, the proposed algorithm is empirically efficient and also theoretically guaranteed to converge to a critical point. Extensive experiments are performed on four machine learning applications using both synthetic and real-world data sets. Results show that the proposed algorithm is not only fast but also has better performance than the state-of-the-arts. [ABSTRACT FROM AUTHOR]

This paper studies an unmanned aerial vehicle (UAV)-assisted wireless powered IoT network, where a rotary-wing UAV adopts fly-hover-communicate protocol to successively visit IoT devices in demand. During the hovering periods, the UAV works on full-duplex mode to simultaneously collect data from the target device and charge other devices within its coverage. Practical propulsion power consumption model and non-linear energy harvesting model are taken into account. We formulate a multi-objective optimization problem to jointly optimize three objectives: maximization of sum data rate, maximization of total harvested energy and minimization of UAV’s energy consumption over a particular mission period. These three objectives are in conflict with each other partly and weight parameters are given to describe associated importance. Since IoT devices keep gathering information from the physical surrounding environment and their requirements to upload data change dynamically, online path planning of the UAV is required. In this paper, we apply deep reinforcement learning algorithm to achieve online decision. An extended deep deterministic policy gradient (DDPG) algorithm is proposed to learn control policies of UAV over multiple objectives. While training, the agent learns to produce optimal policies under given weights conditions on the basis of achieving timely data collection according to the requirement priority and avoiding devices’ data overflow. The verification results show that the proposed MODDPG (multi-objective DDPG) algorithm achieves joint optimization of three objectives and optimal policies can be adjusted according to weight parameters among optimization objectives. [ABSTRACT FROM AUTHOR]

We consider a class of nonconvex regularized optimization problems, which appear frequently in machine learning and data processing. Due to the structure of the problems, the iteratively reweighted algorithm was developed and applied to the consensus optimization. In this paper, we propose the acceleration of this scheme by adding an inertial term in each iteration. The proposed algorithms inherit the advantages of classical decentralized algorithms: they can be implemented over a connected network, in which the agents communicate with their neighbors and perform local computations. We also employ the diminishing stepsizes technique for the iteratively reweighted algorithm and consider its acceleration. In specific cases, our algorithms reduce to existing decentralized schemes and also indicate novel ones. Mathematically, we prove the convergence for both algorithms with several assumptions on the objective functions. With Kurdyka-Łojasiewicz property, convergence rates can be derived for constant stepsize case. Numerical results demonstrate the efficiency of the algorithms. [ABSTRACT FROM AUTHOR]

In this paper, we propose to extend the standard Convolutional Dictionary Learning problem to a tensor representation where the activations are constrained to be “low-rank” through a Canonical Polyadic decomposition. We show that this additional constraint increases the robustness of the CDL with respect to noise and improve the interpretability of the final results. In addition, we discuss in detail the advantages of this representation and introduce two algorithms, based on ADMM or FISTA, that efficiently solve this problem. We show that by exploiting the low rank property of activations, they achieve lower complexity than the main CDL algorithms. Finally, we evaluate our approach on a wide range of experiments, highlighting the modularity and the advantages of this tensorial low-rank formulation. [ABSTRACT FROM AUTHOR]

For vehicle-to-network communications, handover (HO) management enables vehicles to maintain the connection with the network while transiting through coverage areas of different base stations (BSs). However, the high mobility of vehicles means shorter connection periods with each BS that leads to frequent HOs, hence raises the necessity for optimal HO decision making for high quality infotainment services. Machine learning is capable of capturing underlying pattern via data driven methods to find optimal solutions to complex problems, and much learning-based HO optimization research has been conducted focusing on specific network setups. However, attention still needs to be paid to the actual deployment aspect and standardized datasets or simulation environments for evaluation. This paper proposes a deep reinforcement learning-based HO algorithm using the input parameters that are configurable in the existing measurement report of cellular networks. The performance of the proposed algorithm is evaluated using the well-known network simulator ns-3 with its official LTE module. A realistic network setup in the city center of Glasgow (U.K.) is configured with vehicle trajectories generated by the routes mobility model using Google Maps Directions API. Evaluation results reveal that the proposed algorithm significantly outperforms the A3 RSRP baseline with an average of 25.72% packet loss reduction per HO, suggesting significant improvement in quality of service of phone call and video streaming, etc. The proposed algorithm also has a small implementation cost compared to some state-of-the-art and should be deployed by a software update to a local BS controller. [ABSTRACT FROM AUTHOR]

We consider the problem of minimizing the $\ell _1$ norm of a linear map over the sphere, which arises in various machine learning applications such as orthogonal dictionary learning (ODL) and robust subspace recovery (RSR). The problem is numerically challenging due to its nonsmooth objective and nonconvex constraint, and its algorithmic aspects have not been well explored. In this paper, we show how the manifold structure of the sphere can be exploited to design fast algorithms with provable guarantees for tackling this problem. Specifically, our contribution is fourfold. First, we present a manifold proximal point algorithm (ManPPA) for the problem and show that it converges at a global sublinear rate. Furthermore, we show that ManPPA can achieve a local quadratic convergence rate when applied to sharp instances of the problem. Second, we develop a semismooth Newton-based inexact augmented Lagrangian method for computing the search direction in each iteration of ManPPA and show that it has an asymptotic superlinear convergence rate. Third, we propose a stochastic variant of ManPPA called StManPPA, which is well suited for large-scale computation, and establish its sublinear convergence rate. Both ManPPA and StManPPA have provably faster convergence rates than existing subgradient-type methods. Fourth, using ManPPA as a building block, we propose a new heuristic method for solving a matrix analog of the problem, in which the sphere is replaced by the Stiefel manifold. The results from our extensive numerical experiments on the ODL and RSR problems demonstrate the efficiency and efficacy of our proposed methods. [ABSTRACT FROM AUTHOR]

Stochastic unit commitment is an efficient method for grid operation in the presence of significant uncertainties. An example is an operation during a predicted hurricane with uncertain line out-ages. However, the solution quality comes at the cost of substantial computational burden, which makes its adoption challenging. This paper evaluates some possible ways that machine learning can be used to reduce this computational burden. First, a set of feasibility studies is conducted. Results suggest that using machine learning as an assistant to the stochastic unit commitment solver is more advantageous than using it as a standalone solver. In particular, the machine learning model is trained to facilitate solving the problem by determining the unnecessary constraints that can be removed from the original problem without affecting the final accuracy. The variables that can be used as input features/predictors or outputs for the machine learning model are determined through feasibility studies. Then, an algorithm to train and utilize a machine learning model is proposed. The method is tested on a 500-bus synthetic South Carolina system. Various test cases show an average reduction in solution time by more than 90% by using the trained machine learning model to assist the stochastic unit commitment solver. [ABSTRACT FROM AUTHOR]

This paper considers stochastic convex optimization problems with two sets of constraints: (a) deterministic constraints on the domain of the optimization variable, which are difficult to project onto; and (b) deterministic or stochastic constraints that admit efficient projection. Problems of this form arise frequently in the context of semidefinite programming as well as when various NP-hard problems are solved approximately via semidefinite relaxation. Since projection onto the first set of constraints is difficult, it becomes necessary to explore projection-free algorithms, such as the stochastic Frank-Wolfe (FW) algorithm. On the other hand, the second set of constraints cannot be handled in the same way, and must be incorporated as an indicator function within the objective function, thereby complicating the application of FW methods. Similar problems have been studied before; however, they suffer from slow convergence rates. This work, equipped with momentum based gradient tracking technique, guarantees fast convergence rates on par with the best-known rates for problems without the second set of constraints. Zeroth-order variants of the proposed algorithms are also developed and again improve upon the state-of-the-art rate results. We further propose the novel trimmed FW variants that enjoy the same convergence rates as their classical counterparts, but are empirically shown to require significantly fewer calls to the linear minimization oracle speeding up the overall algorithm. The efficacy of the proposed algorithms is tested on relevant applications of sparse matrix estimation, clustering via semidefinite relaxation, and uniform sparsest cut problem. [ABSTRACT FROM AUTHOR]

In this paper, we study the formalism of unsupervised multi-class domain adaptation (multi-class UDA), which underlies a few recent algorithms whose learning objectives are only motivated empirically. Multi-Class Scoring Disagreement (MCSD) divergence is presented by aggregating the absolute margin violations in multi-class classification, and this proposed MCSD is able to fully characterize the relations between any pair of multi-class scoring hypotheses. By using MCSD as a measure of domain distance, we develop a new domain adaptation bound for multi-class UDA; its data-dependent, probably approximately correct bound is also developed that naturally suggests adversarial learning objectives to align conditional feature distributions across source and target domains. Consequently, an algorithmic framework of Multi-class Domain-adversarial learning Networks (McDalNets) is developed, and its different instantiations via surrogate learning objectives either coincide with or resemble a few recently popular methods, thus (partially) underscoring their practical effectiveness. Based on our identical theory for multi-class UDA, we also introduce a new algorithm of Domain-Symmetric Networks (SymmNets), which is featured by a novel adversarial strategy of domain confusion and discrimination. SymmNets affords simple extensions that work equally well under the problem settings of either closed set, partial, or open set UDA. We conduct careful empirical studies to compare different algorithms of McDalNets and our newly introduced SymmNets. Experiments verify our theoretical analysis and show the efficacy of our proposed SymmNets. In addition, we have made our implementation code publicly available. [ABSTRACT FROM AUTHOR]

Dual Coordinate Descent (DCD) is one of the most popular optimization methods. The parallelization of DCD is difficult, as DCD is sequential in nature. As such, simultaneously running multiple DCD threads on batches of data elements causes result inaccuracy and slow convergence, due to the concurrent updates of multiple coordinates. Some parallelization methods adopt separable approximate functions that depend on the degree of parallelism. Such dependencies result in both poor scalability and slow-convergence. To address these challenges, in this paper we present a new parallel primal-dual algorithm for DCD, called PPD. In PPD, the block data distribution is utilized to obtain a new approximate function that is independent from the parallelism. Moreover, PPD is designed with a novel primal-dual acceleration scheme to approach the optimal solution closely and quickly. We demonstrate the advantages of PPD in terms of scalability and efficiency through experiments. [ABSTRACT FROM AUTHOR]

Dictionary learning has achieved remarkable success on a wide range of machine learning-based applications. In this paper, a locality-adaptive structured dictionary learning (LASDL) algorithm for cross-domain recognition is proposed. In the LASDL, a projective structured double reconstruction strategy is developed to train class-oriented sub-dictionaries from the specific classes of cross-domain samples. The strategy benefits to make full advantage of the discriminative information of cross-domain data and bridge the distribution divergence between two different domains. Meanwhile, an adaptive geometrical structure preserving function is designed to not only preserve the local manifold structures spanned by the representation coefficient spaces of the source and target domains, but also impose a constraint that the coefficients should keep closer to their class centers, which is propitious to reduce the distribution divergence and make the representation more accurate. With the structured linear coding technique, the final cross-domain recognition can be efficiently performed by determining class-specific reconstruction error. The optimization of the proposed LASDL model can be efficiently solved by simple least square method and alternating direction method of multipliers (ADMM) algorithm. Extensive experimental results validate the superiority of the proposed algorithm in contrast to other state-of-the-art predecessors. [ABSTRACT FROM AUTHOR]

In this paper, we present a framework for performing automatic analysis of Land Use Zones based on Location-Based Social Networks (LBSNs). We model city areas using a hierarchical structure of POIs extracted from foursquare. We encode such structures in kernel machines, e.g., Support Vector Machines, using a new Tree Kernel, i.e., the Hierarchical POI Kernel (HPK), which can take the importance of the individual POIs into account during the substructure matching. This way, HPK projects structures in the space of all their possible substructures such that each dimension corresponds to a semantic structural feature, weighted according to the discriminative power of POIs. We generated four different datasets for the following cities: Barcelona, Lisbon, Amsterdam and Milan, where we trained and tested our models. The results show that our approach largely outperforms previous work and standard baseline built on simple features, such as counts of different POIs. Finally, we apply a mining algorithm to extract the most relevant features (tree fragments) from the implicit TK space according to the weights the kernel machine assigned to them. Our approach can produce an explicit set of representative features that can be used to classify and characterize urban areas. [ABSTRACT FROM AUTHOR]

Automated machine learning (AutoML) seeks to automatically find so-called machine learning pipelines that maximize the prediction performance when being used to train a model on a given dataset. One of the main and yet open challenges in AutoMLis an effective use of computational resources: An AutoML process involves the evaluation of many candidate pipelines, which are costly but often ineffective because they are canceled due to a timeout. In this paper, we present an approach to predict the runtime of two-step machine learning pipelines with up to one pre-processor, which can be used to anticipate whether or not a pipeline will time out. Separate runtime models are trained offline for each algorithm that may be used in a pipeline, and an overall prediction is derived from these models. We empirically show that the approach increases successful evaluations made by an AutoML tool while preserving or even improving on the previously best solutions. [ABSTRACT FROM AUTHOR]

Temperature rising is an unavoidable effect on VLSI and has always been a critical issue in any system-on-chip – especially when targeting compute-intensive applications. This effect increases the delay in hardware accelerators, resulting in timing errors due to unsustainable clock frequency, whose impact must be carefully evaluated on design time to measure the performance degradation of the hardware accelerator. Further, a hardware operating at a higher temperature accelerates device aging, which incurs in more timing errors. This issue is usually addressed with the inclusion of timing guardbands that compensate for the deleterious effects of temperature, ensuring the hardware accelerator works within a reliable zone, i.e., without any timing errors caused by temperature effects at runtime. However, guardbands directly result in considerable performance and efficiency losses because the circuit will be clocked at a frequency lower than its full potential. Accelerators on edge devices often dismiss such guardbands to explore the full potential of the designed circuits, posing an enormous design challenge as this approach requires a careful evaluation of the impact of timing errors on the quality of the target applications. Many algorithms, such as in multimedia and machine learning applications, are capable of tolerating hardware errors. Yet, these algorithms have a dynamic behavior (i.e., closed-loop) where a timing error can be propagated, affecting subsequent steps. Measuring the degradation-induced errors in these applications is very challenging given that an accurate gate-level simulation to investigate degradation-induced timing errors needs to be coupled dynamically with a system-level simulator to unveil how induced errors in the underlying hardware ultimately impact the algorithm execution in the hardware accelerator. This is the first work to achieve this goal. State-of-the-art works have studied accelerators under timing-errors when removing (or narrowing) guardbands. However, their approach was suitable only for open-loop hardware accelerators which are entirely agnostic of complex interactions of the algorithms. Unlike prior work, this paper investigates temperature- and aging-induced timing-errors in the joint accelerator-algorithm interactions and their runtime impacts. Our framework investigates aging effects across the different layers starting from transistor physics all the way up to the algorithm layer. The hardware accelerator employed as a case study in this work is the sum of absolute differences (SAD), which is the most compute-intensive accelerator on commercial video encoder for mobile applications. Our results demonstrate the runtime behavior impacts of three advanced block-matching algorithms of the video encoder in a joint operation by a SAD accelerator under timing-errors induced by temperature and aging effects considering a 14nm FinFET technology. [ABSTRACT FROM AUTHOR]

Federated learning (FL) involves training machine learning models over distributed edge nodes (i.e., workers) while facing three critical challenges, edge heterogeneity, Non-IID data and communication resource constraint. In the synchronous FL, the parameter server has to wait for the slowest workers, leading to significant waiting time due to edge heterogeneity. Though asynchronous FL can well tackle the edge heterogeneity, it requires frequent model transfers, resulting in massive communication resource consumption. Moreover, the different relative frequency of workers participating in asynchronous updating may seriously hurt training accuracy, especially on Non-IID data. In this paper, we propose a semi-asynchronous federated learning mechanism (FedSA), where the parameter server aggregates a certain number of local models by their arrival order in each round. We theoretically analyze the quantitative relationship between the convergence bound of FedSA and different factors, e.g., the number of participating workers in each round, the degree of data Non-IID and edge heterogeneity. Based on the convergence bound, we present an efficient algorithm to determine the number of participating workers to minimize the training completion time. To further improve the training accuracy on Non-IID data, FedSA deploys adaptive learning rates for workers by their relative participation frequency. We extend our proposed mechanism to the dynamic and multiple learning tasks scenarios. Experimental results on the testbed show that our proposed mechanism and algorithms address the three challenges more effectively than the state-of-the-art solutions. [ABSTRACT FROM AUTHOR]

Federated learning has emerged as a popular technique for distributing machine learning (ML) model training across the wireless edge. In this paper, we propose two timescale hybrid federated learning (TT-HF), a semi-decentralized learning architecture that combines the conventional device-to-server communication paradigm for federated learning with device-to-device (D2D) communications for model training. In TT-HF, during each global aggregation interval, devices (i) perform multiple stochastic gradient descent iterations on their individual datasets, and (ii) aperiodically engage in consensus procedure of their model parameters through cooperative, distributed D2D communications within local clusters. With a new general definition of gradient diversity, we formally study the convergence behavior of TT-HF, resulting in new convergence bounds for distributed ML. We leverage our convergence bounds to develop an adaptive control algorithm that tunes the step size, D2D communication rounds, and global aggregation period of TT-HF over time to target a sublinear convergence rate of $\mathcal {O}(1/t)$ while minimizing network resource utilization. Our subsequent experiments demonstrate that TT-HF significantly outperforms the current art in federated learning in terms of model accuracy and/or network energy consumption in different scenarios where local device datasets exhibit statistical heterogeneity. Finally, our numerical evaluations demonstrate robustness against outages caused by fading channels, as well favorable performance with non-convex loss functions. [ABSTRACT FROM AUTHOR]

Estimating set similarity and detecting highly similar sets are fundamental problems in areas such as databases and machine learning. MinHash is a well-known technique for approximating Jaccard similarity of sets and has been successfully used for many applications. Its two compressed versions, $b$ b -bit MinHash and Odd Sketch, can significantly reduce the memory usage of the MinHash, especially for estimating high similarities (i.e., similarities around 1). Although MinHash can be applied to static sets as well as streaming sets, of which elements are given in a streaming fashion, unfortunately, $b$ b -bit MinHash and Odd Sketch fail to deal with streaming data. To solve this problem, we previously designed a memory-efficient sketch method, MaxLogHash, to accurately estimate Jaccard similarities in streaming sets. Compared with MinHash, our method uses smaller sized registers (each register consists of less than 7 bits) to build a compact sketch for each set. In this paper, we further develop a faster method, MaxLogOPH++. Compared with MaxLogHash, MaxLogOPH++ reduces the time complexity for updating each coming element from $O(k)$ O (k) to $O(1)$ O (1) with a small additional memory. We conduct experiments on a variety of datasets, and experimental results demonstrate the efficiency and effectiveness of our methods. [ABSTRACT FROM AUTHOR]

Recent workat the U.S. Army CCDC Ground Vehicle Systems Center has shown that machine learning classifiers can quickly construct high-fidelity mobility maps. Training these classifiers, on the other hand, is still a challenge, since each data instance is labeled by performing a computationally intensive, physics-based simulation. In this paper we introduce an active learning framework, based on the query-by-bagging algorithm, that substantially reduces the number of simulations needed to train a classifier. Experimental results suggest that our sampling algorithm can train a neural network, with higher accuracy, using less than half the number of simulations when compared to random sampling. [ABSTRACT FROM AUTHOR]

Unmanned aerial vehicle (UAV) technology is recognized as a promising solution to area coverage problems (ACPs) and has been extensively studied recently. In this paper, we study the 3D irregular terrain surface coverage problem with a hierarchical UAV swarm. We first build the 3D model of a random irregular terrain and propose a geometric way to project the 3D terrain surface into many weighted 2D patches. Then we develop a two-level hierarchical UAV swarm architecture, including the low-level follower UAVs (FUAVs) and the high-level leader UAVs (LUAVs). For FUAVs, we design a coverage trajectory algorithm to carry out specific coverage tasks within patches based on the star communication topology. For LUAVs, we propose a swarm deep Q-learning (SDQN) reinforcement learning algorithm to select patches. Moreover, an observation history model based on convolutional neural networks (CNNs) and the mean embedding method is integrated into SDQN to address the communication limitation problems of LUAVs. The numerical results show that FUAVs can cover the entire area of each patch with little redundancies, and the total coverage time of the SDQN is less than that of existing methods, which demonstrates the effectiveness of the proposed algorithms. [ABSTRACT FROM AUTHOR]

Clustering is a widely used machine learning technique for unlabelled data. One of the recently proposed techniques is the twin support vector clustering (TWSVC) algorithm. The idea of TWSVC is to generate hyperplanes for each cluster. TWSVC utilizes the hinge loss function to penalize the misclassification. However, the hinge loss relies on shortest distance between different clusters, and is unstable for noise-corrupted datasets, and for re-sampling. In this paper, we propose a novel Sparse Pinball loss Twin Support Vector Clustering (SPTSVC). The proposed SPTSVC involves the $\epsilon$ -insensitive pinball loss function to formulate a sparse solution. Pinball loss function provides noise-insensitivity and re-sampling stability. The $\epsilon$ -insensitive zone provides sparsity to the model and improves testing time. Numerical experiments on synthetic as well as real world benchmark datasets are performed to show the efficacy of the proposed model. An analysis on the sparsity of various clustering algorithms is presented in this work. In order to show the feasibility and applicability of the proposed SPTSVC on biomedical data, experiments have been performed on epilepsy and breast cancer datasets. [ABSTRACT FROM AUTHOR]

Publishing datasets plays an essential role in open data research and promoting transparency of government agencies. However, such data publication might reveal users’ private information. One of the most sensitive sources of data is spatiotemporal trajectory datasets. Unfortunately, merely removing unique identifiers cannot preserve the privacy of users. Adversaries may know parts of the trajectories or be able to link the published dataset to other sources for the purpose of user identification. Therefore, it is crucial to apply privacy preserving techniques before the publication of spatiotemporal trajectory datasets. In this paper, we propose a robust framework for the anonymization of spatiotemporal trajectory datasets termed as machine learning based anonymization (MLA). By introducing a new formulation of the problem, we are able to apply machine learning algorithms for clustering the trajectories and propose to use k-means algorithm for this purpose. A variation of k-means algorithm is also proposed to preserve the privacy in overly sensitive datasets. Moreover, we improve the alignment process by considering multiple sequence alignment as part of the MLA. The framework and all the proposed algorithms are applied to T-Drive, Geolife, and Gowalla location datasets. The experimental results indicate a significantly higher utility of datasets by anonymization based on MLA framework. [ABSTRACT FROM AUTHOR]

The evolution of the Internet of Things (IoT) is driving an extraordinary growth of traffic and processing demands, persuading 5G players to change their infrastructures. In this context, Fog computing emerges as a potential solution, providing nearby resources to run IoT applications. However, the Fog raises several challenges which hinders its adoption. In this article, we consider the reconfiguration problem, i.e., how to dynamically adapt the placement of IoT applications running on the Fog, depending on application needs and evolution of resource usage. We propose and evaluate a series of reconfiguration algorithms, based on both online scheduling and online learning approaches. Through an extensive set of experiments in a realistic testbed, we demonstrate that the performance strongly depends on the quality and availability of information from both Fog infrastructure and IoT applications. This information mainly concerns the application’s resource usage (estimated by the user during the design of the application) and the availability of resources in the infrastructure (collected by commercial off-the-shelf monitoring tools). Finally, we show that a reactive and greedy strategy, which relies on this additional information, can overcome the performance of state-of-the-art online learning algorithms, even in a scenario with inaccurate information. [ABSTRACT FROM AUTHOR]

To cope with the lack of on-device machine learning samples, this article presents a distributed data augmentation algorithm, coined federated data augmentation (FAug). In FAug, devices share a tiny fraction of their local data, i.e., seed samples, and collectively train a synthetic sample generator that can augment the local datasets of devices. To further improve FAug, we introduce a multihop-based seed sample collection method and an oversampling technique that mixes up collected seed samples. Both approaches enjoy the benefit from the crowd of devices, by hiding data privacy from preceding hops and feeding diverse seed samples. In the image classification tasks, simulations demonstrate that the proposed FAug frameworks yield stronger privacy guarantees, lower communication latency, and higher on-device ML accuracy. [ABSTRACT FROM AUTHOR]

Federated machine learning (FML) is a new machine learning paradigm that is focused on training distributed models, where data are scattered in different places known as data silos, only necessary modeling information (not raw data) is exchanged, and data privacy and security are protected during the modeling. This research area has been growing fast during the past years, but the vision of making it a practical solution is still not fulfilled. Motivated by this, here we introduce an intelligent architecture, termed Federated Digital Gateway. It is designed to help algorithm engineers to easily deploy FML methods for real-life tasks. It provides different modules such as secure communication tools, database interface, authentication center, account system, and user interface. This architecture has been shown to function smoothly in two real-world applications. Overall, the federated digital gateway is practical and deployable for applying federated learning to solve real-life tasks. [ABSTRACT FROM AUTHOR]

The estimation of nested functions (i.e., functions of functions) is one of the central reasons for the success and popularity of machine learning. Today, artificial neural networks are the predominant class of algorithms in this area, known as representational learning. Here, we introduce Representational Gradient Boosting (RGB), a nonparametric algorithm that estimates functions with multi-layer architectures obtained using backpropagation in the space of functions. RGB does not need to assume a functional form in the nodes or output (e.g., linear models or rectified linear units), but rather estimates these transformations. RGB can be seen as an optimized stacking procedure where a meta algorithm learns how to combine different classes of functions (e.g., Neural Networks (NN) and Gradient Boosting (GB)), while building and optimizing them jointly in an attempt to compensate each other’s weaknesses. This highlights a stark difference with current approaches to meta-learning that combine models only after they have been built independently. We showed that providing optimized stacking is one of the main advantages of RGB over current approaches. Additionally, due to the nested nature of RGB we also showed how it improves over GB in problems that have several high-order interactions. Finally, we investigate both theoretically and in practice the problem of recovering nested functions and the value of prior knowledge. [ABSTRACT FROM AUTHOR]

The estimation of nested functions (i.e., functions of functions) is one of the central reasons for the success and popularity of machine learning. Today, artificial neural networks are the predominant class of algorithms in this area, known as representational learning. Here, we introduce Representational Gradient Boosting (RGB), a nonparametric algorithm that estimates functions with multi-layer architectures obtained using backpropagation in the space of functions. RGB does not need to assume a functional form in the nodes or output (e.g., linear models or rectified linear units), but rather estimates these transformations. RGB can be seen as an optimized stacking procedure where a meta algorithm learns how to combine different classes of functions (e.g., Neural Networks (NN) and Gradient Boosting (GB)), while building and optimizing them jointly in an attempt to compensate each other’s weaknesses. This highlights a stark difference with current approaches to meta-learning that combine models only after they have been built independently. We showed that providing optimized stacking is one of the main advantages of RGB over current approaches. Additionally, due to the nested nature of RGB we also showed how it improves over GB in problems that have several high-order interactions. Finally, we investigate both theoretically and in practice the problem of recovering nested functions and the value of prior knowledge. [ABSTRACT FROM AUTHOR]

Feature-based algorithm selection aims to automatically find the best one from a portfolio of optimization algorithms on an unseen problem based on its landscape features. Feature-based algorithm selection has recently received attention in the research field of black-box numerical optimization. However, there is still room for the analysis of algorithm selection for black-box optimization. Most previous studies have focused only on whether an algorithm selection system can outperform the single-best solver (SBS) in a portfolio. In addition, a benchmarking methodology for algorithm selection systems has not been well investigated in the literature. In this context, this article analyzes algorithm selection systems on the 24 noiseless black-box optimization benchmarking functions. First, we demonstrate that the first successful performance measure is more reliable than the expected runtime measure for benchmarking algorithm selection systems. Then, we examine the influence of randomness on the performance of algorithm selection systems. We also show that the performance of algorithm selection systems can be significantly improved by using sequential least squares programming as a presolver. We point out that the difficulty of outperforming the SBS depends on algorithm portfolios, cross-validation methods, and dimensions. Finally, we demonstrate that the effectiveness of algorithm portfolios depends on various factors. These findings provide fundamental insights for algorithm selection for black-box optimization. [ABSTRACT FROM AUTHOR]

This article investigates the pattern synthesis of the conformal phased array antenna (PAA) by using the deep deterministic policy gradient (DDPG) algorithm, and a nearly full solid angle for beam steering is realized. The beam steering capability of the planar and conformal PAAs is theoretically compared at first, and a conclusion is obtained that conformed to the conical-and-cylindrical structure can help to achieve ultrawide-angle beam steering. Next, a typical deep reinforcement learning algorithm, which is the DDPG algorithm, is utilized to deal with the fast beam steering problem of the conformal heterogeneous PAA. By virtue of the strong fitting ability of the DDPG algorithm for high-dimensional continuous nonlinear problems, the performance of fast beam steering is achieved within a wide-angle range within (−150°, 150°). Finally, a prototype of $1\times17$ conformal PAA is fabricated for measurement and verification, and the measured results are in good agreement with the simulation results. [ABSTRACT FROM AUTHOR]

Multi-field packet classification is a crucial component in modern software-defined data center networks. To achieve high throughput and low latency, state-of-the-art algorithms strive to fit the rule lookup data structures into on-die caches; however, they do not scale well with the number of rules. We present a novel approach, NuevoMatch, which improves the memory scaling of existing methods. A new data structure, Range Query Recursive Model Index (RQ-RMI), is the key component that enables NuevoMatch to replace most of the accesses to main memory with model inference computations. We describe an efficient training algorithm that guarantees the correctness of the RQ-RMI-based classification. The use of RQ-RMI allows the rules to be compressed into neural networks that fit into the hardware cache. Further, it takes advantage of the growing support for fast neural network processing in modern CPUs, such as wide vector instructions, achieving a latency of tens of nanoseconds per lookup. Our evaluation using 500K multi-field rules from the standard ClassBench benchmark shows a geometric mean compression factor of $4.9\times $ , $8\times $ , and $82\times $ , and average performance improvement of $2.4\times $ , $2.6\times $ , and $1.6\times $ in throughput compared to CutSplit, NeuroCuts, and TupleMerge, all state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Wireless distributed machine learning is envisaged to facilitate advanced learning services and applications in wireless networks consisting of devices with limited computing capability. Distributed machine learning algorithms are more vulnerable in wireless systems since information exchange in learning is limited by wireless resources and channel conditions. Moreover, their performance can be significantly degraded by attacks of the Byzantine devices, and information distorted by channel fading can be treated as Byzantine attacks. Consequently, protection of wireless distributed machine learning from Byzantine devices is paramount. Leveraging over-the-air computation, we put forth a novel wireless distributed stochastic gradient descent system which is resilient to Byzantine attacks. The proposed learning system is underpinned by two novel and distinct features which enable more accurate and faster distributed machine learning resilient to Byzantine attacks: collecting training data in the PS to obtain its own training results and grouping the distributed devices. We derive upper bounds of the mean square error of the global parameter when the proposed algorithms are used in the cases with and without Byzantine devices, and prove the convergence of the proposed algorithms with the derived bounds. The effectiveness of the algorithms is validated by showing the accuracy and convergence speed. [ABSTRACT FROM AUTHOR]

Machine learning (ML) models have been used to improve the quality of different physical design steps, such as timing analysis, clock tree synthesis, and routing. However, so far very few works have addressed the problem of algorithm selection during physical design, which can drastically reduce the computational effort of some steps. This work proposes a legalization algorithm selection framework using deep convolutional neural networks (CNNs). To extract features, we used snapshots of circuit placements and used transfer learning to train the models using pretrained weights of the Squeezenet architecture. By doing so, we can greatly reduce the training time and required data even though the pretrained weights come from a different problem. We performed extensive experimental analysis of ML models, providing details on how we chose the parameters of our model, such as CNN architecture, learning rate, and number of epochs. We evaluated the proposed framework by training a model to select between different legalization algorithms according to cell displacement and wirelength variation. The trained models achieved an average $F$ -score of 0.98 when predicting cell displacement and 0.83 when predicting wirelength variation. When integrated into the physical design flow, the cell displacement model achieved the best results on 15 out of 16 designs, while the wirelength variation model achieved that for 10 out of 16 designs, being better than any individual legalization algorithm. Finally, using the proposed ML model for algorithm selection resulted in a speedup of up to $10\times $ compared to running all the algorithms separately. [ABSTRACT FROM AUTHOR]

Machine learning (ML) algorithms play a vital role in the brain age estimation frameworks. The impact of regression algorithms on prediction accuracy in the brain age estimation frameworks have not been comprehensively evaluated. Here, we sought to assess the efficiency of different regression algorithms on brain age estimation. To this end, we built a brain age estimation framework based on a large set of cognitively healthy (CH) individuals ($N = 788$) as a training set followed by different regression algorithms (22 different algorithms in total). We then quantified each regression-algorithm on independent test sets composed of 88 CH individuals, 70 mild cognitive impairment patients as well as 30 Alzheimer’s disease patients. The prediction accuracy in the independent test set (i.e., CH set) varied in regression algorithms mean absolute error (MAE) from 4.63 to 7.14 yrs, $R^2$ from 0.76 to 0.88. The highest and lowest prediction accuracies were achieved by Quadratic Support Vector Regression algorithm (MAE $= 4.63$ yrs, $R^2 = 0.88, 95\%$ CI $= [-1.26, 1.42]$) and Binary Decision Tree algorithm (MAE $= 7.14$ yrs, $R^2 = 0.76, 95\%$ CI $= [-1.50, 2.62]$), respectively. Our experimental results demonstrate that the prediction accuracy in brain age frameworks is affected by regression algorithms, indicating that advanced machine learning algorithms can lead to more accurate brain age predictions in clinical settings. [ABSTRACT FROM AUTHOR]

A Dandelion algorithm (DA) inspired by the seed dispersal process of dandelions has been proposed as a newly intelligent optimization algorithm. For improving its exploration ability as well as reducing the probability of its falling into a local optimum, this work proposes to add a novel competition mechanism with historical information feedback to current DA. Specifically, the fitness value of each dandelion in the next generation, which is calculated by linear prediction, is compared with the current best dandelion, and the loser is replaced by a new offspring. Current DA generates new offsprings without considering historical information. This work improves its offspring generation process by exploiting historical information with an estimation-of-distribution algorithm. Three historical information models are designed. They are best, worst, and hybrid historical information feedback models. The experimental results show that the proposed algorithms outperform DA and its variants, and the proposed algorithms are superior or competitive to nine participating algorithms benchmarked on 28 functions from CEC2013. Finally, the proposed algorithms demonstrate the effectiveness on four real-world problems, and the results indicate that the proposed algorithms have better performance than its peers. [ABSTRACT FROM AUTHOR]

Aggressive driving is one of the main causes of fatal crashes. Correctly identifying aggressive driving events still represents a challenge in the literature. Furthermore, datasets available for testing the proposed approaches have some limitations since they generally (a) include only a few types of events, (b) contain data collected with only one device, and (c) are generated in drives that did not fully consider the variety of road characteristics and/or driving conditions. The main objective of this work is to compare the performance of several state-of-the-art algorithms for aggressive driving event detection (belonging to anomaly detection-, threshold- and machine learning-based categories) on multiple datasets containing sensors data collected with different devices (black-boxes and smartphones), on different vehicles and in different locations. A secondary objective is to verify whether smartphones could replace black-boxes in aggressive/non-aggressive classification tasks. To this aim, we propose the AD $^2$ (Aggressive Driving Detection) dataset, which contains (i) data collected using multiple devices to evaluate their influence on the algorithm performance, (ii) geographical data useful to analyze the context in which the events occurred, (iii) events recorded in different situations, and (iv) events generated by traveling the same path with aggressive and non-aggressive driving styles, in order to possibly separate the effects of driving style from those of road characteristics. Our experimental results highlighted the superiority of machine learning-based approaches and underlined the ability of smartphones to ensure a level of performance similar to that of black-boxes. [ABSTRACT FROM AUTHOR]

For the large amount of spatial and spectral information contained in hyperspectral image (HSI), feature description of HSI has attracted widespread concern in recent years. Existing deep learning-based HSI feature description algorithms require a large number of training samples and have poor interpretability. Therefore, it is necessary to develop an efficient HSI features description algorithm with interpretability based on machine learning. Local binary pattern (LBP) is a classical descriptor used to extract the local spatial texture features of images, which has been widely applied to image feature description and matching. However, the existing LBP algorithms for HSI are based on the single-dimensional description, which leads to the limitations on the expression of spatial–spectral information. Therefore, a multidimensional LBP (MDLBP) based on Clifford algebra for HSI is proposed in this article, which is able to extract spatial–spectral feature from multiple dimensions. First, with the theory of the Clifford algebra, a new representation of HSI including spatial and spectral information is built. Second, the geometric relationship between the local geometry of HSI in Clifford algebra space is calculated to realize the local multidimensional description of the local spatial–spectral information. Finally, a novel LBP coding algorithm for HSI is implemented based on the local multidimensional description to calculate the feature descriptor of HSI. The experimental results on HSI classification show that our proposed MDLBP algorithm can achieve higher accuracy than the representative spatial–spectral features and the existing LBP algorithms, especially in the scenery of small-scale training samples. [ABSTRACT FROM AUTHOR]

Federated learning enables machine learning algorithms to be trained over decentralized edge devices without requiring the exchange of local datasets. Successfully deploying federated learning requires ensuring that agents (e.g., mobile devices) faithfully execute the intended algorithm, which has been largely overlooked in the literature. In this study, we first use risk bounds to analyze how the key feature of federated learning, unbalanced and non-i.i.d. data, affects agents’ incentives to voluntarily participate and obediently follow traditional federated learning algorithms. To be more specific, our analysis reveals that agents with less typical data distributions and relatively more samples are more likely to opt out of or tamper with federated learning algorithms. To this end, we formulate the first faithful implementation problem of federated learning and design two faithful federated learning mechanisms which satisfy economic properties, scalability, and privacy. First, we design a Faithful Federated Learning (FFL) mechanism which approximates the Vickrey–Clarke–Groves (VCG) payments via an incremental computation. We show that it achieves (probably approximate) optimality, faithful implementation, voluntary participation, and some other economic properties (such as budget balance). Further, the time complexity in the number of agents $K$ is $\mathcal {O}(\log (K))$. Second, by partitioning agents into several clusters, we present a scalable VCG mechanism approximation. We further design a scalable and Differentially Private FFL (DP-FFL) mechanism, the first differentially private faithful mechanism, that maintains the economic properties. Our DP-FFL mechanism enables one to make three-way performance tradeoffs among privacy, the iterations needed, and payment accuracy loss. [ABSTRACT FROM AUTHOR]

In this work, a gradient-based primal-dual method of multipliers is proposed for solving a class of linearly constrained non-convex problems. We show that with random initialization of the primal and dual variables, the algorithm is able to compute second-order stationary points (SOSPs) with probability one. Further, we present applications of the proposed method in popular signal processing and machine learning problems such as decentralized matrix factorization and decentralized training of overparameterized neural networks. One of the key steps in the analysis is to construct a new loss function for these problems such that the required convergence conditions (especially the gradient Lipschitz conditions) can be satisfied without changing the global optimal points. [ABSTRACT FROM AUTHOR]

We focus on over-the-air (OTA) Federated Learning (FL), which has been suggested recently to reduce the communication overhead of FL due to the repeated transmissions of the model updates by a large number of users over the wireless channel. In OTA FL, all users simultaneously transmit their updates as analog signals over a multiple access channel, and the server receives a superposition of the analog transmitted signals. However, this approach results in the channel noise directly affecting the optimization procedure, which may degrade the accuracy of the trained model. We develop a Convergent OTA FL (COTAF) algorithm which enhances the common local stochastic gradient descent (SGD) FL algorithm, introducing precoding at the users and scaling at the server, which gradually mitigates the effect of noise. We analyze the convergence of COTAF to the loss minimizing model and quantify the effect of a statistically heterogeneous setup, i.e. when the training data of each user obeys a different distribution. Our analysis reveals the ability of COTAF to achieve a convergence rate similar to that achievable over error-free channels. Our simulations demonstrate the improved convergence of COTAF over vanilla OTA local SGD for training using non-synthetic datasets. Furthermore, we numerically show that the precoding induced by COTAF notably improves the convergence rate and the accuracy of models trained via OTA FL. [ABSTRACT FROM AUTHOR]

In this article, we introduce novel reinforced hybrid fuzzy rule-based neural networks (RHFNNs). This article is concerned with the development of the design methodologies of hybrid fuzzy rule-based model for constructing the network structure and enhancing its predictive abilities through the combination of inhomogeneous neurons [i.e., clustering-based polynomial neurons (CPNs) and polynomial neurons (PNs)] and tournament selection. The key points of the proposed RHFNN are enumerated as follows: The first layer of the proposed network consists of CPNs. CPN can effectively reflect the complex nonlinear structure encountered in the data space, and refine (granulate) it with the help of the clustering algorithm. Two types of CPNs including hard C-means (HCM) clustering-based polynomial neuron (HCPN) and fuzzy C-means (FCM) clustering-based polynomial neuron (FCPN) are designed. According to the type of CPN used in the first layer, RHFNN can be categorized into two types, namely, RHFNN based on HCPN (HRHFNN) and RHFNN based on FCPN (FRHFNN). We use PNs to construct the second and consecutive layers. PN can identify and approximate the nonlinear relationship among system's inputs and outputs. A tournament-based performance selection (TPS) algorithm stemming from evolutionary computation is used for selection of neuron. TPS not only ensures that the candidate nodes have sufficient fitting ability but also enhances the individual diversity in the node set and provides the abilities to generate better prediction nodes. In addition, L2-norm regularization is considered to reduce the deviation between coefficients and ameliorate overfitting as well as boost generalization ability. The performance of RHFNN is discussed through a variety of publicly available machine learning datasets. From the experimental results, we conclude that RHFNN achieves the best prediction accuracy on 13 of 15 datasets; the statistical analysis also confirms the superiority of RHFNN. [ABSTRACT FROM AUTHOR]

In data streams, the data distribution of arriving observations at different time points may change—a phenomenon called concept drift. While detecting concept drift is a relatively mature area of study, solutions to the uncertainty introduced by observations with missing values have only been studied in isolation. No one has yet explored whether or how these solutions might impact drift detection performance. We, however, believe that data imputation methods may actually increase uncertainty in the data rather than reducing it. We also conjecture that imputation can introduce bias into the process of estimating distribution changes during drift detection, which can make it more difficult to train a learning model. Our idea is to focus on estimating the distance between observations rather than estimating the missing values, and to define membership functions that allocate observations to histogram bins according to the estimation errors. Our solution comprises a novel masked distance learning (MDL) algorithm to reduce the cumulative errors caused by iteratively estimating each missing value in an observation and a fuzzy-weighted frequency (FWF) method for identifying discrepancies in the data distribution. The concept drift detection algorithm proposed in this article is a singular and unified algorithm that can handle missing values, but not an imputation algorithm combined with a concept drift detection algorithm. Experiments on both synthetic and real-world datasets demonstrate the advantages of this method and show its robustness in detecting drift in data with missing values. The results show that compared to the best-performing algorithm that handles imputation and drift detection separately, MDL-FWF reduced the average drift detection difference from 10.75% to 5.83%. This is a nearly 46% improvement. These findings reveal that missing values exert a profound impact on concept drift detection, but using fuzzy set theory to model observations can produce more reliable results than imputation. [ABSTRACT FROM AUTHOR]

Image partitioning, or segmentation without semantics, is the task of decomposing an image into distinct segments, or equivalently to detect closed contours. Most prior work either requires seeds, one per segment; or a threshold; or formulates the task as multicut / correlation clustering, an NP-hard problem. Here, we propose an efficient algorithm for graph partitioning, the “Mutex Watershed”. Unlike seeded watershed, the algorithm can accommodate not only attractive but also repulsive cues, allowing it to find a previously unspecified number of segments without the need for explicit seeds or a tunable threshold. We also prove that this simple algorithm solves to global optimality an objective function that is intimately related to the multicut / correlation clustering integer linear programming formulation. The algorithm is deterministic, very simple to implement, and has empirically linearithmic complexity. When presented with short-range attractive and long-range repulsive cues from a deep neural network, the Mutex Watershed gives the best results currently known for the competitive ISBI 2012 EM segmentation benchmark. [ABSTRACT FROM AUTHOR]

Kriging models, also known as Gaussian process models, are widely used in surrogate-assisted evolutionary algorithms (SAEAs). However, the cubic time complexity of the standard Kriging models limits their usage in high-dimensional optimization. To tackle this problem, we propose an incremental Kriging model for high-dimensional surrogate-assisted evolutionary computation. The main idea is to update the Kriging model incrementally based on the equations of the previously trained model instead of building the model from scratch when new samples arrive, so that the time complexity of updating the Kriging models can be reduced to quadratic. The proposed incremental learning scheme is very suitable for online SAEAs since they evaluate new samples in each one or several generations. The proposed algorithm is able to achieve competitive optimization results on the test problems compared with the standard Kriging-assisted evolutionary algorithm and is significantly faster than the standard Kriging approach. The proposed algorithm also shows competitive or better performances compared with four fast Kriging-assisted evolutionary algorithms and four state-of-the-art SAEAs. This work provides a fast way of employing Kriging models in high-dimensional surrogate-assisted evolutionary computation. [ABSTRACT FROM AUTHOR]

The ubiquitous Internet of Things (IoT) system is a key component of future 6G networks to realize a fully connected world. Extensive efforts have been made to provide on-demand traffic scheduling in IoT networks through machine learning algorithms. However, the current learning approaches are hindered by the heterogeneous information in ubiquitous IoT systems since the data are collected from different domains (e.g., space, air, ground, and ocean). To uncover the complete picture of ubiquitous IoT, this article presents a novel federated imitation learning framework for traffic prediction without compromising privacy. This framework contains a knowledge-sharing module to imitate the status of cross-domain models. After that, we design a distributed resource allocation algorithm, where the IoT devices cooperatively make association decisions using matching theory. Simulation results reveal that our proposed approach outperforms state-of-the-art federated transfer learning and achieves desirable traffic scheduling performance in a cross-domain environment. [ABSTRACT FROM AUTHOR]

High integration densities and design complexity of printed-circuit boards make board-level functional fault identification extremely difficult. Machine learning provides an opportunity to identify functional faults with high accuracy and thereby reduce repair cost. However, the large volume of manufacturing data comes in a streaming format and exhibits time-dependent concept drift in a production environment. These drawbacks limit the effectiveness of traditional machine-learning algorithms. We propose a diagnosis workflow that utilizes online learning to train classifiers incrementally with a small chunk of data at each step. These online-learning algorithms adapt to concept drift quickly with carefully designed update rules. A hybrid algorithm is also proposed to handle the scenario that data for varying numbers of boards are collected at different times. This hybrid algorithm concurrently implements two basic models. For each data chunk, this algorithm chooses the better model with high probability. The experimental results using two boards in high-volume production show that, with the help of online learning and the proposed hybrid algorithm, the F1-score for diagnosis based on binary classifiers can be improved from 57.3% to 81.0%. The top-3 accuracy for diagnosis based on multiclass classifiers can be improved from 78.3% to 91.4%. [ABSTRACT FROM AUTHOR]

Standard Electromechanical Wave Imaging isochrone generation relies on manual selection of zero-crossing (ZC) locations on incremental strain curves for a number of pixels in the segmented myocardium for each echocardiographic view and patient. When considering large populations, this becomes a time-consuming process, that can be limited by inter-observer variability and operator bias. In this study, we developed and optimized an automated ZC selection algorithm, towards a faster more robust isochrone generation approach. The algorithm either relies on heuristic-based baselines or machine learning classifiers. Manually generated isochrones, previously validated against 3D intracardiac mapping, were considered as ground truth during training and performance evaluation steps. The machine learning models applied herein for the first time were: i) logistic regression; ii) support vector machine (SVM); and iii) Random Forest. The SVM and Random Forest classifiers successfully identified accessory pathways in Wolff-Parkinson-White patients, characterized sinus rhythm in humans, and localized the pacing electrode location in left ventricular paced canines on the resulting isochrones. Nevertheless, the best performing classifier was proven to be Random Forest with a precision rising from 89.5% to 97%, obtained with the voting approach that sets a probability threshold upon ZC candidate selection. Furthermore, the predictivity was not dependent on the type of testing dataset it was applied to, contrary to SVM that exhibited a 5% drop in precision on the canine testing dataset. Finally, these findings indicate that a machine learning approach can reduce user variability and considerably decrease the durations required for isochrone generation, while preserving accurate activation patterns. [ABSTRACT FROM AUTHOR]

Heart-related anomalies are among the most common causes of death worldwide. Patients are often asymptomatic until a fatal event happens, and even when they are under observation, trained personnel is needed in order to identify a heart anomaly. In the last decades, there has been increasing evidence of how Machine Learning can be leveraged to detect such anomalies, thanks to the availability of Electrocardiograms (ECG) in digital format. New developments in technology have allowed to exploit such data to build models able to analyze the patterns in the occurrence of heart beats, and spot anomalies from them. In this work, we propose a novel methodology to extract ECG-related features and predict the type of ECG recorded in real time (less than 30 milliseconds). Our models leverage a collection of almost 40 thousand ECGs labeled by expert cardiologists across different hospitals and countries, and are able to detect 7 types of signals: Normal, AF, Tachycardia, Bradycardia, Arrhythmia, Other or Noisy. We exploit the XGBoost algorithm, a leading machine learning method, to train models achieving out of sample F1 Scores in the range 0.93 – 0.99. To our knowledge, this is the first work reporting high performance across hospitals, countries and recording standards. [ABSTRACT FROM AUTHOR]