Search

The estimation of nested functions (i.e., functions of functions) is one of the central reasons for the success and popularity of machine learning. Today, artificial neural networks are the predominant class of algorithms in this area, known as representational learning. Here, we introduce Representational Gradient Boosting (RGB), a nonparametric algorithm that estimates functions with multi-layer architectures obtained using backpropagation in the space of functions. RGB does not need to assume a functional form in the nodes or output (e.g., linear models or rectified linear units), but rather estimates these transformations. RGB can be seen as an optimized stacking procedure where a meta algorithm learns how to combine different classes of functions (e.g., Neural Networks (NN) and Gradient Boosting (GB)), while building and optimizing them jointly in an attempt to compensate each other’s weaknesses. This highlights a stark difference with current approaches to meta-learning that combine models only after they have been built independently. We showed that providing optimized stacking is one of the main advantages of RGB over current approaches. Additionally, due to the nested nature of RGB we also showed how it improves over GB in problems that have several high-order interactions. Finally, we investigate both theoretically and in practice the problem of recovering nested functions and the value of prior knowledge. [ABSTRACT FROM AUTHOR]

Using cross validation to select the best model from a library is standard practice in machine learning. Similarly, meta learning is a widely used technique where models previously developed are combined (mainly linearly) with the expectation of improving performance with respect to individual models. In this article we consider the Conditional Super Learner (CSL), an algorithm that selects the best model candidate from a library of models conditional on the covariates. The CSL expands the idea of using cross validation to select the best model and merges it with meta learning. We propose an optimization algorithm that finds a local minimum to the problem posed and proves that it converges at a rate faster than $O_p(n^{-1/4})$ O p (n - 1 / 4) . We offer empirical evidence that: (1) CSL is an excellent candidate to substitute stacking and (2) CLS is suitable for the analysis of Hierarchical problems. Additionally, implications for global interpretability are emphasized. [ABSTRACT FROM AUTHOR]

The estimation of nested functions (i.e., functions of functions) is one of the central reasons for the success and popularity of machine learning. Today, artificial neural networks are the predominant class of algorithms in this area, known as representational learning. Here, we introduce Representational Gradient Boosting (RGB), a nonparametric algorithm that estimates functions with multi-layer architectures obtained using backpropagation in the space of functions. RGB does not need to assume a functional form in the nodes or output (e.g., linear models or rectified linear units), but rather estimates these transformations. RGB can be seen as an optimized stacking procedure where a meta algorithm learns how to combine different classes of functions (e.g., Neural Networks (NN) and Gradient Boosting (GB)), while building and optimizing them jointly in an attempt to compensate each other’s weaknesses. This highlights a stark difference with current approaches to meta-learning that combine models only after they have been built independently. We showed that providing optimized stacking is one of the main advantages of RGB over current approaches. Additionally, due to the nested nature of RGB we also showed how it improves over GB in problems that have several high-order interactions. Finally, we investigate both theoretically and in practice the problem of recovering nested functions and the value of prior knowledge. [ABSTRACT FROM AUTHOR]

Using cross validation to select the best model from a library is standard practice in machine learning. Similarly, meta learning is a widely used technique where models previously developed are combined (mainly linearly) with the expectation of improving performance with respect to individual models. In this article we consider the Conditional Super Learner (CSL), an algorithm that selects the best model candidate from a library of models conditional on the covariates. The CSL expands the idea of using cross validation to select the best model and merges it with meta learning. We propose an optimization algorithm that finds a local minimum to the problem posed and proves that it converges at a rate faster than $O_p(n^{-1/4})$ O p (n - 1 / 4) . We offer empirical evidence that: (1) CSL is an excellent candidate to substitute stacking and (2) CLS is suitable for the analysis of Hierarchical problems. Additionally, implications for global interpretability are emphasized. [ABSTRACT FROM AUTHOR]

LSM-tree is widely used in key-value stores for big data storage, but it suffers from write amplification brought by frequent compaction operations. An effective solution for this problem is key-value separation, which decouples values from the LSM-tree and stores them in a separate value log. However, existing key-value separation schemes achieve poor range query performance, especially for small key-value pairs, because they focus on mitigating write amplification but neglect access characteristics of the SSD. In this article, we propose FenceKV, which aims to achieve better range query performance while maintaining reasonable update performance for update-intensive workloads. FenceKV employs a new partition method to map values to the storage space based on the key-range to achieve efficient update and range query. Moreover, it adopts a key-range garbage collection policy to mitigate the garbage collection overhead and maintain sequential access for range queries. We compare FenceKV with modern key-value stores with various workloads, and results show that FenceKV can improve the range query performance significantly, while maintaining reasonable update performance compared to the existing designs of key-value separation. [ABSTRACT FROM AUTHOR]

The intermittent and stochastic nature of solar energy generation systems, climate change, and the inefficiency of modern power systems due to zero inertia have created many challenges for on-grid operators. Solar forecasting systems based on machine learning algorithms are an emerging and effective solution that uses Big Data related to weather phenomena. However, the predictive ability of these algorithms is hampered by the sporadic nature of solar energy generation. In this article, a robust hybrid machine learning system that utilizes multiple linear regression (MLR) and a Pearson correlation coefficient (PCC) was tested on solar power plant sites of varying capacities in Germany (100–8500 kW). The volume of Big Data features can be reduced by focusing on the features that significantly improve the reliability of the mid-term forecasting system. In this way, drastic fluctuations in the prediction of photovoltaic (PV) power generation can be avoided. The results of our approach are evaluated regarding real-world data using the extreme gradient boosting (XGBoost) with feature engineering, and principal component analysis (PCA), in order to forecast PV energy, rank, and track the importance of feature engineering for different PV capacities. Furthermore, we found that the need for selectivity and reduction of performance error was supported by ridge regression. In addition, the proposed novel XGBoost forecast system decreased the root-mean-square error (RMSE) and mean absolute error (MAE) by 30% and 18%, respectively, compared to the Autoencoder and long short-term memory (LSTM) for same datasets. Furthermore, the CoD determination coefficient ($R^{2}$) increased by 85% compared to the statistical model's autoregressive integrated moving average (ARIMA). [ABSTRACT FROM AUTHOR]

Amplifier design, both in traditional approaches and real-time circuit optimization, greatly benefits from fast and thorough extraction of information from measurement data. Using only a few performance samples at varying impedances, deep learning image completion techniques can be utilized to extrapolate an entire set of Smith chart load-pull contours. In addition to speeding nonlinear device characterizations, this extrapolation can be performed in an iterative fashion for use as a circuit optimization algorithm with a very low number of measurements. The techniques of this work have been tested in the measurement of a nonlinear, large-signal amplifier. The load impedance can be estimated with a typical error of < 0.1 linear units using as few as seven impedances and yields even better accuracy with larger sample sizes. [ABSTRACT FROM AUTHOR]

With the maturity and popularity of Internet of Things (IoT), the notion of Social Internet of Things (SIoT) has been proposed to support novel applications and networking services for the IoT in more effective and efficient ways. Although there are many works for SIoT, they focus on designing the architectures and protocols for SIoT under the specific schemes. How to efficiently utilize the collaboration capability of SIoT to complete complex tasks remains unexplored. Therefore, we propose a new problem family, namely, Task-Optimized SIoT Selection (TOSS), to find the best group of IoT objects for a given set of tasks in the task pool. TOSS aims to select the target SIoT group such that the target SIoT group is able to easily communicate with each other while maximizing the accuracy of performing the given tasks. We propose two problem formulations, named Bounded Communication-loss TOSS (BC-TOSS) and Robustness Guaranteed TOSS (RG-TOSS), for different scenarios and prove that they are both NP-hard and inapproximable. We propose a polynomial-time algorithm with a performance guarantee for BC-TOSS, and an efficient polynomial-time algorithm to obtain good solutions for RG-TOSS. Moreover, as RG-TOSS is NP-hard and inapproximable within any factor, we further propose Structure-Aware Reinforcement Learning (SARL) to leverage the Graph Convolutional Networks (GCN) and Deep Reinforcement Learning (DRL) to effectively solve RG-TOSS. Further, since we use graph models to simulate the problem instance for DRL, which is different from the real ones, we propose Structure-Aware Meta Reinforcement Learning (SAMRL) for fast adapting to new domains. Experimental results on multiple real datasets indicate that our proposed algorithms outperform the other deterministic and learning-based baseline approaches. [ABSTRACT FROM AUTHOR]

AutoML aims at best configuring learning systems automatically. It contains core subtasks of algorithm selection and hyper-parameter tuning. Previous approaches considered searching in the joint hyper-parameter space of all algorithms, which forms a huge but redundant space and causes an inefficient search. We tackle this issue in a cascaded algorithm selection way, which contains an upper-level process of algorithm selection and a lower-level process of hyper-parameter tuning for algorithms. While the lower-level process employs an anytime tuning approach, the upper-level process is naturally formulated as a multi-armed bandit, deciding which algorithm should be allocated one more piece of time for the lower-level tuning. To achieve the goal of finding the best configuration, we propose the Extreme-Region Upper Confidence Bound (ER-UCB) strategy. Unlike UCB bandits that maximize the mean of feedback distribution, ER-UCB maximizes the extreme-region of feedback distribution. We first consider stationary distributions and propose the ER-UCB-S algorithm that has $O(K\ln n)$ O (K ln n) regret upper bound with $K$ K arms and $n$ n trials. We then extend to non-stationary settings and propose the ER-UCB-N algorithm that has $O(Kn^\nu)$ O (K n ν) regret upper bound, where $\frac{2}{3}<\nu <1$ 2 3 < ν < 1 . Finally, empirical studies on synthetic and AutoML tasks verify the effectiveness of ER-UCB-S/N by their outperformance in corresponding settings. [ABSTRACT FROM AUTHOR]

In real-world applications, it is important for machine learning algorithms to be robust against data outliers or corruptions. In this paper, we focus on improving the robustness of a large class of learning algorithms that are formulated as low-rank semi-definite programming (SDP) problems. Traditional formulations use the square loss, which is notorious for being sensitive to outliers. We propose to replace this with more robust noise models, including the $\ell _1$ ℓ 1 -loss and other nonconvex losses. However, the resultant optimization problem becomes difficult as the objective is no longer convex or smooth. To alleviate this problem, we design an efficient algorithm based on majorization-minimization. The crux is on constructing a good optimization surrogate, and we show that this surrogate can be efficiently obtained by the alternating direction method of multipliers (ADMM). By properly monitoring ADMM's convergence, the proposed algorithm is empirically efficient and also theoretically guaranteed to converge to a critical point. Extensive experiments are performed on four machine learning applications using both synthetic and real-world data sets. Results show that the proposed algorithm is not only fast but also has better performance than the state-of-the-arts. [ABSTRACT FROM AUTHOR]

The foundational assumption of machine learning is that the data under consideration is separable into classes; while intuitively reasonable, separability constraints have proven remarkably difficult to formulate mathematically. We believe this problem is rooted in the mismatch between existing statistical techniques and commonly encountered data; object representations are typically high dimensional but statistical techniques tend to treat high dimensions a degenerate case. To address this problem, we develop a dedicated statistical framework for machine learning in high dimensions. The framework derives from the observation that object relations form a natural hierarchy; this leads us to model objects as instances of a high dimensional, hierarchal generative processes. Using a distance based statistical technique, also developed in this paper, we show that in such generative processes, instances of each process in the hierarchy, are almost-always encapsulated by a distinctive-shell that excludes almost-all other instances. The result is shell theory, a statistical machine learning framework in which separability constraints (distinctive-shells) are formally derived from the assumed generative process. [ABSTRACT FROM AUTHOR]

Owing to the considerations of cell density and low static power consumption, nonvolatile random-access memory (NVRAM) has been a promising candidate for collaborating with a dynamic random-access memory (DRAM) as the main memory in modern computer systems. As NVRAM also brings technical challenges (e.g., limited endurance and high writing cost) to computer system developers, the concept of write reduction becomes the famous doctrine in NVRAM-based system design. Unfortunately, a well-known machine learning algorithm, random forest, will generate a massive amount of write traffic to the main memory space during its construction phase. In other words, a random forest hits the Achilles’ heel of NVRAM-based systems. For remedying this pain, our work proposes an NVRAM-friendly random forest algorithm, namely, Amine, for an NVRAM-based system. The design principle of Amine is to replace write operations with read accesses without raising the read complexity of the random forest algorithm. According to experimental results, Amine can effectively decrease the latency of random forest construction by 64%, compared with the original random forest algorithm. [ABSTRACT FROM AUTHOR]

Due to the increasing size of integrated circuits (ICs), their design and optimization phases (i.e., computer-aided design, CAD) grow increasingly complex. At design time, a large design space needs to be explored to find an implementation that fulfills all specifications and then optimizes metrics like energy, area, delay, reliability, etc. At run time, a large configuration space needs to be searched to find the best set of parameters (e.g., voltage/frequency) to further optimize the system. Both spaces are infeasible for exhaustive search typically leading to heuristic optimization algorithms that find some tradeoff between design quality and computational overhead. Machine learning (ML) can build powerful models that have successfully been employed in related domains. In this survey, we categorize how ML may be used and is used for design-time and run-time optimization and exploration strategies of ICs. A metastudy of published techniques unveils areas in CAD that are well explored and underexplored with ML, as well as trends in the employed ML algorithms. We present a comprehensive categorization and summary of the state of the art on ML for CAD. Finally, we summarize the remaining challenges and promising open research directions. [ABSTRACT FROM AUTHOR]

Edge intelligence enables edge devices to collaboratively perform computations and generate knowledge where distributed computations act as the main innovative technologies. The existing solutions for distributed computations mainly focus on the designs of computations, such as distributed machine learning algorithms. By contrast, edge intelligence poses the design requirements for efficient networking technology, including one request multiple reply in a time Interval (ORMRI), computation aggregation, and chained computation, which have not been well addressed. To satisfy those requirements, we design a Computation-Oriented Network (CON) to enable the efficient discovery, task allocation, and collaborative computations for edge devices in this article. In CON, model and datasets are afforded with the names for direct resource discovery, and a hybrid computation oriented routing (HCOR) mechanism has been proposed, which combines proactive and reactive routing approaches. Through CON, computation requests and replies can be sent to and collected from multiple suitable edge devices, respectively; computation aggregation is enabled during the computation result replying process; and chained computation is supported through sequentially popping out model names. We examine the performance of CON through analysis and implementation, which shows that the proposed CON can greatly reduce the communication bandwidth consumption compared to the existing end-to-end approach. [ABSTRACT FROM AUTHOR]

The field of meta-learning, or learning-to-learn, has seen a dramatic rise in interest in recent years. Contrary to conventional approaches to AI where tasks are solved from scratch using a fixed learning algorithm, meta-learning aims to improve the learning algorithm itself, given the experience of multiple learning episodes. This paradigm provides an opportunity to tackle many conventional challenges of deep learning, including data and computation bottlenecks, as well as generalization. This survey describes the contemporary meta-learning landscape. We first discuss definitions of meta-learning and position it with respect to related fields, such as transfer learning and hyperparameter optimization. We then propose a new taxonomy that provides a more comprehensive breakdown of the space of meta-learning methods today. We survey promising applications and successes of meta-learning such as few-shot learning and reinforcement learning. Finally, we discuss outstanding challenges and promising areas for future research. [ABSTRACT FROM AUTHOR]

Stereo matching is one of the most popular techniques to estimate dense depth maps by finding the disparity between matching pixels on two, synchronized and rectified images. Alongside with the development of more accurate algorithms, the research community focused on finding good strategies to estimate the reliability, i.e., the confidence, of estimated disparity maps. This information proves to be a powerful cue to naively find wrong matches as well as to improve the overall effectiveness of a variety of stereo algorithms according to different strategies. In this paper, we review more than ten years of developments in the field of confidence estimation for stereo matching. We extensively discuss and evaluate existing confidence measures and their variants, from hand-crafted ones to the most recent, state-of-the-art learning based methods. We study the different behaviors of each measure when applied to a pool of different stereo algorithms and, for the first time in literature, when paired with a state-of-the-art deep stereo network. Our experiments, carried out on five different standard datasets, provide a comprehensive overview of the field, highlighting in particular both strengths and limitations of learning-based strategies. [ABSTRACT FROM AUTHOR]

While most existing multilabel ranking methods assume the availability of a single objective label ranking for each instance in the training set, this paper deals with a more common case where only subjective inconsistent rankings from multiple rankers are associated with each instance. Two ranking methods are proposed from the perspective of instances and rankers, respectively. The first method, Instance-oriented Preference Distribution Learning (IPDL), is to learn a latent preference distribution for each instance. IPDL generates a common preference distribution that is most compatible to all the personal rankings, and then learns a mapping from the instances to the preference distributions. The second method, Ranker-oriented Preference Distribution Learning (RPDL), is proposed by leveraging interpersonal inconsistency among rankers, to learn a unified model from personal preference distribution models of all rankers. These two methods are applied to natural scene images dataset and 3D facial expression dataset BU_3DFE. Experimental results show that IPDL and RPDL can effectively incorporate the information given by the inconsistent rankers, and perform remarkably better than the compared state-of-the-art multilabel ranking algorithms. [ABSTRACT FROM AUTHOR]

Curriculum learning (CL) is a training strategy that trains a machine learning model from easier data to harder data, which imitates the meaningful learning order in human curricula. As an easy-to-use plug-in, the CL strategy has demonstrated its power in improving the generalization capacity and convergence rate of various models in a wide range of scenarios such as computer vision and natural language processing etc. In this survey article, we comprehensively review CL from various aspects including motivations, definitions, theories, and applications. We discuss works on curriculum learning within a general CL framework, elaborating on how to design a manually predefined curriculum or an automatic curriculum. In particular, we summarize existing CL designs based on the general framework of Difficulty Measurer $+$ + Training Scheduler and further categorize the methodologies for automatic CL into four groups, i.e., Self-paced Learning, Transfer Teacher, RL Teacher, and Other Automatic CL. We also analyze principles to select different CL designs that may benefit practical applications. Finally, we present our insights on the relationships connecting CL and other machine learning concepts including transfer learning, meta-learning, continual learning and active learning, etc., then point out challenges in CL as well as potential future research directions deserving further investigations. [ABSTRACT FROM AUTHOR]

More program functions are no longer written in code but learned from a huge number of data samples using a machine learning (ML) algorithm. We present an overview of current techniques to manage complex software and discuss how this applies to ML models. [ABSTRACT FROM AUTHOR]

In the 2015 Canadian case of Ewert v. Canada, risk assessment tools were put on trial in Canada’s Federal Court and eventually at the Supreme Court of Canada, their efficacy was challenged, and their reliability was upended ,. Risk assessment tools are used by the justice system to present a calculated prediction of an offender’s risk of future criminal behavior. These tools are viewed by the legal community as precursors to machine learning and technological advancements in artificial intelligence (AI) in the criminal justice system, using algorithmic and data-driven decision-making to provide courts with predictions regarding an offender’s risk if released. [ABSTRACT FROM AUTHOR]

Autonomous vehicles (AVs) generate a massive amount of multi-modal data that once collected and processed through Machine Learning algorithms, enable AI-based services at the Edge. In fact, only a subset of the collected data present informative attributes to be exploited at the Edge. Therefore, extracting such a subset is of utmost importance to limit computation and communication workloads. Doing that in a distributed manner imposes the AVs to cooperate in finding an agreement on which attributes should be sent to the Edge. In this work, we address such a problem by proposing a federated feature selection (FFS) algorithm where the AVs collaborate to filter out, iteratively, the less relevant attributes in a distributed manner, without any exchange of raw data, thought two different components: a Mutual-Information-based feature selection algorithm run by the AVs and a novel aggregation function based on the Bayes theorem executed on the Edge. The FFS algorithm has been tested on two reference datasets: MAV with images and inertial measurements of a monitored vehicle, WESAD with a collection of samples from biophysical sensors to monitor a relative passenger. The numerical results show that the AVs converge to a minimum achievable subset of features with both the datasets, i.e., 24 out of 2166 (99%) in MAV and 4 out of 8 (50%) in WESAD, respectively, preserving the informative content of data. [ABSTRACT FROM AUTHOR]

Artificial intelligence and machine learning use a wide variety of algorithms. More than 60 years of research have shown that intelligent behavior requires a variety of methods. We discuss the major classes of algorithms and their uses. [ABSTRACT FROM AUTHOR]

Multi-view subspace clustering has attracted intensive attention to effectively fuse multi-view information by exploring appropriate graph structures. Although existing works have made impressive progress in clustering performance, most of them suffer from the cubic time complexity which could prevent them from being efficiently applied into large-scale applications. To improve the efficiency, anchor sampling mechanism has been proposed to select vital landmarks to represent the whole data. However, existing anchor selecting usually follows the heuristic sampling strategy, e.g. $k$ -means or uniform sampling. As a result, the procedures of anchor selecting and subsequent subspace graph construction are separated from each other which may adversely affect clustering performance. Moreover, the involved hyper-parameters further limit the application of traditional algorithms. To address these issues, we propose a novel subspace clustering method termed Fast Parameter-free Multi-view Subspace Clustering with Consensus Anchor Guidance (FPMVS-CAG). Firstly, we jointly conduct anchor selection and subspace graph construction into a unified optimization formulation. By this way, the two processes can be negotiated with each other to promote clustering quality. Moreover, our proposed FPMVS-CAG is proved to have linear time complexity with respect to the sample number. In addition, FPMVS-CAG can automatically learn an optimal anchor subspace graph without any extra hyper-parameters. Extensive experimental results on various benchmark datasets demonstrate the effectiveness and efficiency of the proposed method against the existing state-of-the-art multi-view subspace clustering competitors. These merits make FPMVS-CAG more suitable for large-scale subspace clustering. The code of FPMVS-CAG is publicly available at https://github.com/wangsiwei2010/FPMVS-CAG. [ABSTRACT FROM AUTHOR]

We present a novel method to detect underground cavities using cross-borehole ground-penetrating radar (GPR). We model the propagation of a GPR signal across a cavity and find that the spectrum of the signal will be distorted in a specific low-pass manner. Comparing the spectra of received signals with the predicted spectrum using a hypothesis-testing approach allows us to detect cavities with high probability. The proposed method is validated on both simulated and real measurements. Our approach generally remains effective even when conventional methods fail; our method can also be combined with conventional methods to more successfully distinguish between cavities and geologic clutter. [ABSTRACT FROM AUTHOR]

Adaptive gradient algorithms have recently become extremely popular because they have been applied successfully in training deep neural networks, such as Adam, AMSGrad, and AdaBound. Despite their success, however, the distributed variant of the adaptive method, which is expected to possess a rapid training speed at the beginning and a good generalization capacity at the end, is rarely studied. To fill the gap, a distributed adaptive subgradient algorithm is presented, called D-AdaBound, where the learning rates are dynamically bounded by clipping the learning rates. Moreover, we obtain the regret bound of D-AdaBound, in which the objective functions are convex. Finally, we confirm the effectiveness of D-AdaBound by simulation experiments on different datasets. The results show the performance improvement of D-AdaBound relative to existing distributed online learning algorithms. [ABSTRACT FROM AUTHOR]

The problem of representative selection amounts to sampling few informative exemplars from large datasets. Existing approaches to data selection often fall short of simultaneously handling non-linear data structures, sampling concise and non-redundant subsets, rejecting outliers, and yielding interpretable outcomes. This paper presents a novel representative selection approach, dubbed MOSAIC, for drawing descriptive sketches of arbitrary manifold structures. Resting upon a novel quadratic formulation, MOSAIC advances a multi-criteria selection approach that maximizes the global representation power of the sampled subset, ensures novelty of the samples by minimizing redundancy, and rejects disruptive information by effectively detecting outliers. Theoretical analyses shed light on geometrical characterization of the obtained sketch and reveal that the sampled representatives maximize a well-defined notion of data coverage in a transformed space. In addition, we present a highly scalable randomized implementation of the proposed algorithm shown to bring about substantial speedups. MOSAIC’s superiority in achieving the desired characteristics of a representative subset all at once while exhibiting remarkable robustness to various outlier types is demonstrated via extensive experiments conducted on both real and synthetic data with comparisons to state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Static and dynamic clustering algorithms are a fundamental tool in any machine learning library. Most of the efforts in developing dynamic machine learning and data mining algorithms have been focusing on the sliding window model or more simplistic models. However, in many real-world applications one might need to deal with arbitrary deletions and insertions. For example, one might need to remove data items that are not necessarily the oldest ones, because they have been flagged as containing inappropriate content or due to privacy concerns. Clustering trajectory data might also require to deal with more general update operations. We develop a $(2+\epsilon)$ (2 + ε) -approximation algorithm for the $k$ k -center clustering problem with “small” amortized cost under the fully dynamic adversarial model. In such a model, points can be added or removed arbitrarily, provided that the adversary does not have access to the random choices of our algorithm. The amortized cost of our algorithm is poly-logarithmic when the ratio between the maximum and minimum distance between any two points in input is bounded by a polynomial, while $k$ k and $\epsilon$ ε are constant. Furthermore, we significantly improve the memory requirement of our fully dynamic algorithm, although at the cost of a worse approximation ratio of $4 +\epsilon$ 4 + ε . Our theoretical results are complemented with an extensive experimental evaluation on dynamic data from Twitter, Flickr, as well as trajectory data, demonstrating the effectiveness of our approach. [ABSTRACT FROM AUTHOR]

As multipath components (MPCs) are experimentally observed to appear in clusters, cluster-based channel models have been focused in the wireless channel study. However, most of the MPC clustering algorithms for multi-in–multiout (MIMO) channels with delay and angle information of MPCs are based on the distance metric that quantifies the similarity of two MPCs and determines the preferred cluster shape, greatly impacting MPC clustering quality. In this article, a general framework of Mahalanobis-distance metric is proposed for MPC clustering in the MIMO channel analysis, without user-specified parameters. Remarkably, the popular multipath component distance (MCD) is proven to be a special case of the proposed distance metric framework. Furthermore, two machine learning algorithms, namely, weak-supervised Mahalanobis metric for clustering and supervised large margin nearest neighbor, are introduced to learn the distance metric. To evaluate the effectiveness, a modified channel model is proposed based on the Third Generation Partnership Project (3GPP) spatial channel model (SCM) to generate clustered MPCs with delay and angular information since the original 3GPP SCM is incapable to evaluate clustering quality. Experiment results show that the proposed distance metric can significantly improve the clustering quality of existing clustering algorithms, while the learning phase requires considerably limited efforts of labeling MPCs. [ABSTRACT FROM AUTHOR]

Industrial Internet of Things (IoT) is the most rapidly developing industry in the current IoT industry, and the intrusion detection system (IDS) remains one of the key technologies for industrial IoT security protection. Researchers have considered applying algorithms such as machine learning and deep learning to network IDSs to cope with complex and changing network environments and to automatically extract key features from high-dimensional feature data. However, in the real industrial IoT environment, data imbalance is the main factor that affects the performance of the deep-learning-based IDS. In this article, we study the network intrusion detection model based on data level. Three data-based research schemes are constructed step by step in this article, which are a data augmentation scheme based on the variational autoencoder (VAE), a data-balancing scheme based on the conditional VAE, and a data-balancing scheme based on random undersampling and conditional VAE. The three data-level-based schemes are combined with the deep-learning-based IDS. In this article, we build experiments based on the CSE-CIC-IDS2018 dataset to verify the effectiveness of three data processing schemes. After data enhancement through the third scheme, the Macro-F1-score of the convolutional-neural-network-based IDS model improved by 3.75% and the Macro-F1-score of the gated-recurrent-unit-based IDS model improved by 5.32%. [ABSTRACT FROM AUTHOR]

Spectre attacks can be catastrophic and widespread because they exploit common design flaws caused by the speculative capabilities in modern processors to leak sensitive data through side channels. Completely fixing the problem would require a redesign of the architecture for transient execution or the implementation of a new design on re-configurable hardware. However, such fixes cannot be backported to old machines with fixed hardware design. Completely replacing those machines will take a long time. Moreover, existing software patches may cause significant performance overhead. This paper proposes to detect Spectre by monitoring deviations in microarchitectural events using hardware performance counters with promising accuracy above 90 percent under a variety of workload conditions. However, the attacker may attempt to evade detection by slowing down the attack or mimicking benign programs. This paper thus compares different evasion strategies quantitatively and demonstrates that it is possible for the attacker to avoid detection when operating the attacks at a lower speed while maintaining a reasonable attack success rate. Then, we show that, in order to resist evasion, the original detector must be enhanced by randomly switching between a set of detectors using different features and sampling periods so we can keep the detection accuracy above 80 percent. [ABSTRACT FROM AUTHOR]

The Hadamard Extension $\mathbb H({\mathrm {m}})$ of an $n \times k$ matrix m is the collection of all Hadamard products of subsets of its rows. This construction is essential for source identification (parameter estimation) of a mixture of $k$ product distributions over $n$ binary random variables. A necessary requirement for such identification is that $\mathbb H({\mathrm {m}})$ have full column rank; conversely, identification is possible if apart from each row there exist two disjoint sets of rows of m, each of whose extension has full column rank. It is necessary therefore to understand when $\mathbb H({\mathrm {m}})$ has full column rank; we provide two results in this direction. The first is that if $\mathbb H({\mathrm {m}})$ has full column rank then there exists a set of at most $k-1$ rows of m, whose extension already has full column rank. The second is a Hall-type condition on the values in the rows of m, that suffices to ensure full column rank of $\mathbb H({\mathrm {m}})$. [ABSTRACT FROM AUTHOR]

When demonstrating the effectiveness of a new algorithm, researchers are traditionally encouraged to compare their algorithm's performance against existing algorithms on well-studied benchmark test suites. In the absence of more nuanced methodologies, algorithm performance is typically summarized on average across the test suite examples. This paper highlights the potential bias of conclusions drawn by analyzing “on average” performance, and the opportunities offered by a recent testing methodology known as instance space analysis. To illustrate, we revisit our 2007 comparative study of algorithms for facial age estimation, and rigorously stress-test to challenge the original conclusions. The case study demonstrates how powerful visualizations offered by instance space analysis enable greater insights into unique strengths and weaknesses, and which algorithm should be used when and why. Inspired by such insights, a new algorithm is proposed, and its unique advantage is demonstrated. The bias often hidden in well-studied datasets, and the ramifications for drawing biased conclusions, are also illustrated in this case study. While focused on facial age estimation, the methodology and lessons learned from the case study are broadly applicable to any study seeking to draw conclusions about algorithm performance based on empirical results. [ABSTRACT FROM AUTHOR]

Spectral unmixing expresses the mixed pixels existing in hyperspectral images as the product of endmembers and their corresponding fractional abundances, which has been widely used in hyperspectral imagery analysis. However, the endmember spectra even for pixels from the same material of an image may include variability due to the influence of lighting conditions and inherent properties of materials within different pixels. Though the in situ spectral library has been used to accommodate such variability by using multiple in situ spectra to represent each kind of material, the performance improvement may be restricted due to the limited number of endmembers for each material. Therefore, in this article, spectral variability is directly extracted from an in situ endmember library and considered to be transferable among different endmembers for the first time. Furthermore, such a spectral variability is further used to augment sparse unmixing by synchronously performing endmember-based reconstruction and spectral variability-augmented reconstruction in the sparse unmixing model. By, respectively, imposing sparse and smoothness regularization over abundances and variability coefficients, a convex optimization-based spectral variability augmented sparse unmixing (SVASU) is finally proposed, and its convergence performance is also analyzed. Experiments conducted over synthetic and real-world datasets demonstrate that the proposed SVASU method not only significantly improves the unmixing performance of conventional spectral library-based unmixing but also outperforms several state-of-the-art sparse unmixing algorithms. [ABSTRACT FROM AUTHOR]

As a promising solution to boost the performance of distance-related algorithms (e.g., $K$ -means and KNN), FPGA-based acceleration attracts lots of attention, but also comes with numerous challenges. In this work, we propose, STPAcc, an optimization framework based on structural triangle-inequality (TI)-based pruning (STP) for accelerating distance-related algorithms on CPU-FPGA platforms. STPAcc provides a domain-specific language to unify distance-related algorithms effectively, a structural TI-based pruning strategy to remove unnecessary distance computations, a coarse-grained workload partitioning and mapping strategy to fully exploit the potentials of the CPU-FPGA platform, and fine-grained hardware optimizations to further improve performance on the FPGA. Intensive experiments show that STPAcc designs achieve $31.42\times $ speedup and $99.63\times $ better energy efficiency on average over standard CPU-based implementations. [ABSTRACT FROM AUTHOR]

Early fault detection in power electronic systems (PESs) to maintain reliability is one of the most important issues that has been significantly addressed in recent years. In this article, after reviewing various works of literature based on fault detection in PESs, data mining-based techniques including artificial neural network, machine learning, and deep learning algorithms are introduced. Then, the fault detection routine in PESs is expressed by introducing signal measurement sensors and how to extract the feature from them. Finally, based on studies, the performance of various data mining methods in detecting PESs faults is evaluated. The results of evaluations show that the deep learning-based techniques given the ability of feature extraction from measured signals are significantly more effective than other methods and as an ideal tool for future applications in the power electronics industry are introduced. [ABSTRACT FROM AUTHOR]

Real-time forecasting of the financial time-series data is challenging for many machine learning (ML) algorithms. First, many ML models operate offline, where they need a batch of data, which may not be available during training. Besides, due to a fixed architecture of the majority of the offline-based ML models, they suffer to deal with the uncertain nature of financial time-series data. In contrast, online learning mode evolving-structured ML models could be promising for financial time-series forecasting. For real-time deployment of such models, low memory demand is a must. Besides, the model’s explainability plays a crucial role in forecasting financial time-series. Considering all the requirements, a rule-based autonomous neuro-fuzzy learning algorithm called the parsimonious learning machine (PALM) is proposed here to forecast time-varying stock indices. To provide efficient automation of the proposed algorithm by maintaining the model explainability in terms of limited number linguistic IF-THEN rules, two popular multiobjective evolutionary algorithms (MEAs), such as a real-coded genetic algorithm (GA) and a self-adaptive differential evolution (DE) algorithm are utilized here. In addition, fuzzy type-2 variants of PALMs’ are considered here due to better uncertainty handling capacity than their type-1 counterparts. To evaluate the proposed algorithm’s performance, the closing stock price of fifteen (15) different stock market indices are predicted here. From the results, it is observed that the MEA-based PALMs are performing better than the state-of-the-art benchmark online ML models and providing a rule-based explainable model to the end-user. [ABSTRACT FROM AUTHOR]

Class imbalance problem that characterized with the skew distribution towards the majority arises as one challenge in recent years. Many oversampling techniques have been proposed to cope with this problem and some of them combine the oversampling procedure with the clustering algorithm which guaranteeing new synthetic samples being generated in clusters. However far-away samples but with the same minority sub-region are generally clustered into different groups owing to the characteristic of clustering algorithm itself. Therefore, the following oversampling procedure is mostly carried in incomplete minority sub-regions that synthetic samples not well cover the integral minority region. And to our best knowledge, none of existing algorithm is designed to directly estimate minority sub-regions for class imbalance problem. Thus, one new grouping algorithm, named Direction Distribution-based Minority Sub-region Estimation (DDMSE), is first proposed. The new algorithm exploits the intuitive observation, that the minority with the same sub-region almost distribute within the same direction when compared to other majority, to estimate minority sub-regions that tactfully ignoring negative impacts brought by the distance factor like in clustering algorithms. Finally, new synthetic samples are generated in those minority sub-regions. And experimental results on real-world datasets show the comparable performance with other state-of-the-art oversampling methods. [ABSTRACT FROM AUTHOR]

Deep learning is recognized to be capable of discovering deep features for representation learning and pattern recognition without requiring elegant feature engineering techniques by taking advantages of human ingenuity and prior knowledge. Thus it has triggered enormous research activities in machine learning and pattern recognition. One of the most important challenges of deep learning is to figure out relations between a feature and the depth of deep neural networks (deep nets for short) to reflect the necessity of depth. Our purpose is to quantify this feature-depth correspondence in feature extraction and generalization. We present the adaptivity of features to depths and vice-verse via showing a depth-parameter trade-off in extracting both single feature and composite features. Based on these results, we prove that implementing the classical empirical risk minimization on deep nets can achieve the optimal generalization performance for numerous learning tasks. Our theoretical results are verified by a series of numerical experiments including toy simulations and a real application of earthquake seismic intensity prediction. [ABSTRACT FROM AUTHOR]

Algorithms are becoming ubiquitous. However, companies are increasingly alarmed about their algorithms causing major financial or reputational damage. A new industry is envisaged: auditing and assurance of algorithms with the remit to validate artificial intelligence, machine learning, and associated algorithms. [ABSTRACT FROM AUTHOR]

A growing specter in the rise of machine learning is whether the decisions made by machine learning models are fair. While research is already underway to formalize a machine-learning concept of fairness and to design frameworks for building fair models with sacrifice in accuracy, most are geared toward either supervised or unsupervised learning. Yet two observations inspired us to wonder whether semi-supervised learning might be useful to solve discrimination problems. First, previous study showed that increasing the size of the training set may lead to a better trade-off between fairness and accuracy. Second, the most powerful models today require an enormous of data to train which, in practical terms, is likely possible from a combination of labeled and unlabeled data. Hence, in this paper, we present a framework of fair semi-supervised learning in the pre-processing phase, including pseudo labeling to predict labels for unlabeled data, a re-sampling method to obtain multiple fair datasets and lastly, ensemble learning to improve accuracy and decrease discrimination. A theoretical decomposition analysis of bias, variance and noise highlights the different sources of discrimination and the impact they have on fairness in semi-supervised learning. A set of experiments on real-world and synthetic datasets show that our method is able to use unlabeled data to achieve a better trade-off between accuracy and discrimination. [ABSTRACT FROM AUTHOR]

This paper focuses on improving indoor Wi-Fi localization by mitigating the effect of fluctuation of received signal strength indication (RSSI). The RSSI data collected at each reference point is first transformed through translation and scaling. The BP (Back Propagation) neural network is then used to construct the ranging model using the transformed RSSI to determine the distances between the target point and each reference point. A genetic algorithm (GA) is developed to optimize the initial values of weights and biases of the BP neural network. For convenience, our proposed ranging model is denoted as GTBPD. A new localization algorithm is then proposed, which uses the GTBPD model and the sequential quadratic programming (SQP, an iterative nonlinear optimization algorithm), and the algorithm is denoted as GTBPD-LSQP for simplicity. Experiments were conducted in three areas of two different teaching buildings with complex environments. The performance of the proposed GTBPD-LSQP algorithm is evaluated and compared with four existing algorithms. The experimental results show that our proposed GTBPD-LSQP algorithm achieves significantly higher location accuracy than the four existing algorithms. [ABSTRACT FROM AUTHOR]

We consider a class of nonconvex regularized optimization problems, which appear frequently in machine learning and data processing. Due to the structure of the problems, the iteratively reweighted algorithm was developed and applied to the consensus optimization. In this paper, we propose the acceleration of this scheme by adding an inertial term in each iteration. The proposed algorithms inherit the advantages of classical decentralized algorithms: they can be implemented over a connected network, in which the agents communicate with their neighbors and perform local computations. We also employ the diminishing stepsizes technique for the iteratively reweighted algorithm and consider its acceleration. In specific cases, our algorithms reduce to existing decentralized schemes and also indicate novel ones. Mathematically, we prove the convergence for both algorithms with several assumptions on the objective functions. With Kurdyka-Łojasiewicz property, convergence rates can be derived for constant stepsize case. Numerical results demonstrate the efficiency of the algorithms. [ABSTRACT FROM AUTHOR]