Search

Due to the increasing size of integrated circuits (ICs), their design and optimization phases (i.e., computer-aided design, CAD) grow increasingly complex. At design time, a large design space needs to be explored to find an implementation that fulfills all specifications and then optimizes metrics like energy, area, delay, reliability, etc. At run time, a large configuration space needs to be searched to find the best set of parameters (e.g., voltage/frequency) to further optimize the system. Both spaces are infeasible for exhaustive search typically leading to heuristic optimization algorithms that find some tradeoff between design quality and computational overhead. Machine learning (ML) can build powerful models that have successfully been employed in related domains. In this survey, we categorize how ML may be used and is used for design-time and run-time optimization and exploration strategies of ICs. A metastudy of published techniques unveils areas in CAD that are well explored and underexplored with ML, as well as trends in the employed ML algorithms. We present a comprehensive categorization and summary of the state of the art on ML for CAD. Finally, we summarize the remaining challenges and promising open research directions. [ABSTRACT FROM AUTHOR]

The traditional method for transformer moisture diagnosis is to establish empirical equations between feature parameters extracted from frequency domain spectroscopy (FDS) and the transformer’s moisture content. However, the established empirical equation may not be applicable to a novel testing environment, resulting in an unreliable evaluation result. In this regard, it is acknowledged that FDS combined with machine learning is more suitable for estimating moisture content in a variety of test environments. Nonetheless, the accuracy of the estimation results obtained using the existing method is limited by the algorithm’s inability to generalize. To address this issue, we propose an AdaBoost algorithm-enhanced back-propagation neural network (BP_AdaBoost). This study creates a database by extracting feature parameters from the FDS that characterize the insulation states of the prepared samples. Then, using the BP_AdaBoost algorithm and the newly constructed database, the moisture estimation models are trained. Finally, the results of the estimation are discussed in terms of laboratory and field transformers. By comparing the proposed BP_AdaBoost algorithm to other intelligence algorithms, it is demonstrated that it not only performs better in generalization, but also maintains a high level of accuracy. [ABSTRACT FROM AUTHOR]

The article presents the discussion on cloud computing playing a dominant role in supporting the applications. Topics include vehicular networking, automated manufacturing, smart cities, drones, smart grids, e-health, and the many emerging AI-enabled applications such as Virtual Reality (VR) and Augmented Reality (AR); and establishing radio connections more timely and adjusting radio channel coding dynamically in response to changing user needs and communication environments.

Prospective authors are requested to submit new, unpublished manuscripts for inclusion in the upcoming event described in this call for papers. [ABSTRACT FROM AUTHOR]

The article reports that With the continued growth of IoT devices and their deployment, manually managing and connecting them is impractical and presents multiple challenges. To that end, Zero Touch Networks that rely on software-based modules instead of dedicated propriety hardware become a viable potential solution. The overall aim of zero-touch networks is for machines to learn how to become more autonomous so that we can delegate complex, mundane tasks to them.

Prospective authors are requested to submit new, unpublished manuscripts for inclusion in the upcoming event described in this call for papers. [ABSTRACT FROM AUTHOR]

This paper discusses the optimization of an advanced process monitoring scheme with interdicting fault detection and classification (FDC) capabilities that improved the control over the process development of Tantalum Nitride thin film resistors (TaN TFR). Its implementation in a high-volume manufacturing environment resulted in a reduction of misprocessed wafers, shorter equipment downtime, higher throughput, enhanced process visibility, and yield improvement. Along with optimizing FDC capabilities, implementing a short loop sampling plan reduced the turnaround time for early electrical characterization by a factor of four; this allowed for timely inline adjustments within the fabrication process to tighten the statistical process control (SPC) over the distribution of the process control monitor (PCM) TaN resistor value and improve thin film uniformity. [ABSTRACT FROM AUTHOR]

Semiconductor manufacturing has been extensively exploiting machine-learning (ML) to process equipment sensory data (ESD) for near-real time anomaly detection (AD). ESD characteristics are highly diversified and data lengths vary among processing steps and cycles. Cleansing ESD with minimum distortion (CMD) to fit the fixed-length input requirement by ML-based AD is critical to AD effectiveness and is challenging. This paper presents a novel CMD method of four innovations: i) statistical mode-based equalization of step data lengths for the least number of step data length changes, ii) importance indicator value (IIV) of a data sample based on its relative difference with the subsequent sample, and iii) step data segmentation into groups based on samples of significant IIVs and the least-entropy-group-to-cleanse-first rule, and iv) cleansing the least IIV sample(s) in the selected group for step data length equalization. CMD application to ESD demonstrates its characteristics preservation property. Simulation experiments are on an integration of data cleansing with an unsupervised ML-based AD system, STALAD. Comparisons with two benchmark methods over AD scenarios of small-scale drifts and shifts show that CMD not only is superior in facilitating accurate detection by STALAD but also helps detect anomaly much earlier than using the two benchmarks. [ABSTRACT FROM AUTHOR]

Quality monitoring is an essential element of defect detection in semiconductor manufacturing processes, but semiconductor companies use virtual metrology (VM) in addition to actual metrology to prevent productivity degradation due to the time and costs required to obtain measurements. Past VM studies aimed to predict average wafer measurement values via equipment sensor data and focused on achieving improved predictive performance by selecting or extracting important variables among high-dimensional variables such as equipment sensor data. However, the management of wafer chip quality requires not only average measurement values but also measurement value predictions for chips located at the edges, which are vulnerable to defects. In this paper, we therefore propose a graph attention (GAT) network-based VM model that predicts the measurement values of chips located at wafer edges by constructing graph data with measurement data (i.e., the measurement location information in wafers and the measurement values). To verify the performance of the proposed model, we conduct a comparative experiment with conventional machine learning methods. The experimental results show that the proposed VM model contributes to a predictive performance improvement in terms of the measurement values of chips located at wafer edges. [ABSTRACT FROM AUTHOR]

Data quality plays an important role during the training stage of machine/deep learning models. The annotation hinges on the experiences of domain experts. To acquire the expert’s knowledge in the context of machine learning, manual data labeling, a tedious and time-consuming task in supervised learning, should be given a top priority. However, the domain experts in the line of plentiful manual annotation may easily get distracted or fatigued after long-time work, causing judgment errors, mislabeling, etc. The pattern recognition of wafer defect map is investigated in this paper, the primary goal of which is to train the convolutional neural network (CNN) model through a very limited number of manually labeled data so that the trained model is capable of performing pseudo labeling. Subsequently, a self-assured adaptive ensemble learner in terms of a series of shallow neural networks is proposed to filter wafer map samples with untrusted pseudo-labels. In the result, the amount of human annotations is significantly reduced by 61% for training a highly accurate classifier. A minimum number of manually labeled data is suggested while the equally high classification performance of wafer defect pattern is maintained. For the evaluation purpose, the proposed self-assured learning is compared with the confidence learning. [ABSTRACT FROM AUTHOR]

Bi-Level Optimization (BLO) is originated from the area of economic game theory and then introduced into the optimization community. BLO is able to handle problems with a hierarchical structure, involving two levels of optimization tasks, where one task is nested inside the other. In machine learning and computer vision fields, despite the different motivations and mechanisms, a lot of complex problems, such as hyper-parameter optimization, multi-task and meta learning, neural architecture search, adversarial learning and deep reinforcement learning, actually all contain a series of closely related subproblms. In this paper, we first uniformly express these complex learning and vision problems from the perspective of BLO. Then we construct a best-response-based single-level reformulation and establish a unified algorithmic framework to understand and formulate mainstream gradient-based BLO methodologies, covering aspects ranging from fundamental automatic differentiation schemes to various accelerations, simplifications, extensions and their convergence and complexity properties. Last but not least, we discuss the potentials of our unified BLO framework for designing new algorithms and point out some promising directions for future research. A list of important papers discussed in this survey, corresponding codes, and additional resources on BLOs are publicly available at: https://github.com/vis-opt-group/BLO. [ABSTRACT FROM AUTHOR]

Bi-Level Optimization (BLO) is originated from the area of economic game theory and then introduced into the optimization community. BLO is able to handle problems with a hierarchical structure, involving two levels of optimization tasks, where one task is nested inside the other. In machine learning and computer vision fields, despite the different motivations and mechanisms, a lot of complex problems, such as hyper-parameter optimization, multi-task and meta learning, neural architecture search, adversarial learning and deep reinforcement learning, actually all contain a series of closely related subproblms. In this paper, we first uniformly express these complex learning and vision problems from the perspective of BLO. Then we construct a best-response-based single-level reformulation and establish a unified algorithmic framework to understand and formulate mainstream gradient-based BLO methodologies, covering aspects ranging from fundamental automatic differentiation schemes to various accelerations, simplifications, extensions and their convergence and complexity properties. Last but not least, we discuss the potentials of our unified BLO framework for designing new algorithms and point out some promising directions for future research. A list of important papers discussed in this survey, corresponding codes, and additional resources on BLOs are publicly available at: https://github.com/vis-opt-group/BLO. [ABSTRACT FROM AUTHOR]

Multi-Task Learning (MTL) is a learning paradigm in machine learning and its aim is to leverage useful information contained in multiple related tasks to help improve the generalization performance of all the tasks. In this paper, we give a survey for MTL from the perspective of algorithmic modeling, applications and theoretical analyses. For algorithmic modeling, we give a definition of MTL and then classify different MTL algorithms into five categories, including feature learning approach, low-rank approach, task clustering approach, task relation learning approach and decomposition approach as well as discussing the characteristics of each approach. In order to improve the performance of learning tasks further, MTL can be combined with other learning paradigms including semi-supervised learning, active learning, unsupervised learning, reinforcement learning, multi-view learning and graphical models. When the number of tasks is large or the data dimensionality is high, we review online, parallel and distributed MTL models as well as dimensionality reduction and feature hashing to reveal their computational and storage advantages. Many real-world applications use MTL to boost their performance and we review representative works in this paper. Finally, we present theoretical analyses and discuss several future directions for MTL. [ABSTRACT FROM AUTHOR]

Malicious software, popularly known as malware, is widely acknowledged as a serious threat to modern computing systems. Software-based solutions, such as anti-virus software (AVS), are not effective since they rely on matching patterns that can be easily fooled by carefully crafted malware with obfuscation or other deviation capabilities. While recent malware detection methods provide promising results through an effective utilization of hardware features, the detection results cannot be interpreted in a meaningful way. In this paper, we propose a hardware-assisted malware detection framework using explainable machine learning. This paper makes three important contributions. First, we theoretically establish that our proposed method can provide an interpretable explanation of classification results to address the challenge of transparency. Next, we show that the explainable outcome through effective utilization of hardware performance counters and embedded trace buffer can lead to accurate localization of malicious behavior. Finally, we have performed efficiency versus accuracy trade-off analysis using decision tree and recurrent neural networks. Extensive evaluation using a wide variety of real-world malware dataset demonstrates that our framework can produce accurate and human-understandable malware detection results with provable guarantees. [ABSTRACT FROM AUTHOR]

As an aerial base station, unmanned aerial vehicle (UAV) has been considered as a promising technology to assist future wireless communications due to its flexible, swift and low cost features, where resource allocation is the basis for ensuring energy-efficient UAV-assisted networks. This paper formulates a joint optimization problem of user association, UAV trajectory design and power control to maximize the channel capacity among all ground users at a limited power level in a downlink transmission. To tackle the mixed-integer non-linear programming problem, this paper proposes a clustering-aided reinforcement learning approach consisting of three consecutive stages. Firstly, modified expectation-maximization unsupervised learning algorithm is investigated to cluster the ground users, which reduces the dimensions and hence, the association complexity is reduced as well. Then, Kuhn-Munkres algorithm is incorporated for user association, which associates a UAV with the ground users via matching to the cluster, and assigns the UAVs to the centroid of the matching cluster for pre-placement, with the aim of speeding up the convergence of the following deep reinforcement learning algorithm. Finally, a multi-agent twin delayed deep deterministic (MATD3) policy gradient is proposed to solve the non-convex sub-problem, which determines the transmit power and designs the fine-tuned trajectory of UAVs. By incorporating low-bias value estimation technique, the reward of the proposed MATD3 algorithm is improved. Simulation results have demonstrated that our proposed approach achieves higher reward as well as converging faster than existing reinforcement algorithms. Besides, the clustering-aided reinforcement learning has lower computational complexity than the benchmark schemes. [ABSTRACT FROM AUTHOR]

Reinforcement learning (RL) has demonstrated its ability to solve high dimensional tasks by leveraging non-linear function approximators. However, these successes are mostly achieved by ’black-box’ policies in simulated domains. When deploying RL to the real world, several concerns regarding the use of a ’black-box’ policy might be raised. In order to make the learned policies more transparent, we propose in this paper a policy iteration scheme that retains a complex function approximator for its internal value predictions but constrains the policy to have a concise, hierarchical, and human-readable structure, based on a mixture of interpretable experts. Each expert selects a primitive action according to a distance to a prototypical state. A key design decision to keep such experts interpretable is to select the prototypical states from trajectory data. The main technical contribution of the paper is to address the challenges introduced by this non-differentiable prototypical state selection procedure. Experimentally, we show that our proposed algorithm can learn compelling policies on continuous action deep RL benchmarks, matching the performance of neural network based policies, but returning policies that are more amenable to human inspection than neural network or linear-in-feature policies. [ABSTRACT FROM AUTHOR]

This paper serves as a survey of recent advances in large margin training and its theoretical foundations, mostly for (nonlinear) deep neural networks (DNNs) that are probably the most prominent machine learning models for large-scale data in the community over the past decade. We generalize the formulation of classification margins from classical research to latest DNNs, summarize theoretical connections between the margin, network generalization, and robustness, and introduce recent efforts in enlarging the margins for DNNs comprehensively. Since the viewpoint of different methods is discrepant, we categorize them into groups for ease of comparison and discussion in the paper. Hopefully, our discussions and overview inspire new research work in the community that aim to improve the performance of DNNs, and we also point to directions where the large margin principle can be verified to provide theoretical evidence why certain regularizations for DNNs function well in practice. We managed to shorten the paper such that the crucial spirit of large margin learning and related methods are better emphasized. [ABSTRACT FROM AUTHOR]

Contrastive self-supervised representation learning methods maximize the similarity between the positive pairs, and at the same time tend to minimize the similarity between the negative pairs. However, in general the interplay between the negative pairs is ignored as they do not put in place special mechanisms to treat negative pairs differently according to their specific differences and similarities. In this paper, we present Extended Momentum Contrast (XMoCo), a self-supervised representation learning method founded upon the legacy of the momentum-encoder unit proposed in the MoCo family configurations. To this end, we introduce a cross consistency regularization loss, with which we extend the transformation consistency to dissimilar images (negative pairs). Under the cross consistency regularization rule, we argue that semantic representations associated with any pair of images (positive or negative) should preserve their cross-similarity under pretext transformations. Moreover, we further regularize the training loss by enforcing a uniform distribution of similarity over the negative pairs across a batch. The proposed regularization can easily be added to existing self-supervised learning algorithms in a plug-and-play fashion. Empirically, we report a competitive performance on the standard Imagenet-1K linear head classification benchmark. In addition, by transferring the learned representations to common downstream tasks, we show that using XMoCo with the prevalently utilized augmentations can lead to improvements in the performance of such tasks. We hope the findings of this paper serve as a motivation for researchers to take into consideration the important interplay among the negative examples in self-supervised learning. [ABSTRACT FROM AUTHOR]

Due to high biological adaptability and flexibility, pneumatic artificial muscle (PAM) systems are widely employed in exoskeleton robots to accomplish rehabilitation training with repetitive motions. However, some intrinsic characteristics of PAMs and inevitable practical factors, e.g., high nonlinearity, hysteresis, uncertain dynamics, and limited working space, may badly degrade tracking performance and safety. Hence, this paper designs a new learning-based motion controller for PAMs, to simultaneously compensate for model uncertainties, eliminate tracking errors, and satisfy preset motion constraints. Particularly, when PAMs suffer from periodically non-parametric uncertainties, the elaborately designed continuous update algorithm can repetitively learn them online to enhance tracking accuracy, without employing upper/lower bounds of unknown parts for controller design and gain selections. Meanwhile, some non-periodic uncertainties are handled by a robust term, whose value is only related to the initial states of PAMs, instead of exact upper bounds of unknown dynamics. From safety concerns, we introduce error-related saturation terms to limit initial amplitudes of control inputs within saturation constraints and avoid overlarge errors inducing overlarge acceleration. Meanwhile, the constraint-related auxiliary term is utilized to keep tracking errors within allowable ranges. To the best of our knowledge, this paper presents the first learning-based error-constrained controller for uncertain PAM-actuated exoskeleton robots, to realize high-precision tracking control and improve safety without additional gain conditions. Moreover, the asymptotic convergence of tracking errors is strictly proven by Lyapunov-based stability analysis. Finally, based on a self-built exoskeleton robot, the effectiveness of the proposed controller is verified by hardware experiments. Note to Practitioners—This work is motivated by the practical requirements of exoskeleton robots in rehabilitation training and exploration fields. Currently, PAM systems, as a kind of new flexible actuator equipment, are playing increasingly important roles in the development of exoskeleton robot control. However, uncertain (or time-varying) parameters/structures and highly nonlinear dynamics, such as creep and hysteresis, may badly increase the control difficulty of PAMs. Moreover, higher and higher tracking accuracy and safety requirements also induce urgently solved problems to practical PAM-actuated exoskeleton robots, e.g., smooth start, motion constraints, and rapid error elimination. To this end, this paper proposes a new learning-based adaptive controller, which realizes accurate tracking control for PAM-actuated exoskeleton robots by utilizing an elaborately designed repetitive learning algorithm and a robust term to handle periodic and non-periodic uncertainties, respectively. More importantly, the proposed controller simultaneously enhances transient performance of PAMs, including gradually improved tracking accuracy, effective constraints for startup acceleration and tracking errors. Additionally, it is not required to consider the upper bounds of unknown dynamics and additional gain selection conditions, which is theoretically and practically important for PAM systems. Some hardware experiments further verify the effectiveness and robustness of the suggested controller. In our future work, we intend to design more effective methods for PAMs with unmeasurable states and time-delay. [ABSTRACT FROM AUTHOR]

With the abundance of data, machine learning applications engaged increased attention in the last decade. An attractive approach to robustify the statistical analysis is to preprocess the data through clustering. This paper develops a low-complexity Riemannian optimization framework for solving optimization problems on the set of positive semidefinite stochastic matrices. The low-complexity feature of the proposed algorithms stems from the factorization of the optimization variable $\mathbf {X}=\mathbf {Y}\mathbf {Y}^{\mathrm{T}}$ X = Y Y T and deriving conditions on the number of columns of $\mathbf {Y}$ Y under which the factorization yields a satisfactory solution. The paper further investigates the embedded and quotient geometries of the resulting Riemannian manifolds. In particular, the paper explicitly derives the tangent space, Riemannian gradients and Hessians, and a retraction operator allowing the design of efficient first and second-order optimization methods for the graph-based clustering applications of interest. The numerical results reveal that the resulting algorithms present a clear complexity advantage as compared with state-of-the-art euclidean and Riemannian approaches for graph clustering applications. [ABSTRACT FROM AUTHOR]

This paper presents the overview and rationale behind the Decoder-Side Depth Estimation (DSDE) mode of the MPEG Immersive Video (MIV) standard, using the Geometry Absent profile, for efficient compression of immersive multiview video. A MIV bitstream generated by an encoder operating in the DSDE mode does not include depth maps. It only contains the information required to reconstruct them in the client or in the cloud: decoded views and metadata. The paper explains the technical details and techniques supported by this novel MIV DSDE mode. The description additionally includes the specification on Geometry Assistance Supplemental Enhancement Information which helps to reduce the complexity of depth estimation, when performed in the cloud or at the decoder side. The depth estimation in MIV is a non-normative part of the decoding process, therefore, any method can be used to compute the depth maps. This paper lists a set of requirements for depth estimation, induced by the specific characteristics of the DSDE. The depth estimation reference software, continuously and collaboratively developed with MIV to meet these requirements, is presented in this paper. Several original experimental results are presented. The efficiency of the DSDE is compared to two MIV profiles. The combined non-transmission of depth maps and efficient coding of textures enabled by the DSDE leads to efficient compression and rendering quality improvement compared to the usual encoder-side depth estimation. Moreover, results of the first evaluation of state-of-the-art multiview depth estimators in the DSDE context, including machine learning techniques, are presented. [ABSTRACT FROM AUTHOR]

This paper attacks the challenging problem of video retrieval by text. In such a retrieval paradigm, an end user searches for unlabeled videos by ad-hoc queries described exclusively in the form of a natural-language sentence, with no visual example provided. Given videos as sequences of frames and queries as sequences of words, an effective sequence-to-sequence cross-modal matching is crucial. To that end, the two modalities need to be first encoded into real-valued vectors and then projected into a common space. In this paper we achieve this by proposing a dual deep encoding network that encodes videos and queries into powerful dense representations of their own. Our novelty is two-fold. First, different from prior art that resorts to a specific single-level encoder, the proposed network performs multi-level encoding that represents the rich content of both modalities in a coarse-to-fine fashion. Second, different from a conventional common space learning algorithm which is either concept based or latent space based, we introduce hybrid space learning which combines the high performance of the latent space and the good interpretability of the concept space. Dual encoding is conceptually simple, practically effective and end-to-end trained with hybrid space learning. Extensive experiments on four challenging video datasets show the viability of the new method. Code and data are available at https://github.com/danieljf24/hybrid_space. [ABSTRACT FROM AUTHOR]

To solve the information explosion problem and enhance user experience in various online applications, recommender systems have been developed to model users’ preferences. Although numerous efforts have been made toward more personalized recommendations, recommender systems still suffer from several challenges, such as data sparsity and cold-start problems. In recent years, generating recommendations with the knowledge graph as side information has attracted considerable interest. Such an approach can not only alleviate the above mentioned issues for a more accurate recommendation, but also provide explanations for recommended items. In this paper, we conduct a systematical survey of knowledge graph-based recommender systems. We collect recently published papers in this field, and group them into three categories, i.e., embedding-based methods, connection-based methods, and propagation-based methods. Also, we further subdivide each category according to the characteristics of these approaches. Moreover, we investigate the proposed algorithms by focusing on how the papers utilize the knowledge graph for accurate and explainable recommendation. Finally, we propose several potential research directions in this field. [ABSTRACT FROM AUTHOR]

This paper proposes a Principal Component Analysis (PCA)-based method to construct composite indices to rapidly assess power system transient stability. By assigning appropriate weights to a few selected sub-indicators, a composite index can be constructed to guarantee a better coverage of worst-case stability scenarios of the power system. This paper provides an automatic method to construct a composite index with four sub-indicators calculated by the integral of acceleration, speed deviation and rotor angle of generators over a certain integration period using PCA. The effectiveness of the sub-indicators is validated first, followed by a comprehensive example of process of composite index construction, and the assessment of the influence of the effect of integration period on the performance of the composite index. High accuracy of classification has been demonstrated using composite index with appropriate settings of the integration periods of the sub-indicators. [ABSTRACT FROM AUTHOR]

Coded caching is a promising technique to smooth out network traffic by storing part of the library content at the users’ local caches. The seminal work on coded caching for single file retrieval by Maddah-Ali and Niesen (MAN) showed the existence of a global caching gain that scales with the total memory in the system, in addition to the known local caching gain in uncoded systems. This paper formulates a novel cache-aided matrix multiplication retrieval problem, relevant for data analytics and machine learning applications. In the considered problem, each cache-aided user requests the product of two matrices from the library. A structure-agnostic solution is to treat each possible matrix product as an independent file and use the MAN coded caching scheme for single file retrieval. This paper proposes two structure-aware schemes, which partition each matrix in the library by either rows or columns and let a subset of users cache some sub-matrices, that improve on the structure-agnostic scheme. For the case where the library matrices are “fat” matrices, the structure-aware row-partition scheme is shown to be order optimal under some constraint. [ABSTRACT FROM AUTHOR]

Objective: Long-term monitoring of epilepsy patients outside of hospital settings is impractical due to the complexity and costs associated with electroencephalogram (EEG) systems. Alternative sensing modalities that can acquire, and automatically interpret signals through easy-to-use wearable devices, are needed to help with at-home management of the disease. In this paper, a novel machine learning algorithm is presented for detecting epileptic seizures using acoustic physiological signals acquired from the neck using a wearable device. Methods: Acoustic signals from an existing database, were processed, to extract their Mel-frequency Cepstral Coefficients (MFCCs) which were used to train RUSBoost classifiers to identify ictal and non-ictal acoustic segments. A postprocessing stage was then applied to the segment classification results to identify seizures episodes. Results: Tested on 667 hours of acoustic data acquired from 15 patients with at least one seizure, the algorithm achieved a detection sensitivity of 88.1% (95% CI: 79%–97%) from a total of 36 seizures, out of which 24 had no motor manifestations, with a FPR of 0.83/h, and a median detection latency of −42 s. Conclusion: The results demonstrated for the first time the ability to identify seizures using acoustic internal body signals acquired on the neck. Significance: The results of this paper validate the feasibility of using internal physiological sounds for seizure detection, which could potentially be of use for the development of novel, wearable, very simple to use, long term monitoring, or seizure detection systems; circumventing the practical limitations of EEG monitoring outside hospital settings, or systems based on sensing modalities that work on convulsive seizures only. [ABSTRACT FROM AUTHOR]

The wide deployment of advanced robots with industrial IoT (IIoT) technologies in smart factories generates a large volume of data during production and a wide variety of data processing workloads are launched to maintain productivity and safety of smart manufacture. The emerging fog computing paradigm offers a promising solution to enhancing data processing performance in a smart factory environment while on the other hand brings in new challenges to resource management, which call for a more effective approach for recommending resource configurations to heterogeneous workloads. In this paper, we propose an Optimized Recommendations of Heterogeneous Resource Configurations (ORHRC) model that employs machine learning techniques to provide resource configuration recommendations for the heterogeneous workloads in a fog computing-based smart factory environment. ORHRC learns a recommendation model by leveraging the operating characteristics and execution time of workloads on fog servers with different configurations. We also design a decision model in ORHRC to further improve prediction accuracy and reduce operational overheads. Experiment results show that ORHRC outperforms the state of art configuration recommendation methods in terms of average prediction accuracy. Note to Practitioners—The various data processing workloads in a smart factory environment need to be processed by the computational resources with optimal configurations for meeting their performance requirements. In this paper, we employ machine learning technologies for enabling automatic recommendation of resource configurations to heterogeneous workloads. Specifically, we develop an Optimized Recommendations of Heterogeneous Resource Configurations (ORHRC) model that can identify the optimal resource configurations for various workloads. We also conducted extensive experiments that verify the effectiveness of the proposed ORHRC model. [ABSTRACT FROM AUTHOR]

In manufacturing industries, it is of fundamental importance to detect anomalies in production in order to meet the required quality goals and to limit the number of defective products that are accidentally delivered to the customers. Nevertheless, monitoring systems currently employed in production are typically very simple and rely on a set of univariate control charts that fail to capture the multivariate and complex nature of real-world industrial systems. In such context, Machine Learning (ML)-based approaches for Anomaly Detection (AD) have proven to be extremely effective in increasing anomalies detectability and, in general, in enhancing monitoring procedures. However, industrial data are typically very complex and not suitable to be fed directly to classical ML-based AD tools making feature extraction procedures a necessary step that unfortunately may lead to information loss and low scalability. Deep Learning, has proven very effective at learning useful representations of complex data in an automatic way. In this paper, we propose an AD pipeline that makes use of convolutional autoencoders to extract useful features from two-dimensional, non-image, data. We test our approach on real world Optical Emission Spectroscopy data that are typical of semiconductor manufacturing and we achieve improved performance over classical monitoring methods. Note to Practitioners—Advanced monitoring is one of the most important task in the context of Industry 4.0. Some of the main issues in developing Machine Learning-based solutions in industrial environment are: (i) the lack of reliable tagged data; (ii) the complexity of data structures present in real-world scenarios. In this paper we investigate unsupervised anomaly detection for 2-dimensional data in manufacturing environment: we provide an approach that exploit Deep Learning-based architecture for handling the data at hand. We show the effectiveness of the proposed approach in a real world case study related to optical emission spectroscopy data in semiconductor manufacturing process providing satisfactory classification accuracy. [ABSTRACT FROM AUTHOR]

In this paper, we study an automatic hypothesis generation (HG) problem, which refers to the discovery of meaningful implicit connections between scientific terms, including but not limited to diseases, chemicals, drugs, and genes extracted from databases of biomedical publications. Most prior studies of this problem focused on the use of static information of terms and largely ignored the temporal dynamics of scientific term relations. Even when the dynamics were considered in a few recent studies, they learned the representations for the scientific terms, rather than focusing on the term-pair relations. Since the HG problem is to predict term-pair connections, it is not enough to know with whom the terms are connected, it is more important to know how the connections have been formed (in a dynamic process). We formulate this HG problem as a future connectivity prediction in a dynamic attributed graph. The key is to capture the temporal evolution of node-pair (term-pair) relations. We propose an inductive edge (node-pair) embedding method named T-PAIR, utilizing both the graphical structure and node attribute to encode the temporal node-pair relationship. We demonstrate the efficiency of the proposed model on three real-world datasets, which are three graphs constructed from Pubmed papers published until 2019 in Neurology, Immunotherapy, and Virology, respectively. Evaluations were conducted on predicting future term-pair relations between millions of seen terms (in the transductive setting), as well as on the relations involving unseen terms (in the inductive setting). Experiment results and case study analyses show the effectiveness of the proposed model. [ABSTRACT FROM AUTHOR]

There is an opportunity in modern power systems to explore the demand flexibility by incentivizing consumers with dynamic prices. In this paper, we quantify demand flexibility using an efficient tool called time-varying elasticity, whose value may change depending on the prices and decision dynamics. This tool is particularly useful for evaluating the demand response potential and system reliability. Recent empirical evidences have highlighted some abnormal features when studying demand flexibility, such as delayed responses and vanishing elasticities after price spikes. Existing methods fail to capture these complicated features because they heavily rely on some predefined (often over-simplified) regression expressions. Instead, this paper proposes a model-free methodology to automatically and accurately derive the optimal estimation pattern. We further develop a two-stage estimation process with Siamese long short-term memory (LSTM) networks. Here, a LSTM network encodes the price response, while the other network estimates the time-varying elasticities. In the case study, the proposed framework and models are validated to achieve higher overall estimation accuracy and better description for various abnormal features when compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Spatial failure patterns in wafer defect maps are very useful for root cause analysis, which is essential for yield optimization in the manufacturing process. Especially, the scratch defect type is the most challenging pattern to recognize because the position, shape, size and curvature vary widely from one scratch to another. Discontinuity points within scratches also contribute to the low recognition rate, and such points are often hidden defective dies that become reliability threat. Previous studies on defect pattern identification show that the recognition rate for scratches is among the lowest in all patterns even if the overall accuracy is high. In this paper, we propose a novel scratch pattern recognition method. In this paper, image processing techniques are applied to recognize scratch patterns in wafer maps. The Hough transform is first employed to identify line segments, from which scratches can be reconstructed. In contrast to previous machine-learning based methods, there is no need to train a complicated prediction model so that the computation time is small. The method is validated by using wafer maps from six real products and public data. Experimental results show that the proposed method achieves high recall, precision and accuracy in all cases. [ABSTRACT FROM AUTHOR]

Estimating depth from RGB images is a long-standing ill-posed problem, which has been explored for decades by the computer vision, graphics, and machine learning communities. Among the existing techniques, stereo matching remains one of the most widely used in the literature due to its strong connection to the human binocular system. Traditionally, stereo-based depth estimation has been addressed through matching hand-crafted features across multiple images. Despite the extensive amount of research, these traditional techniques still suffer in the presence of highly textured areas, large uniform regions, and occlusions. Motivated by their growing success in solving various 2D and 3D vision problems, deep learning for stereo-based depth estimation has attracted a growing interest from the community, with more than 150 papers published in this area between 2014 and 2019. This new generation of methods has demonstrated a significant leap in performance, enabling applications such as autonomous driving and augmented reality. In this paper, we provide a comprehensive survey of this new and continuously growing field of research, summarize the most commonly used pipelines, and discuss their benefits and limitations. In retrospect of what has been achieved so far, we also conjecture what the future may hold for deep learning-based stereo for depth estimation research. [ABSTRACT FROM AUTHOR]

With the rapid development of intelligent transportation, automated traffic surveillance is considered as an important component. In the field of traffic surveillance, it is particularly important to achieve robust and real-time tracking of vehicles in complex scenes. In this paper, a robust real-time vehicle tracking model named NoisyOTNet is proposed, which formulates tracking as reinforcement learning with parameter space noise. In this formulation, the exploration ability of the model is enhanced to improve the robustness of tracking. Specifically, we develop a new implementation for noisy network based on deep deterministic policy gradients (DDPGs) with parameter noise, which can better cope with the tracking task and directly predict the tracking result. To improve the tracking accuracy in complex conditions, e.g. fast motion and large deformation, this paper presents an adaptive update strategy that can exploit the vehicle spatial-temporal information based on Upper Confidence Bound (UCB) algorithm by exploiting. Moreover, as for the recovery of the lost target, a relocation algorithm based on incremental learning is developed. The results of extensive experiments demonstrate that the proposed NoisyOTNet can effectively track vehicles in complex scenes and achieve competitive performance compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Brain surface analysis is essential to neuroscience, however, the complex geometry of the brain cortex hinders computational methods for this task. The difficulty arises from a discrepancy between 3D imaging data, which is represented in Euclidean space, and the non-Euclidean geometry of the highly-convoluted brain surface. Recent advances in machine learning have enabled the use of neural networks for non-Euclidean spaces. These facilitate the learning of surface data, yet pooling strategies often remain constrained to a single fixed-graph. This paper proposes a new learnable graph pooling method for processing multiple surface-valued data to output subject-based information. The proposed method innovates by learning an intrinsic aggregation of graph nodes based on graph spectral embedding. We illustrate the advantages of our approach with in-depth experiments on two large-scale benchmark datasets. The ablation study in the paper illustrates the impact of various factors affecting our learnable pooling method. The flexibility of the pooling strategy is evaluated on four different prediction tasks, namely, subject-sex classification, regression of cortical region sizes, classification of Alzheimer’s disease stages, and brain age regression. Our experiments demonstrate the superiority of our learnable pooling approach compared to other pooling techniques for graph convolutional networks, with results improving the state-of-the-art in brain surface analysis. [ABSTRACT FROM AUTHOR]

This paper presents an asset health index (HI) prediction methodology for high voltage transmission overhead lines (OHLs) using supervised machine learning and structured, unambiguous visual inspections. We propose a framework for asset HI predictions to determine the technical condition of individual OHL towers to improve grid reliability in a cost-effective manner. The paper focuses on asset HI prediction and the selection of the most parsimonious model. Based on the technical specifications and HI data, our methodology allows for the prediction of a HI for OHLs without HI data, and models asset aging behaviour. Technical specifications and the HI as defined in this paper are taken from the Estonian TSO periodical visual inspections implemented in 2018. The case study successfully demonstrates that the proposed methodology can predict tower HI values for a single OHL with nearly 80 percent accuracy without the need for additional measurements. [ABSTRACT FROM AUTHOR]

As the semiconductor manufacturing process is moving towards the 3 nm node, there is a crucial need to reduce the edge placement error (EPE) to ensure proper functioning of the integrated circuit (IC) devices. EPE is the most important metric that quantify the fidelity of fabricated patterns in multi-patterning processes, and it is the combination of overlay errors and critical dimension (CD) errors. Recent advances in machine learning have enabled many new possibilities to improve the performance and efficiency of EPE optimization techniques. In this paper, we conducted a survey of recent research work that applied machine learning/ deep learning techniques for the purposes of enhancing virtual overlay metrology, reducing overlay error, and improving mask optimization methods for EPE reduction. Thorough discussions about the objectives, datasets, input features, models, key findings, and limitations are provided. In general, the results of the review work show a great potential of machine learning techniques in aiding the improvement of EPE in the field of semiconductor manufacturing. [ABSTRACT FROM AUTHOR]

This paper addresses the issue of detecting the use of information hiding at anti-copying 2D barcodes. Prior hidden information detection schemes have their roots in either heuristic-based or Machine Learning (ML). However, prior heuristics-based schemes lack a rigorous theoretical analysis. Prior ML-based information schemes lack robustness because a printed 2D barcode is very much environmentally dependent. Thus, an information hiding detection scheme trained in one environment often does not work well in another environment. In this paper, we propose two hidden information detection schemes for existing anti-copying 2D barcodes. The first scheme directly uses the pixel distance to detect the use of an information hiding scheme in a 2D barcode, referred to as the Pixel Distance Based Detection (PDBD) scheme. The second scheme first calculates the variance of raw signal and the covariance between the recovered and raw signals, and then based on the variance results, detects the use of information hiding scheme in a 2D barcode, referred to as the Pixel Variance Based Detection (PVBD) scheme. Moreover, we design advanced Illegitimately-Copying (IC) attacks to evaluate the security of two existing anti-copying 2D barcodes. We conduct extensive performance comparisons among the proposed schemes and prior schemes under different capturing devices, e.g., different scanners or camera phones. Our experimental results show that the PVBD scheme can correctly detect the existence of hidden information at both the 2LQR code and the LCAC 2D barcode. Moreover, the successful attacking probability of the proposed IC attacks achieves 0.6538 for the 2LQR code and 1 for the LCAC 2D barcode. [ABSTRACT FROM AUTHOR]

In the field of smart agriculture health monitoring of livestock is an important field of research. Maintaining the good health of cows is very much essential for the steady growth of milk production. Unfortunately, in a large dairy cow farm, day-to-day monitoring of the health status of individual cows is a complex and time-consuming activity. This paper proposed LiveCare, an IoT-based framework that automatically monitors the health of cows in a large cow farm. It tracks the cow’s behavioral changes on a daily basis. This paper also proposed the Cow Disease Prediction (CDP) algorithm, which is an unsupervised multi-class classifier that serves as the LiveCare framework’s central component. The CDP algorithm can predict several cow diseases by analyzing the cows’ behavioral changes. In this framework, we have also tabulated a few common cow diseases, their measurable symptoms, and the various sensors used to record them. We compared the efficiency of the proposed CDP algorithm to that of other machine learning algorithms. [ABSTRACT FROM AUTHOR]

Non-volatile memory express (NVMe) solid-state drives (SSDs) have been widely adopted in multi-tenant cloud computing environments or multi-programming systems. The on-board DRAM cache inside NVMe SSDs can efficiently reduce the disk accesses and extend the lifetime of SSDs. Current SSD cache management research either improves cache hit ratio while ignoring fairness, or improves fairness while sacrificing overall performance. In this paper, we present MLCache, a space-efficient shared cache management scheme for NVMe SSDs. By learning the impact of reuse distance on cache allocation, a workload-generic neural network model is built. At runtime, MLCache continuously monitors the reuse distance distribution for the neural network module to obtain space-efficient allocation decisions. MLCache also proposes an efficient parallel writing back strategy based on hit ratio and response time, to improve fairness. Experimental results show MLCache improves the write hit ratio when compared to baseline, and MLCache strongly safeguards the fairness of SSDs with parallel write-back and maintains a low level of degradation. [ABSTRACT FROM AUTHOR]

Dependency-aware jobs, such as the big data analytic workflows, are commonly executed on the cloud. They are compiled to directed acyclic graphs, with tasks linked in regarding the dependency. The cloud scheduler, which maintains a large number of resources, is responsible to execute tasks in parallel. To resolve the complex dependencies, Deep Reinforcement Learning (DRL) based schedulers are widely applied. However, we find that the DRL-based schedulers are vulnerable to the perturbations in the input jobs and may generate falsified decisions to benefit a particular job while delaying the others. By perturbation, we mean a slight adjustment to the job's node features or dependencies, while not changing its functionality. In this paper, we first explore the vulnerability of DRL-based schedulers to job perturbations without accessing the information of the DRL models used in the scheduler. We devise the black-box perturbation system, in which, a proxy model is trained to mimic the DRL-based scheduling policy. We show that the high-faith proxy model can help to craft effective perturbations. The DRL-based schedulers can be as high as 60% likely to be badly affected by the perturbations. Then, we investigate the solution to improve the robustness of DRL-based schedulers to such perturbations. We propose an adversarial training framework to force the neural model to adapt to the perturbation patterns during training so as to eliminate the potential damage during applications. Experiments show that the adversarial-trained scheduler is more robust, reducing the chance of being affected to 3-fold less and the potential bad effects halved. [ABSTRACT FROM AUTHOR]

Adversarial attacks on machine learning-based classifiers, along with defense mechanisms, have been widely studied in the context of single-label classification problems. In this paper, we shift the attention to multi-label classification, where the availability of domain knowledge on the relationships among the considered classes may offer a natural way to spot incoherent predictions, i.e., predictions associated to adversarial examples lying outside of the training data distribution. We explore this intuition in a framework in which first-order logic knowledge is converted into constraints and injected into a semi-supervised learning problem. Within this setting, the constrained classifier learns to fulfill the domain knowledge over the marginal distribution, and can naturally reject samples with incoherent predictions. Even though our method does not exploit any knowledge of attacks during training, our experimental analysis surprisingly unveils that domain-knowledge constraints can help detect adversarial examples effectively, especially if such constraints are not known to the attacker. We show how to implement an adaptive attack exploiting knowledge of the constraints and, in a specifically-designed setting, we provide experimental comparisons with popular state-of-the-art attacks. We believe that our approach may provide a significant step towards designing more robust multi-label classifiers. [ABSTRACT FROM AUTHOR]

In multi-label classification, the strategy of label-specific features has been shown to be effective to learn from multi-label examples by accounting for the distinct discriminative properties of each class label. However, most existing approaches exploit the semantic relations among labels as immutable prior knowledge, which may not be appropriate to constrain the learning process of label-specific features. In this paper, we propose to learn label semantics and label-specific features in a collaborative way. Accordingly, a deep neural network (DNN) based approach named Clif, i.e., CollaborativeLearning of label semantIcs and deep label-specificFeatures for multi-label classification, is proposed. By integrating a graph autoencoder for encoding semantic relations in the label space and a tailored feature-disentangling module for extracting label-specific features, Clif is able to employ the learned label semantics to guide mining label-specific features and propagate label-specific discriminative properties to the learning process of the label semantics. In such a way, the learning of label semantics and label-specific features interact and facilitate with each other so that label semantics can provide more accurate guidance to label-specific feature learning. Comprehensive experiments on 14 benchmark data sets show that our approach outperforms other well-established multi-label classification algorithms. [ABSTRACT FROM AUTHOR]

This paper aims at proposing an unsupervised hierarchical nonparametric Bayesian framework for modeling axial data (i.e., observations are axes of direction) that can be partitioned into multiple groups, where each observation within a group is sampled from a mixture of Watson distributions with an infinite number of components that are allowed to be shared across different groups. First, we propose a hierarchical nonparametric Bayesian model for modeling grouped axial data based on the hierarchical Pitman-Yor process mixture model of Watson distributions. Then, we demonstrate that by setting the discount parameters of the proposed model to 0, another hierarchical nonparametric Bayesian model based on hierarchical Dirichlet process can be derived for modeling axial data. To learn the proposed models, we systematically develop a closed-form optimization algorithm based on the collapsed variational Bayes (CVB) inference. Furthermore, to ensure the convergence of the proposed learning algorithm, an annealing mechanism is introduced to the framework of CVB inference, leading to an averaged collapsed variational Bayes inference strategy. The merits of the proposed models for modeling grouped axial data are demonstrated through experiments on both synthetic data and real-world applications involving gene expression data clustering and depth image analysis. [ABSTRACT FROM AUTHOR]

Available data in machine learning applications is becoming increasingly complex, due to higher dimensionality and difficult classes. There exists a wide variety of approaches to measuring complexity of labeled data, according to class overlap, separability or boundary shapes, as well as group morphology. Many techniques can transform the data in order to find better features, but few focus on specifically reducing data complexity. Most data transformation methods mainly treat the dimensionality aspect, leaving aside the available information within class labels which can be useful when classes are somehow complex. This paper proposes an autoencoder-based approach to complexity reduction, using class labels in order to inform the loss function about the adequacy of the generated variables. This leads to three different new feature learners, Scorer, Skaler and Slicer. They are based on Fisher’s discriminant ratio, the Kullback-Leibler divergence and least-squares support vector machines, respectively. They can be applied as a preprocessing stage for a binary classification problem. A thorough experimentation across a collection of 27 datasets and a range of complexity and classification metrics shows that class-informed autoencoders perform better than 4 other popular unsupervised feature extraction techniques, especially when the final objective is using the data for a classification task. [ABSTRACT FROM AUTHOR]