Search

As a follow-up to the first IEEE Transactions on Medical Imaging (TMI) special issue on the theme of deep tomographic reconstruction, the second special issue is assembled to reflect the status and momentum of this rapidly emerging field. In this editorial, we provide a brief background illustrating the motivation for the development of network-based, data-driven, and learning-oriented reconstruction methods, summarize the included papers, and report our verification of the shared deep learning codes. Finally, we discuss several important research topics to facilitate further investigation and collaboration. [ABSTRACT FROM AUTHOR]

This paper discusses the optimization of an advanced process monitoring scheme with interdicting fault detection and classification (FDC) capabilities that improved the control over the process development of Tantalum Nitride thin film resistors (TaN TFR). Its implementation in a high-volume manufacturing environment resulted in a reduction of misprocessed wafers, shorter equipment downtime, higher throughput, enhanced process visibility, and yield improvement. Along with optimizing FDC capabilities, implementing a short loop sampling plan reduced the turnaround time for early electrical characterization by a factor of four; this allowed for timely inline adjustments within the fabrication process to tighten the statistical process control (SPC) over the distribution of the process control monitor (PCM) TaN resistor value and improve thin film uniformity. [ABSTRACT FROM AUTHOR]

Semiconductor manufacturing has been extensively exploiting machine-learning (ML) to process equipment sensory data (ESD) for near-real time anomaly detection (AD). ESD characteristics are highly diversified and data lengths vary among processing steps and cycles. Cleansing ESD with minimum distortion (CMD) to fit the fixed-length input requirement by ML-based AD is critical to AD effectiveness and is challenging. This paper presents a novel CMD method of four innovations: i) statistical mode-based equalization of step data lengths for the least number of step data length changes, ii) importance indicator value (IIV) of a data sample based on its relative difference with the subsequent sample, and iii) step data segmentation into groups based on samples of significant IIVs and the least-entropy-group-to-cleanse-first rule, and iv) cleansing the least IIV sample(s) in the selected group for step data length equalization. CMD application to ESD demonstrates its characteristics preservation property. Simulation experiments are on an integration of data cleansing with an unsupervised ML-based AD system, STALAD. Comparisons with two benchmark methods over AD scenarios of small-scale drifts and shifts show that CMD not only is superior in facilitating accurate detection by STALAD but also helps detect anomaly much earlier than using the two benchmarks. [ABSTRACT FROM AUTHOR]

Thanks to the many submissions we have received, we can present this second part of our special issue on "AI-eXplained." In here, we continue our mission to demystify the intricate world of artificial intelligence and make it accessible to a broader audience. As AI continues to evolve and integrate into various aspects of our lives, it becomes increasingly important to bridge the gap between experts and those eager to understand the inner workings of AI systems. Our goal remains unchanged—to present AI concepts in a comprehensible, engaging, and interactive manner, empowering our readers to explore and grasp the enchanting world of AI. [ABSTRACT FROM AUTHOR]

Since its beginning in 1992 in Japan, International Symposium on Semiconductor Manufacturing (ISSM) has provided unique opportunities to share the best practices of semiconductor manufacturing technologies for professionals. At the symposiums, semiconductor manufacturing professionals discussed the technologies developed to meet the worldwide requirements for advanced manufacturing. It is becoming crucial to re-examine semiconductor manufacturing in terms of fundamental principles to improve the performance of semiconductor devices. Moreover, utilizing artificial intelligence and machine learning technologies to improve semiconductor manufacturing have become a new challenge. These manufacturing technology challenges are showing the need for drastic revolutionary concept and stronger collaborative efforts to find solutions to the precompetitive challenges. [ABSTRACT FROM AUTHOR]

In active and assisted living environments, a major service that can be provided is the automated assessment of elderly people’s well-being. Therefore, activity recognition is required to detect what types of help disabled persons need to support them in their daily life activities. Unfortunately, it is still a difficult task to estimate the size of the required window for online sensor data streams to recognize a specific activity, especially when new sensor events are recorded. This paper proposes a windowing algorithm, which presents promising results to recognize complex human activities for multi-resident homes. The approach is based on the analysis of the sensor data to identify the best fitting sensors that should be considered in a specified window. Moreover, the second part of this paper proposes a set of different statistical spatio-temporal features to recognize human activities. In order to check the overall performance, this approach is tested using the CASAS data set and artificially generated laboratory data using our HBMS simulator. The results show high performance based on different evaluation metrics compared to other approaches. We believe that the proposed windowing approach provides a good approximation of the required window size in order to robustly detect human activities in comparison to other windowing approaches. [ABSTRACT FROM AUTHOR]

This paper is concerned with scheduling of a set of ${n}$ single-operation tasks/orders on a set of $m$ unrelated parallel machines where subcontracting is allowed. When a machine/subcontractor is chosen to do a set of orders/tasks, it incurs a one-time fixed cost. When a job/order is performed by a machine/subcontractor, there is a cost that depends on the machine/subcontractor. The objective is to choose a subset of ${k}$ machines and/or subcontractors from the set of all ${m}$ available machines and/or subcontractors to perform all jobs to minimize the sum of total workload costs and total fixed costs. We discuss the complexity of the problem, and prove NP-hardness of the problem. Simplified mathematical development is provided that allows efficient implementation of two-exchange algorithms. An efficient tabu search heuristic with a diversification generation component is developed. An extensive computational experiment of the heuristic for large-scale problems with comparison to the results from CPLEX software is presented. We also solved 40 benchmark ${k}$ -median problems available on the Internet that have been used by many researchers. Note to Practitioners—To be competitive in the global market, companies must be prudent in the use of their resources. This paper considers a parallel scheduling environment where choosing in-house machines and/or subcontractors as resources to perform the orders/jobs is the main objective. Processing time (or cost) of a job to be performed by different machines or subcontractors can be different. Furthermore, if a machine or a subcontractor is chosen to perform a set of orders, there is a one-time fixed cost (in the case of subcontractor it can be considered transportation cost) that depends on the machine or subcontractor. The scheduling criteria are to choose a subset of $k$ machines and/or subcontractors to do all orders/jobs while minimizing the sum of the total workload and total fixed costs. The complexity of the problem is discussed and shown to be NP-hard. An efficient tabu search that solves large-scale problems in fraction of a second of CPU time is presented and an extensive computational experiment is provided. [ABSTRACT FROM AUTHOR]

Evolving fuzzy systems (EFSs) are now well developed and widely used, thanks to their ability to self-adapt both their structures and parameters online. Since the concept was first introduced two decades ago, many different types of EFSs have been successfully implemented. However, there are only very few works considering the stability of the EFSs, and these studies were limited to certain types of membership functions with specifically predefined parameters, which largely increases the complexity of the learning process. At the same time, stability analysis is of paramount importance for control applications and provides the theoretical guarantees for the convergence of the learning algorithms. In this paper, we introduce the stability proof of a class of EFSs based on data clouds, which are grounded at the AnYa type fuzzy systems and the recently introduced empirical data analytics (EDA) methodological framework. By employing data clouds, the class of EFSs of AnYa type considered in this paper avoids the traditional way of defining membership functions for each input variable in an explicit manner and its learning process is entirely data driven. The stability of the considered EFS of AnYa type is proven through the Lyapunov theory, and the proof of stability shows that the average identification error converges to a small neighborhood of zero. Although, the stability proof presented in this paper is specially elaborated for the considered EFS, it is also applicable to general EFSs. The proposed method is illustrated with Box–Jenkins gas furnace problem, one nonlinear system identification problem, Mackey–Glass time series prediction problem, eight real-world benchmark regression problems as well as a high-frequency trading prediction problem. Compared with other EFSs, the numerical examples show that the considered EFS in this paper provides guaranteed stability as well as a better approximation accuracy. [ABSTRACT FROM AUTHOR]

Electrical capacitance tomography exhibits great potentials in the visualization measurement of industrial processes, and high-precision images are of great significance for the reliability and usefulness of measurement results. In this paper, we propose a deep learning-based inversion method to ameliorate the reconstruction accuracy. With the aid of the deep learning methodology, the prior from the images reconstructed by a certain imaging technique to the true images is abstracted and stored in the deep extreme learning machine. A new cost function is constructed to encapsulate the prior from the proposed deep learning model and the domain expertise about imaging targets, and the split Bregman algorithm and the fast iterative shrinkage thresholding technique are combined into a new numerical method to effectively solve it to get the final reconstruction. The numerical and experimental results validate that the inversion method proposed in this paper reduces the reconstruction artifacts and deformations and leads to the much improvement in the imaging quality. [ABSTRACT FROM AUTHOR]

Robust principal component analysis (RPCA) has drawn significant attention in the last decade due to its success in numerous application domains, ranging from bioinformatics, statistics, and machine learning to image and video processing in computer vision. RPCA and its variants such as sparse PCA and stable PCA can be formulated as optimization problems with exploitable special structures. Many specialized efficient optimization methods have been proposed to solve robust PCA and related problems. In this paper, we review existing optimization methods for solving convex and nonconvex relaxations/variants of RPCA, discuss their advantages and disadvantages, and elaborate on their convergence behaviors. We also provide some insights for possible future research directions including new algorithmic frameworks that might be suitable for implementing on multiprocessor setting to handle large-scale problems. [ABSTRACT FROM AUTHOR]

This papers in this special issue mainly focus on deep learning models for industry informatics, addressing both original algorithmic development and new applications of deep learning. [ABSTRACT FROM AUTHOR]

Although ultra wideband (UWB) positioning is considered as one of the most promising solutions for indoor positioning due to its high positioning accuracy, the accuracy in situations with a large number of users will be reduced because the time between two UWB position updates can be very high. To obtain a position estimate in between these updates, we can combine the UWB positioning with a different technology, e.g., an inertial measurement unit (IMU) that captures data from an accelerometer, gyroscope, and magnetometer. Previous research using the IMU outputs for location-based services employs the periodic behaviour of the accelerometer signal to count steps. However, most of these algorithms require extensive manual tuning of multiple parameters to obtain satisfactory accuracy. To overcome these practical issues, step counting algorithms using machine learning (ML) principles can be developed. In this paper, we consider accelerometer-based step counters using ML. As the performance and complexity of such algorithms depend on the features used in the training and inference phase, proper selection of the employed features is important. Therefore, in this paper, we propose a novel feature selection algorithm, where we first rank the features based on their Bhattacharyya coefficients and then systematically construct a subset of these ranked features. In this paper, we compare three ranking approaches and apply our algorithm to different ML algorithms employing an experimental set. Although the performance of the evaluated combinations slightly varies for different ML algorithms, their performance is comparable to state-of-the-art step counters, without the need to tune parameters manually. [ABSTRACT FROM PUBLISHER]

Using a network of cache enabled small cells, traffic during peak hours can be reduced by proactively fetching the content that is most likely to be requested. In this paper, we aim to explore the impact of proactive caching on an important metric for future generation networks, namely, energy efficiency (EE). We argue that, exploiting the spatial repartitions of users in addition to the correlation in their content popularity profiles, can result in considerable improvement of the achievable EE. In this paper, the optimization of EE is decoupled into two related subproblems. The first one addresses the issue of content popularity modeling. While most existing works assume similar popularity profiles for all users, we consider an alternative framework in which, users are clustered according to their popularity profiles. In order to showcase the utility of the proposed clustering, we use a statistical model selection criterion, namely, Akaike information criterion. Using stochastic geometry, we derive a closed-form expression of the achievable EE and we find the optimal active small cell density vector that maximizes it. The second subproblem investigates the impact of exploiting the spatial repartitions of users. After considering a snapshot of the network, we formulate a combinatorial problem that optimizes content placement in order to minimize the transmission power. Numerical results show that the clustering scheme considerably improves the cache hit probability and consequently the EE, compared with an unclustered approach. Simulations also show that the small base station allocation algorithm improves the energy efficiency and hit probability. [ABSTRACT FROM PUBLISHER]

An autonomous framework to evaluate layout of a piping design in the form of piping and instrumentation diagram (P&ID) according to a set of standards of marine and offshore industry is proposed. The method starts with transforming a P&ID into a vector x in R^d . Transformation is done based on a concept introduced for piping known as Histogram of Connectivity. The proposed descriptor captures two essential properties of P&ID: attributes of each component and connectivity among the components. Next, linear support vector machine (SVM) is used to learn a classifier from existing compliant and noncompliant designs. Subsequently, the linear classifier can be used to check if an unseen design complies with the standards. In addition, to enable follow up on noncompliant design including correction or modification, a method to analyze the reason of noncompliance prediction by the learned SVM model is introduced. The method has demonstrated encouraging performance in two challenging data sets of designs created with advice from experienced engineers in the industry, based on International Convention for the Prevention of Pollution from Ships (MARPOL) and Rules for Classification of Ships of Lloyd’s Register. Note to Practitioners—This paper is motivated by need of marine and offshore industry for automated solution for design appraisal. This paper aims to address this issue by using a machine learning-based approach. Some compliant and noncompliant designs are provided to a developed algorithm for a machine (or computer) to learn. After learning is completed, the machine is able to classify unseen designs as compliant or noncompliant. As highlighted in this paper, the developed method has demonstrated encouraging performance in two case studies, including specific parts in MARPOL and Rules of Lloyd’s Register. For adoption by industry, necessary steps include collecting some designs (compliant and noncompliant) available in an organization and feeding these into the developed method for learning by machine before it can predict. With ability of highlighting possible connections that cause noncompliance, follow up and correction on a noncompliant design is made possible. [ABSTRACT FROM PUBLISHER]

The fourth call for papers of the Series on Machine Learning in Communications and Networks has continued to receive a great number of high-quality papers covering various aspects of intelligent communications, from which we have included 16 original contributions in this issue. In the following, we provide a brief review of these papers according to their topics. [ABSTRACT FROM AUTHOR]

Non-volatile memory express (NVMe) solid-state drives (SSDs) have been widely adopted in multi-tenant cloud computing environments or multi-programming systems. The on-board DRAM cache inside NVMe SSDs can efficiently reduce the disk accesses and extend the lifetime of SSDs. Current SSD cache management research either improves cache hit ratio while ignoring fairness, or improves fairness while sacrificing overall performance. In this paper, we present MLCache, a space-efficient shared cache management scheme for NVMe SSDs. By learning the impact of reuse distance on cache allocation, a workload-generic neural network model is built. At runtime, MLCache continuously monitors the reuse distance distribution for the neural network module to obtain space-efficient allocation decisions. MLCache also proposes an efficient parallel writing back strategy based on hit ratio and response time, to improve fairness. Experimental results show MLCache improves the write hit ratio when compared to baseline, and MLCache strongly safeguards the fairness of SSDs with parallel write-back and maintains a low level of degradation. [ABSTRACT FROM AUTHOR]

Time series classification models have been garnering significant importance in the research community. However, not much research has been done on generating adversarial samples for these models. These adversarial samples can become a security concern. In this paper, we propose utilizing an adversarial transformation network (ATN) on a distilled model to attack various time series classification models. The proposed attack on the classification model utilizes a distilled model as a surrogate that mimics the behavior of the attacked classical time series classification models. Our proposed methodology is applied onto 1-nearest neighbor dynamic time warping (1-NN DTW) and a fully convolutional network (FCN), all of which are trained on 42 University of California Riverside (UCR) datasets. In this paper, we show both models were susceptible to attacks on all 42 datasets. When compared to Fast Gradient Sign Method, the proposed attack generates a larger faction of successful adversarial black-box attacks. A simple defense mechanism is successfully devised to reduce the fraction of successful adversarial samples. Finally, we recommend future researchers that develop time series classification models to incorporating adversarial data samples into their training data sets to improve resilience on adversarial samples. [ABSTRACT FROM AUTHOR]

This paper studies an unmanned aerial vehicle (UAV)-assisted wireless powered IoT network, where a rotary-wing UAV adopts fly-hover-communicate protocol to successively visit IoT devices in demand. During the hovering periods, the UAV works on full-duplex mode to simultaneously collect data from the target device and charge other devices within its coverage. Practical propulsion power consumption model and non-linear energy harvesting model are taken into account. We formulate a multi-objective optimization problem to jointly optimize three objectives: maximization of sum data rate, maximization of total harvested energy and minimization of UAV’s energy consumption over a particular mission period. These three objectives are in conflict with each other partly and weight parameters are given to describe associated importance. Since IoT devices keep gathering information from the physical surrounding environment and their requirements to upload data change dynamically, online path planning of the UAV is required. In this paper, we apply deep reinforcement learning algorithm to achieve online decision. An extended deep deterministic policy gradient (DDPG) algorithm is proposed to learn control policies of UAV over multiple objectives. While training, the agent learns to produce optimal policies under given weights conditions on the basis of achieving timely data collection according to the requirement priority and avoiding devices’ data overflow. The verification results show that the proposed MODDPG (multi-objective DDPG) algorithm achieves joint optimization of three objectives and optimal policies can be adjusted according to weight parameters among optimization objectives. [ABSTRACT FROM AUTHOR]

This paper applies machine learning to optimize the transmission policies of cognitive radio inspired non-orthogonal multiple access (CR-NOMA) networks, where time-division multiple access (TDMA) is used to serve multiple primary users and an energy-constrained secondary user is admitted to the primary users’ time slots via NOMA. During each time slot, the secondary user performs the two tasks: data transmission and energy harvesting based on the signals received from the primary users. The goal of the paper is to maximize the secondary user’s long-term throughput, by optimizing its transmit power and the time-sharing coefficient for its two tasks. The long-term throughput maximization problem is challenging due to the need for making decisions that yield long-term gains but might result in short-term losses. For example, when in a given time slot, a primary user with large channel gains transmits, intuition suggests that the secondary user should not carry out data transmission due to the strong interference from the primary user but perform energy harvesting only, which results in zero data rate for this time slot but yields potential long-term benefits. In this paper, a deep reinforcement learning (DRL) approach is applied to emulate this intuition, where the deep deterministic policy gradient (DDPG) algorithm is employed together with convex optimization. Our simulation results demonstrate that the proposed DRL assisted NOMA transmission scheme can yield significant performance gains over two benchmark schemes. [ABSTRACT FROM AUTHOR]

While early AutoML frameworks focused on optimizing traditional ML pipelines and their hyperparameters, a recent trend in AutoML is to focus on neural architecture search. In this paper, we introduce Auto-PyTorch, which brings together the best of these two worlds by jointly and robustly optimizing the network architecture and the training hyperparameters to enable fully automated deep learning (AutoDL). Auto-PyTorch achieves state-of-the-art performance on several tabular benchmarks by combining multi-fidelity optimization with portfolio construction for warmstarting and ensembling of deep neural networks (DNNs) and common baselines for tabular data. To thoroughly study our assumptions on how to design such an AutoDL system, we additionally introduce a new benchmark on learning curves for DNNs, dubbed LCBench, and run extensive ablation studies of the full Auto-PyTorch on typical AutoML benchmarks, eventually showing that Auto-PyTorch performs better than several state-of-the-art competitors. [ABSTRACT FROM AUTHOR]

This Special Issue contains a collection of outstanding papers covering several recent advances in optical networks for 5G communications and beyond. Papers are organized into four categories: network resource planning; optical access networks; optical fronthaul solutions; and autonomous and data-driven network management. In this introduction, a brief overview of the field is given, followed by a summary of the seventeen papers of this Special Issue, and a discussion of future directions in the field. [ABSTRACT FROM AUTHOR]

Agriculture Cyber-Physical System (A-CPS) is becoming increasingly important in enhancing crop quality and productivity by utilizing minimum cropland. This paper introduces the innovative idea of the Internet-of-Agro-Things (IoAT) with an explanation of the automatic detection of plant disease for the development of ACPS. Majority of the crops were infected by microbial diseases in conventional agriculture. Also, the constantly mutating pathogens cannot be known to the knowledge of the farmer, due to which, there arises a demand to develop a disease prediction system. To prevent this, we use a trained Convolutional Neural Network (CNN) model to perform an analysis of the crop image captured by a health maintenance system. The image capturing along with continuous sensing and intelligent automation is performed by the solar sensor node. The sensor node houses a developed soil moisture sensor which has a high longevity compared to its peers. A real time implementation of the proposed system is demonstrated using a solar sensor node with a camera module, a microcontroller and a smartphone application using which a farmer can monitor the field. The prototype was deployed for three months and has achieved a robust performance by remaining rust-free and sustaining the varied weather conditions. An accuracy of 99.24% is achieved by the proposed plant disease prediction framework. [ABSTRACT FROM AUTHOR]

Reducing the energy consumption of mobile phones is a crucial design goal for cellular modem solutions for LTE and 5G NR standards. Most dynamic power management techniques targeting mobile devices proposed so far, however, are purely reactive in powering down and up system components. Promising approaches extend this, by predicting information from the cell and the communication protocol to take decisions proactively. In this paper, we present a complete proactive power management approach for the modem based on on-line grant prediction. In this context, we define proactive policies that allow a mobile device to go to sleep states more often compared to reactive power management systems, e.g., in time slots of predicted transmission inactivity in a cell. Furthermore, we propose and compare two algorithmic solutions to this proactive grant prediction problem, one a feed-forward neural network and one a SARSA- $\lambda$ λ reinforcement agent. As the implementation of these machine learning techniques also creates additional energy and resource costs, both approaches are carefully designed, optimized, and evaluated not only in terms of prediction accuracy, but also in terms of overall energy savings. Notably, our predictor implementations are able to achieve up to 17 percent in overall energy savings on real-world traces. [ABSTRACT FROM AUTHOR]

The densification of wireless infrastructure to meet ever-increasing quality of service (QoS) demands, and the ever-growing number of wireless devices may lead to higher levels of electromagnetic field (EMF) exposure in the environment, in the 5G era. The possible long term health effects related to the EMF radiation are still an open debate and requires attention. Therefore, in this paper, we propose a novel EMF-aware resource allocation scheme based on the power domain non-orthogonal multiple access (PD-NOMA) and machine learning (ML) technologies for reducing the EMF exposure in the uplink of cellular systems. More specifically, we use the K-means approach (an unsupervised ML approach) to create clusters of users to be allocated together and to then strategically group and assign them on the subcarriers, based on their associated channel properties. Finding the best number of clusters in the PD-NOMA environment is a key challenge, and in this paper, we have used the elbow method in conjunction with the F-test method to effectively control the maximum number of users to be allocated at the same time per subcarrier. We have also derived an EMF-aware power allocation by formulating and solving a convex optimization problem. Based on the simulation results, our proposed ML-based strategy effectively reduces the EMF exposure, in comparison with the state-of-the-art techniques. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of instrument current transformers technical state assessment based on machine learning methods. The introductory parts of the paper provide a detailed analysis of modern methods and approaches for technical state assessment of high-voltage power equipment of power plants and substations as well as a review of modern software tools and the latest trends in the given field of study. Justification of the relevance of the presented research aimed at instrument current transformers technical state assessment is provided along with the motivation for machine learning methods application for improvement of the accuracy and quality of high-voltage equipment state classification. Within the framework of the study, a comparative analysis of gradient boosting on decision trees and random forest algorithms was carried out for a given mathematical problem formulation. The main stages of processing the initial dataset are proposed as a step-by-step procedure, including feature extraction, feature transformation, feature interactions, etc. The outperforming efficiency of gradient boosting on decision trees algorithm was validated for real power equipment fleet. The resulting classification quality metrics of current transformers technical state assessment, Precision and Recall, are estimated to be 87.1% and 83.7%, correspondingly. [ABSTRACT FROM AUTHOR]

Adversarial attacks on machine learning-based classifiers, along with defense mechanisms, have been widely studied in the context of single-label classification problems. In this paper, we shift the attention to multi-label classification, where the availability of domain knowledge on the relationships among the considered classes may offer a natural way to spot incoherent predictions, i.e., predictions associated to adversarial examples lying outside of the training data distribution. We explore this intuition in a framework in which first-order logic knowledge is converted into constraints and injected into a semi-supervised learning problem. Within this setting, the constrained classifier learns to fulfill the domain knowledge over the marginal distribution, and can naturally reject samples with incoherent predictions. Even though our method does not exploit any knowledge of attacks during training, our experimental analysis surprisingly unveils that domain-knowledge constraints can help detect adversarial examples effectively, especially if such constraints are not known to the attacker. We show how to implement an adaptive attack exploiting knowledge of the constraints and, in a specifically-designed setting, we provide experimental comparisons with popular state-of-the-art attacks. We believe that our approach may provide a significant step towards designing more robust multi-label classifiers. [ABSTRACT FROM AUTHOR]

This paper aims at proposing an unsupervised hierarchical nonparametric Bayesian framework for modeling axial data (i.e., observations are axes of direction) that can be partitioned into multiple groups, where each observation within a group is sampled from a mixture of Watson distributions with an infinite number of components that are allowed to be shared across different groups. First, we propose a hierarchical nonparametric Bayesian model for modeling grouped axial data based on the hierarchical Pitman-Yor process mixture model of Watson distributions. Then, we demonstrate that by setting the discount parameters of the proposed model to 0, another hierarchical nonparametric Bayesian model based on hierarchical Dirichlet process can be derived for modeling axial data. To learn the proposed models, we systematically develop a closed-form optimization algorithm based on the collapsed variational Bayes (CVB) inference. Furthermore, to ensure the convergence of the proposed learning algorithm, an annealing mechanism is introduced to the framework of CVB inference, leading to an averaged collapsed variational Bayes inference strategy. The merits of the proposed models for modeling grouped axial data are demonstrated through experiments on both synthetic data and real-world applications involving gene expression data clustering and depth image analysis. [ABSTRACT FROM AUTHOR]

Correspondence analysis (CA) is a multivariate statistical tool used to visualize and interpret data dependencies by finding maximally correlated embeddings of pairs of random variables. CA has found applications in fields ranging from epidemiology to social sciences. However, current methods for CA do not scale to large, high-dimensional datasets. In this paper, we provide a novel interpretation of CA in terms of an information-theoretic quantity called the principal inertia components. We show that estimating the principal inertia components, which consists in solving a functional optimization problem over the space of finite variance functions of two random variable, is equivalent to performing CA. We then leverage this insight to design algorithms to perform CA at scale. Specifically, we demonstrate how the principal inertia components can be reliably approximated from data using deep neural networks. Finally, we show how the maximally correlated embeddings of pairs of random variables in CA further play a central role in several learning problems including multi-view and multi-modal learning methods and visualization of classification boundaries. [ABSTRACT FROM AUTHOR]

With the rapid advancement of intelligent transportation systems (ITS) and vehicular communications, vehicular edge computing (VEC) is emerging as a promising technology to support low-latency ITS applications and services. In this paper, we consider the computation offloading problem from mobile vehicles/users in a heterogeneous VEC scenario, and focus on the network- and base station selection problems, where different networks have different traffic loads. In a fast-varying vehicular environment, computation offloading experience of users is strongly affected by the latency due to the congestion at the edge computing servers co-located with the base stations. However, as a result of the non-stationary property of such an environment and also information shortage, predicting this congestion is an involved task. To address this challenge, we propose an on-line learning algorithm and an off-policy learning algorithm based on multi-armed bandit theory. To dynamically select the least congested network in a piece-wise stationary environment, these algorithms predict the latency that the offloaded tasks experience using the offloading history. In addition, to minimize the task loss due to the mobility of the vehicles, we develop a method for base station selection. Moreover, we propose a relaying mechanism for the selected network, which operates based on the sojourn time of the vehicles. Through intensive numerical analysis, we demonstrate that the proposed learning-based solutions adapt to the traffic changes of the network by selecting the least congested network, thereby reducing the latency of offloaded tasks. Moreover, we demonstrate that the proposed joint base station selection and the relaying mechanism minimize the task loss in a vehicular environment. [ABSTRACT FROM AUTHOR]

It is widely acknowledged that the success of deep learning is built upon large-scale training data and tremendous computing power. However, the data and computing power are not always available for many real-world applications. In this paper, we address the machine learning problem where it lacks training data and limits computing power. Specifically, we investigate domain adaptation which is able to transfer knowledge from one labeled source domain to an unlabeled target domain, so that we do not need much training data from the target domain. At the same time, we consider the situation that the running environment is confined, e.g., in edge computing the end device has very limited running resources. Technically, we present the Faster Domain Adaptation (FDA) protocol and further report two paradigms of FDA: early stopping and amid skipping. The former accelerates domain adaptation by multiple early exit points. The latter speeds up the adaptation by wisely skip several amid neural network blocks. Extensive experiments on standard benchmarks verify that our method is able to achieve the comparable and even better accuracy but employ much less computing resources. To the best of our knowledge, there are very few works which investigated accelerating knowledge adaptation in the community. This work is expected to inspire the topic for more discussion. [ABSTRACT FROM AUTHOR]

Inspired by the great success of machine learning (ML), researchers have applied ML techniques to visualizations to achieve a better design, development, and evaluation of visualizations. This branch of studies, known as ML4VIS, is gaining increasing research attention in recent years. To successfully adapt ML techniques for visualizations, a structured understanding of the integration of ML4VIS is needed. In this article, we systematically survey 88 ML4VIS studies, aiming to answer two motivating questions: “what visualization processes can be assisted by ML?” and “how ML techniques can be used to solve visualization problems? ”This survey reveals seven main processes where the employment of ML techniques can benefit visualizations: Data Processing4VIS, Data-VIS Mapping, Insight Communication, Style Imitation, VIS Interaction, VIS Reading, and User Profiling. The seven processes are related to existing visualization theoretical models in an ML4VIS pipeline, aiming to illuminate the role of ML-assisted visualization in general visualizations. Meanwhile, the seven processes are mapped into main learning tasks in ML to align the capabilities of ML with the needs in visualization. Current practices and future opportunities of ML4VIS are discussed in the context of the ML4VIS pipeline and the ML-VIS mapping. While more studies are still needed in the area of ML4VIS, we hope this article can provide a stepping-stone for future exploration. A web-based interactive browser of this survey is available at https://ml4vis.github.io. [ABSTRACT FROM AUTHOR]

Visualizations themselves have become a data format. Akin to other data formats such as text and images, visualizations are increasingly created, stored, shared, and (re-)used with artificial intelligence (AI) techniques. In this survey, we probe the underlying vision of formalizing visualizations as an emerging data format and review the recent advance in applying AI techniques to visualization data (AI4VIS). We define visualization data as the digital representations of visualizations in computers and focus on data visualization (e.g., charts and infographics). We build our survey upon a corpus spanning ten different fields in computer science with an eye toward identifying important common interests. Our resulting taxonomy is organized around WHAT is visualization data and its representation, WHY and HOW to apply AI to visualization data. We highlight a set of common tasks that researchers apply to the visualization data and present a detailed discussion of AI approaches developed to accomplish those tasks. Drawing upon our literature review, we discuss several important research questions surrounding the management and exploitation of visualization data, as well as the role of AI in support of those processes. We make the list of surveyed papers and related material available online at. [ABSTRACT FROM AUTHOR]

Depth maps acquired with ToF cameras have a limited accuracy due to the high noise level and to the multi-path interference. Deep networks can be used for refining ToF depth, but their training requires real world acquisitions with ground truth, which is complex and expensive to collect. A possible workaround is to train networks on synthetic data, but the domain shift between the real and synthetic data reduces the performances. In this paper, we propose three approaches to perform unsupervised domain adaptation of a depth denoising network from synthetic to real data. These approaches are respectively acting at the input, at the feature and at the output level of the network. The first approach uses domain translation networks to transform labeled synthetic ToF data into a representation closer to real data, that is then used to train the denoiser. The second approach tries to align the network internal features related to synthetic and real data. The third approach uses an adversarial loss, implemented with a discriminator trained to recognize the ground truth statistic, to train the denoiser on unlabeled real data. Experimental results show that the considered approaches are able to outperform other state-of-the-art techniques and achieve superior denoising performances. [ABSTRACT FROM AUTHOR]

This paper revisits the temporal difference (TD) learning algorithm for the policy evaluation tasks in reinforcement learning. Typically, the performance of TD(0) and TD($\lambda$ λ ) is very sensitive to the choice of stepsizes. Oftentimes, TD(0) suffers from slow convergence. Motivated by the tight link between the TD(0) learning algorithm and the stochastic gradient methods, we develop a provably convergent adaptive projected variant of the TD(0) learning algorithm with linear function approximation that we term AdaTD(0). In contrast to the TD(0), AdaTD(0) is robust or less sensitive to the choice of stepsizes. Analytically, we establish that to reach an $\epsilon$ ε accuracy, the number of iterations needed is $\tilde{O}(\epsilon ^{-2}\ln ^4\frac{1}{\epsilon }/\ln ^4\frac{1}{\rho })$ O ˜ (ε - 2 ln 4 1 ε / ln 4 1 ρ) in the general case, where $\rho$ ρ represents the speed of the underlying Markov chain converges to the stationary distribution. This implies that the iteration complexity of AdaTD(0) is no worse than that of TD(0) in the worst case. When the stochastic semi-gradients are sparse, we provide theoretical acceleration of AdaTD(0). Going beyond TD(0), we develop an adaptive variant of TD($\lambda$ λ ), which is referred to as AdaTD($\lambda$ λ ). Empirically, we evaluate the performance of AdaTD(0) and AdaTD($\lambda$ λ ) on several standard reinforcement learning tasks, which demonstrate the effectiveness of our new approaches. [ABSTRACT FROM AUTHOR]

Malicious software, popularly known as malware, is widely acknowledged as a serious threat to modern computing systems. Software-based solutions, such as anti-virus software (AVS), are not effective since they rely on matching patterns that can be easily fooled by carefully crafted malware with obfuscation or other deviation capabilities. While recent malware detection methods provide promising results through an effective utilization of hardware features, the detection results cannot be interpreted in a meaningful way. In this paper, we propose a hardware-assisted malware detection framework using explainable machine learning. This paper makes three important contributions. First, we theoretically establish that our proposed method can provide an interpretable explanation of classification results to address the challenge of transparency. Next, we show that the explainable outcome through effective utilization of hardware performance counters and embedded trace buffer can lead to accurate localization of malicious behavior. Finally, we have performed efficiency versus accuracy trade-off analysis using decision tree and recurrent neural networks. Extensive evaluation using a wide variety of real-world malware dataset demonstrates that our framework can produce accurate and human-understandable malware detection results with provable guarantees. [ABSTRACT FROM AUTHOR]

The growing number of Internet of Thing (IoT) and Ultra-Reliable Low Latency Communications (URLCC) use cases in next generation communication networks calls for the development of efficient Forward Error Correction (FEC) mechanisms. These use cases usually imply using short to mid-sized information blocks and requires low-complexity and/or fast decoding procedures. This paper investigates the joint learning of short to mid block-length coding schemes and associated Belief-Propagation (BP) like decoders using Machine Learning (ML) techniques. An interpretable auto-encoder (AE) architecture is proposed, ensuring scalability to block sizes currently challenging for ML-based linear block code design approaches. By optimizing a coding scheme w.r.t. the targeted decoder, the proposed system offers a good complexity/performance trade-off compared to various codes from literature with length up to 128 bits. [ABSTRACT FROM AUTHOR]

The Czochralski (CZ) process is the core technology for producing semiconductor silicon monocrystalline (SMC), and it is a complex batch process. However, the crystal growth rate and crystal diameter, which are key quality indicators, are difficult to detect directly online, and the offline calculation process lags seriously, which easily causes blind crystal quality control. Therefore, this paper proposes a data-driven and mechanism-based hybrid model for semiconductor SMC quality variables prediction in the CZ process. Firstly, a data-driven model JITL-SAE-ELM based on just-in-time learning (JITL) fine-tuning strategy is proposed. This model is used to solve the nonlinear and time-varying relationship of the energy transfer process in the CZ process that cannot be accurately described by traditional mechanism models. Here, the SAE-ELM model integrates a stacked autoencoder (SAE) and an extreme learning machine (ELM), which are used to deeply capture the nonlinear and time-varying features of process data. Secondly, according to the hydrodynamics and geometric behavior, a crystal pulling dynamic mechanism model based on the crystal growth mechanism is constructed, which avoids the complicated heat transfer link. Further, considering the unmodeled dynamics caused by model parameter uncertainty during the combination of the energy transfer model and crystal pulling dynamic mechanism model, a crystal diameter compensation model SAE-ELM was developed to improve the prediction accuracy of the CZ process hybrid model. Finally, an industrial data experiment based on a CZ monocrystal furnace illustrates the proposed method. [ABSTRACT FROM AUTHOR]

Capturing the dynamics in user preference is crucial to better predict user future behaviors because user preferences often drift over time. Many existing recommendation algorithms – including both shallow and deep ones – often model such dynamics independently, i.e., user static and dynamic preferences are not modeled under the same latent space, which makes it difficult to fuse them for recommendation. This paper considers the problem of embedding a user's sequential behavior into the latent space of user preferences, namely translating sequence to preference. To this end, we formulate the sequential recommendation task as a dictionary learning problem, which learns: 1) a shared dictionary matrix, each row of which represents a partial signal of user dynamic preferences shared across users; and 2) a posterior distribution estimator using a deep autoregressive model integrated with Gated Recurrent Unit (GRU), which can select related rows of the dictionary to represent a user's dynamic preferences conditioned on his/her past behaviors. Qualitative studies on the Netflix dataset demonstrate that the proposed method can capture the user preference drifts over time and quantitative studies on multiple real-world datasets demonstrate that the proposed method can achieve higher accuracy compared with state-of-the-art factorization and neural sequential recommendation methods. [ABSTRACT FROM AUTHOR]

As an aerial base station, unmanned aerial vehicle (UAV) has been considered as a promising technology to assist future wireless communications due to its flexible, swift and low cost features, where resource allocation is the basis for ensuring energy-efficient UAV-assisted networks. This paper formulates a joint optimization problem of user association, UAV trajectory design and power control to maximize the channel capacity among all ground users at a limited power level in a downlink transmission. To tackle the mixed-integer non-linear programming problem, this paper proposes a clustering-aided reinforcement learning approach consisting of three consecutive stages. Firstly, modified expectation-maximization unsupervised learning algorithm is investigated to cluster the ground users, which reduces the dimensions and hence, the association complexity is reduced as well. Then, Kuhn-Munkres algorithm is incorporated for user association, which associates a UAV with the ground users via matching to the cluster, and assigns the UAVs to the centroid of the matching cluster for pre-placement, with the aim of speeding up the convergence of the following deep reinforcement learning algorithm. Finally, a multi-agent twin delayed deep deterministic (MATD3) policy gradient is proposed to solve the non-convex sub-problem, which determines the transmit power and designs the fine-tuned trajectory of UAVs. By incorporating low-bias value estimation technique, the reward of the proposed MATD3 algorithm is improved. Simulation results have demonstrated that our proposed approach achieves higher reward as well as converging faster than existing reinforcement algorithms. Besides, the clustering-aided reinforcement learning has lower computational complexity than the benchmark schemes. [ABSTRACT FROM AUTHOR]

Neural networks have achieved remarkable successes in machine learning tasks. This has recently been extended to graph learning using neural networks. However, there is limited theoretical work in understanding how and when they perform well, especially relative to established statistical learning techniques such as spectral embedding. In this short paper, we present a simple generative model where unsupervised graph convolutional network fails, while the adjacency spectral embedding succeeds. Specifically, unsupervised graph convolutional network is unable to look beyond the first eigenvector in certain approximately regular graphs, thus missing inference signals in non-leading eigenvectors. The phenomenon is demonstrated by visual illustrations and comprehensive simulations. [ABSTRACT FROM AUTHOR]

Deep neural networks have been tremendously successful at segmenting objects in images. However, it has been shown they still have limitations on challenging problems such as the segmentation of medical images. The main reason behind this lower success resides in the reduced size of the object in the image. In this paper we overcome this limitation through a cyclic collaborative framework, CyCoSeg. The proposed framework is based on a deep active shape model (D-ASM), which provides prior information about the shape of the object, and a semantic segmentation network (SSN). These two models collaborate to reach the desired segmentation by influencing each other: SSN helps D-ASM identify relevant keypoints in the image through an Expectation Maximization formulation, while D-ASM provides a segmentation proposal that guides the SSN. This cycle is repeated until both models converge. Extensive experimental evaluation shows CyCoSeg boosts the performance of the baseline models, including several popular SSNs, while avoiding major architectural modifications. The effectiveness of our method is demonstrated on the left ventricle segmentation on two benchmark datasets, where our approach achieves one of the most competitive results in segmentation accuracy. Furthermore, its generalization is demonstrated for lungs and kidneys segmentation in CT scans. [ABSTRACT FROM AUTHOR]

Although deep face recognition has achieved impressive progress in recent years, controversy has arisen regarding discrimination based on skin tone, questioning their deployment into real-world scenarios. In this paper, we aim to systematically and scientifically study this bias from both data and algorithm aspects. First, using the dermatologist approved Fitzpatrick Skin Type classification system and Individual Typology Angle, we contribute a benchmark called Identity Shades (IDS) database, which effectively quantifies the degree of the bias with respect to skin tone in existing face recognition algorithms and commercial APIs. Further, we provide two skin-tone aware training datasets, called BUPT-Globalface dataset and BUPT-Balancedface dataset, to remove bias in training data. Finally, to mitigate the algorithmic bias, we propose a novel meta-learning algorithm, called Meta Balanced Network (MBN), which learns adaptive margins in large margin loss such that the model optimized by this loss can perform fairly across people with different skin tones. To determine the margins, our method optimizes a meta skewness loss on a clean and unbiased meta set and utilizes backward-on-backward automatic differentiation to perform a second order gradient descent step on the current margins. Extensive experiments show that MBN successfully mitigates bias and learns more balanced performance for people with different skin tones in face recognition. The proposed datasets are available at http://www.whdeng.cn/RFW/index.html. [ABSTRACT FROM AUTHOR]

Online transient analysis plays an increasingly important role in dynamic power grids as the renewable generation continues growing. Traditional numerical methods for transient analysis not only are computationally intensive but also require precise contingency information as input, and therefore, are not suitable for online applications. Existing online transient assessment studies focus on the determination of post-contingency system stability or stability margin. This paper develops a novel graph-learning framework, Deep-learning Neural Representation or DNR, for online prediction, of the time-series trajectories of the system states using initial system responses that can be measured by phasor measurement units (PMUs). The proposed DNR framework consists of two sequential modules: a Network Constructor that captures network dependencies among generators, and a Dynamics Predictor that predicts the system trajectories. The key to improved prediction performance is the introduction of the spatio-temporal message-passing operations into graph neural networks with structural knowledge. Its effectiveness and scalability are validated through comparative studies, demonstrating the prediction performance under different contingency scenarios for systems of different sizes. This framework provides a solution to online predicting post-fault system dynamics based on real-time PMU measurements. Additionally, it can also be applied to facilitate the offline transient simulation without simulating the entire trajectories. [ABSTRACT FROM AUTHOR]

Due to the diversity of power consumption patterns, the false positive rate (FPR) of data-driven electricity theft detection (ETD) methods is too high to meet practical needs, which severely restricts the engineering application of data-based methods. To reduce FPR of ETD methods based on advanced metering infrastructure (AMI), a deep neural network with low FPR (LFPR-DNN) is proposed in this paper. First, a deep model is constructed based on one-dimensional convolution and residual network, which can automatically extract features from consumption data. Then, a two-stage training scheme is used to train the network. In the first stage, the conventional gradient descent algorithm is used to update the network weights. To minimize the impact of data imbalance on detection performance, focal loss is used. Besides, grid search is used to optimize the hyper-parameters of the model. In the second stage, with FPR as the optimization objective, the particle swarm optimization (PSO) algorithm is used to train the network. Finally, the proposed LFPR-DNN is verified by using the open Irish data set. Compared to other state-of-the-art classifiers, LFPR-DNN has the lowest FPR with 0.29% and the highest AUC with 99.42%. The FPR is reduced by an order of magnitude, which verifies the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

This paper proposes a robust method to screen patients with sleep apnea syndrome (SAS) using a single-lead electrocardiogram (ECG). This method consists of minute-by-minute abnormal breathing detection and apnea-hypopnea index (AHI) estimation. Heartbeat interval and ECG-derived respiration (EDR) are calculated using the single-lead ECG and used to train the models, including ResNet18, ResNet34, and ResNet50. The proposed method, using data from 1232 subjects, was developed with two open datasets and experimental data and evaluated using two additional open datasets and data acquired from an abdomen-attached wearable device (in total, data from 189 subjects). ResNet18 showed the best results, having an average Cohen's kappa coefficient of 0.57, in the abnormal breathing detection. Moreover, SAS patient classification, with 15 as the AHI threshold, yielded an average Cohen's kappa coefficient of 0.71. The results of patient classification were biased toward data from the wearable patch-type device, which may be influenced by different ECG waveforms. The proposed method is tuned with a sample of the data from the device, and the performance result of Cohen's kappa increased from 0.54 to 0.91 for SAS patient classification. Our method, proposed in this paper, achieved equivalent performance results with data recorded using an abdomen-attached wearable device and two open datasets used in previous studies, although the method had not used those data during model training. The proposed method could reduce the development costs of commercial software, as it was developed using open datasets, has robust performance throughout all datasets. [ABSTRACT FROM AUTHOR]

This paper analyzes the application of machine learning techniques to the control of a soft, peristaltic, $xy$ -sorting table. In particular, we address peristaltic tables made of a soft upper silicone layer and actuated by an array of integrated air-filled chambers. The chambers are pneumatically inflated in order to deform the table and move objects on the table. To control the robot precisely, it is necessary to model both the inverse mapping between the control signals of the actuators and the resulting surface deformation. There is currently no parametric model available. In this paper, we, therefore, study if nonparametric approaches are applicable. In these approaches, the mapping would be learned from a database of input signals and observed behaviors. From our analysis, we conclude that the most promising research direction consists in the nonparametric modeling of a limited set of peristaltic actuation patterns. However, the nonlinear hardware design that impedes a parametric model also affects the nonparametric optimization process. Our simulation suggests that the optimization is nonconvex, approximately monotonous, and feasible in terms of the number of observations of the physical robot. [ABSTRACT FROM PUBLISHER]