Search

Due to the increasing size of integrated circuits (ICs), their design and optimization phases (i.e., computer-aided design, CAD) grow increasingly complex. At design time, a large design space needs to be explored to find an implementation that fulfills all specifications and then optimizes metrics like energy, area, delay, reliability, etc. At run time, a large configuration space needs to be searched to find the best set of parameters (e.g., voltage/frequency) to further optimize the system. Both spaces are infeasible for exhaustive search typically leading to heuristic optimization algorithms that find some tradeoff between design quality and computational overhead. Machine learning (ML) can build powerful models that have successfully been employed in related domains. In this survey, we categorize how ML may be used and is used for design-time and run-time optimization and exploration strategies of ICs. A metastudy of published techniques unveils areas in CAD that are well explored and underexplored with ML, as well as trends in the employed ML algorithms. We present a comprehensive categorization and summary of the state of the art on ML for CAD. Finally, we summarize the remaining challenges and promising open research directions. [ABSTRACT FROM AUTHOR]

The document is the table of contents for the August 2024 issue of the IEEE Transactions on Semiconductor Manufacturing. It includes a special section on the 2023 Advanced Semiconductor Manufacturing Conference, as well as regular issue papers on topics such as wafer-scale 3D integration, chemical uniformity impacts on electrodeposition, and thermal study of polymers for microelectronics. The document also includes letters and announcements, such as calls for papers and nominations for awards. [Extracted from the article]

This paper serves as a survey of recent advances in large margin training and its theoretical foundations, mostly for (nonlinear) deep neural networks (DNNs) that are probably the most prominent machine learning models for large-scale data in the community over the past decade. We generalize the formulation of classification margins from classical research to latest DNNs, summarize theoretical connections between the margin, network generalization, and robustness, and introduce recent efforts in enlarging the margins for DNNs comprehensively. Since the viewpoint of different methods is discrepant, we categorize them into groups for ease of comparison and discussion in the paper. Hopefully, our discussions and overview inspire new research work in the community that aim to improve the performance of DNNs, and we also point to directions where the large margin principle can be verified to provide theoretical evidence why certain regularizations for DNNs function well in practice. We managed to shorten the paper such that the crucial spirit of large margin learning and related methods are better emphasized. [ABSTRACT FROM AUTHOR]

There is an opportunity in modern power systems to explore the demand flexibility by incentivizing consumers with dynamic prices. In this paper, we quantify demand flexibility using an efficient tool called time-varying elasticity, whose value may change depending on the prices and decision dynamics. This tool is particularly useful for evaluating the demand response potential and system reliability. Recent empirical evidences have highlighted some abnormal features when studying demand flexibility, such as delayed responses and vanishing elasticities after price spikes. Existing methods fail to capture these complicated features because they heavily rely on some predefined (often over-simplified) regression expressions. Instead, this paper proposes a model-free methodology to automatically and accurately derive the optimal estimation pattern. We further develop a two-stage estimation process with Siamese long short-term memory (LSTM) networks. Here, a LSTM network encodes the price response, while the other network estimates the time-varying elasticities. In the case study, the proposed framework and models are validated to achieve higher overall estimation accuracy and better description for various abnormal features when compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Linear spectral unmixing is the practice of decomposing the mixed pixel into a linear combination of the constituent endmembers and the estimated abundances. This paper focuses on unsupervised spectral unmixing where the endmembers are unknown a priori. Conventional approaches use either geometrical- or statistical-based approaches. In this paper, we address the challenges of spectral unmixing with unsupervised deep learning models, in specific, the autoencoder models, where the decoder serves as the endmembers and the hidden layer output serves as the abundances. In several recent attempts, part-based autoencoders have been designed to solve the unsupervised spectral unmixing problem. However, the performance has not been satisfactory. In this paper, we first discuss some important findings we make on issues with part-based autoencoders. By proof of counterexample, we show that all existing part-based autoencoder networks with nonnegative and tied encoder and decoder are inherently defective by making these inappropriate assumptions on the network structure. As a result, they are not suitable for solving the spectral unmixing problem. We propose a so-called untied denoising autoencoder with sparsity, in which the encoder and decoder of the network are independent, and only the decoder of the network is enforced to be nonnegative. Furthermore, we make two critical additions to the network design. First, since denoising is an essential step for spectral unmixing, we propose to incorporate the denoising capacity into the network optimization in the format of a denoising constraint rather than cascading another denoising preprocessor in order to avoid the introduction of additional reconstruction error. Second, to be more robust to the inaccurate estimation of a number of endmembers, we adopt an $l_{21}$ -norm on the encoder of the network to reduce the redundant endmembers while decreasing the reconstruction error simultaneously. The experimental results demonstrate that the proposed approach outperforms several state-of-the-art methods, especially for highly noisy data. [ABSTRACT FROM AUTHOR]

Distribution system state estimation (DSSE) is a core task for monitoring and control of distribution networks. Widely used algorithms such as Gauss–Newton perform poorly with the limited number of measurements typically available for DSSE, often require many iterations to obtain reasonable results, and sometimes fail to converge. DSSE is a non-convex problem, and working with a limited number of measurements further aggravates the situation, as indeterminacy induces multiple global (in addition to local) minima. Gauss–Newton is also known to be sensitive to initialization. Hence, the situation is far from ideal. It is therefore natural to ask if there is a smart way of initializing Gauss–Newton that will avoid these DSSE-specific pitfalls. This paper proposes using historical or simulation-derived data to train a shallow neural network to “learn to initialize,” that is, map the available measurements to a point in the neighborhood of the true latent states (network voltages), which is used to initialize Gauss–Newton. It is shown that this hybrid machine learning/optimization approach yields superior performance in terms of stability, accuracy, and runtime efficiency, compared to conventional optimization-only approaches. It is also shown that judicious design of the neural network training cost function helps to improve the overall DSSE performance. [ABSTRACT FROM AUTHOR]

Correspondence analysis (CA) is a multivariate statistical tool used to visualize and interpret data dependencies by finding maximally correlated embeddings of pairs of random variables. CA has found applications in fields ranging from epidemiology to social sciences. However, current methods for CA do not scale to large, high-dimensional datasets. In this paper, we provide a novel interpretation of CA in terms of an information-theoretic quantity called the principal inertia components. We show that estimating the principal inertia components, which consists in solving a functional optimization problem over the space of finite variance functions of two random variable, is equivalent to performing CA. We then leverage this insight to design algorithms to perform CA at scale. Specifically, we demonstrate how the principal inertia components can be reliably approximated from data using deep neural networks. Finally, we show how the maximally correlated embeddings of pairs of random variables in CA further play a central role in several learning problems including multi-view and multi-modal learning methods and visualization of classification boundaries. [ABSTRACT FROM AUTHOR]

Deep neural networks have been tremendously successful at segmenting objects in images. However, it has been shown they still have limitations on challenging problems such as the segmentation of medical images. The main reason behind this lower success resides in the reduced size of the object in the image. In this paper we overcome this limitation through a cyclic collaborative framework, CyCoSeg. The proposed framework is based on a deep active shape model (D-ASM), which provides prior information about the shape of the object, and a semantic segmentation network (SSN). These two models collaborate to reach the desired segmentation by influencing each other: SSN helps D-ASM identify relevant keypoints in the image through an Expectation Maximization formulation, while D-ASM provides a segmentation proposal that guides the SSN. This cycle is repeated until both models converge. Extensive experimental evaluation shows CyCoSeg boosts the performance of the baseline models, including several popular SSNs, while avoiding major architectural modifications. The effectiveness of our method is demonstrated on the left ventricle segmentation on two benchmark datasets, where our approach achieves one of the most competitive results in segmentation accuracy. Furthermore, its generalization is demonstrated for lungs and kidneys segmentation in CT scans. [ABSTRACT FROM AUTHOR]

This paper investigates the stochastic optimization problem focusing on developing scalable parallel algorithms for deep learning tasks. Our solution involves a reformation of the objective function for stochastic optimization in neural network models, along with a novel parallel computing strategy, coined the weighted aggregating stochastic gradient descent (WASGD). Following a theoretical analysis on the characteristics of the new objective function, WASGD introduces a decentralized weighted aggregating scheme based on the performance of local workers. Without any center variable, the new method automatically gauges the importance of local workers and accepts them by their contributions. Furthermore, we have developed an enhanced version of the method, WASGD+, by (1) implementing a designed sample order and (2) upgrading the weight evaluation function. To validate the new method, we benchmark our pipeline against several popular algorithms including the state-of-the-art deep neural network classifier training techniques (e.g., elastic averaging SGD). Comprehensive validation studies have been conducted on four classic datasets: CIFAR-100, CIFAR-10, Fashion-MNIST, and MNIST. Subsequent results have firmly validated the superiority of the WASGD scheme in accelerating the training of deep architecture. Better still, the enhanced version, WASGD+, is shown to be a significant improvement over its prototype. [ABSTRACT FROM AUTHOR]

Condition monitoring of power transformers, which are key components of electrical power systems, is essential to identify incipient faults and avoid catastrophic failures. In this paper machine learning algorithms, i.e., nonlinear autoregressive neural networks and support vector machines, are proposed to model the transformer thermal behavior for the purpose of monitoring. The thermal models are developed based on the historical measurements from nine transformers comprised of two 180-MVA units, four 240-MVA units and three 1000-MVA units. The data consist of load profile, tap position, winding indicator temperature (WTI) measurement, ambient temperature, wind speed and solar radiation. The results are validated against field measurements, and it is clearly demonstrated that the alternative algorithms surpass the IEEE Annex G thermal model. An incipient thermal fault identification algorithm is then proposed and successfully used to identify an issue using measurements taken in the field. This algorithm could be used to alert the operator and plan intervention accordingly. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (DCNNs) are currently the method of choice both for generative, as well as for discriminative learning in computer vision and machine learning. The success of DCNNs can be attributed to the careful selection of their building blocks (e.g., residual blocks, rectifiers, sophisticated normalization schemes, to mention but a few). In this paper, we propose $\Pi$ Π -Nets, a new class of function approximators based on polynomial expansions. $\Pi$ Π -Nets are polynomial neural networks, i.e., the output is a high-order polynomial of the input. The unknown parameters, which are naturally represented by high-order tensors, are estimated through a collective tensor factorization with factors sharing. We introduce three tensor decompositions that significantly reduce the number of parameters and show how they can be efficiently implemented by hierarchical neural networks. We empirically demonstrate that $\Pi$ Π -Nets are very expressive and they even produce good results without the use of non-linear activation functions in a large battery of tasks and signals, i.e., images, graphs, and audio. When used in conjunction with activation functions, $\Pi$ Π -Nets produce state-of-the-art results in three challenging tasks, i.e., image generation, face verification and 3D mesh representation learning. The source code is available at https://github.com/grigorisg9gr/polynomial_nets. [ABSTRACT FROM AUTHOR]

A general deep learning (DL) mechanism for a multiple hidden layer feed-forward neural network contains two parts, i.e., 1) an unsupervised greedy layer-wise training and 2) a supervised fine-tuning which is usually an iterative process. Although this mechanism has been demonstrated in many fields to be able to significantly improve the generalization of neural network, there is no clear evidence to show which one of the two parts plays the essential role for the generalization improvement, resulting in an argument within the DL community. Focusing on this argument, this paper proposes a new DL approach to train multilayer feed-forward neural networks. This approach uses restricted Boltzmann machine (RBM) as the layer-wise training and uses the generalized inverse of a matrix as the supervised fine-tuning. Different from the general deep training mechanism like back-propagation (BP), the proposed approach does not need to iteratively tune the weights, and therefore, has many advantages such as quick training, better generalization, and high understandability, etc. Experimentally, the proposed approach demonstrates an excellent performance in comparison with BP-based DL and the traditional training method for multilayer random weight neural networks. To a great extent, this paper demonstrates that the supervised part plays a more important role than the unsupervised part in DL, which provides some new viewpoints for exploring the essence of DL. [ABSTRACT FROM AUTHOR]

This paper proposes energy-efficient approximate multipliers based on the Mitchell's log multiplication, optimized for performing inferences on convolutional neural networks (CNN). Various design techniques are applied to the log multiplier, including a fully-parallel LOD, efficient shift amount calculation, and exact zero computation. Additionally, the truncation of the operands is studied to create the customizable log multiplier that further reduces energy consumption. The paper also proposes using the one's complements to handle negative numbers, as an approximation of the two's complements that had been used in the prior works. The viability of the proposed designs is supported by the detailed formal analysis as well as the experimental results on CNNs. The experiments also provide insights into the effect of approximate multiplication in CNNs, identifying the importance of minimizing the range of error.The proposed customizable design at $w$w = 8 saves up to 88 percent energy compared to the exact fixed-point multiplier at 32 bits with just a performance degradation of 0.2 percent for the ImageNet ILSVRC2012 dataset. [ABSTRACT FROM AUTHOR]

In this paper, we present a two-speed, radix-4, serial-parallel multiplier for accelerating applications such as digital filters, artificial neural networks, and other machine learning algorithms. Our multiplier is a variant of the serial–parallel (SP) modified radix-4 Booth multiplier that adds only the nonzero Booth encodings and skips over the zero operations, making the latency dependent on the multiplier value. Two subcircuits with different critical paths are utilized so that throughput and latency are improved for a subset of multiplier values. The multiplier is evaluated on an Intel Cyclone V field-programmable gate array against standard parallel–parallel and SP multipliers across four different process–voltage–temperature corners. We show that for bit widths of 32 and 64, our optimizations can result in a $1.42\times $ – $3.36\times $ improvement over the standard parallel Booth multiplier in terms of area–time depending on the input set. [ABSTRACT FROM AUTHOR]

This paper presents an adaptive neural control approach for nonstrict-feedback nonlinear systems in presence of unmodeled dynamics, unknown control directions and input dead-zone nonlinearity. To handle the difficulty due to uncertain control directions, Nussbaum gain functions are applied. Based on the structural characteristic of radial basis function neural networks, a backstepping-based adaptive neural control algorithm is developed. The main contributions of this paper lie in the fact that a backstepping-based neural control algorithm is developed for nonstrict-feedback nonlinear systems with unmodeled dynamics, unknown control directions and actuator dead-zone, and the total number of adaptive laws is not greater than the order of control system. As a beneficial result, the controller is much easier to be implemented in practice with less computational burden. A simulation example is given to reveal the viability of the presented approach. It is demonstrated by both theoretical analysis and simulation study that the presented control strategy ensures the semiglobally uniform ultimate boundedness of all closed-loop system signals. [ABSTRACT FROM AUTHOR]

A sum-product network (SPN) is a probabilistic model, based on a rooted acyclic directed graph, in which terminal nodes represent probability distributions and non-terminal nodes represent convex sums (weighted averages) and products of probability distributions. They are closely related to probabilistic graphical models, in particular to Bayesian networks with multiple context-specific independencies. Their main advantage is the possibility of building tractable models from data, i.e., models that can perform several inference tasks in time proportional to the number of edges in the graph. They are somewhat similar to neural networks and can address the same kinds of problems, such as image processing and natural language understanding. This paper offers a survey of SPNs, including their definition, the main algorithms for inference and learning from data, several applications, a brief review of software libraries, and a comparison with related models. [ABSTRACT FROM AUTHOR]

There has been a resurgence of research in the design of deep architecture models and learning algorithms, i.e., methods that rely on the extraction of a multilayer representation of the data. Often referred to as deep learning, this topic of research has been building on and contributing to many different research topics, such as neural networks, graphical models, feature learning, unsupervised learning, optimization, pattern recognition, and signal processing. Deep learning is also motivated and inspired by neuroscience and has had a tremendous impact on various applications such as computer vision, speech recognition, and natural language processing. The clearly multidisciplinary nature of deep learning led to a call for papers for a special issue dedicated to learning deep architectures. [ABSTRACT FROM PUBLISHER]

Efficient nonlinearity compensation in fiber-optic communication systems is considered a key element to go beyond the “capacity crunch”. One guiding principle for previous work on the design of practical nonlinearity compensation schemes is that fewer steps lead to better systems. In this paper, we challenge this assumption and show how to carefully design multi-step approaches that provide better performance–complexity trade-offs than their few-step counterparts. We consider the recently proposed learned digital backpropagation (LDBP) approach, where the linear steps in the split-step method are re-interpreted as general linear functions, similar to the weight matrices in a deep neural network. Our main contribution lies in an experimental demonstration of this approach for a 25 Gbaud single-channel optical transmission system. It is shown how LDBP can be integrated into a coherent receiver DSP chain and successfully trained in the presence of various hardware impairments. Our results show that LDBP with limited complexity can achieve better performance than standard DBP by using very short, but jointly optimized, finite-impulse response filters in each step. This paper also provides an overview of recently proposed extensions of LDBP and we comment on potentially interesting avenues for future work. [ABSTRACT FROM AUTHOR]

The classification of distinct problems of insulators in the distribution networks is a task that requires operator's experience. The applications of techniques to automate the inspection of electrical systems with the objective of detecting faults in insulators have shown to be reasonable alternatives to improve reliability in power grid. In this paper, based on the development of an experimental setup, signals are acquired considering three distinct faults in insulators. In this case, 13.8 kV (rms) is applied in drilled, contaminated, and good insulators considering an ultrasound detector connected to a computer. In the sequence, a multiclass classification method is proposed considering the ensemble of classifiers. The method considers the association of five distinct techniques, Bottom-Up segmentation, wavelet energy coefficient, principal component analysis, and particle swarm optimization associated with ensemble extreme learning machine (EN-ELM). Named optimized ensemble extreme learning machine, the present approach outperforms the original EN-ELM method. Finally, results show significant increase in robustness and faster training procedure when compared to classical approaches. [ABSTRACT FROM AUTHOR]

Stereo matching methods consist of matching cost computation and several post processing steps. Deep learning methods have greatly raised the accuracy of matching cost and achieved the lowest error rate on several public datasets. However, their generality capabilities are not the best due to potential overfitting, which is the common problem of supervised learning approaches. This paper proposes a convolutional neural network (CNN) based cost estimation method for computing the similarity of image patches. In consideration of accuracy and generalization capability, small size convolution kernels are chosen in the convolution layer and dropout in the decision layer is used for preventing overfitting. After obtaining stereo matching cost from the output of the CNN, several post-processing operations are adopted for disparity optimization, which includes semi-global matching in 1-D from different directions, a left-right consistency check, and the slanted plane smoothing method. The method is evaluated on KITTI 2012, KITTI 2015, and Middlebury stereo datasets and the experimental results on the KITTI benchmark demonstrate the competitive accuracy performance of the approach. Additionally, to test the generalization of the method, a series of extended crossover experiments are conducted in which the training samples and testing samples come from different datasets. The results indicate superior generalization capability of our method than other supervised learning methods. [ABSTRACT FROM AUTHOR]

Machine learning algorithms such as those for object classification in images, video content analysis, and human action recognition are used to extract meaningful information from data recorded by image sensors and cameras. Among the existing machine learning algorithms for such purposes, extreme learning machines (ELMs) and online sequential ELMs (OS-ELMs) are well known for their computational efficiency and performance when processing large datasets. The latter approach was derived from the ELM approach and optimized for real-time application. However, OS-ELM classifiers are computationally demanding, and the existing state-of-the-art computing platforms are not efficient enough for embedded systems, especially for applications with strict requirements in terms of low power consumption, high throughput, and low latency. This paper presents the implementation of an ELM/OS-ELM in a customized system-on-a-chip field-programmable gate array-based architecture to ensure efficient hardware acceleration. The acceleration process comprises parallel extraction, deep pipelining, and efficient shared memory communication. [ABSTRACT FROM AUTHOR]

Significance: The performance of traditional approaches to decoding movement intent from electromyograms (EMGs) and other biological signals commonly degrade over time. Furthermore, conventional algorithms for training neural network based decoders may not perform well outside the domain of the state transitions observed during training. The work presented in this paper mitigates both these problems, resulting in an approach that has the potential to substantially improve the quality of life of the people with limb loss. Objective: This paper presents and evaluates the performance of four decoding methods for volitional movement intent from intramuscular EMG signals. Methods: The decoders are trained using the dataset aggregation (DAgger) algorithm, in which the training dataset is augmented during each training iteration based on the decoded estimates from previous iterations. Four competing decoding methods, namely polynomial Kalman filters (KFs), multilayer perceptron (MLP) networks, convolutional neural networks (CNN), and long short-term memory (LSTM) networks, were developed. The performances of the four decoding methods were evaluated using EMG datasets recorded from two human volunteers with transradial amputation. Short-term analyses, in which the training and cross-validation data came from the same dataset, and long-term analyses, in which the training and testing were done in different datasets, were performed. Results: Short-term analyses of the decoders demonstrated that CNN and MLP decoders performed significantly better than KF and LSTM decoders, showing an improvement of up to 60% in the normalized mean-square decoding error in cross-validation tests. Long-term analyses indicated that the CNN, MLP, and LSTM decoders performed significantly better than a KF-based decoder at most analyzed cases of temporal separations (0–150 days) between the acquisition of the training and testing datasets. Conclusion: The short-term and long-term performances of MLP- and CNN-based decoders trained with DAgger demonstrated their potential to provide more accurate and naturalistic control of prosthetic hands than alternate approaches. [ABSTRACT FROM AUTHOR]

The extreme learning machine (ELM), which was originally proposed for “generalized” single-hidden layer feedforward neural networks, provides efficient unified learning solutions for the applications of regression and classification. Although, it provides promising performance and robustness and has been used for various applications, the single-layer architecture possibly lacks the effectiveness when applied for natural signals. In order to over come this shortcoming, the following work indicates a new architecture based on multilayer network framework. The significant contribution of this paper are as follows: 1) unlike existing multilayer ELM, in which hidden nodes are obtained randomly, in this paper all hidden layers with invertible functions are calculated by pulling the network output back and putting it into hidden layers. Thus, the feature learning is enriched by additional information, which results in better performance; 2) in contrast to the existing multilayer network methods, which are usually efficient for classification applications, the proposed architecture is implemented for dimension reduction and image reconstruction; and 3) unlike other iterative learning-based deep networks (DL), the hidden layers of the proposed method are obtained via four steps. Therefore, it has much better learning efficiency than DL. Experimental results on 33 datasets indicate that, in comparison to the other existing dimension reduction techniques, the proposed method performs competitively better with fast training speeds. [ABSTRACT FROM AUTHOR]

In this paper, we experimentally and analytically evaluate the reliability of two state-of-the-art neural networks for linear regression and pattern recognition (multilayer perceptron and single-layer perceptron) implemented in a system-on-chip composed of a field-programmable gate array (FPGA) and a microprocessor. We have considered, for each neural network, three different activation function complexities, to evaluate how the implementation affects FPGAs reliability. As we show in this paper, higher complexity increases the exposed area but reduces the probability of one failure to impact the network output. In addition, we propose to distinguish between critical and tolerable errors in artificial neural networks. Experiments using a controlled heavy-ions beam show that, for both networks, only about 30% of the observed output errors actually affect the outputs correctness. We identify the causes of critical errors through fault injection, and found that faults in initial layers are more likely to significantly affect the output. [ABSTRACT FROM PUBLISHER]

Supervised approaches classify input data using a set of representative samples for each class, known as training samples. The collection of such samples is expensive and time demanding. Hence, unsupervised feature learning, which has a quick access to arbitrary amounts of unlabeled data, is conceptually of high interest. In this paper, we propose a novel network architecture, fully Conv-Deconv network, for unsupervised spectral-spatial feature learning of hyperspectral images, which is able to be trained in an end-to-end manner. Specifically, our network is based on the so-called encoder-decoder paradigm, i.e., the input 3-D hyperspectral patch is first transformed into a typically lower dimensional space via a convolutional subnetwork (encoder), and then expanded to reproduce the initial data by a deconvolutional subnetwork (decoder). However, during the experiment, we found that such a network is not easy to be optimized. To address this problem, we refine the proposed network architecture by incorporating: 1) residual learning and 2) a new unpooling operation that can use memorized max-pooling indexes. Moreover, to understand the "black box," we make an in-depth study of the learned feature maps in the experimental analysis. A very interesting discovery is that some specific "neurons" in the first residual block of the proposed network own good description power for semantic visual patterns in the object level, which provide an opportunity to achieve "free" object detection. This paper, for the first time in the remote sensing community, proposes an end-to-end fully Conv-Deconv network for unsupervised spectral-spatial feature learning. Moreover, this paper also introduces an in-depth investigation of learned features. Experimental results on two widely used hyperspectral data, Indian Pines and Pavia University, demonstrate competitive performance obtained by the proposed methodology compared with other studied approaches. [ABSTRACT FROM AUTHOR]

In order to further improve the productivity of manufacturing equipment, it is indispensable to monitor the conditions of all the manufacturing equipment and not just the processing chambers. In this paper, we present a robust machine learning based deterioration diagnosis technology with harmonic sensor, which has frequency characteristics that have high sensitivity to deterioration and low sensitivity to torque. The robust deterioration diagnosis using a small amount of data is possible by limiting the input data to be trained and inferred to signals and motion patterns, and inferring with neural network models that do not require developments for manufacturing equipment. The example of wafer transfer robots in the ion implanter and the Low-Pressure Chemical Vapor Deposition (LP-CVD) and the coater/developer show that wear deterioration of machine components can be detected from the level of deterioration output and it is possible to prevent errors of the wafer transfer robots because of maintenance based on increases in the level of deterioration. [ABSTRACT FROM AUTHOR]

Nowadays, an Internet-of-Things (IoT) connected system brings a tremendous paradigm shift into the medium access control (MAC) design. In this paper, we present a distributed MAC framework assisted by machine learning for the Heterogeneous IoT system, where the IoT devices coexist with the WiFi users in the unlicensed industrial, scientific, and medical (ISM) spectrum. Specifically, the superframe is divided into two phases: a rendezvous phase and a transmission phase. During the rendezvous phase, the gateway that is capable of machine learning predicts the number of WiFi users and the IoT devices by performing a triangular handshake on the primary channel. The prediction takes advantage of the deep neural network (DNN) model which is pretrained on our universal software radio peripheral (USRP2) testbed offline. The gateway allocates the frequency channels to the WiFi and IoT systems based on the inference results. Then, the IoT devices and WiFi users initiate data transmissions during the transmission phase. Furthermore, system throughput is analyzed and optimized in two typical scenarios, respectively. An optimized MAC framework is proposed to maximize the total system throughput by finding the key design parameters. The analytical and simulation results that are conducted using the ns-2 demonstrate the effectiveness of the proposed MAC framework. [ABSTRACT FROM AUTHOR]

This paper advocates a data summarization approach based on distance rather than the traditional time period when developing individualized machine learning models for fuel consumption. This approach is used in conjunction with seven predictors derived from vehicle speed and road grade to produce a highly predictive neural network model for average fuel consumption in heavy vehicles. The proposed model can easily be developed and deployed for each individual vehicle in a fleet in order to optimize fuel consumption over the entire fleet. The predictors of the model are aggregated over fixed window sizes of distance traveled. Different window sizes are evaluated and the results show that a 1 km window is able to predict fuel consumption with a 0.91 coefficient of determination and mean absolute peak-to-peak percent error less than 4% for routes that include both city and highway duty cycle segments. [ABSTRACT FROM AUTHOR]

Time series classification is an important research topic in machine learning and data mining communities, since time series data exist in many application domains. Recent studies have shown that machine learning algorithms could benefit from good feature representation, explaining why deep learning has achieved breakthrough performance in many tasks. In deep learning, the convolutional neural network (CNN) is one of the most well-known approaches, since it incorporates feature learning and classification task in a unified network architecture. Although CNN has been successfully applied to image and text domains, it is still a challenge to apply CNN to time series data. This paper proposes a tensor scheme along with a novel deep learning architecture called multivariate convolutional neural network (MVCNN) for multivariate time series classification, in which the proposed architecture considers multivariate and lag-feature characteristics. We evaluate our proposed method with the prognostics and health management (PHM) 2015 challenge data, and compare with several algorithms. The experimental results indicate that the proposed method outperforms the other alternatives using the prediction score, which is the evaluation metric used by the PHM Society 2015 data challenge. Besides performance evaluation, we provide detailed analysis about the proposed method. [ABSTRACT FROM AUTHOR]

Machine learning-guided optical proximity correction, called ML-OPC in this paper, has recently been proposed to alleviate long runtime of model-based OPC. ML-OPC using regression methods has been presented but with limited prediction accuracy. We propose neural network classifier-based OPC (NNC-OPC), in which a neural network classifier serves as a mask bias model. A few techniques are applied to enhance basic NNC-OPC: parameterization of layout segments using polar Fourier transform signals, dimensionality reduction through weighted principal component analysis, and sampling of training layout segments. Training segments are typically imbalanced over the range of mask biases, which may cause large prediction error for segments that appear less frequently. This is resolved by three techniques: 1) synthetic data generation; 2) class reorganization; and 3) an adaptive learning rate. Experiments with NNC-OPC with all techniques applied indicate that prediction error of mask bias and training time are reduced by 29% and 80%, respectively, compared to state-of-the-art ML-OPC with regression methods. [ABSTRACT FROM AUTHOR]

As microprocessor design scales to the 10-nm technology and beyond, traditional pre- and post-silicon validation techniques are unsuitable to get a full system functional coverage. Physical complexity and extreme technology process variations severely limits the effectiveness and reliability of presilicon validation techniques. This scenario imposes the need of sophisticated post-silicon validation approaches to consider complex electromagnetic phenomena and large manufacturing fluctuations observed in actual physical platforms. One of the major challenges in electrical validation of high-speed input/output (HSIO) links in modern computer platforms lies in the physical layer (PHY) tuning process, where equalization techniques are used to cancel undesired effects induced by the channels. Current industrial practices for PHY tuning in HSIO links are very time consuming since they require massive laboratory measurements. An alternative is to use machine learning techniques to model the PHY, and then perform equalization using the resultant surrogate model. In this paper, a metamodeling approach based on neural networks is proposed to efficiently simulate the effects of a receiver equalizer PHY tuning settings. We use several design of experiments techniques to find a neural model capable of approximating the real system behavior without requiring a large amount of actual measurements. We evaluate the models performance by comparing with measured responses on a real server HSIO link. [ABSTRACT FROM AUTHOR]

The multimode power-split architecture for hybrid electric vehicle (HEV) powertrains is generally known for the complexity of its operation. This paper first addresses the challenge of developing an automated on-line near-optimal control strategy for these systems. Particularly, a machine learning logic based on supervised learning is developed for on-line selection of the HEV operating mode, thus the particular set of clutches to be engaged. An efficiency-based approach is then adopted to determine the optimal power-split between powertrain components. Later, the developed strategy finds integration in an optimal design methodology for multimode power-split HEVs considering the effectiveness and ease of on-line controllability. The obtained results are compared with the ones by the traditional HEV design methodology that only considers off-line energy management. The illustrated design methodology with on-line control reveals efficient at identifying the multimode HEV design that demonstrates optimal predicted fuel economy values and ease of on-line controllability simultaneously. Results suggest that the HEV design optimization procedure may produce different outcomes and demonstrate more effective when the evaluation of the on-line controlled operation is integrated. [ABSTRACT FROM AUTHOR]

This paper proposes a novel intelligent fault diagnosis method to automatically identify different health conditions of wind turbine (WT) gearbox. Unlike traditional approaches, where feature extraction and classification are separately designed and performed, this paper aims to automatically learn effective fault features directly from raw vibration signals while classify the type of faults in a single framework, thus providing an end-to-end learning-based fault diagnosis system for WT gearbox without additional signal processing and diagnostic expertise. Considering the multiscale characteristics inherent in vibration signals of a gearbox, a new multiscale convolutional neural network (MSCNN) architecture is proposed to perform multiscale feature extraction and classification simultaneously. The proposed MSCNN incorporates multiscale learning into the traditional CNN architecture, which has two merits: 1) high-level fault features can be effectively learned by the hierarchical learning structure with multiple pairs of convolutional and pooling layers; and 2) multiscale learning scheme can capture complementary and rich diagnosis information at different scales. This greatly improves the feature learning ability and enables better diagnosis performance. The proposed MSCNN approach is evaluated through experiments on a WT gearbox test rig. Experimental results and comprehensive comparison analysis with respect to the traditional CNN and traditional multiscale feature extractors have demonstrated the superiority of the proposed method. [ABSTRACT FROM AUTHOR]

Responding to the growing bandwidth demand by emerging applications, such as fixed-mobile convergence for fifth generation (5G) and beyond 5G, 100 Gb/s/λ access network becomes the next research focus of passive optical network (PON) roadmap. Intensity modulation and direct detection (IMDD) technology is still considered as a promising candidate for 100 Gb/s/λ PON attributed to its low cost, low power consumption, and small footprint. In this paper, we achieve 100 Gb/s/λ IMDD PON by using 20G-class optical and electrical devices due to its commercial availability. To mitigate the system linear and nonlinear distortions, neural network (NN) based equalizer is used and the performance is compared with feedforward equalizer (FFE) and Volterra nonlinear equalizer (VNE). We introduce the rules to train and test the data while using NN-based equalizer to guarantee a fair comparison with FFE and VNE. Random data have to be used for training, but for test, both random data and pseudorandom bit sequence are applicable. We found that the NN-based equalizer has the same performance with FFE and VNE in the case of linear distortion, but outperforms them in a strong nonlinearity case. In the experiment, to improve the loss budget, we increase the launch power to 18 dBm, achieving a 30-dB loss budget for 33 GBd/s PAM8 signal at the system frequency response of 16.2 GHz, attributed to the strong nonlinear equalization capability of NN. [ABSTRACT FROM AUTHOR]

Machine learning algorithms can have difficulties adapting to data from different sources, for example from different imaging modalities. We present and analyze three techniques for unsupervised cross-modality feature learning, using a shared autoencoder-like convolutional network that learns a common representation from multi-modal data. We investigate a form of feature normalization, a learning objective that minimizes cross-modality differences, and modality dropout, in which the network is trained with varying subsets of modalities. We measure the same-modality and cross-modality classification accuracies and explore whether the models learn modality-specific or shared features. This paper presents experiments on two public data sets, with knee images from two MRI modalities, provided by the Osteoarthritis Initiative, and brain tumor segmentation on four MRI modalities from the BRATS challenge. All three approaches improved the cross-modality classification accuracy, with modality dropout and per-feature normalization giving the largest improvement. We observed that the networks tend to learn a combination of cross-modality and modality-specific features. Overall, a combination of all three methods produced the most cross-modality features and the highest cross-modality classification accuracy, while maintaining most of the same-modality accuracy. [ABSTRACT FROM AUTHOR]

Deep learning (DL) aims at learning the meaningful representations. A meaningful representation gives rise to significant performance improvement of associated machine learning (ML) tasks by replacing the raw data as the input. However, optimal architecture design and model parameter estimation in DL algorithms are widely considered to be intractable. Evolutionary algorithms are much preferable for complex and nonconvex problems due to its inherent characteristics of gradient-free and insensitivity to the local optimal. In this paper, we propose a computationally economical algorithm for evolving unsupervised deep neural networks to efficiently learn meaningful representations, which is very suitable in the current big data era where sufficient labeled data for training is often expensive to acquire. In the proposed algorithm, finding an appropriate architecture and the initialized parameter values for an ML task at hand is modeled by one computational efficient gene encoding approach, which is employed to effectively model the task with a large number of parameters. In addition, a local search strategy is incorporated to facilitate the exploitation search for further improving the performance. Furthermore, a small proportion labeled data is utilized during evolution search to guarantee the learned representations to be meaningful. The performance of the proposed algorithm has been thoroughly investigated over classification tasks. Specifically, error classification rate on MNIST with 1.15% is reached by the proposed algorithm consistently, which is considered a very promising result against state-of-the-art unsupervised DL algorithms. [ABSTRACT FROM AUTHOR]