Search

Due to the increasing size of integrated circuits (ICs), their design and optimization phases (i.e., computer-aided design, CAD) grow increasingly complex. At design time, a large design space needs to be explored to find an implementation that fulfills all specifications and then optimizes metrics like energy, area, delay, reliability, etc. At run time, a large configuration space needs to be searched to find the best set of parameters (e.g., voltage/frequency) to further optimize the system. Both spaces are infeasible for exhaustive search typically leading to heuristic optimization algorithms that find some tradeoff between design quality and computational overhead. Machine learning (ML) can build powerful models that have successfully been employed in related domains. In this survey, we categorize how ML may be used and is used for design-time and run-time optimization and exploration strategies of ICs. A metastudy of published techniques unveils areas in CAD that are well explored and underexplored with ML, as well as trends in the employed ML algorithms. We present a comprehensive categorization and summary of the state of the art on ML for CAD. Finally, we summarize the remaining challenges and promising open research directions. [ABSTRACT FROM AUTHOR]

Presents an approach to the controllable simulation of paper forming using artificial neural network methods. Problem of simulating the dynamics of paper forming; Model for simulation of the dynamics of paper forming; Artificial neural networks and simulation of paper forming; Experimental results of the study; Discussion of the results; Conclusions.

This paper presents a study of the impact of two important parameters, moisture and aging of the oil/paper dielectric used as insulation in power transformers.The way in which these two parameters influence different parameters of the Frequency Domain Spectroscopy (FDS) measurements, is emphasized.Different FDS parameters were measured by varying the moisturecontent and the aging degree of the oil impregnated paper.The use of two types of neural networks for analysis of the results was necessary in order to help discriminating the impact of moisture and aging on the FDS measurements and, in some cases, to estimate the aging duration of the paper impregnated with oil. [ABSTRACT FROM PUBLISHER]

Presents information on a study which described the application of neural network techniques to the papermaking industry. Information on the database provided by Tillis Russell for the study; Preprocessing and coding of data; Neural-network training; Results of the study; Conclusion.

Describes the neural-based recognition and verification techniques used in a banknote machine implemented for accepting paper currency of different countries. Basis of the perception mechanism; Sensors for banknote perception; Connectionist models of banknote verification; Results showing the effectiveness of neural technologies.

In this paper, we develop a method for recognizing face images by combining wavelet decomposition, Fisherface method, and fuzzy integral. The proposed approach is comprised of four main stages. The first stage uses the wavelet decomposition that helps extract Intrinsic features of face images. As a result of this decomposition, we obtain four subimages (namely approximation, horizontal, vertical, and diagonal detailed images). The second stage of the approach concerns the application of the Fisherface method to these four decompositions. The choice of the Fisherface method in this setting is motivated by its insensitivity to large variation in light direction, face pose, and facial expression. The two last phases are concerned with the aggregation of the individual classifiers by means of the fuzzy integral. Both Sugeno and Choquet type of fuzzy integral are considered as the aggregalion method. In the experiments we use n-fold cross-validation to assure high consistency of the produced classification outcomes. The experimental results obtained for the Chungbuk National University (CNU) and Yale University face databases reveal that the approach presented in this paper yields better classification performance in comparison to the results obtained by other classifiers. [ABSTRACT FROM AUTHOR]

This paper presents a novel framework to recover detailed avatar from a single image. It is a challenging task due to factors such as variations in human shapes, body poses, texture, and viewpoints. Prior methods typically attempt to recover the human body shape using a parametric-based template that lacks the surface details. As such resulting body shape appears to be without clothing. In this paper, we propose a novel learning-based framework that combines the robustness of the parametric model with the flexibility of free-form 3D deformation. We use the deep neural networks to refine the 3D shape in a Hierarchical Mesh Deformation (HMD) framework, utilizing the constraints from body joints, silhouettes, and per-pixel shading information. Our method can restore detailed human body shapes with complete textures beyond skinned models. Experiments demonstrate that our method has outperformed previous state-of-the-art approaches, achieving better accuracy in terms of both 2D IoU number and 3D metric distance. [ABSTRACT FROM AUTHOR]

This paper serves as a survey of recent advances in large margin training and its theoretical foundations, mostly for (nonlinear) deep neural networks (DNNs) that are probably the most prominent machine learning models for large-scale data in the community over the past decade. We generalize the formulation of classification margins from classical research to latest DNNs, summarize theoretical connections between the margin, network generalization, and robustness, and introduce recent efforts in enlarging the margins for DNNs comprehensively. Since the viewpoint of different methods is discrepant, we categorize them into groups for ease of comparison and discussion in the paper. Hopefully, our discussions and overview inspire new research work in the community that aim to improve the performance of DNNs, and we also point to directions where the large margin principle can be verified to provide theoretical evidence why certain regularizations for DNNs function well in practice. We managed to shorten the paper such that the crucial spirit of large margin learning and related methods are better emphasized. [ABSTRACT FROM AUTHOR]

It is known that the determination of the good-dice-in-bad-neighborhoods (GDBNs) has been regarded as an effective technique to reduce the value of the defect parts per million (DPPM) by identifying and rejecting the suspicious dice even though they are good in testing. Instead of examining eight immediate neighbors in a small-sized $3\times 3$ window or exploiting simple linear regression, a large-sized window can be used to recognize the broad-sighted neighborhoods and accurately infer the suspiciousness level for any given die. In this paper, the artificial neural networks (ANN)-based method can be proposed to solve the GDBN identification. Furthermore, two enhanced techniques can be further presented to improve the inference accuracy of the original ANN-based method by considering the variation of the time-dependent wafer patterns and the wafer-to-wafer relationship between two adjacent wafers. After applying the two enhanced techniques, the business profits can be improved in the new ANN-based method. Various experiments on two datasets clearly reveal the superiority of the proposed ANN-based method over the other existing methods. In addition to the reduction of the DPPM value, the new ANN-based method can achieve the 1.5X–2X better reduction in the cost of the return merchandise authorization (RMA). On the other hand, the experimental results show that the similar result can also be obtained in the other lower-yield products. By using the new ANN-based method, the relationships on bad dice cross wafers can be captured and the highly-accurate inference results can be simultaneously maintained. [ABSTRACT FROM AUTHOR]

Unmanned Aerial Vehicle (UAV) inspection has gradually replaced manual inspection of transmission tower, which produces many images. While it is laborious and time-consuming to manually analyze these images, there are also challenges in automatically detecting small-size defective components such as bolts in transmission tower images, due to problems including complex background, small size, and many similar objects of bolts. In this paper, by virtue of multi-scale features and context information, we propose a deep neural network named Camp-Net (Context Information and Multi-Scale Pyramid Network) to identify bolts defect in transmission tower images. First, multi-scale feature fusion combines deep features and shallow features in convolutional networks to detect small-size bolts. Second, context information fusion puts the information around bolts into the detection network to remove the disturbance of complex background and similar objects. An image dataset containing defective bolts and normal bolts is constructed for model training and testing. Experimental results show that bolts with loose pins and bolts without pins among fittings in transmission tower can be accurately identified with the proposed model. The Average Precision (AP) of defective bolts detection of this model can be 11.4% higher than that of the commonly used high performance model, Faster R-CNN. [ABSTRACT FROM AUTHOR]

Lists names of various papers, published in the journal "IEE Transaction on Neutral Networks" as of September 01, 2003, which exclusively deal with neural network system and the related works on it.

This paper investigates the output consensus control problem of uncertain second-order nonlinear multiagent systems with unknown nonlinear dead zone, where the communication networks can be described as a directed graph with a fixed topology. The control input in each follower is assumed to be subject to unknown nonlinear dead zone. By the function approximation capability of neural networks (NNs), a distributed adaptive NN-based controller is designed for each follower in the graph such that all the outputs of the followers can asymptotically synchronize to the output of the leader with synchronization errors being semiglobally uniformly ultimately bounded. Based on Lyapunov stability theory and algebraic graph theory, analysis is conducted on stability and parameter convergence of the proposed algorithm. Finally, an example is provided to validate the theoretical results. [ABSTRACT FROM PUBLISHER]

The optimal scheduling of interfering links in a dense wireless network with full frequency reuse is a challenging task. The traditional method involves first estimating all the interfering channel strengths and then optimizing the scheduling based on the model. This model-based method is, however, resource intensive and computationally hard because channel estimation is expensive in dense networks; furthermore, finding even a locally optimal solution of the resulting optimization problem may be computationally complex. This paper shows that by using a deep learning approach, it is possible to bypass the channel estimation and to schedule links efficiently based solely on the geographic locations of the transmitters and the receivers due to the fact that in many propagation environments, the wireless channel strength is largely a function of the distance-dependent path-loss. This is accomplished by unsupervised training over randomly deployed networks and by using a novel neural network architecture that computes the geographic spatial convolutions of the interfering or interfered neighboring nodes along with subsequent multiple feedback stages to learn the optimum solution. The resulting neural network gives a near-optimal performance for sum-rate maximization and is capable of generalizing to larger deployment areas and to deployments of different link densities. Moreover, to provide fairness, this paper proposes a novel scheduling approach that utilizes the sum-rate optimal scheduling algorithm over judiciously chosen subsets of links for maximizing a proportional fairness objective over the network. The proposed approach shows highly competitive and generalizable network utility maximization results. [ABSTRACT FROM AUTHOR]

This paper proposes Value-Aware solid-state drive (SSD) with fast access speed and low power consumption by eliminating error-correcting code (ECC). Value-Aware SSD utilizes the error tolerance of image recognition application using a deep neural network (DNN) to enhance reliability. In a previous paper, which proposes Value-Aware SSD, fast ECC decoder is implemented and SSD is evaluated with the 32-bit floating-point data format. On the other hand, in this paper, the proposed Value-Aware SSD is analyzed with 32-bit and 8-bit fixed-point data format and achieves the higher reliability even without ECC by newly proposed two techniques, Critical Bit Error Reduction (CBER) and Middle & Lower Page Error Reduction (M&L-PER). CBER and M&L-PER are proposed for 32-bit and 8-bit data format of application, respectively. These techniques modulate the threshold voltage ($V_{\mathrm {TH}}$) distribution of memory cells by recognizing the importance of each stored bit. By proposed CBER, as much as 15% bit error rate (BER) of NAND flash is allowed while the application provides high image recognition accuracy. Even if bit precision is truncated to 8 bit, 3.9% BER is accepted by M&L-PER. The fast read access and low power consumption are realized because ECC is not required. Finally, this paper analyzes the Value-Aware techniques with 3-D multi-level cell (MLC) NAND flash to compare the effects for 3D-MLC and triple-level cell (TLC) NAND flash. [ABSTRACT FROM AUTHOR]

Resistive random access memory (ReRAM) is often considered as a strong candidate for storing the weights in non-von Neumann neuromorphic computing systems. This paper studies how nonideal memory characteristics, which include programing error, read fluctuation, and retention, affect the inference accuracy of the analog ReRAM neural networks by incorporating memory characteristics extracted from 1-Mb ReRAM into a simulated inference-only neural network. This paper also shows that the different layer in the network can tolerate different amount of such imperfects. We learned four key points: 1) the conductance range of memory with less relative fluctuation is preferred for designing the weight-conductance mapping; 2) the control of programing error is essential for high inference accuracy; 3) retention-induced conductance drift can be fatal to the neuromorphic system. A compensation scheme is proposed in this paper which can effectively recover the inference accuracy; and 4) for multilayer networks, avoiding weight errors in the front layers can help to maintain the inference accuracy by reducing calculation error which may otherwise accumulate and pass down the networks. The concepts and approaches of this paper can also be applied to evaluate other types of nonvolatile memories for artificial neural networks. [ABSTRACT FROM AUTHOR]

Human beings tend to learn unknown knowledge in a gradual process, from the basic to the complex. Based on this point, we propose a progressive learning method for producing service strategies according to requests, with a hierarchical priori knowledge and reinforcement learning. Service strategy aims to guide how to perform home services and takes into consideration the relationship between actions and objects in home environment. In this paper, strategy generation is regarded as a text generation problem in question answering (QA). Firstly, a hierarchical priori knowledge with service-object correlation at the bottom and action-object correlation at the top is constructed to assist the understanding on the relationship of objects and actions in service strategies. Service-object correlation guides how to select proper objects with the correct order, while action-object correlation associates actions in strategies according to selected objects. Based on the hierarchical priori knowledge, a progressive learning method is proposed to make the model produce effective strategies with a sequential cognition, from service-object correlation (objects) to action-object correlation (actions). After that, reinforcement learning is employed to enhance the progressive guidance, by designing rewards in terms of the hierarchical priori knowledge. Finally, the proposed method is tested with both comparative experiments and ablation studies, and the experimental results demonstrate the superiority in producing comprehensive and logical strategies, indicating that the progressive learning method in our paper can further improve the QA performance. [ABSTRACT FROM AUTHOR]

To accurately pour drinks into various containers is an essential skill for service robots. However, drink pouring is a dynamic process and difficult to model. Traditional deep imitation learning techniques for implementing autonomous robotic pouring have an inherent black-box effect and require a large amount of demonstration data for model training. To address these issues, an Explainable Hierarchical Imitation Learning (EHIL) method is proposed in this paper such that a robot can learn high-level general knowledge and execute low-level actions across multiple drink pouring scenarios. Moreover, with the EHIL method, a logical graph can be constructed for task execution, through which the decision-making process for action generation can be made explainable to users and the causes of failure can be traced out. Based on the logical graph, the framework is manipulable to achieve different targets while the adaptability to unseen scenarios can be achieved in an explainable manner. A series of experiments have been conducted to verify the effectiveness of the proposed method. Results indicate that EHIL outperforms the traditional behavior cloning method in terms of success rate, adaptability, manipulability, and explainability. Note to Practitioners—Pouring liquids is a common activity in people’s daily lives and all wet-lab industries. Drink pouring dynamic control is difficult to model, while the accurate perception of flow is challenging. To enable the robot to learn under unknown dynamics via observing the human demonstration, deep imitation learning can be used. To address the limitations of traditional deep neural networks, an Explainable Hierarchical Imitation Learning (EHIL) method is proposed in this paper. The proposed method enables the robot to learn a sequence of reasonable pouring phases for performing the task rather than simply execute the task via traditional behavior cloning. In this way, explainability and safety can be ensured. Manipulability can be achieved by reconstructing the logical graph. The target of this research is to obtain pouring dynamics via the learning method and realize the precise and quick pouring of drink from the source containers to various targeted containers with reliable performance, adaptability, manipulability, and explainability. [ABSTRACT FROM AUTHOR]

The variety of arc fault induced by different load types makes residential series arc fault detection complicated and challengeable. This paper proposes a hybrid time and frequency analysis and fully connected neural network (HTFNN) based method to identify series ac arc fault. The HTFNN method recognizes samples state by broadly classifying them into resistive (Re) category, capacitive–inductive (CI) category, and switching (Sw) category according to the fundamental frequency components of series current. In each category, separate fully connected neural network (NN) with customized time and frequency indicators being the input is employed for fine class recognition and state identification. This method construction addresses the feature overlap among various cases by separate categories and enables suitable indicator selection within each category. Since the identification complexity in each category is cheaper than identifying all cases together, concise NNs are applied in this paper to reduce computing cost. The general accuracies of normal and arcing state identification in Re, CI, and Sw category are 99.64%, 100%, and 98.45%, respectively, based on 3950 test samples. Method comparison shows that HTFNN achieves higher detection precision at lower computational complexity for generalized load types. This paper also evaluates the feasibility of implementing the method in hardware for arc fault detection device application. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs), including convolutional neural networks (CNNs) and residual networks (ResNets) models, are able to learn abstract representations from the input data by considering a deep hierarchy of layers that perform advanced feature extraction. The combination of these models with visual attention techniques can assist with the identification of the most representative parts of the data from a visual standpoint, obtained through more detailed filtering of the features extracted by the operational layers of the network. This is of significant interest for analyzing remotely sensed hyperspectral images (HSIs), characterized by their very high spectral dimensionality. However, few efforts have been conducted in the literature in order to adapt visual attention methods to remotely sensed HSI data analysis. In this paper, we introduce a new visual attention-driven technique for the HSI classification. Specifically, we incorporate attention mechanisms to a ResNet in order to better characterize the spectral–spatial information contained in the data. Our newly proposed method calculates a mask that is applied to the features obtained by the network in order to identify the most desirable ones for classification purposes. Our experiments, conducted using four widely used HSI data sets, reveal that the proposed deep attention model provides competitive advantages in terms of classification accuracy when compared to other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

To convey neural network architectures in publications, appropriate visualizations are of great importance. While most current deep learning papers contain such visualizations, these are usually handcrafted just before publication, which results in a lack of a common visual grammar, significant time investment, errors, and ambiguities. Current automatic network visualization tools focus on debugging the network itself and are not ideal for generating publication visualizations. Therefore, we present an approach to automate this process by translating network architectures specified in Keras into visualizations that can directly be embedded into any publication. To do so, we propose a visual grammar for convolutional neural networks (CNNs), which has been derived from an analysis of such figures extracted from all ICCV and CVPR papers published between 2013 and 2019. The proposed grammar incorporates visual encoding, network layout, layer aggregation, and legend generation. We have further realized our approach in an online system available to the community, which we have evaluated through expert feedback, and a quantitative study. It not only reduces the time needed to generate network visualizations for publications, but also enables a unified and unambiguous visualization design. [ABSTRACT FROM AUTHOR]

A problem of image denoising, when images are corrupted by a non-stationary noise, is considered in this paper. Since, in practice, no a priori information on noise is available, noise statistics should be pre-estimated prior to image denoising. In this paper, deep convolutional neural network (CNN) based method for estimation of a map of local, patch-wise, standard deviations of noise (so-called sigma-map) is proposed. It achieves the state-of-the-art performance in accuracy of estimation of sigma-map for the case of non-stationary noise, as well as estimation of a noise variance for the case of an additive white Gaussian noise. Extensive experiments on image denoising using estimated sigma-maps demonstrate that our method outperforms recent CNN-based blind image denoising methods by up to 6 dB in PSNR, as well as other state-of-the-art methods based on sigma-map estimation by up to 0.5 dB, providing, at the same time, better usage flexibility. A comparison with the ideal case, when denoising is applied using ground-truth sigma-map, shows that a difference of corresponding PSNR values for the most of noise levels is within 0.1-0.2 dB, and does not exceed 0.6 dB. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new state representation method, called encoding sum and concatenation (ESC), to describe the environment observation for decision-making in autonomous driving. Unlike existing state representation methods, ESC is applicable to the situation where the number of surrounding vehicles is variable and eliminates the need for manually pre-designed sorting rules, leading to higher representation ability and generality. The proposed ESC method introduces a feature neural network (NN) to encode the real-valued feature of each surrounding vehicle into an encoding vector, and then adds these vectors up to obtain the representation vector of the set of surrounding vehicles. Then, a fixed-dimensional and permutation-invariance state representation can be obtained by concatenating the set representation with other variables, such as indicators of the ego vehicle and road. By introducing the sum-of-power mapping, this paper has further proved that the injectivity of the ESC state representation can be guaranteed if the output dimension of the feature NN is greater than the number of variables of all surrounding vehicles. This means that the ESC representation can be used to describe the environment and taken as the inputs of learning-based policy functions. Experiments demonstrate that compared with the fixed-permutation representation method, the policy learning accuracy based on ESC representation is improved by 62.2%. [ABSTRACT FROM AUTHOR]

Network data in real-world tends to be error-prone due to incomplete sampling, imperfect measurements, etc.; this in turn results in inaccurate results when performing network analysis or modeling, such as node classification and link prediction, on these flawed networks. In this paper, we aim to reconstruct a reliable network from a flawed, undirected, unweighted network, a process referred to network enhancement. More specifically, network enhancement aims to detect the noisy links that are observed in the network but should not exist in the real world, as well as to predict the missing links that do indeed exist in the real world yet remain unobserved. While some attempts have been made to detect either noisy links or missing links, few of these works have considered unifying these two tasks, even though they are inter-dependent and capable of mutually boosting each others’ performance. In this paper, we therefore propose E-Net, an end-to-end graph neural network model, to leverage the mutual influence of these two tasks in order to achieve both goals more effectively. On one hand, detecting noisy links can benefit the performance of missing link prediction, while on the other hand, predicting missing links can provide indirect supervision for detecting noisy link detection when the labels of these noisy links are unavailable. Moreover, by proposing a subgraph extraction mechanism based on random walk with restart, the model can be scaled up to large networks and is able to preserve the local and global structural characteristics. The experimental results on several types of large networks demonstrate that the proposed model obtains an improvement of 10.7 percent on average in terms of F1 for predicting missing links, along with an average of 3.7 percent improvement in terms of precision for detecting noisy links compared with the state-of-the-art baselines. [ABSTRACT FROM AUTHOR]

This paper proposes to administer a multi-step artificial intelligence approach with an ensemble of adaptive neural networks (NNs) trained on 50000 samples to identify partial discharge (PD) diagnostic measurements for in-service medium voltage (MV) power cables. To evaluate the performance of the algorithm, a case study was performed on cables deliberately selected to contain both uncomplicated measurements and disruptive irregularities representative of conditions during field testing. Experimental test results prove that the proposed cable-specific adaptation improves PD identification accuracy, with further increment through the NN ensembles. The main contribution of the approach is in both the cable-specific adaption and the NN ensemble being applied to MV cable field measurements. [ABSTRACT FROM AUTHOR]

There is an opportunity in modern power systems to explore the demand flexibility by incentivizing consumers with dynamic prices. In this paper, we quantify demand flexibility using an efficient tool called time-varying elasticity, whose value may change depending on the prices and decision dynamics. This tool is particularly useful for evaluating the demand response potential and system reliability. Recent empirical evidences have highlighted some abnormal features when studying demand flexibility, such as delayed responses and vanishing elasticities after price spikes. Existing methods fail to capture these complicated features because they heavily rely on some predefined (often over-simplified) regression expressions. Instead, this paper proposes a model-free methodology to automatically and accurately derive the optimal estimation pattern. We further develop a two-stage estimation process with Siamese long short-term memory (LSTM) networks. Here, a LSTM network encodes the price response, while the other network estimates the time-varying elasticities. In the case study, the proposed framework and models are validated to achieve higher overall estimation accuracy and better description for various abnormal features when compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

This paper deals with the problem of controller design for normal nonlinear switched systems with unknown dynamics. The situation assumes that the nonlinear component of the system dynamics is unknown and there are extra uncertain terms and external perturbations affecting the system responses. Unlike previous methods that usually apply artificial neural networks to obtain an approximate model for an unknown dynamics, we use simple adaptive laws to surmount such fluctuations as well as to circumvent the steady state errors in the approaches provided by neural networks or fuzzy logics. No information on the bounds and parameters of the uncertain parts is needed. We design a twofold sliding mode control methodology to stabilize the whole switched system with arbitrary switching signals. To handle the consequences of the inherent chattering, a smooth integral type control signal is constructed. It is proved that once the system states attain the developed sliding manifold, the system becomes insensitive to both the lumped uncertainties and the dynamics of the switched system. The common assumption of a known Lyapunov functions for the subsystems is relaxed and the derived adaptive approach provides nonconservative circumstances for guaranteeing the global stability of the equilibrium state. The theoretical results of this paper are generalized for the canonical nonlinear systems with mismatched uncertainties and a novel adaptive sliding manifold is proposed to suppress the effects of the system mismatching uncertain parts. A comparative computer simulation is developed to examine the effective performance of the introduced controllers. [ABSTRACT FROM AUTHOR]

This paper focuses on the shingled magnetic recording (SMR) as the recording system for further high-density hard disk drives. Previously, we applied a log-likelihood ratio (LLR) modulator to the low-density parity check coding and iterative decoding system in SMR to improve the decoding performance. In this paper, we propose a new LLR modulator using neural network. Furthermore, we clarify that the proposed LLR modulator using neural network realizes effective iterative decoding in a read/write channel with pattern-dependent medium noise. [ABSTRACT FROM AUTHOR]

This paper investigates the feasibility of novel data-driven deep learning (DL) models to predict the solution of Maxwell’s equations for low-frequency electromagnetic (EM) devices. With ground truth (empirical evidence) data being generated from a finite-element analysis solver, a deep convolutional neural network is trained in a supervised manner to learn a mapping for magnetic field distribution for topologies of different complexities of geometry, material, and excitation, including a simple coil, a transformer, and a permanent magnet motor. Preliminary experiments show DL model predictions in close agreement with the ground truth. A probabilistic model is introduced to improve the accuracy and to quantify the uncertainty in the prediction, based on Monte Carlo dropout. This paper establishes a basis for a fast and generalizable data-driven model used in the analysis, design, and optimization of EM devices. [ABSTRACT FROM AUTHOR]

In this paper, a model identification method based on a long short-term memory (LSTM) neural network composed of a network structure and training algorithm is used to build a thermal field model that accurately simulates the crystal growth process. The support vector machine (SVM) approach is then adopted to identify model order and lag to determine network input and to improve precision. The thermal field model reflecting the growth process in the Czochralski crystal furnace is simulated. Experimental results and comparative analysis results both suggest that the method proposed by this paper can build an efficient thermal field model which outperforms other methods in terms of precision. [ABSTRACT FROM AUTHOR]

Linear spectral unmixing is the practice of decomposing the mixed pixel into a linear combination of the constituent endmembers and the estimated abundances. This paper focuses on unsupervised spectral unmixing where the endmembers are unknown a priori. Conventional approaches use either geometrical- or statistical-based approaches. In this paper, we address the challenges of spectral unmixing with unsupervised deep learning models, in specific, the autoencoder models, where the decoder serves as the endmembers and the hidden layer output serves as the abundances. In several recent attempts, part-based autoencoders have been designed to solve the unsupervised spectral unmixing problem. However, the performance has not been satisfactory. In this paper, we first discuss some important findings we make on issues with part-based autoencoders. By proof of counterexample, we show that all existing part-based autoencoder networks with nonnegative and tied encoder and decoder are inherently defective by making these inappropriate assumptions on the network structure. As a result, they are not suitable for solving the spectral unmixing problem. We propose a so-called untied denoising autoencoder with sparsity, in which the encoder and decoder of the network are independent, and only the decoder of the network is enforced to be nonnegative. Furthermore, we make two critical additions to the network design. First, since denoising is an essential step for spectral unmixing, we propose to incorporate the denoising capacity into the network optimization in the format of a denoising constraint rather than cascading another denoising preprocessor in order to avoid the introduction of additional reconstruction error. Second, to be more robust to the inaccurate estimation of a number of endmembers, we adopt an $l_{21}$ -norm on the encoder of the network to reduce the redundant endmembers while decreasing the reconstruction error simultaneously. The experimental results demonstrate that the proposed approach outperforms several state-of-the-art methods, especially for highly noisy data. [ABSTRACT FROM AUTHOR]

The earth’s magnetosphere is an intricate input–output system, where the inputs are the solar wind parameters and the output measure are the geomagnetic. The disturbance storm time (DST) index is one such quantity measure for the intensity of disturbance in the geomagnetic field due to the magnetic storm. The geomagnetic activities have catastrophic effects on several communication networks and harmful for biological livings. Therefore, it is significant to develop a model for the prediction of disturbance in the geomagnetic field. This paper presents linear as well as nonlinear parametric techniques to model the complex magnetosphere dynamics. Neural network (NN)-based modeling techniques, such as feed-forward NN (FFNN), NN integrated with nonlinear autoregression with exogenous (NARX) inputs, adaptive neuro-fuzzy inference system (ANFIS), and recurrent NN are developed for the prediction of the magnetic storm. Global search algorithms, such as particle swarm optimization and genetic algorithm, also train the developed FFNNs. All the models are trained on 15 years’ real-time series data of solar wind parameters and DST index on a frequency of 1 h. The accuracy of the predicted DST index for multiple intensity levels of the magnetic storm using all developed linear and nonlinear modeling techniques is presented, compared, and analyzed thoroughly. The performance of NN integrated with NARX inputs and ANFIS is higher than rest of methods developed in the paper. Both techniques have the capability to predict mild, moderate, and intense magnetic storm with high degree of accuracy. The comparative analysis with state of the art shows the enhanced accuracy and robustness of developed models in this paper. [ABSTRACT FROM AUTHOR]

Object detection is important in car sharing services. Accuracy, efficiency, and low memory consumption are desirable for object detection in car sharing services. This paper presents a network system that satisfies all these requirements. Our approach first divides the object detection task into multiple simpler local regression tasks. Then, we propose the generalized Haar filter-based convolutional neural network to reduce the consumption of memory and computing resource. To achieve real-time performance, we introduce a sparse window generation strategy to reduce the number of input image patches without sacrificing accuracy. We perform experiments on both vehicle and pedestrian data sets. Experimental results demonstrate that our approach can accurately detect objects under challenging conditions. Note to Practitioners—Object detection is an important part of intelligent vehicle technologies, which play an important role in car sharing services. Object detection provides metadata for collision avoidance, self-driving systems, and driver-assistance systems, which can result in better safety and consumer experiences in car sharing services. Although deep learning has achieved an excellent performance in object detection, they consume a large amount of storage and computing resource, which makes them difficult to be deployed for car sharing services. This paper suggests a novel approach which is based on the generalized Haar filter and the local regression strategy. Our approach is accurate, efficient, and light. The experimental results verify the effectiveness of the proposed approach in car sharing services. [ABSTRACT FROM AUTHOR]

This paper presents an efficient method, which reconstructs the temperature field around the tool/chip interface from infrared (IR) thermal images, for online monitoring the internal peak temperature of the cutting tool. The tool temperature field is divided into two regions; namely, a far field for solving the heat-transfer coefficient between the tool and ambient temperature, and a near field where an artificial neural network (ANN) is trained to account for the unknown heat variations at the frictional contact interface. Methods to extract physics-based feature points from the IR image as ANN inputs are discussed. The effects of image resolution, feature selection, chip occlusion, contact heat variation, and measurement noises on the estimated contact temperature are analyzed numerically and experimentally. The proposed method has been verified by comparing the ANN-estimated surface temperatures against “true values” experimentally obtained using a high-resolution IR imager on a custom-designed testbed as well as numerically simulated using finite-element analysis. The concept feasibility of the temperature monitoring method is demonstrated on an industrial lathe-turning center with a commercial tool insert. Note to Practitioners—The internal tool-temperature field around the tool/chip interface during cutting offers essential information to monitor tool-wear and ensure surface-quality particularly in finishing cuts, which is difficult to be monitored because of the stringent real-time requirements (including low cost and high accuracy) in harsh working environments. This paper presents a potentially low-cost solution that combines noncontact surface temperature measurements with high-fidelity physics-based computational models for monitoring the internal peak temperature at the tool/chip frictional contact during cutting. This physics-based method requires only a small number of preselected features, and uses thermal isotherms and streamlines to detect and substitute any occlusions in the IR images. The method does not rely on high-resolution (HR) images to infer the steep temperature gradient near the tool-tip where the peak temperature occurs, and thus can be implemented with a relatively low-cost IR imager. Unlike traditional least-square methods that bases solutions to a heat-conduction partial differential equation (PDE), the ANN is trained with precomputed physics-based models based on a novel dual-field (far-field and near-field) approach. The ANN-based method not only eliminates the need to solve the time-demanding PDE in real time, but also effectively accounts for parameter variations due to the uncertain frictional heat-fluxes at the contact interface during cutting. [ABSTRACT FROM AUTHOR]

Most of the existing single object trackers track the target in a unitary local search window, making them particularly vulnerable to challenging factors such as heavy occlusions and out-of-view movements. Despite the attempts to further incorporate global search, prevailing mechanisms that cooperate local and global search are relatively static, thus are still sub-optimal for improving tracking performance. By further studying the local and global search results, we raise a question: can we allow more dynamics for cooperating both results? In this paper, we propose to introduce more dynamics by devising a dynamic attention-guided multi-trajectory tracking strategy. In particular, we construct dynamic appearance model that contains multiple target templates, each of which provides its own attention for locating the target in the new frame. Guided by different attention, we maintain diversified tracking results for the target to build multi-trajectory tracking history, allowing more candidates to represent the true target trajectory. After spanning the whole sequence, we introduce a multi-trajectory selection network to find the best trajectory that deliver improved tracking performance. Extensive experimental results show that our proposed tracking strategy achieves compelling performance on various large-scale tracking benchmarks. The project page of this paper can be found at https://sites.google.com/view/mt-track/. [ABSTRACT FROM AUTHOR]

Correspondence analysis (CA) is a multivariate statistical tool used to visualize and interpret data dependencies by finding maximally correlated embeddings of pairs of random variables. CA has found applications in fields ranging from epidemiology to social sciences. However, current methods for CA do not scale to large, high-dimensional datasets. In this paper, we provide a novel interpretation of CA in terms of an information-theoretic quantity called the principal inertia components. We show that estimating the principal inertia components, which consists in solving a functional optimization problem over the space of finite variance functions of two random variable, is equivalent to performing CA. We then leverage this insight to design algorithms to perform CA at scale. Specifically, we demonstrate how the principal inertia components can be reliably approximated from data using deep neural networks. Finally, we show how the maximally correlated embeddings of pairs of random variables in CA further play a central role in several learning problems including multi-view and multi-modal learning methods and visualization of classification boundaries. [ABSTRACT FROM AUTHOR]

Some electromagnetic pulses, such as the high-altitude electromagnetic pulse (HEMP), can couple with power lines in the range of thousands of kilometers. Conventional numerical methods cannot efficiently predict the voltage induced by the early-time (E1) HEMP on the power lines in a large area. This paper presents an efficient model based on an artificial neural network (ANN) which can predict the coupled voltage induced by the E1 HEMP on distribution power lines quickly. Based on Agrawal's model of the electromagnetic field coupling with transmission lines, the important parameters which affect the coupled voltages of distribution power lines are analyzed firstly. The Baum-Liu-Tesche equation and inverse Fourier transform is applied to compute the voltage magnitude when the elevation angle, azimuth angle, and the polarization angle of the E1 HEMP change and more than twenty thousand numerical results are obtained. Then an ANN is developed and trained by the numerical results. Once the ANN is trained, the prediction time is on the order of milliseconds. The results show that the proposed model based on the ANN can provide accurate prediction results for the E1 HEMP coupling with power lines at different observation points. [ABSTRACT FROM AUTHOR]

While recent developments of neural network (NN) models have led to a series of record-breaking achievements in many applications, the lack of sufficiently good datasets remains a problem for some applications. For such a problem, we can however exploit a large number of unstructured text corpora as an external knowledge to complement the training data, and most prevailing neural network solutions employ word embedding methods for such purposes. In this paper, we propose LDA-Reg, a novel knowledge driven regularization framework based on Latent Dirichlet Allocation (LDA) as an alternative to the word embedding methods to adaptively utilize abundant external knowledge and to interpret the NN model. For the joint learning of the parameters, we propose EM-SGD, an effective update method which incorporates Expectation Maximization (EM) and Stochastic Gradient Descent (SGD) to update parameters iteratively. Moreover, we also devise a lazy update and sparse update method for the high-dimensional inputs and sparse inputs respectively. We validate the effectiveness of our regularization framework through an extensive experimental study over real world and standard benchmark datasets. The results show that our proposed framework not only achieves significant improvement over state-of-the-art word embedding methods but also learns interpretable and significant topics for various tasks. [ABSTRACT FROM AUTHOR]

The paper proposes a novel block-wise prediction paradigm based on Convolutional Neural Networks (CNNs) for lossless video coding. A deep neural network model which follows a multi-resolution design is employed for block-wise prediction. Several contributions are proposed to improve neural network training. A first contribution proposes a novel loss function formulation for an efficient network training based on a new approach for patch selection. Another contribution consists in replacing all HEVC-based angular intra-prediction modes with a CNN-based intra-prediction method, where each angular prediction mode is complemented by a CNN-based prediction mode using a specifically trained model. Another contribution consists in an efficient adaptation of the CNN-based intra-prediction residual for lossless video coding. Experimental results on standard test sequences show that the proposed coding system outperforms the HEVC standard with an average bitrate improvement of around 5%. To our knowledge, the paper is the first to replace all the traditional HEVC-based angular intra-prediction modes with an intra-prediction method based on modern Machine Learning techniques for lossless video coding applications. [ABSTRACT FROM AUTHOR]

This paper aims to investigate how adequate cognitive functions for recognizing, predicting, and generating a variety of actions can be developed through iterative learning of action-caused dynamic perceptual patterns. Particularly, we examined the capabilities of mental simulation of one’s own actions as well as the inference of others’ intention because they play a crucial role, especially in social cognition. We propose a dynamic neural network model based on predictive coding which can generate and recognize dynamic visuo-proprioceptive patterns. The proposed model was examined by conducting a set of robotic simulation experiments in which a robot was trained to imitate visually perceived gesture patterns of human subjects in a simulation environment. The experimental results showed that the proposed model was able to develop a predictive model of imitative interaction through iterative learning of large-scale spatio-temporal patterns in visuo-proprioceptive input streams. Also, the experiment verified that the model was able to generate mental imagery of dynamic visuo-proprioceptive patterns without feeding the external inputs. Furthermore, the model was able to recognize the intention of others by minimizing prediction error in the observations of the others’ action patterns in an online manner. These findings suggest that the error minimization principle in predictive coding could provide a primal account for the mirror neuron functions for generating actions as well as recognizing those generated by others in a social cognitive context. [ABSTRACT FROM AUTHOR]

In this paper, the stability problem of delayed neural networks is investigated by adopting some switching methods. First, the delay interval is divided into many smaller variable intervals, and when the smaller variable interval is regarded as a mode of the delay, delayed neural networks are modeled as switched systems. Then, by using some switching methods, less conservative stability criteria are derived to ensure the stability of delayed neural networks. Finally, one example is provided to show the effectiveness of the obtained criteria. [ABSTRACT FROM AUTHOR]

Being able to detect, identify, and diagnose a fault is a key feature of industrial supervision systems, which enables advance asset management, in particular, predictive maintenance, which greatly increases efficiency and productivity. In this paper, an Industrial Internet app for real-time fault detection and diagnosis is implemented and tested in a pilot scale industrial motor. Real-time fault detection and identification is based on dynamic incremental principal component analysis (DIPCA) and reconstruction-based contribution (RBC). When the analysis indicates that one of the vibration measurements is responsible for the fault, a convolutional neural network (CNN) is used to identify the unbalance or bearing fault type. The application was evaluated in its three functionalities: fault detection, fault identification, and fault identification of vibration-related faults, yielding a fault detection rate over 99%, a false alarm rate below 5%, and an identification accuracy over 90%. Note to Practitioners—This paper focuses on designing and evaluating a real-time fault diagnosis application in an industrial setup. To this end, this paper also tackles the problem of developing a methodology for implementing advanced state-of-the-art fault detection techniques in real machinery, following industry standards and using a modern informatics architecture. The application here developed uses a statistical data-driven fault diagnosis technique, hence it requires a training stage using historical data to learn patterns and estimate parameters. A proof of concept in fault diagnosis for industrial motors is given; however, it should be noted that both the methodology and the deployed architecture are scalable and flexible enough to facilitate the implementation in other industrial environments. The implementation here presented was deployed using only open-source tools, which allows evaluating this tool without incurring in high expenses. [ABSTRACT FROM AUTHOR]

This paper proposes and experimentally evaluates a method, which is able to measure the refrigeration capacity of compressors in extremely short times. Traditionally, the measurement of this parameter is done using specialized and expensive test rigs and can take up to 4.5 h for the measurement of a single compressor. To overcome this long measurement time, it is proposed to use a simpler measuring system, in which, instead of measuring the refrigeration capacity directly, the rate of pressure rise imposed by the compressor to a cylinder with fixed volume is measured. This paper shows that both variables have a high correlation factor when the test and compressor conditions are taken into account. As the correlation among the variables is nonlinear, it is not possible to use simple mathematical correlation functions, so neural networks are used for that purpose. By using the proposed method, the refrigeration capacity of a compressor can be determined in less than 7 s, which is approximately 0.05% of the time required by traditional methods. To guarantee the metrological reliability of the results presented by the proposed neural model, this paper proposes a novel approach based on the Monte Carlo and the bootstrap methods. [ABSTRACT FROM AUTHOR]

Distribution system state estimation (DSSE) is a core task for monitoring and control of distribution networks. Widely used algorithms such as Gauss–Newton perform poorly with the limited number of measurements typically available for DSSE, often require many iterations to obtain reasonable results, and sometimes fail to converge. DSSE is a non-convex problem, and working with a limited number of measurements further aggravates the situation, as indeterminacy induces multiple global (in addition to local) minima. Gauss–Newton is also known to be sensitive to initialization. Hence, the situation is far from ideal. It is therefore natural to ask if there is a smart way of initializing Gauss–Newton that will avoid these DSSE-specific pitfalls. This paper proposes using historical or simulation-derived data to train a shallow neural network to “learn to initialize,” that is, map the available measurements to a point in the neighborhood of the true latent states (network voltages), which is used to initialize Gauss–Newton. It is shown that this hybrid machine learning/optimization approach yields superior performance in terms of stability, accuracy, and runtime efficiency, compared to conventional optimization-only approaches. It is also shown that judicious design of the neural network training cost function helps to improve the overall DSSE performance. [ABSTRACT FROM AUTHOR]

This paper proposes a complex-valued convolutional neural network for land form classification and discovery in interferometric synthetic aperture radar (InSAR). Since the amount of satellite-borne SAR data has been increasing drastically, it is necessary to structurize the local features contained in observation data prior to utilization in the so-called big data framework for higher usability. Convolutional neural networks have such potential in general. However, there exists no network that can deal with complex amplitude data obtained in InSAR consistently. In this paper, we propose a complex-valued convolutional neural network to deal with InSAR. We demonstrate that the network classifies slopes and plains adaptively and, moreover, indicates small volcanos similar to a sample volcano (Omuroyama) included in the InSAR data. We also find their characteristic features emerging in the kernels in the convolution layers. These results reveal that the proposed complex-valued convolutional neural network is capable of successfully discovering unidentified lands similar to a prepared sample, which is highly useful for the InSAR data structurization. [ABSTRACT FROM AUTHOR]