Search

Due to the increasing size of integrated circuits (ICs), their design and optimization phases (i.e., computer-aided design, CAD) grow increasingly complex. At design time, a large design space needs to be explored to find an implementation that fulfills all specifications and then optimizes metrics like energy, area, delay, reliability, etc. At run time, a large configuration space needs to be searched to find the best set of parameters (e.g., voltage/frequency) to further optimize the system. Both spaces are infeasible for exhaustive search typically leading to heuristic optimization algorithms that find some tradeoff between design quality and computational overhead. Machine learning (ML) can build powerful models that have successfully been employed in related domains. In this survey, we categorize how ML may be used and is used for design-time and run-time optimization and exploration strategies of ICs. A metastudy of published techniques unveils areas in CAD that are well explored and underexplored with ML, as well as trends in the employed ML algorithms. We present a comprehensive categorization and summary of the state of the art on ML for CAD. Finally, we summarize the remaining challenges and promising open research directions. [ABSTRACT FROM AUTHOR]

Unmanned Aerial Vehicle (UAV) inspection has gradually replaced manual inspection of transmission tower, which produces many images. While it is laborious and time-consuming to manually analyze these images, there are also challenges in automatically detecting small-size defective components such as bolts in transmission tower images, due to problems including complex background, small size, and many similar objects of bolts. In this paper, by virtue of multi-scale features and context information, we propose a deep neural network named Camp-Net (Context Information and Multi-Scale Pyramid Network) to identify bolts defect in transmission tower images. First, multi-scale feature fusion combines deep features and shallow features in convolutional networks to detect small-size bolts. Second, context information fusion puts the information around bolts into the detection network to remove the disturbance of complex background and similar objects. An image dataset containing defective bolts and normal bolts is constructed for model training and testing. Experimental results show that bolts with loose pins and bolts without pins among fittings in transmission tower can be accurately identified with the proposed model. The Average Precision (AP) of defective bolts detection of this model can be 11.4% higher than that of the commonly used high performance model, Faster R-CNN. [ABSTRACT FROM AUTHOR]

The optimal scheduling of interfering links in a dense wireless network with full frequency reuse is a challenging task. The traditional method involves first estimating all the interfering channel strengths and then optimizing the scheduling based on the model. This model-based method is, however, resource intensive and computationally hard because channel estimation is expensive in dense networks; furthermore, finding even a locally optimal solution of the resulting optimization problem may be computationally complex. This paper shows that by using a deep learning approach, it is possible to bypass the channel estimation and to schedule links efficiently based solely on the geographic locations of the transmitters and the receivers due to the fact that in many propagation environments, the wireless channel strength is largely a function of the distance-dependent path-loss. This is accomplished by unsupervised training over randomly deployed networks and by using a novel neural network architecture that computes the geographic spatial convolutions of the interfering or interfered neighboring nodes along with subsequent multiple feedback stages to learn the optimum solution. The resulting neural network gives a near-optimal performance for sum-rate maximization and is capable of generalizing to larger deployment areas and to deployments of different link densities. Moreover, to provide fairness, this paper proposes a novel scheduling approach that utilizes the sum-rate optimal scheduling algorithm over judiciously chosen subsets of links for maximizing a proportional fairness objective over the network. The proposed approach shows highly competitive and generalizable network utility maximization results. [ABSTRACT FROM AUTHOR]

To accurately pour drinks into various containers is an essential skill for service robots. However, drink pouring is a dynamic process and difficult to model. Traditional deep imitation learning techniques for implementing autonomous robotic pouring have an inherent black-box effect and require a large amount of demonstration data for model training. To address these issues, an Explainable Hierarchical Imitation Learning (EHIL) method is proposed in this paper such that a robot can learn high-level general knowledge and execute low-level actions across multiple drink pouring scenarios. Moreover, with the EHIL method, a logical graph can be constructed for task execution, through which the decision-making process for action generation can be made explainable to users and the causes of failure can be traced out. Based on the logical graph, the framework is manipulable to achieve different targets while the adaptability to unseen scenarios can be achieved in an explainable manner. A series of experiments have been conducted to verify the effectiveness of the proposed method. Results indicate that EHIL outperforms the traditional behavior cloning method in terms of success rate, adaptability, manipulability, and explainability. Note to Practitioners—Pouring liquids is a common activity in people’s daily lives and all wet-lab industries. Drink pouring dynamic control is difficult to model, while the accurate perception of flow is challenging. To enable the robot to learn under unknown dynamics via observing the human demonstration, deep imitation learning can be used. To address the limitations of traditional deep neural networks, an Explainable Hierarchical Imitation Learning (EHIL) method is proposed in this paper. The proposed method enables the robot to learn a sequence of reasonable pouring phases for performing the task rather than simply execute the task via traditional behavior cloning. In this way, explainability and safety can be ensured. Manipulability can be achieved by reconstructing the logical graph. The target of this research is to obtain pouring dynamics via the learning method and realize the precise and quick pouring of drink from the source containers to various targeted containers with reliable performance, adaptability, manipulability, and explainability. [ABSTRACT FROM AUTHOR]

To convey neural network architectures in publications, appropriate visualizations are of great importance. While most current deep learning papers contain such visualizations, these are usually handcrafted just before publication, which results in a lack of a common visual grammar, significant time investment, errors, and ambiguities. Current automatic network visualization tools focus on debugging the network itself and are not ideal for generating publication visualizations. Therefore, we present an approach to automate this process by translating network architectures specified in Keras into visualizations that can directly be embedded into any publication. To do so, we propose a visual grammar for convolutional neural networks (CNNs), which has been derived from an analysis of such figures extracted from all ICCV and CVPR papers published between 2013 and 2019. The proposed grammar incorporates visual encoding, network layout, layer aggregation, and legend generation. We have further realized our approach in an online system available to the community, which we have evaluated through expert feedback, and a quantitative study. It not only reduces the time needed to generate network visualizations for publications, but also enables a unified and unambiguous visualization design. [ABSTRACT FROM AUTHOR]

This paper investigates the feasibility of novel data-driven deep learning (DL) models to predict the solution of Maxwell’s equations for low-frequency electromagnetic (EM) devices. With ground truth (empirical evidence) data being generated from a finite-element analysis solver, a deep convolutional neural network is trained in a supervised manner to learn a mapping for magnetic field distribution for topologies of different complexities of geometry, material, and excitation, including a simple coil, a transformer, and a permanent magnet motor. Preliminary experiments show DL model predictions in close agreement with the ground truth. A probabilistic model is introduced to improve the accuracy and to quantify the uncertainty in the prediction, based on Monte Carlo dropout. This paper establishes a basis for a fast and generalizable data-driven model used in the analysis, design, and optimization of EM devices. [ABSTRACT FROM AUTHOR]

Linear spectral unmixing is the practice of decomposing the mixed pixel into a linear combination of the constituent endmembers and the estimated abundances. This paper focuses on unsupervised spectral unmixing where the endmembers are unknown a priori. Conventional approaches use either geometrical- or statistical-based approaches. In this paper, we address the challenges of spectral unmixing with unsupervised deep learning models, in specific, the autoencoder models, where the decoder serves as the endmembers and the hidden layer output serves as the abundances. In several recent attempts, part-based autoencoders have been designed to solve the unsupervised spectral unmixing problem. However, the performance has not been satisfactory. In this paper, we first discuss some important findings we make on issues with part-based autoencoders. By proof of counterexample, we show that all existing part-based autoencoder networks with nonnegative and tied encoder and decoder are inherently defective by making these inappropriate assumptions on the network structure. As a result, they are not suitable for solving the spectral unmixing problem. We propose a so-called untied denoising autoencoder with sparsity, in which the encoder and decoder of the network are independent, and only the decoder of the network is enforced to be nonnegative. Furthermore, we make two critical additions to the network design. First, since denoising is an essential step for spectral unmixing, we propose to incorporate the denoising capacity into the network optimization in the format of a denoising constraint rather than cascading another denoising preprocessor in order to avoid the introduction of additional reconstruction error. Second, to be more robust to the inaccurate estimation of a number of endmembers, we adopt an $l_{21}$ -norm on the encoder of the network to reduce the redundant endmembers while decreasing the reconstruction error simultaneously. The experimental results demonstrate that the proposed approach outperforms several state-of-the-art methods, especially for highly noisy data. [ABSTRACT FROM AUTHOR]

Light-fidelity (LiFi) is a fully-networked bidirectional optical wireless communication (OWC) technology that is considered as a promising solution for high-speed indoor connectivity. In this paper, the joint estimation of user 3D position and user equipment (UE) orientation in indoor LiFi systems with unknown emission power is investigated. Existing solutions for this problem assume either ideal LiFi system settings or perfect knowledge of the UE states, rendering them unsuitable for realistic LiFi systems. In addition, these solutions consider the non-line-of-sight (NLOS) links of the LiFi channel gain as a source of deterioration for the estimation performance instead of harnessing these components in improving the position and the orientation estimation performance. This is mainly due to the lack of appropriate estimation techniques that can extract the position and orientation information hidden in these components. In this paper, and against the above limitations, the UE is assumed to be connected with at least one access point (AP), i.e., at least one active LiFi link. Fingerprinting is employed as an estimation technique and the received signal-to-noise ratio (SNR) is used as an estimation metric, where both the line-of-sight (LOS) and NLOS components of the LiFi channel are considered. Motivated by the success of deep learning techniques in solving several complex estimation and prediction problems, we employ two deep artificial neural network (ANN) models, one based on the multilayer perceptron (MLP) and the second on the convolutional neural network (CNN), that can map efficiently the instantaneous received SNR with the user 3D position and the UE orientation. Through numerous examples, we investigate the performance of the proposed schemes in terms of the average estimation error, precision, computational time, and the bit error rate. We also compare this performance to that of the k-nearest neighbours (KNN) scheme, which is widely used in solving wireless localization problems. It is demonstrated that the proposed schemes achieve significant gains and are superior to the KNN scheme. [ABSTRACT FROM AUTHOR]

In conventional multi-user multiple-input multiple-output (MU-MIMO) systems with frequency division duplexing (FDD), channel acquisition and precoder optimization processes have been designed separately although they are highly coupled. This paper studies an end-to-end design of downlink MU-MIMO systems which include pilot sequences, limited feedback, and precoding. To address this problem, we propose a novel deep learning (DL) framework which jointly optimizes the feedback information generation at users and the precoder design at a base station (BS). Each procedure in the MU-MIMO systems is replaced by intelligently designed multiple deep neural networks (DNN) units. At the BS, a neural network generates pilot sequences and helps the users obtain accurate channel state information. At each user, the channel feedback operation is carried out in a distributed manner by an individual user DNN. Then, another BS DNN collects feedback information from the users and determines the MIMO precoding matrices. A joint training algorithm is proposed to optimize all DNN units in an end-to-end manner. In addition, a training strategy which can avoid retraining for different network sizes for a scalable design is proposed. Numerical results demonstrate the effectiveness of the proposed DL framework compared to classical optimization techniques and other conventional DNN schemes. [ABSTRACT FROM AUTHOR]

This paper investigates the stochastic optimization problem focusing on developing scalable parallel algorithms for deep learning tasks. Our solution involves a reformation of the objective function for stochastic optimization in neural network models, along with a novel parallel computing strategy, coined the weighted aggregating stochastic gradient descent (WASGD). Following a theoretical analysis on the characteristics of the new objective function, WASGD introduces a decentralized weighted aggregating scheme based on the performance of local workers. Without any center variable, the new method automatically gauges the importance of local workers and accepts them by their contributions. Furthermore, we have developed an enhanced version of the method, WASGD+, by (1) implementing a designed sample order and (2) upgrading the weight evaluation function. To validate the new method, we benchmark our pipeline against several popular algorithms including the state-of-the-art deep neural network classifier training techniques (e.g., elastic averaging SGD). Comprehensive validation studies have been conducted on four classic datasets: CIFAR-100, CIFAR-10, Fashion-MNIST, and MNIST. Subsequent results have firmly validated the superiority of the WASGD scheme in accelerating the training of deep architecture. Better still, the enhanced version, WASGD+, is shown to be a significant improvement over its prototype. [ABSTRACT FROM AUTHOR]

In this paper, we consider how to incorporate psychophysical measurements of human visual perception into the loss function of a deep neural network being trained for a recognition task, under the assumption that such information can reduce errors. As a case study to assess the viability of this approach, we look at the problem of handwritten document transcription. While good progress has been made towards automatically transcribing modern handwriting, significant challenges remain in transcribing historical documents. Here we describe a general enhancement strategy, underpinned by the new loss formulation, which can be applied to the training regime of any deep learning-based document transcription system. Through experimentation, reliable performance improvement is demonstrated for the standard IAM and RIMES datasets for three different network architectures. Further, we go on to show feasibility for our approach on a new dataset of digitized Latin manuscripts, originally produced by scribes in the Cloister of St. Gall in the the 9th century. [ABSTRACT FROM AUTHOR]

Most deep learning models for temporal regression directly output the estimation based on single input images, ignoring the relationships between different images. In this paper, we propose deep relation learning for regression, aiming to learn different relations between a pair of input images. Four non-linear relations are considered: “cumulative relation,” “relative relation,” “maximal relation” and “minimal relation.” These four relations are learned simultaneously from one deep neural network which has two parts: feature extraction and relation regression. We use an efficient convolutional neural network to extract deep features from the pair of input images and apply a Transformer for relation learning. The proposed method is evaluated on a merged dataset with 6,049 subjects with ages of 0–97 years using 5-fold cross-validation for the task of brain age estimation. The experimental results have shown that the proposed method achieved a mean absolute error (MAE) of 2.38 years, which is lower than the MAEs of 8 other state-of-the-art algorithms with statistical significance (p<0.05) in paired T-test (two-side). [ABSTRACT FROM AUTHOR]

Deep learning (DL) techniques have shown promising performance for designing data-driven power system transient stability assessment (TSA) models. However, due to the deep structure of the DL, the resulting model is always a black-box and hard to explain, which hinders its practical adoption by the industry. This paper proposes an interpretable DL-based TSA model to balance the TSA accuracy and transparency. The proposed method combines the strong nonlinear modelling capability of a deep neural network and the interpretability of a decision tree (DT). Through regularizing DL-based model with the average DT path length in the training process, the proposed interpretable DL-based TSA method can visually explain the TSA decision-making process. Simulation results have shown that the proposed method can deliver highly accurate TSA results and interpretable TSA decision-making rules, which can be used for designing preventive control actions. [ABSTRACT FROM AUTHOR]

Multiple-input multiple-output orthogonal frequency division multiplexing (MIMO-OFDM), a fundamental transmission scheme, promises high throughput and robustness against multipath fading. However, these benefits rely on the efficient detection strategy at the receiver and come at the expense of the extra bandwidth consumed by the cyclic prefix (CP). We use the iterative orthogonal approximate message passing (OAMP) algorithm in this paper as the prototype of the detector because of its remarkable potential for interference suppression. However, OAMP is computationally expensive for the matrix inversion per iteration. We replace the matrix inversion with the conjugate gradient (CG) method to reduce the complexity of OAMP. We further unfold the CG-based OAMP algorithm into a network and tune the critical parameters through deep learning (DL) to enhance detection performance. Simulation results and complexity analysis show that the proposed scheme has significant gain over other iterative detection methods and exhibits comparable performance to the state-of-the-art DL-based detector at a reduced computational cost. Furthermore, we design a highly efficient CP-free MIMO-OFDM receiver architecture to remove the CP overhead. This architecture first eliminates the intersymbol interference by buffering the previously recovered data and then detects the signal using the proposed detector. Numerical experiments demonstrate that the designed receiver offers a higher spectral efficiency than traditional receivers. Finally, over-the-air tests verify the effectiveness and robustness of the proposed scheme in realistic environments. [ABSTRACT FROM AUTHOR]

The analysis of over-parameterized neural networks has drawn significant attention in recent years. It was shown that such systems behave like convex systems under various restricted settings, such as for two-layer neural networks, and when learning is only restricted locally in the so-called neural tangent kernel space around specialized initializations. However, there is a lack of powerful theoretical techniques that can analyze fully trained deep neural networks under general conditions. This paper considers this fundamental problem by investigating such overparameterized deep neural networks when fully trained. Specifically, we characterize a deep neural network by its features’ distributions and propose a metric to intuitively measure the usefulness of feature representations. Under certain regularizers that bounds the metric, we show deep neural networks can be reformulated as a convex optimization and the system can guarantee effective feature representations in terms of the metric. Our new analysis is more consistent with empirical observations that deep neural networks are capable of learning efficient feature representations. Empirical studies confirm that predictions of our theory are consistent with results observed in practice. [ABSTRACT FROM AUTHOR]

Text clustering is a critical step in text data analysis and has been extensively studied by the text mining community. Most existing text clustering algorithms are based on the bag-of-words model, which faces the high-dimensional and sparsity problems and ignores text structural and sequence information. Deep learning-based models such as convolutional neural networks and recurrent neural networks regard texts as sequences but lack supervised signals and explainable results. In this paper, we propose a deep feature-based text clustering (DFTC) framework that incorporates pretrained text encoders into text clustering tasks. This model, which is based on sequence representations, breaks the dependency on supervision. The experimental results show that our model outperforms classic text clustering algorithms and the state-of-the-art pretrained language model, i.e., BERT, on almost all the considered datasets. In addition, the explanation of the clustering results is significant for understanding the principles of the deep learning approach. Our proposed clustering framework includes an explanation module that can help users understand the meaning and quality of the clustering results. [ABSTRACT FROM AUTHOR]

This paper proposes a sound separation methodology based on deep learning neural network (DLNN) to extract useful diagnostic material in non-invasive audio-based On-Load Tap Changer (OLTC) diagnostics. The proposed methodology has been experimentally verified on both artificial mixtures (created by reproducing the targeted data by the speakers placed next to the active transformers) and actual mixtures (recorded by the microphone during OLTC live operation in the field). The results show that the method produces high-quality estimates (correlation to referent fingerprints $\rho > 0.9$) compared to other sound separation methods, e.g. Independent Component Analysis (ICA). The proposed framework can also perform source separation from a monaural mixture (mixture recorded with a single microphone only), which is impossible for ICA methods. Moreover, the results show that DLNN trained with healthy OLTC data produces diagnostically valuable estimates even when fed with a faulty OLTC audio mixture. For that reason, once trained, the DLNN can produce the diagnostic signal estimates from monaural mixtures that can be used with existing vibration-based diagnostic methods. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (DCNNs) are currently the method of choice both for generative, as well as for discriminative learning in computer vision and machine learning. The success of DCNNs can be attributed to the careful selection of their building blocks (e.g., residual blocks, rectifiers, sophisticated normalization schemes, to mention but a few). In this paper, we propose $\Pi$ Π -Nets, a new class of function approximators based on polynomial expansions. $\Pi$ Π -Nets are polynomial neural networks, i.e., the output is a high-order polynomial of the input. The unknown parameters, which are naturally represented by high-order tensors, are estimated through a collective tensor factorization with factors sharing. We introduce three tensor decompositions that significantly reduce the number of parameters and show how they can be efficiently implemented by hierarchical neural networks. We empirically demonstrate that $\Pi$ Π -Nets are very expressive and they even produce good results without the use of non-linear activation functions in a large battery of tasks and signals, i.e., images, graphs, and audio. When used in conjunction with activation functions, $\Pi$ Π -Nets produce state-of-the-art results in three challenging tasks, i.e., image generation, face verification and 3D mesh representation learning. The source code is available at https://github.com/grigorisg9gr/polynomial_nets. [ABSTRACT FROM AUTHOR]

A general deep learning (DL) mechanism for a multiple hidden layer feed-forward neural network contains two parts, i.e., 1) an unsupervised greedy layer-wise training and 2) a supervised fine-tuning which is usually an iterative process. Although this mechanism has been demonstrated in many fields to be able to significantly improve the generalization of neural network, there is no clear evidence to show which one of the two parts plays the essential role for the generalization improvement, resulting in an argument within the DL community. Focusing on this argument, this paper proposes a new DL approach to train multilayer feed-forward neural networks. This approach uses restricted Boltzmann machine (RBM) as the layer-wise training and uses the generalized inverse of a matrix as the supervised fine-tuning. Different from the general deep training mechanism like back-propagation (BP), the proposed approach does not need to iteratively tune the weights, and therefore, has many advantages such as quick training, better generalization, and high understandability, etc. Experimentally, the proposed approach demonstrates an excellent performance in comparison with BP-based DL and the traditional training method for multilayer random weight neural networks. To a great extent, this paper demonstrates that the supervised part plays a more important role than the unsupervised part in DL, which provides some new viewpoints for exploring the essence of DL. [ABSTRACT FROM AUTHOR]

This paper investigates a deep reinforcement learning (DRL)-based MAC protocol for heterogeneous wireless networking, referred to as a Deep-reinforcement Learning Multiple Access (DLMA). Specifically, we consider the scenario of a number of networks operating different MAC protocols trying to access the time slots of a common wireless medium. A key challenge in our problem formulation is that we assume our DLMA network does not know the operating principles of the MACs of the other networks—i.e., DLMA does not know how the other MACs make decisions on when to transmit and when not to. The goal of DLMA is to be able to learn an optimal channel access strategy to achieve a certain pre-specified global objective. Possible objectives include maximizing the sum throughput and maximizing $\alpha $ -fairness among all networks. The underpinning learning process of DLMA is based on DRL. With proper definitions of the state space, action space, and rewards in DRL, we show that DLMA can easily maximize the sum throughput by judiciously selecting certain time slots to transmit. Maximizing general $\alpha $ -fairness, however, is beyond the means of the conventional reinforcement learning (RL) framework. We put forth a new multi-dimensional RL framework that enables DLMA to maximize general $\alpha $ -fairness. Our extensive simulation results show that DLMA can maximize sum throughput or achieve proportional fairness (two special classes of $\alpha $ -fairness) when coexisting with TDMA and ALOHA MAC protocols without knowing they are TDMA or ALOHA. Importantly, we show the merit of incorporating the use of neural networks into the RL framework (i.e., why DRL and not just traditional RL): specifically, the use of DRL allows DLMA (i) to learn the optimal strategy with much faster speed and (ii) to be more robust in that it can still learn a near-optimal strategy even when the parameters in the RL framework are not optimally set. [ABSTRACT FROM AUTHOR]

The trend of pushing inference from cloud to edge due to concerns of latency, bandwidth, and privacy has created demand for energy-efficient neural network hardware. This paper presents a mixed-signal binary convolutional neural network (CNN) processor for always-on inference applications that achieves 3.8 $\mu \text{J}$ /classification at 86% accuracy on the CIFAR-10 image classification data set. The goal of this paper is to establish the minimum-energy point for the representative CIFAR-10 inference task, using the available design tradeoffs. The BinaryNet algorithm for training neural networks with weights and activations constrained to +1 and −1 drastically simplifies multiplications to XNOR and allows integrating all memory on-chip. A weight-stationary, data-parallel architecture with input reuse amortizes memory access across many computations, leaving wide vector summation as the remaining energy bottleneck. This design features an energy-efficient switched-capacitor (SC) neuron that addresses this challenge, employing a 1024-bit thermometer-coded capacitive digital-to-analog converter (CDAC) section for summing pointwise products of CNN filter weights and activations and a 9-bit binary-weighted section for adding the filter bias. The design occupies 6 mm2 in 28-nm CMOS, contains 328 kB of on-chip SRAM, operates at 237 frames/s (FPS), and consumes 0.9 mW from 0.6 V/0.8 V supplies. The corresponding energy per classification (3.8 $\mu \text{J}$) amounts to a 40 $\times $ improvement over the previous low-energy benchmark on CIFAR-10, achieved in part by sacrificing some programmability. The SC neuron array is 12.9 $\times $ more energy efficient than a synthesized digital implementation, which amounts to a 4 $\times $ advantage in system-level energy per classification. [ABSTRACT FROM AUTHOR]

Most of the existing blind omnidirectional image quality assessment (BOIQA) methods are based on data-driven approach where the end-to-end neural network or deep learning tools are mainly used for feature extraction. However, it usually lacks interpretability and is difficult to discover the perceptual mechanism behind. In this paper, from the perspective of perception modeling, we propose a novel multi-angle projection based BOIQA (MP-BOIQA) method. Considering the omnibearing and near eye display characteristics with head mounted display, multiple color cubemap projection images with respect to different viewpoints are grouped as the color omnidirectional distortion (COD) units so as to simulate the user’s viewing behavior in subjective quality assessment. In the designed multi-angle projection based feature extractor, tensor decomposition is implemented on each COD unit for dimensionality reduction, and piecewise exponential fitting is used to get the distribution of mean subtracted contrast normalized coefficients of the unit’s feature matrices in tensor domain. Finally, the extracted features are pooled with random forest. The experimental results on three omnidirectional image quality datasets show that the MP-BOIQA method can deliver highly competitive performance compared with some representative full-reference quality assessment methods, as well as some state-of-the-art BOIQA methods. [ABSTRACT FROM AUTHOR]

Despite encouraging results have been achieved in human pose estimation in recent years, it remains challenging problems. When the background is similar to the human body parts, and there are small persons with low-resolution in the image, the performance may degrade dramatically. This paper addresses problems in background-inference and small-person pose estimation. To achieve this, a novel pose estimation algorithm is proposed on the basis of person semantic segmentation deep neural network. Different from most previous methods with a single pose estimation model, we generate mixture models with pose estimation and semantic segmentation. We introduce novel generative adversarial model and auxiliary model to realize the semantic segmentation network, which can handle the confusion of the similar regions in the background. In addition, to address the problem of the scale differences between big and small persons’ keypoints, we add additional position and channel attention modules to the first two stages of OpenPose. We conduct extensive experiments on COCO and VOC datasets. And we compare the proposed method with the most popular state-of-the-art human pose estimation and semantic segmentation frameworks, including MultiPoseNet, Deterton2 and DeepLab V3. Our experimental results show that the proposed method is more accurate than the state-of-the-art algorithms and performs effectively in tackling the complex situations. [ABSTRACT FROM AUTHOR]

Efficient nonlinearity compensation in fiber-optic communication systems is considered a key element to go beyond the “capacity crunch”. One guiding principle for previous work on the design of practical nonlinearity compensation schemes is that fewer steps lead to better systems. In this paper, we challenge this assumption and show how to carefully design multi-step approaches that provide better performance–complexity trade-offs than their few-step counterparts. We consider the recently proposed learned digital backpropagation (LDBP) approach, where the linear steps in the split-step method are re-interpreted as general linear functions, similar to the weight matrices in a deep neural network. Our main contribution lies in an experimental demonstration of this approach for a 25 Gbaud single-channel optical transmission system. It is shown how LDBP can be integrated into a coherent receiver DSP chain and successfully trained in the presence of various hardware impairments. Our results show that LDBP with limited complexity can achieve better performance than standard DBP by using very short, but jointly optimized, finite-impulse response filters in each step. This paper also provides an overview of recently proposed extensions of LDBP and we comment on potentially interesting avenues for future work. [ABSTRACT FROM AUTHOR]

Stereo matching methods consist of matching cost computation and several post processing steps. Deep learning methods have greatly raised the accuracy of matching cost and achieved the lowest error rate on several public datasets. However, their generality capabilities are not the best due to potential overfitting, which is the common problem of supervised learning approaches. This paper proposes a convolutional neural network (CNN) based cost estimation method for computing the similarity of image patches. In consideration of accuracy and generalization capability, small size convolution kernels are chosen in the convolution layer and dropout in the decision layer is used for preventing overfitting. After obtaining stereo matching cost from the output of the CNN, several post-processing operations are adopted for disparity optimization, which includes semi-global matching in 1-D from different directions, a left-right consistency check, and the slanted plane smoothing method. The method is evaluated on KITTI 2012, KITTI 2015, and Middlebury stereo datasets and the experimental results on the KITTI benchmark demonstrate the competitive accuracy performance of the approach. Additionally, to test the generalization of the method, a series of extended crossover experiments are conducted in which the training samples and testing samples come from different datasets. The results indicate superior generalization capability of our method than other supervised learning methods. [ABSTRACT FROM AUTHOR]

Remaining useful life (RUL) prediction has been increasingly considered in many industrial fields for the reliability and safety of their systems. As a data analysis tool of deep learning, deep convolutional neural network (CNN) shows great potential for RUL prediction. This paper proposes an intelligent RUL prediction method based on a double-CNN model architecture. Given the powerful feature extraction capability of CNN, the proposed method is fed with original vibration signals with no need to resort to any feature extractor, which can also retain the useful information in maximum. The prediction includes two stages: first, incipient fault point is identified by the first CNN model and a proposed “3/5” principle; then, the second CNN model is constructed for RUL prediction. In practice, RULs of identical components are different from each other, which poses a major challenge in RUL prediction. To overcome this problem, an intermediate reliability variable is first calculated in this paper, instead of directly predicting the RUL value. Then, a mapping algorithm is proposed to map reliability to RUL. To demonstrate the effectiveness of the proposed method, data of four tests of bearing degradation are utilized for RUL prediction. Compared with state-of-the-art methods, the proposed method shows higher prediction accuracy and robustness. The prediction results and evaluation indexes demonstrated the effectiveness and superiority of the proposed method. [ABSTRACT FROM AUTHOR]

Automatic target recognition (ATR) is an important part for many computer vision applications. Despite the extensive research which has been carried out in this area for many years, there is no ATR system which performs well on all applications. Recently, different object recognition frameworks have been proposed which yield a high performance in baseline databases. However, our experiments showed that they can fail in real-world scenarios, when dealing with a limited number of data samples. In this paper, we propose a new ATR system, based on deep convolutional neural network (DCNN), to detect the targets in forward looking infrared (FLIR) scenes and recognize their classes. In our proposed ATR framework, a fully convolutional network is trained to map the input FLIR imagery data to a fixed stride correspondingly-sized target score map. The potential targets are identified by applying a threshold on the target score map. Finally, the corresponding regions centered at these target points are fed to a DCNN to classify them into different target types while at the same time rejecting the false alarms. The proposed architecture achieves a significantly better performance in comparison with that of the state-of-the-art methods on two large FLIR image databases. [ABSTRACT FROM AUTHOR]

Significance: The performance of traditional approaches to decoding movement intent from electromyograms (EMGs) and other biological signals commonly degrade over time. Furthermore, conventional algorithms for training neural network based decoders may not perform well outside the domain of the state transitions observed during training. The work presented in this paper mitigates both these problems, resulting in an approach that has the potential to substantially improve the quality of life of the people with limb loss. Objective: This paper presents and evaluates the performance of four decoding methods for volitional movement intent from intramuscular EMG signals. Methods: The decoders are trained using the dataset aggregation (DAgger) algorithm, in which the training dataset is augmented during each training iteration based on the decoded estimates from previous iterations. Four competing decoding methods, namely polynomial Kalman filters (KFs), multilayer perceptron (MLP) networks, convolutional neural networks (CNN), and long short-term memory (LSTM) networks, were developed. The performances of the four decoding methods were evaluated using EMG datasets recorded from two human volunteers with transradial amputation. Short-term analyses, in which the training and cross-validation data came from the same dataset, and long-term analyses, in which the training and testing were done in different datasets, were performed. Results: Short-term analyses of the decoders demonstrated that CNN and MLP decoders performed significantly better than KF and LSTM decoders, showing an improvement of up to 60% in the normalized mean-square decoding error in cross-validation tests. Long-term analyses indicated that the CNN, MLP, and LSTM decoders performed significantly better than a KF-based decoder at most analyzed cases of temporal separations (0–150 days) between the acquisition of the training and testing datasets. Conclusion: The short-term and long-term performances of MLP- and CNN-based decoders trained with DAgger demonstrated their potential to provide more accurate and naturalistic control of prosthetic hands than alternate approaches. [ABSTRACT FROM AUTHOR]

This paper proposes a novel facial landmark detection (FLD) algorithm for use in real driving situations. The proposed algorithm is based on an ensemble of local weighted random forest regressor (WRFR) with random sampling consensus (RANSAC) and explicit global shape models, and considers the dynamic and irregular characteristics of driving. In this paper, to estimate the offset distance from a landmark and a reference point, we first detect the nose region as a reference point. Next, we propose a local WRFR to maintain the generality with a small number of regression trees. With the WRFR, we adopt RANSAC instead of the averaging or median of offsets to handle the problem of sensitivity to outlier offsets, and we estimate the accurate 2D offset vector. To identify the erroneous positions of local landmarks and rearrange the overall landmark layout, we adopt the global face models based on the spatial relation between landmarks. Using the unified framework of the proposed FLD, our proposed algorithm is robust to large head poses and partial occlusions caused by a driver’s hair or sunglasses. For a benchmark data set considering real driving situations, we construct a data set called a face alignment data set used in driving (FADID) using a near-infrared camera for FLD under real driving situations. We apply the proposed algorithm to various driving sequences in FADID, and the results show that its FLD performance is better than that of other state-of-the-art methods, while the computational speed is high for real-time applications such as driver-state monitoring systems. [ABSTRACT FROM AUTHOR]

With the development of video surveillance in public safety field, there is an increasing research on person re-identification (re-id). In this paper, we address the image-to-video person re-id, in which the probe is an image and the gallery is consists of videos captured by nonoverlapping cameras. Compared with image, video sequence contains more temporal information that can be explored to improve the performance of re-identification system. However, it is challenging to model temporal information in the matching process of image-to-video person re-id. In this paper, we proposed a novel temporally memorized similarity learning neural network for this problem. In specific, the proposed network mainly consisted of two parts, including feature representation sub-network and similarity sub-network. In the first part, we adopted a convolutional neural network (CNN) to extract features from the input image. Given a video sequence of a person, features were first extracted from each its frame by using CNN and further forward to a long shot term memory (LSTM) network to encode the temporal information of video sequence. The outputs of LSTM were concatenated together as the feature vector of video sequences. Finally, the feature vectors of probe image and the video sequence were further forward to the similarity sub-network for distance metric learning. In the proposed framework, the feature representation and the similarity metric learning can be learned and optimized simultaneously. We evaluated the proposed framework on three public person re-id data sets, and the experimental results showed that the proposed approach is effective for the image-to-video person re-id. [ABSTRACT FROM AUTHOR]

This paper presents a scalable inference accelerator called a deep-learning specific instruction-set processor (DSIP) to support various convolutional neural networks (CNNs). For CNNs requiring a large amount of computations and memory accesses, a programmable inference system called master–slave instruction set architecture (ISA) is newly proposed to achieve high flexibility, processing speed, and energy efficiency. The master is responsible for sending and receiving feature maps in order to deal with neural networks in a scalable way, and the slave performs CNN operations, such as multiply accumulate, max pooling, and activation functions, on the features received from the master. The master–slave ISA maximizes computation speed by overlapping the off-chip data transmission and the CNN operations, and reduces power consumption by performing the convolution incrementally to reuse input and partial-sum data as maximally as possible. An inference system can be configured by connecting multiple DSIPs in a form of either 1-D or 2-D chain structure in order to enhance computation speed further. To evaluate the proposed accelerator, a prototype chip is implemented and evaluated for AlexNet. Compared to the state-of-the-art accelerator, the DSIP-based system enhances the energy efficiency by 2.17 $\times $ . [ABSTRACT FROM AUTHOR]

In this paper, we experimentally and analytically evaluate the reliability of two state-of-the-art neural networks for linear regression and pattern recognition (multilayer perceptron and single-layer perceptron) implemented in a system-on-chip composed of a field-programmable gate array (FPGA) and a microprocessor. We have considered, for each neural network, three different activation function complexities, to evaluate how the implementation affects FPGAs reliability. As we show in this paper, higher complexity increases the exposed area but reduces the probability of one failure to impact the network output. In addition, we propose to distinguish between critical and tolerable errors in artificial neural networks. Experiments using a controlled heavy-ions beam show that, for both networks, only about 30% of the observed output errors actually affect the outputs correctness. We identify the causes of critical errors through fault injection, and found that faults in initial layers are more likely to significantly affect the output. [ABSTRACT FROM PUBLISHER]

Recent studies have shown that a Deep Convolutional Neural Network (DCNN) trained on a large image dataset can be used as a universal image descriptor and that doing so leads to impressive performance for a variety of image recognition tasks. Most of these studies adopt activations from a single DCNN layer, usually a fully-connected layer, as the image representation. In this paper, we proposed a novel way to extract image representations from two consecutive convolutional layers: one layer is used for local feature extraction and the other serves as guidance to pool the extracted features. By taking different viewpoints of convolutional layers, we further develop two schemes to realize this idea. The first directly uses convolutional layers from a DCNN. The second applies the pre-trained CNN on densely sampled image regions and treats the fully-connected activations of each image region as a convolutional layer's feature activations. We then train another convolutional layer on top of that as the pooling-guidance convolutional layer. By applying our method to three popular visual classification tasks, we find that our first scheme tends to perform better on applications which demand strong discrimination on lower-level visual patterns while the latter excels in cases that require discrimination on category-level patterns. Overall, the proposed method achieves superior performance over existing approaches for extracting image representations from a DCNN. In addition, we apply cross-layer pooling to the problem of image retrieval and propose schemes to reduce the computational cost. Experimental results suggest that the proposed method achieves promising results for the image retrieval task. [ABSTRACT FROM AUTHOR]

Massive multiple-input multiple-output can obtain more performance gain by exploiting the downlink channel state information (CSI) at the base station (BS). Therefore, studying CSI feedback with limited communication resources in frequency-division duplexing systems is of great importance. Recently, deep learning (DL)-based CSI feedback has shown considerable potential. However, the existing DL-based explicit feedback schemes are difficult to deploy because current fifth-generation mobile communication protocols and systems are designed based on an implicit feedback mechanism. In this paper, we propose a DL-based implicit feedback architecture to inherit the low-overhead characteristic, which uses neural networks (NNs) to replace the precoding matrix indicator (PMI) encoding and decoding modules. By using environment information, the NNs can achieve a more refined mapping between the precoding matrix and the PMI compared with codebooks. The correlation between subbands is also used to further improve the feedback performance. Simulation results show that, for a single resource block (RB), the proposed architecture can save 25.0% – 40.0% of overhead compared with the Type I codebook under different antenna configurations. For a wideband system with 52 RBs, overhead can be saved by 30.7% and 48.0% compared with the Type II codebook when ignoring and considering extracting subband correlation, respectively. [ABSTRACT FROM AUTHOR]

Using deep neural networks (DNNs) to explore spatial patterns and temporal dynamics of human brain activities has been an important yet challenging problem because the artificial neural networks are hard to be designed manually. There have been several promising deep learning methods, e.g., deep belief network (DBN), convolutional neural network (CNN), and deep sparse recurrent auto-encoder (DSRAE), that can decompose neuroscientific and meaningful spatiotemporal patterns from 4D functional Magnetic Resonance Imaging (fMRI) data. However, those previous studies still depend on hand-crafted neural network architectures and hyperparameters, which are not optimal in various senses. In this paper, we employ the evolutionary algorithms (EA) to optimize the deep neural architecture of DSRAE by minimizing the expected loss of initialized models, named eNAS-DSRAE (evolutionary Neural Architecture Search on Deep Sparse Recurrent Auto-Encoder). Also, validation experiments are designed and performed on the publicly available human connectome project (HCP) 900 datasets, and the results achieved by the optimized eNAS-DSRAE suggested that our framework can successfully identify the spatiotemporal features and perform better than the hand-crafted neural network models. To our best knowledge, the proposed eNAS-DSRAE is not only among the earliest NAS models that can extract connectome-scale meaningful spatiotemporal brain networks from 4D fMRI data, but also is an effective framework to optimize the RNN-based models. [ABSTRACT FROM AUTHOR]

Inter prediction is a crucial part of hybrid video coding frameworks, utilized to exploit the temporal redundancy in video sequences and improve the coding performance. During inter prediction, a predicted block is typically derived from reference pictures using motion estimation and motion compensation. To improve the coding performance of inter prediction, a neural network based enhancement to inter prediction (NNIP) is proposed in this paper. NNIP is composed of three networks, namely residue estimation network, combination network, and deep refinement network. Specifically, first, a residue estimation network is designed to estimate the residue between current block and its predicted block using their available spatial neighbors. Second, the feature maps of the estimated residue and the predicted block are extracted and concatenated in a combination network. Finally, the concatenated feature maps are fed into a deep refinement network to generate a refined residue, which is added back to the predicted block to derive a more accurate predicted block. NNIP is integrated in HEVC to evaluate its efficiency. The experimental results demonstrate that NNIP can achieve 4.6%, 3.0%, and 2.7% BD-rate reduction on average under LDP, LDB, and RA configurations compared to HEVC. [ABSTRACT FROM AUTHOR]

Knowledge transfer is a promising concept to achieve real-time decision-making for autonomous vehicles. This paper constructs a transfer deep reinforcement learning (RL) framework to transform the driving tasks in the intersection environments. The driving missions at the unsignalized intersection are cast into a left turn, right turn, and running straight for automated vehicles. The goal of the autonomous ego vehicle (AEV) is to drive through the intersection situation efficiently and safely. This objective promotes the studied vehicle to increase its speed and avoid crashing other vehicles. The decision-making policy learned from one driving task is transferred through three transfer rules in another driving mission and evaluated. Simulation results reveal that the decision-making strategies related to similar tasks are transferable and have a high success rate. It indicates that the presented control framework could reduce time consumption and realize online implementation. Therefore, the transfer RL concept is helpful for establishing the real-time decision-making policy for autonomous vehicles. [ABSTRACT FROM AUTHOR]

In this paper, we propose an end-to-end deep learning architecture that generates 3D triangular meshes from single color images. Restricted by the nature of prevalent deep learning techniques, the majority of previous works represent 3D shapes in volumes or point clouds. However, it is non-trivial to convert these representations to compact and ready-to-use mesh models. Unlike the existing methods, our network represents 3D shapes in meshes, which are essentially graphs and well suited for graph-based convolutional neural networks. Leveraging perceptual features extracted from an input image, our network produces the correct geometry by progressively deforming an ellipsoid. To make the whole deformation procedure stable, we adopt a coarse-to-fine strategy, and define various mesh/surface related losses to capture properties of various aspects, which benefits producing the visually appealing and physically accurate 3D geometry. In addition, our model by nature can be adapted to objects in specific domains, e.g., human faces, and be easily extended to learn per-vertex properties, e.g., color. Extensive experiments show that our method not only qualitatively produces the mesh model with better details, but also achieves the higher 3D shape estimation accuracy compared against the state-of-the-arts. [ABSTRACT FROM AUTHOR]

The key role of spectrum sensing in cognitive radios attracted substantial research attention to improve the performance of detectors. We consider a general model for the receiver noise with the potential of generalization towards modeling noise in various environments. This general noise model describes accurately noise characteristics ranging from the Gaussian noise to the severe impulsive noise. However, many previous studies are based on the ideal Gaussian noise, and they cannot capture the non-Gaussian models. To provide a robust detector against different behaviors of the noise in various environments, we employ a convolutional neural network (CNN) compatible with different noise models. The proposed CNN detector is data-driven, and due to its single-dimensional input layer, it is consistent with the received signal and requires no pre-processing. The likelihood ratio test (LRT), the Wald, and the Rao tests for this problem are derived to enrich the paper with comparative evaluations of the proposed CNN and conventional model-based approaches and other neural networks. Although various simulated scenarios substantiate the general superiority and robustness of the CNN-based against impulsive noise and mismatch of parameters, it requires higher computational complexity than other discussed detectors. [ABSTRACT FROM AUTHOR]

Mobile sensing is a promising sensing paradigm in the era of Internet of Things (IoT) that utilizes mobile device sensors to collect sensory data about sensing targets and further applies learning techniques to recognize the sensed targets to correct classes or categories. Due to the recent great success of deep learning, an emerging trend is to adopt deep learning in this recognition process, while we find an overlooked yet crucial issue to be solved in this paper — The size of deep learning models should be sufficiently large for reliably classifying various types of recognition targets, while the achieved processing delay may fail to satisfy the stringent latency requirement from applications. If we blindly shrink the deep learning model for acceleration, the performance cannot be guaranteed. To cope with this challenge, this paper presents a compact deep neural network architecture, namely cDeepArch. The key idea of the cDeepArch design is to decompose the entire recognition task into two lightweight sub-problems: context recognition and the context-oriented target recognitions. This decomposition essentially utilizes the adequate storage to trade for the CPU and memory resource consumptions during execution. In addition, we further formulate the execution latency for decomposed deep learning models and propose a set of enhancement techniques, so that system performance and resource consumption can be quantitatively balanced. We implement a cDeepArch prototype system and conduct extensive experiments. The result shows that cDeepArch achieves excellent recognition performance and the execution latency is also lightweight. [ABSTRACT FROM AUTHOR]

Residual networks (ResNets) have introduced a milestone for the deep learning community due to their outstanding performance in diverse applications. They enable efficient training of increasingly deep networks, reducing the training difficulty and error. The main intuition behind them is that, instead of mapping the input information, they are mapping a residual part of it. Since the original work, a lot of extensions have been proposed to improve information mapping. In this paper, a novel extension of the residual block is proposed inspired by linear dynamical systems (LDSs), called LDS-ResNet. Specifically, a new module is presented that improves mapping of residual information by transforming it in a hidden state and then mapping it back to the desired feature space using convolutional layers. The proposed module is utilized to construct multi-branch residual blocks for convolutional neural networks. An exploration of possible architectural choices is presented and evaluated. Experimental results show that LDS-ResNet outperforms the original ResNet in image classification and object detection tasks on public datasets such as CIFAR-10/100, ImageNet, VOC, and MOT2017. Moreover, its performance boost is complementary to other extensions of the original network such as pre-activation and bottleneck, as well as stochastic training and Squeeze-Excitation. [ABSTRACT FROM AUTHOR]

Existing graph layout algorithms are usually not able to optimize all the aesthetic properties desired in a graph layout. To evaluate how well the desired visual features are reflected in a graph layout, many readability metrics have been proposed in the past decades. However, the calculation of these readability metrics often requires access to the node and edge coordinates and is usually computationally inefficient, especially for dense graphs. Importantly, when the node and edge coordinates are not accessible, it becomes impossible to evaluate the graph layouts quantitatively. In this paper, we present a novel deep learning-based approach to evaluate the readability of graph layouts by directly using graph images. A convolutional neural network architecture is proposed and trained on a benchmark dataset of graph images, which is composed of synthetically generated graphs and graphs created by sampling from real large networks. Multiple representative readability metrics (including edge crossing, node spread, and group overlap) are considered in the proposed approach. We quantitatively compare our approach to traditional methods and qualitatively evaluate our approach by showing usage scenarios and visualizing convolutional layers. This paper is a first step towards using deep learning based methods to quantitatively evaluate images from the visualization field. [ABSTRACT FROM AUTHOR]

Video super-resolution (VSR) has become one of the most critical problems in video processing. In the deep learning literature, recent works have shown the benefits of using adversarial-based and perceptual losses to improve the performance on various image restoration tasks; however, these have yet to be applied for video super-resolution. In this paper, we propose a generative adversarial network (GAN)-based formulation for VSR. We introduce a new generator network optimized for the VSR problem, named VSRResNet, along with new discriminator architecture to properly guide VSRResNet during the GAN training. We further enhance our VSR GAN formulation with two regularizers, a distance loss in feature-space and pixel-space, to obtain our final VSRResFeatGAN model. We show that pre-training our generator with the mean-squared-error loss only quantitatively surpasses the current state-of-the-art VSR models. Finally, we employ the PercepDist metric to compare the state-of-the-art VSR models. We show that this metric more accurately evaluates the perceptual quality of SR solutions obtained from neural networks, compared with the commonly used PSNR/SSIM metrics. Finally, we show that our proposed model, the VSRResFeatGAN model, outperforms the current state-of-the-art SR models, both quantitatively and qualitatively. [ABSTRACT FROM AUTHOR]

Time series classification is an important research topic in machine learning and data mining communities, since time series data exist in many application domains. Recent studies have shown that machine learning algorithms could benefit from good feature representation, explaining why deep learning has achieved breakthrough performance in many tasks. In deep learning, the convolutional neural network (CNN) is one of the most well-known approaches, since it incorporates feature learning and classification task in a unified network architecture. Although CNN has been successfully applied to image and text domains, it is still a challenge to apply CNN to time series data. This paper proposes a tensor scheme along with a novel deep learning architecture called multivariate convolutional neural network (MVCNN) for multivariate time series classification, in which the proposed architecture considers multivariate and lag-feature characteristics. We evaluate our proposed method with the prognostics and health management (PHM) 2015 challenge data, and compare with several algorithms. The experimental results indicate that the proposed method outperforms the other alternatives using the prediction score, which is the evaluation metric used by the PHM Society 2015 data challenge. Besides performance evaluation, we provide detailed analysis about the proposed method. [ABSTRACT FROM AUTHOR]

This paper proposes a novel intelligent fault diagnosis method to automatically identify different health conditions of wind turbine (WT) gearbox. Unlike traditional approaches, where feature extraction and classification are separately designed and performed, this paper aims to automatically learn effective fault features directly from raw vibration signals while classify the type of faults in a single framework, thus providing an end-to-end learning-based fault diagnosis system for WT gearbox without additional signal processing and diagnostic expertise. Considering the multiscale characteristics inherent in vibration signals of a gearbox, a new multiscale convolutional neural network (MSCNN) architecture is proposed to perform multiscale feature extraction and classification simultaneously. The proposed MSCNN incorporates multiscale learning into the traditional CNN architecture, which has two merits: 1) high-level fault features can be effectively learned by the hierarchical learning structure with multiple pairs of convolutional and pooling layers; and 2) multiscale learning scheme can capture complementary and rich diagnosis information at different scales. This greatly improves the feature learning ability and enables better diagnosis performance. The proposed MSCNN approach is evaluated through experiments on a WT gearbox test rig. Experimental results and comprehensive comparison analysis with respect to the traditional CNN and traditional multiscale feature extractors have demonstrated the superiority of the proposed method. [ABSTRACT FROM AUTHOR]

Machine learning algorithms can have difficulties adapting to data from different sources, for example from different imaging modalities. We present and analyze three techniques for unsupervised cross-modality feature learning, using a shared autoencoder-like convolutional network that learns a common representation from multi-modal data. We investigate a form of feature normalization, a learning objective that minimizes cross-modality differences, and modality dropout, in which the network is trained with varying subsets of modalities. We measure the same-modality and cross-modality classification accuracies and explore whether the models learn modality-specific or shared features. This paper presents experiments on two public data sets, with knee images from two MRI modalities, provided by the Osteoarthritis Initiative, and brain tumor segmentation on four MRI modalities from the BRATS challenge. All three approaches improved the cross-modality classification accuracy, with modality dropout and per-feature normalization giving the largest improvement. We observed that the networks tend to learn a combination of cross-modality and modality-specific features. Overall, a combination of all three methods produced the most cross-modality features and the highest cross-modality classification accuracy, while maintaining most of the same-modality accuracy. [ABSTRACT FROM AUTHOR]

The image classification accuracy of a TaOx ReRAM-based neuromorphic computing accelerator is evaluated after intentionally inducing a displacement damage up to a fluence of 1014 2.5-MeV Si ions/cm2 on the analog devices that are used to store weights. Results are consistent with a radiation-induced oxygen vacancy production mechanism. When the device is in the high-resistance state during heavy ion radiation, the device resistance, linearity, and accuracy after training are only affected by high fluence levels. The findings in this paper are in accordance with the results of previous studies on TaOx-based digital resistive random access memory. When the device is in the low-resistance state during irradiation, no resistance change was detected, but devices with a 4- $\text{k}\Omega $ inline resistor did show a reduction in accuracy after training at 1014 2.5-MeV Si ions/cm2. This indicates that changes in resistance can only be somewhat correlated with changes to devices’ analog properties. This paper demonstrates that TaOx devices are radiation tolerant not only for high radiation environment digital memory applications but also when operated in an analog mode suitable for neuromorphic computation and training on new data sets. [ABSTRACT FROM AUTHOR]

A system to perform video analytics is proposed using a dynamically tuned convolutional network. Videos are fetched from cloud storage, preprocessed, and a model for supporting classification is developed on these video streams using cloud-based infrastructure. A key focus in this paper is on tuning hyper-parameters associated with the deep learning algorithm used to construct the model. We further propose an automatic video object classification pipeline to validate the system. The mathematical model used to support hyper-parameter tuning improves performance of the proposed pipeline, and outcomes of various parameters on system’s performance is compared. Subsequently, the parameters that contribute toward the most optimal performance are selected for the video object classification pipeline. Our experiment-based validation reveals an accuracy and precision of 97% and 96%, respectively. The system proved to be scalable, robust, and customizable for a variety of different applications. [ABSTRACT FROM AUTHOR]

This paper proposes a novel method for voltage dip classification using deep convolutional neural networks. The main contributions of this paper include: 1) to propose a new effective deep convolutional neural network architecture for automatically learning voltage dip features, rather than extracting hand-crafted features; 2) to employ the deep learning in an effective two-dimensional (2-D) transform domain, under space-phasor model (SPM), for efficient learning of dip features; 3) to characterize voltage dips by 2-D SPM-based deep learning, which leads to voltage dip features independent of the duration and sampling frequency of dip recordings; and 4) to develop robust automatically-extracted features that are insensitive to training and test datasets measured from different countries/regions. Experiments were conducted on datasets containing about 6000 measured voltage dips spread over seven classes measured from several different countries. Results have shown good performance of the proposed method: average classification rate is about 97% and false alarm rate is about 0.50%. The test results from the proposed method are compared with the results from two existing dip classification methods. The proposed method is shown to outperform these existing methods. [ABSTRACT FROM AUTHOR]

Neural networks (NNs) have become the go-to tool for solving many real-world recognition and classification tasks with massive and complex data sets. These networks require large data sets for training, which is usually performed on GPUs and CPUs in either a cloud or edge computing setting. No matter where the training is performed, it is subject to tight power/energy and data storage/transfer constraints. While these issues can be mitigated by replacing SRAM/DRAM with nonvolatile memories (NVMs) which offer near-zero leakage power and high scalability, the massive weight updates performed during training shorten NVM endurance and engender high write energy. In this paper, an NVM-friendly NN training approach is proposed. Weight update is redesigned to reduce bit flips in NVM cells. Moreover, two techniques, namely, filter exchange and bitwise rotation, are proposed to respectively balance writes to different weights and to different bits of one weight. The proposed techniques are integrated and evaluated in Caffe. Experimental results show significant power savings and endurance improvements, while maintaining high inference accuracy. [ABSTRACT FROM AUTHOR]