Search

Recently, fault-aware retraining has emerged as a promising approach to improve the error resilience of Deep Neu-ral Networks (DNNs) against manufacturing-induced defects in DNN accelerators. However, state-of-the-art fault-aware training techniques incur a gigantic retraining overhead due to their per-chip retraining nature for the chip’s unique fault map, which may render it practically infeasible if retraining is done on large datasets. To address this major limitation and improve the practicability of the fault-aware retraining methodology, this work proposes a novel concept of merging fault maps to effectively retrain a DNN for a group of faulty chips in a single fault-aware retraining round. The merging of fault maps enables to avoid per chip retraining and thereby reduces the retraining overhead significantly. However, the merging of fault maps brings in new challenges such as training divergence (accuracy collapse) if a high number of accumulated faults are injected into the network in the first epoch. To address these challenges, we propose a methodology for effective merging of fault maps and then retraining of DNNs. Experimental results show that our methodology offers at least 1.4x retraining speedup on average while improving the error resilience of the network (depending on the DNN models and the number of merged fault maps). For example, for the Resnet-32 model using fault map generated from 5 fault maps at the fault rate 6e-3, our methodology offers 2x retraining speedup and 0.6% classification accuracy drop against per-chip retraining.

This paper proposes a method to enhance the quality of detecting and classifying surface vehicle propeller cavitation noise (VPCN) in shallow water by using the improved Detection Envelope Modulation On Noise (DEMON) algorithm in combination with the modified Convolution Neural Network (CNN). To improve the quality of the VPCN spectrogram signal, we apply the DEMON algorithm while analyzing the amplitude variation (AV) to detect the fundamental frequencies of the VPCN signal. To enhance the performance of the traditional CNN, we adapt the size of the sliding window in accordance with the properties of the VPCN spectrogram data, and also reconstruct the CNN layer structure. As for the results, the fundamental frequencies contented in the VPCN spectrogram data can be detected. The analytical results based on the measured data show that the accuracy of the VPCN classification obtained by the proposed method is above 90%, which is higher than those obtained by traditional methods.

In this paper, we propose a digital network coding protocol to enhance data rate for the secondary network in underlay cognitive radio environment. In the proposed protocol, transmit antenna selection and selection combining techniques are respectively used by a primary transmitter and a primary receiver to enhance performance for a primary network. In a secondary network, two secondary sources exchange their information via assistance of a common secondary relay that employs interference cancellation method to reduce one time slot, as compared with the conventional digital network coding protocol. We derive expressions of transmit power of the secondary nodes under a constraint of outage probability (OP) given by the primary network. Exact and asymptotic closed-form formulas of OP of the secondary network over Rayleigh fading channel are derived, and verified by Monte-Carlo simulations.

Deep learning has demonstrated the important roles in improving the system performance and reducing computational complexity for $5$G-and-beyond networks. In this paper, we propose a new channel estimation method with the assistance of deep learning in order to support the least squares estimation, which is a low-cost method but having relatively high channel estimation errors. This goal is achieved by utilizing a MIMO (multiple-input multiple-output) system with a multi-path channel profile used for simulations in the 5G networks under the severity of Doppler effects. Numerical results demonstrate the superiority of the proposed deep learning-assisted channel estimation method over the other channel estimation methods in previous works in terms of mean square errors., Comment: 7 pages, 8 figures, and 3 tables. This paper was presented at IMCOM 2021

This paper contains the description for the submission of Karlsruhe Institute of Technology (KIT) for the multilingual TEDx translation task in the IWSLT 2021 evaluation campaign. Our main approach is to develop both cascade and end-to-end systems and eventually combine them together to achieve the best possible results for this extremely low-resource setting. The report also confirms certain consistent architectural improvement added to the Transformer architecture, for all tasks: translation, transcription and speech translation.

This paper describes KIT’submission to the IWSLT 2021 Offline Speech Translation Task. We describe a system in both cascaded condition and end-to-end condition. In the cascaded condition, we investigated different end-to-end architectures for the speech recognition module. For the text segmentation module, we trained a small transformer-based model on high-quality monolingual data. For the translation module, our last year’s neural machine translation model was reused. In the end-to-end condition, we improved our Speech Relative Transformer architecture to reach or even surpass the result of the cascade system.

Transformer models are powerful sequence-to-sequence architectures that are capable of directly mapping speech inputs to transcriptions or translations. However, the mechanism for modeling positions in this model was tailored for text modeling, and thus is less ideal for acoustic inputs. In this work, we adapt the relative position encoding scheme to the Speech Transformer, where the key addition is relative distance between input states in the self-attention network. As a result, the network can better adapt to the variable distributions present in speech data. Our experiments show that our resulting model achieves the best recognition result on the Switchboard benchmark in the non-augmentation condition, and the best published result in the MuST-C speech translation benchmark. We also show that this model is able to better utilize synthetic data than the Transformer, and adapts better to variable sentence segmentation quality for speech translation., Submitted to Interspeech 2020

This paper demonstrates the performance evaluation of UAV detection based on micro-Doppler radar image data with the proposed inception-residual neural network (IRNN). Accordingly, the network is designed and analyzed by changing network hyper-parameters through experiment with the Real Doppler RAD-DAR (RDRD) dataset that is collected by the practical measurements. Numerical analysis results show that the proposed network with 16 filters yield a good trade-off between accuracy and time-consuming performances. Moreover, the network is taken into account for competing with three other networks. Due to inception-residual structure, the proposed network remarkably outperforms other ones.

In recent years, with the increasing proportion of renewable energy, some system problems have gradually emerged. To reduce the economic cost of system operations and improve power system reliability, renewable power forecasting is an indispensable part. Compared with the deterministic prediction, the probabilistic forecast considers the uncertainty, which helps manage risks and make decisions for power grids. This study proposes a novel probabilistic forecasting method for wind power generation, which includes data preprocessing, adaptive neuro fuzzy inference system (ANFIS) training model with fuzzy c-means clustering algorithm, and post processing of predicted-interval. The input data of the proposed probabilistic forecasting model include the numerical weather prediction (NWP) ensemble wind speeds, NWP spot wind-speed forecasts, and historical wind power measurements. For practical applications, measured data of power generation at actual wind farms were used to compare different forecasting models. The research results demonstrate that the proposed model supports better performance and prediction stability. Furthermore, this work reveals the importance of both data preprocessing and post processing of predicted interval on wind power forecasting. These essential processes greatly improve the performance of the probabilistic wind power forecasts.

Deep Neural Networks (DNNs) - the state-of-the-art computational models for many Artificial Intelligence (AI) applications - are inherently compute and resource-intensive and, hence, cannot exploit traditional redundancy-based fault mitigation techniques for enhancing the dependability of DNN-based systems. Therefore, there is a dire need to search for alternate methods that can improve their reliability without high expenditure of resources by exploiting the intrinsic characteristics of these networks. In this paper, we present cross-layer approaches that, based on the intrinsic characteristics of DNNs, employ software and hardware-level modifications for improving the resilience of DNN-based systems to hardware-level faults, e.g., soft errors and permanent faults.

20 years of MovieLens datasets have witnessed a blossom of research that is garnering a remarkable significance with the advent of e-commerce and the whole industry. Four variations of the dataset have been downloaded hundreds of thousands of times, reflecting their popularity and distinctive contribution in the field of recommendation systems and connected subjects. This paper exploits the movie recommender system based on movies' genres and actors/actresses themselves as the input tags, or tag interpolation. The problem has not been addressed in the literature, especially for the 100K variations of the MovieLens datasets. We apply tag-based filtering and collaborative filtering that can effectively predict a list of movies that is similar to the movie that a user has been watched. Due to not depending on users' profiles, our model has eliminated the e.ect of the cold-start problem. The experimental results provide us much better recommendations to users because it utilizes the underlying relation between movies based on their similar genres and actors/actresses. A movie recommender system has been deployed to demonstrate our work.

This paper describes KIT’s submissions to the IWSLT2020 Speech Translation evaluation campaign. We first participate in the simultaneous translation task, in which our simultaneous models are Transformer based and can be efficiently trained to obtain low latency with minimized compromise in quality. On the offline speech translation task, we applied our new Speech Transformer architecture to end-to-end speech translation. The obtained model can provide translation quality which is competitive to a complicated cascade. The latter still has the upper hand, thanks to the ability to transparently access to the transcription, and resegment the inputs to avoid fragmentation.

—A movie recommender system has been proven to be a convincing implement on carrying out comprehensive and complicated recommendation which helps users find appropriate movies conveniently. It follows a mechanism that a user can be accurately recommended movies based on other similar interests, e.g. collaborative filtering, and the movies themselves, e.g. content-based filtering. Therefore, the systems should come with predeter-mined information either by users or by movies. One interesting research question should be asked: “what if this information is missing or not manually manipulated?” The problem has not been addressed in the literature, especially for the 100K and 1M variations of the MovieLens datasets. This paper exploits the movie recommender system based on movies’ genres and actors/actresses themselves as the input tags or tag interpolation. We apply tag-based filtering and collaborative filtering that can effectively predict a list of movies that is similar to the movie that a user has been watched. Due to not depending on users’ profiles, our approach has eliminated the effect of the cold-start problem. The experiment results obtained on MovieLens datasets indicate that the proposed model may contribute ade-quate performance regarding efficiency and reliability, and thus provide better-personalized movie recommendations. A movie recommender system has been deployed to demonstrate our work. The collected datasets have been published on our Github repository to encourage further reproducibility and improvement.

Deep Neural Networks (DNNs) are widely being adopted for safety-critical applications, e.g., healthcare and autonomous driving. Inherently, they are considered to be highly error-tolerant. However, recent studies have shown that hardware faults that impact the parameters of a DNN (e.g., weights) can have drastic impacts on its classification accuracy. In this paper, we perform a comprehensive error resilience analysis of DNNs subjected to hardware faults (e.g., permanent faults) in the weight memory. The outcome of this analysis is leveraged to propose a novel error mitigation technique which squashes the high-intensity faulty activation values to alleviate their impact. We achieve this by replacing the unbounded activation functions with their clipped versions. We also present a method to systematically define the clipping values of the activation functions that result in increased resilience of the networks against faults. We evaluate our technique on the AlexNet and the VGG-16 DNNs trained for the CIFAR-10 dataset. The experimental results show that our mitigation technique significantly improves the resilience of the DNNs to faults. For example, the proposed technique offers on average 68.92% improvement in the classification accuracy of resilience-optimized VGG-16 model at 1e-5 fault rate, when compared to the base network without any fault mitigation., The 23rd Design, Automation and Test in Europe (DATE 2020)

Among the six challenges of neural machine translation (NMT) coined by (Koehn and Knowles, 2017), rare-word problem is considered the most severe one, especially in translation of low-resource languages. In this paper, we propose three solutions to address the rare words in neural machine translation systems. First, we enhance source context to predict the target words by connecting directly the source embeddings to the output of the attention component in NMT. Second, we propose an algorithm to learn morphology of unknown words for English in supervised way in order to minimize the adverse effect of rare-word problem. Finally, we exploit synonymous relation from the WordNet to overcome out-of-vocabulary (OOV) problem of NMT. We evaluate our approaches on two low-resource language pairs: English-Vietnamese and Japanese-Vietnamese. In our experiments, we have achieved significant improvements of up to roughly +1.0 BLEU points in both language pairs.

The standardization of the first phase 5G system in Release 15 named New Radio (NR) will soon move on to the stand-alone version with intention to bring it commercially to the market around 2020. This paper investigates the impact of power amplifier in OFDM-based waveforms including CP-OFDM, F-OFDM and W-OFDM. We first derive a mathematical model for these OFDM-based waveforms. Then we analyze the impact of power amplifier (PA) to the out-of-band (OOB) waveform, which is a critical component in the densely spectrum with different channel widths using scalable numerologies. It is shown that W-OFDM and F-OFDM are more suitable than the conventional CP-OFDM. The provided results of this paper thus facilitate knowledge of design and practice for further NR features. As the simulation results, without PA clipping, the W-OFDM and F-OFDM present better OOB characteristic than CP-OFDM, while applying PA clipping both of them show the similar performance. Additionally, another W-OFDM design with PA clipping shows advantages to implement for 5G NR waveform.

Transmission Control Protocol (TCP) in infrastructure based vehicular networks is dedicated to support reliable Internet services for mobile users. However, an end-toend TCP flow not only experiences some common challenges in wireless mobile networks, such as high packet loss rate, medium access competition, unstable wireless bandwidth, and dynamic topology, etc., but also suffers from performance degradation due to traffic congestion at the Road-Side Units (RSUs) that connect the wireline and wireless networks. In order to address the challenging issues related to reliable TCP transmissions in infrastructure based vehicular networks, we propose an RSU based TCP (R-TCP) scheme. For wireline source nodes, R-TCP adopts a novel flow control mechanism to adjust transmission rates according to the status of bottleneck link. Specifically, during the short wireless connection time in Infrastructure based vehicular networks, R-TCP quickly chooses an ideal transmission rate for data transmissions instead of activating the slow start algorithm after the connection is established, and successfully avoids the oscillation of the transmission rate. Simulation results show that R-TCP achieves great advantages than some relate proposals in terms of throughput, end-to-end delay, and packet loss rate.

Neural machine translation (NMT) systems have recently obtained state-of-the art in many machine translation systems between popular language pairs because of the availability of data. For low-resourced language pairs, there are few researches in this field due to the lack of bilingual data. In this paper, we attempt to build the first NMT systems for a low-resourced language pairs:Japanese-Vietnamese. We have also shown significant improvements when combining advanced methods to reduce the adverse impacts of data sparsity and improve the quality of NMT systems. In addition, we proposed a variant of Byte-Pair Encoding algorithm to perform effective word segmentation for Vietnamese texts and alleviate the rare-word problem that persists in NMT systems.

Through the development of neural machine translation, the quality of machine translation systems has been improved significantly. By exploiting advancements in deep learning, systems are now able to better approximate the complex mapping from source sentences to target sentences. But with this ability, new challenges also arise. An example is the translation of partial sentences in low-latency speech translation. Since the model has only seen complete sentences in training, it will always try to generate a complete sentence, though the input may only be a partial sentence. We show that NMT systems can be adapted to scenarios where no task-specific training data is available. Furthermore, this is possible without losing performance on the original training data. We achieve this by creating artificial data and by using multi-task learning. After adaptation, we are able to reduce the number of corrections displayed during incremental output construction by 45%, without a decrease in translation quality., Comment: 5 Pages; Interspeech

In the project of testing platform, characteristics of the measured object are framework. In view of the difficulties of non-contact high-precision measurement of the planar motion objects, an innovative measuring method is proposed which is based on the combined using of high-precision laser distance sensors. This paper presents the structure of the new sensor and its measuring principle, and established a mathematical model of the measurement object and its error model by using differential method. We use the Monte Carlo method to random sample the random sampling of random variables of the error model; finally combining with the instance; we use the Monte Carlo method to get frequency distribution histogram of the measurement error. Through analysis and calculation of this paper, we prepare for theory for the practical application of this new type of sensor.

In this paper, we offer an in-depth analysis about the modeling and search performance. We address the question if a more complex search algorithm is necessary. Furthermore, we investigate the question if more complex models which might only be applicable during rescoring are promising. By separating the search space and the modeling using $n$-best list reranking, we analyze the influence of both parts of an NMT system independently. By comparing differently performing NMT systems, we show that the better translation is already in the search space of the translation systems with less performance. This results indicate that the current search algorithms are sufficient for the NMT systems. Furthermore, we could show that even a relatively small $n$-best list of $50$ hypotheses already contain notably better translations., Comment: 7 pages, First Workshop on Neural Machine Translation

In mobile base station antennas, linear array antennas are employed to achieve shaped beams in the vertical plane. Also, antennas are designed to achieve multi-frequency use. In the case of expanding multi-frequency range, conventional equal element spacing configuration shows a defect of producing grating lobes. In order to suppress grating lobes, unequal element spacing is promising. In this paper, a radiation pattern synthesis based on a least mean square method (LMS) is applied to unequally spaced array configurations. In frequencies of 900 MHz, 1.5 GHz, 2.0 GHz, and 2.5 GHz, calculated results are presented. Suppression of grating lobes and increase of antenna gain are shown. Also, simulation results by FEKO using electric point source and dipole as array elements are shown to confirm accuracy of this method.

This paper presented the joined submission of KIT and LIMSI to the English to German translation task of WMT 2015. In this year submission, we integrated a neural network-based translation model into a phrase-based translation model by rescoring the n-best lists. Since the computation complexity is one of the main issues for continuous space models, we compared two techniques to reduce the computation cost. We investigated models using a structured output layer as well as models trained with noise contrastive estimation. Furthermore, we evaluated a new method to obtain the best log-linear combination in the rescoring phase. Using these techniques, we were able to improve the BLEU score of the baseline phrase-based system by 1.4 BLEU points.

In this paper, the KIT systems submitted to the Shared Translation Task are presented. We participated in two translation directions: from German to English and from English to German. Both translations are generated using phrase-based translation systems. The performance of the systems was boosted by using language models built based on different tokens such as word, part-of-speech, and automacally generated word clusters. The difference in word order between German and English is addressed by part-of-speech and syntactic tree-based reordering models. In addition to a discriminative word lexicon, we used hypothesis rescoring using the ListNet algorithm after generating the translation with the phrase-based system. We evaluated the rescoring using only the baseline features as well as using additional computational complex features.

Flocking behavior is a common phenomenon in nature, such as flocks of birds and groups of fish. In order to make the agents effectively avoid obstacles and fast form flocking towards the direction of destination point, this paper proposes a fast multiagent obstacle avoidance (FMOA) algorithm. FMOA is illustrated based on the status of whether the flocking has formed. If flocking has not formed, agents should avoid the obstacles toward the direction of target. If otherwise, these agents have reached the state of lattice and then these agents only need to avoid the obstacles and ignore the direction of target. The experimental results show that the proposed FMOA algorithm has better performance in terms of flocking path length. Furthermore, the proposed FMOA algorithm is applied to the formation flying of quad-rotor helicopters. Compared with other technologies to perform the localization of quad-rotor helicopter, this paper innovatively constructs a smart environment by deploying some wireless sensor network (WSN) nodes using the proposed localization algorithm. Finally, the proposed FMOA algorithm is used to conduct the formation flying of these quad-rotor helicopters in the smart environment.

In this paper, we present the KIT systems participating in the Shared Translation Task translating between English$German and English$French. All translations are generated using phrase-based translation systems, using different kinds of word-based, part-ofspeech-based and cluster-based language models trained on the provided data. Additional models include bilingual language models, reordering models based on part-of-speech tags and syntactic parse trees, as well as a lexicalized reordering model. In order to make use of noisy web-crawled data, we apply filtering and data selection methods for language modeling. A discriminative word lexicon using source context information proved beneficial for all translation directions.

Provides insights into the process of knowledge construction in EFL/ESL writing - from classrooms to research sites, from the dilemmas and risks NNEST student writers experience in the pursuit of true agency to the confusions and conflicts academics experience in their own writing practices.

A general statement of face recognition problem (in computer vision) can be formulated as follows: Given still or video images of a scene, identify or verify one or more persons in the scene using a database of stored faces. For the fully automatic system of face recognition, a lot of researches have been done in many directions that eliminate the difficulties of finding and identifying faces with the illumination and pose problems. However, most of them just concentrate on these issues separately instead of consider the efficiency of the whole system while both method in face detection and recognition work together. This research tried to make compensation between those two main components. In the recognition step, eigenface technique was used to normalize faces in the database. At first, we calculate the face covariance matrix and the associated face eigenvectors. These will be the face prints. Different vectors of weights can represent different faces. After that, any face can be reconstructed from a set of weights. So we can recognize a new picture of a familiar face. For the face detection process, skin-color model was used to separate skin areas from the background in order to cut out the face block for recognizing. The whole process of face recognition can be separate into two phases: Training phase and Recognition phase. At first, we train the recognition system by using a sequence of images for each person to create the alternative face print. In the second phase, the result of the first phase will be used to match with the input face image in the video stream and then show the conclusion on whether people in video is granted or not. The experiments included both aspects: independent component and holistic system on the popular databases of face detection, recognition in still image and video. The result achieved a recognition rate at from 80 percents to 100 percents with a small number of 10 or 24 training samples.