Search

The attention mechanism of the Listen, Attend and Spell (LAS) model requires the whole input sequence to calculate the attention context and thus is not suitable for online speech recognition. To deal with this problem, we propose multi-head monotonic chunk-wise attention (MTH-MoChA), an improved version of MoChA. MTH-MoChA splits the input sequence into small chunks and computes multi-head attentions over the chunks. We also explore useful training strategies such as LSTM pooling, minimum world error rate training and SpecAugment to further improve the performance of MTH-MoChA. Experiments on AISHELL-1 data show that the proposed model, along with the training strategies, improve the character error rate (CER) of MoChA from 8.96% to 7.68% on test set. On another 18000 hours in-car speech data set, MTH-MoChA obtains 7.28% CER, which is significantly better than a state-of-the-art hybrid system.

Vehicle tracking has become one of the key applications of wireless sensor networks (WSNs) in the fields of rescue, surveillance, traffic monitoring, etc. However, the increased tracking accuracy requires more energy consumption. In this letter, a decentralized vehicle tracking strategy is conceived for improving both tracking accuracy and energy saving, which is based on adjusting the intersection area between the fixed sensing area and the dynamic activation area. Then, two deep reinforcement learning (DRL) aided solutions are proposed relying on the dynamic selection of the activation area radius. Finally, simulation results show the superiority of our DRL aided design.

In this paper, we make an investigation of receive antenna selection (RAS) strategies in the secure pre-coding aided spatial modulation (PSM) system with the aid of artificial noise. Due to a lack of the closed-form expression for secrecy rate (SR) in secure PSM systems, it is hard to optimize the RAS. To address this issue, the cut-off rate is used as an approximation of the SR. Further, two low-complexity RAS schemes for maximizing SR, called Max-SR-L and Max-SR-H, are derived in the low and high signal-to-noise ratio (SNR) regions, respectively. Due to the fact that the former works well in the low SNR region but becomes worse in the medium and high SNR regions while the latter also has the similar problem, a novel RAS strategy Max-SR-A is proposed to cover all SNR regions. Simulation results show that the proposed Max-SR-H and Max-SR-L schemes approach the optimal SR performances of the exhaustive search (ES) in the high and low SNR regions, respectively. In particular, the SR performance of the proposed Max-SR-A is close to that of the optimal ES and better than that of the random method in almost all SNR regions.

Unlike typical phase transitions of first and second order, a system displaying the Thouless effect exhibits characteristics of both at the critical point (jumps in the order parameter and anomalously large fluctuations). An $extreme$ Thousless effect was observed in a recently introduced model of social networks consisting of `introverts and extroverts' ($XIE$). We study the fluctuations and correlations of this system using both Monte Carlo simulations and analytic methods based on a self-consistent mean field theory. Due to the symmetries in the model, we derive identities between all independent two point correlations and fluctuations in three quantities (degrees of individuals and the total number of links between the two subgroups) in the stationary state. As simulations confirm these identities, we study only the fluctuations in detail. Though qualitatively similar to those in the 2D Ising model, there are several unusual aspects, due to the extreme Thouless effect. All these anomalous fluctuations can be quantitatively understood with our theory, despite the mean-field aspects in the approximations. In our theory, we frequently encounter the `finite Poisson distribution' (i.e., $x^{n}/n!$ for $n\in \left[ 0,N\right] $ and zero otherwise). Since its properties appear to be quite obscure, we include an Appendix on the details and the related `finite exponential series' $\sum_{0}^{N}x^{n}/n!$. Some simulation studies of joint degree distributions, which provide a different perspective on correlations, have also been carried out.

It is well known that a speech recognition system that combines multiple acoustic models trained on the same data significantly outperforms a single-model system. Unfortunately, real time speech recognition using a whole ensemble of models is too computationally expensive. In this paper, we propose to distill the knowledge of essence in an ensemble of models (i.e. the teacher model) to a single model (i.e. the student model) that needs much less computation to deploy. Previously, all the soften outputs of the teacher model are used to optimize the student model. We argue that not all the outputs of the ensemble are necessary to be distilled. Some of the outputs may even contain noisy information that is useless or even harmful to the training of the student model. In addition, we propose to train the student model with a multitask learning approach by utilizing both the soften outputs of the teacher model and the correct hard labels. The proposed method achieves some surprising results on the Switchboard data set. When the student model is trained together with the correct labels and the essence knowledge from the teacher model, it not only significantly outperforms another single model with the same architecture that is trained only with the correct labels, but also consistently outperforms the teacher model that is used to generate the soft labels., Comment: 5 pages, 2 figures