Search

Long-term visual place recognition (VPR) is challenging as the environment is subject to drastic appearance changes across different temporal resolutions, such as time of the day, month, and season. A wide variety of existing methods address the problem by means of feature disentangling or image style transfer but ignore the structural information that often remains stable even under environmental condition changes. To overcome this limitation, this article presents a novel structure-aware feature disentanglement network (SFDNet) based on knowledge transfer and adversarial learning. Explicitly, probabilistic knowledge transfer (PKT) is employed to transfer knowledge obtained from the Canny edge detector to the structure encoder. An appearance teacher module is then designed to ensure that the learning of appearance encoder does not only rely on metric learning. The generated content features with structural information are used to measure the similarity of images. We finally evaluate the proposed approach and compare it to state-of-the-art place recognition methods using six datasets with extreme environmental changes. Experimental results demonstrate the effectiveness and improvements achieved using the proposed framework. Source code and some trained models will be available at http://www.tianshu.org.cn.

Traffic forecasting has attracted considerable attention due to its importance in proactive urban traffic control and management. Scholars and engineers have exerted considerable efforts in improving the performance of traffic forecasting algorithms in terms of accuracy, reliability, and efficiency. Spatial feature representation of traffic flow is a core component that greatly influences traffic forecasting performance. In previous studies, several spatial attributes of traffic flow are ignored due to the following issues: a) traffic flow propagation does not comply with the road network, b) the spatial pattern of traffic flow varies over time, and c) single adjacent matrix cannot handle the complex and hierarchical urban traffic flow. To address the abovementioned issues, this study proposes a novel traffic forecasting algorithm called traffic transformer, which achieves great success in natural language processing. The multihead attention mechanism and stacking layers enable the transformer to learn dynamic and hierarchical features in sequential data. Two components, namely, global encoder and global-local decoder, are proposed to extract and fuse the spatial patterns globally and locally. Experimental results indicate that the proposed traffic transformer outperforms state-of-the-art methods. The learned dynamic and hierarchical features of traffic flow can help achieve a better understanding of spatial dependency of traffic flow for effective and efficient traffic control and management strategies.

In this paper, we propose a novel deep Efficient Relational Sentence Ordering Network (referred to as ERSON) by leveraging pre-trained language model in both encoder and decoder architectures to strengthen the coherence modeling of the entire model. Specifically, we first introduce a divide-and-fuse BERT (referred to as DF-BERT), a new refactor of BERT network, where lower layers in the improved model encode each sentence in the paragraph independently, which are shared by different sentence pairs, and the higher layers learn the cross-attention between sentence pairs jointly. It enables us to capture the semantic concepts and contextual information between the sentences of the paragraph, while significantly reducing the runtime and memory consumption without sacrificing the model performance. Besides, a Relational Pointer Decoder (referred to as RPD) is developed, which utilizes the pre-trained Next Sentence Prediction (NSP) task of BERT to capture the useful relative ordering information between sentences to enhance the order predictions. In addition, a variety of knowledge distillation based losses are added as auxiliary supervision to further improve the ordering performance. The extensive evaluations on Sentence Ordering, Order Discrimination, and Multi-Document Summarization tasks show the superiority of ERSON to the state-of-the-art ordering methods.

Unsupervised domain adaptation (UDA) aims at learning a classifier for an unlabeled target domain by transferring knowledge from a labeled source domain with a related but different distribution. Most existing approaches learn domain-invariant features by adapting the entire information of the images. However, forcing adaptation of domain-specific variations undermines the effectiveness of the learned features. To address this problem, we propose a novel, yet elegant module, called the deep ladder-suppression network (DLSN), which is designed to better learn the cross-domain shared content by suppressing domain-specific variations. Our proposed DLSN is an autoencoder with lateral connections from the encoder to the decoder. By this design, the domain-specific details, which are only necessary for reconstructing the unlabeled target data, are directly fed to the decoder to complete the reconstruction task, relieving the pressure of learning domain-specific variations at the later layers of the shared encoder. As a result, DLSN allows the shared encoder to focus on learning cross-domain shared content and ignores the domain-specific variations. Notably, the proposed DLSN can be used as a standard module to be integrated with various existing UDA frameworks to further boost performance. Without whistles and bells, extensive experimental results on four gold-standard domain adaptation datasets, for example: 1) Digits; 2) Office31; 3) Office-Home; and 4) VisDA-C, demonstrate that the proposed DLSN can consistently and significantly improve the performance of various popular UDA frameworks.

Part-of-Speech (POS) tagging is a fundamental sequence labeling problem in Natural Language Processing. Recent deep learning sequential models combine the forward and backward word informatio for POS tagging. The information of contextual words to the current word play a vital role in capturing the non-continuous relationship. We have proposed Monotonic chunk-wise attention with CNN-GRU-Softmax (MCCGS), a deep learning architecture that adheres to these essential information. This architecture consists of Input Encoder (IE), encodes word and character-level, Contextual Encoder (CE), assigns the weightage to adjacent word and Disambiguator (D), which resolves intra-label dependencies as core components. Moreover, different morphological features have been integrated into the core components of MCCGS architecture as MCCGS-IE, MCCGS-CE and MCCGS-D. The MCCGS architecture is validated on the 21 languages from Universal Dependency (UD) treebank. The state-of-the-art models, Type constraints, Retrofitting, Distant Supervision from Disparate Sources and Position-aware Self Attention, MCCGS and its variants such as MCCGS-IE, MCCGS-CE and MCCGS-D are obtained mean accuracy 83.65 % , 81.29 % , 84.10 % , 90.18 % , 90.40 % , 91.40 % , 90.90 % , 92.30 % , respectively. The proposed model architecture provides state-of-the-art accuracy on the low resource languages as Marathi ( 93.58 % ), Tamil ( 87.50 % ), Telugu ( 96.69 % ) and Sanskrit ( 97.28 % ) from UD treebank and Hindi ( 95.64 % ) and Urdu ( 87.47 % ) from Hindi-Urdu multi-representational treebank.

Phrase-indexed question answering (PIQA) seeks to improve the inference speed of question answering (QA) models by enforcing complete independence of the document encoder from the question encoder, and it shows that the constrained model can achieve significant efficiency at the cost of its accuracy. In this paper, we aim to build a model under the PIQA constraint while reducing its accuracy gap with the unconstrained QA models. We propose a novel framework—AnsDR, which consists of an answer boundary detector (AnsD) and an answer candidate ranker (AnsR). More specifically, AnsD is a QA model under the PIQA architecture and it is designed to identify the rough answer boundaries; and AnsR is a lightweight ranking model to finely re-rank the potential candidates without losing the efficiency. We perform the extensive experiments on public datasets. The experimental results show that the proposed method achieves the state of the art on the PIQA task.

In this work, a reverse saliency (RS) spoke-type permanent magnet (PM) synchronous motor (PMSM) besides its hybrid adaptive notch filter (HANF) based self-sensing for driving a solar photovoltaic water pump system (WPS) is presented. The conventional spoke-PMSM driven WPS experiences two types of problems. Firstly, the motor is operated with a flux-weakening current to produce positive reluctance torque, thereby increasing the PM demagnetization risk. Secondly, an encoder is used to sense the rotor angle, which in turn affects the cost and reliability. To solve the first problem, the RS-PMSM is presented, which operates with a flux-intensifying current to produce positive reluctance torque. While second issue is resolved by using the HANF based rotor angle estimation. Although conventional methods give an estimate of rotor angle, they do not eliminate both the DC-offset and dominant harmonics. The superiority of both the RS-PMSM and HANF over existing solutions, is shown respectively by using finite-element methods and MATLAB/ Simulink based performances. Besides, a prototype motor is also manufactured. The performance of RS-PMSM prototype along with its HANF based self-sensing is obtained in the laboratory and various test results are reported. Meanwhile, a comparison of the HANF with available self-sensing techniques, is also performed.

In the natural language processing family, learning representations is a pioneering study, especially in sequence-to-sequence tasks where outputs are generated, totally relying on the learning representations of source sequence. Generally, classic methods infer that each word occurring in the source sequence, having more or less influence on the target sequence, should all be considered when generating outputs. As the summarization task requires the output sequence to only retain the essence, classic full consideration of the source sequence may not work well on it, which calls for more suitable methods with the ability to discard the misleading noise words. Motivated by this, with both relevance retaining and redundancy removal in mind, we propose a summarization learning model by implementing an encoder with copious contextual information represented and a decoder with a selecting mechanism integrated. Specifically, we equip the encoder with an asynchronous bi directional parallel structure, in order to obtain abundant semantic representation. The decoder, different from the classic attention-based works, employs a self-aware context selecting mechanism to generate summary in a more productive way. We evaluate the proposed methods on three benchmark summarization corpora. The experimental results demonstrate the effectiveness and applicability of the proposed framework in relation to several well-practiced and state-of-the-art summarization methods.

In today's digital world, we are equipped with modern computer-based data collection sources and feature extraction methods. It enhances the availability of the multi-view data and corresponding researches. Multi-view prediction models form a mainstream research direction in the healthcare and bioinformatics domain. While these models are designed with the assumption that there is no missing data for any views, in the real world, certain views of the data are often not having the same number of samples, resulting in the incomplete multi-view dataset. The studies performed over these datasets are termed incomplete multi-view clustering or prediction. Here, we develop a two-stage generative incomplete multi-view prediction model named GIMPP to address the missing view problem of breast cancer prognosis prediction by explicitly generating the missing data. The first stage incorporates the multi-view encoder networks and the bi-modal attention scheme to learn common latent space representations by leveraging complementary knowledge between different views. The second stage generates missing view data using view-specific generative adversarial networks conditioned on the shared representations and encoded features given by other views. Experimental results on TCGA-BRCA and METABRIC datasets proves the usefulness of the developed method over the state-of-the-art methods.

Traffic flow prediction is an important part of the intelligent transportation system. Accurate multi-step traffic flow prediction plays an important role in improving the operational efficiency of the traffic network. Since traffic flow data has complex spatio-temporal correlation and non-linearity, existing prediction methods are mainly accomplished through a combination of a Graph Convolutional Network (GCN) and a recurrent neural network. The combination strategy has an excellent performance in traffic prediction tasks. However, multi-step prediction error accumulates with the predicted step size. Some scholars use multiple sampling sequences to achieve more accurate prediction results. But it requires high hardware conditions and multiplied training time. Considering the spatiotemporal correlation of traffic flow and influence of external factors, we propose an Attention Based Spatio-Temporal Graph Convolutional Network considering External Factors (ABSTGCN-EF) for multi-step traffic flow prediction. This model models the traffic flow as diffusion on a digraph and extracts the spatial characteristics of traffic flow through GCN. We add meaningful time-slots attention to the encoder-decoder to form an Attention Encoder Network (AEN) to handle temporal correlation. The attention vector is used as a competitive choice to draw the correlation between predicted states and historical states. We considered the impact of three external factors (daytime, weekdays, and traffic accident markers) on the traffic flow prediction tasks. Experiments on two public data sets show that it makes sense to consider external factors. The prediction performance of our ABSTGCN-EF model achieves 7.2%–8.7% higher than the state-of-the-art baselines.

The perception and expression of emotion, as well as the content of a text, are key factors in the success of conversational agents. However, previous models for conversation generation handled single-language pairs during training and testing, neglecting the complementary information from different languages. In this article, we propose a bilingual-aided interactive approach that can simultaneously and interactively generate bilingual emotional replies to monolingual posts. Specifically, the generation of one emotional reply relies on the output of the encoder, the generated tokens, and the interactive information from the other language decoder. The interactive approach includes 1) internal interaction to capture the change in implicit contextual information and 2) external interaction to balance grammaticality and the expression of emotion. Qualitative and quantitative experiments with NLPCC2017 show that our model performs better in terms of the content and emotion of replies than several state-of-the-art approaches.

Understanding human language is one of the key themes of artificial intelligence. For language representation, the capacity of effectively modeling the linguistic knowledge from the detail-riddled and lengthy texts and getting rid of the noises is essential to improve its performance. Traditional attentive models attend to all words without explicit constraint, which results in inaccurate concentration on some dispensable words. In this work, we propose using syntax to guide the text modeling by incorporating explicit syntactic constraints into attention mechanisms for better linguistically motivated word representations. In detail, for self-attention network (SAN) sponsored Transformer-based encoder, we introduce syntactic dependency of interest (SDOI) design into the SAN to form an SDOI-SAN with syntax-guided self-attention. Syntax-guided network (SG-Net) is then composed of this extra SDOI-SAN and the SAN from the original Transformer encoder through a dual contextual architecture for better linguistics inspired representation. The proposed SG-Net is applied to typical Transformer encoders. Extensive experiments on popular benchmark tasks, including machine reading comprehension, natural language inference, and neural machine translation show the effectiveness of the proposed SG-Net design., The early version accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI). Journal extension of arXiv:1908.05147 (AAAI 2020)

According to a recent Deloitte study, the COVID-19 pandemic continues to place a huge strain on the global health care sector. Covid-19 has also catalysed digital transformation across the sector for improving operational efficiencies. As a result, the amount of digitally stored patient data such as discharge letters, scan images, test results or free text entries by doctors has grown significantly. In 2020, 2314 exabytes of medical data was generated globally. This medical data does not conform to a generic structure and is mostly in the form of unstructured digitally generated or scanned paper documents stored as part of a patient’s medical reports. This unstructured data is digitised using Optical Character Recognition (OCR) process. A key challenge here is that the accuracy of the OCR process varies due to the inability of current OCR engines to correctly transcribe scanned or handwritten documents in which text may be skewed, obscured or illegible. This is compounded by the fact that processed text is comprised of specific medical terminologies that do not necessarily form part of general language lexicons. The proposed work uses a deep neural network based self-supervised pre-training technique: Robustly Optimized Bidirectional Encoder Representations from Transformers (RoBERTa) that can learn to predict hidden (masked) sections of text to fill in the gaps of non-transcribable parts of the documents being processed. Evaluating the proposed method on domain-specific datasets which include real medical documents, shows a significantly reduced word error rate demonstrating the effectiveness of the approach.

Due to the importance of practical applications, unsupervised domain adaptation (UDA) person re-identification (re-ID) has attracted increasing attention. However, most of existing methods often lack the multi-view information reasoning and ignore the domain discrepancy of the pedestrian images with the same identity, which constrain the further improvement of recognition performance. So, this paper proposes a triple adversarial learning and multi-view imaginative reasoning network (TAL-MIRN) for UDA person re-ID, which consists of a multi-view imaginative reasoning module (IRM) and a triple adversarial learning module (TALM). IRM makes the classified pedestrian identity features from a single-view image extracted by a feature encoder consistent with the classification results of the aggregated multi-view pedestrian identity features, so the strong multi-view imaginative reasoning ability of the feature encoder is obtained. TALM is composed by the adversarial learning between the camera classifier and feature encoder, adversarial learning of joint distribution alignment, and adversarial learning of the difference between two classifiers used in classification. In particular, the domain-invariant features at camera level are guaranteed by the adversarial learning between the feature extractor and camera classifier. The joint alignment of identity and domain is achieved by the competition between the feature extractor and classifier integrated with identity and domain. The discriminability and robustness of the learned features are enhanced by playing a MinMax game between two different identity classifiers. Furthermore, a simple normalization operation named as cross normalization (CN) is proposed to increase both modeling and generalization capability of the proposed TAL-MIRN across multiple domains. The proposed TAL-MIRN is applied to five benchmark datasets, and the comparative experimental results confirm its superiority over the state-of-the-art methods. The related source codes is available at https://github.com/lhf12278/TALM-IRM.

It has been proved that long noncoding RNA (lncRNA) plays critical roles in many human diseases. Therefore, inferring associations between lncRNAs and diseases can contribute to disease diagnosis, prognosis and treatment. To overcome the limitation of traditional experimental methods such as expensive and time-consuming, several computational methods have been proposed to predict lncRNA-disease associations by fusing different biological data. However, the prediction performance of lncRNA-disease associations identification needs to be improved. In this study, we propose a computational model (named LDICDL) to identify lncRNA-disease associations based on collaborative deep learning. It uses an automatic encoder to denoise multiple lncRNA feature information and multiple disease feature information, respectively. Then, the matrix decomposition algorithm is employed to predict the potential lncRNA-disease associations. In addition, to overcome the limitation of matrix decomposition, the hybrid model is developed to predict associations between new lncRNA (or disease) and diseases (or lncRNA). The ten-fold cross validation and de novo test are applied to evaluate the performance of method. The experimental results show LDICDL outperforms than other state-of-the-art methods in prediction performance.

The accurate forecast of the number of effective telegram frames (NETF) received by the balise transmission module (BTM) at different speeds is the key to assessing its speed adaptability. However, the NETF index has a complex correlation with multiple BTM transmission parameters and exhibits highly nonlinear variations in the speed dimension, which poses a challenge to the NETF series prediction. Thus, a deep-adaptive-attention-based model is proposed in this study, which combines two enhanced attention mechanisms into the long short-term memory encoder-decoder network to thoroughly explore the spatial-speed relationship between the historical multivariate series and the future NETF series. Specifically, the proposed model consists of an encoder with a dual-phase spatial-attention mechanism and a decoder with an adaptive speed-attention mechanism, which can jointly capture the stable spatial correlation and relevant long-term speed-dependent information of the input multivariate series across all speed steps for the robust and accurate prediction of the NETF series. Experimental results on real-world BTM datasets demonstrated that the proposed model can predict the NETF series with the lowest local predicted error and highest overall prediction accuracy, compared with other state-of-the-art attention-based methods.

Along with the rapid-growth number of Internet of-Things(IoT) devices, significant security concerns are raised due to the hidden vulnerabilities among them. Illuminating the characteristics of online devices would shed a light on protecting these potential vulnerable devices. State-of-arts methodologies enumerate devices characteristics as keywords and rules and match them with IoT network data. However, the heterogeneous implementations of IoT devices introduce intricate characteristics features, which impede the large-scale identification. In this work, we close this gap and present a semantic extraction based approach that can automatically and effectively characterize online devices. We leverage the observation that IoT devices can be identified by analyzing the semantic information of the network packets. Specifically, we first collect the network data of IoT devices and utilize a co-training algorithm to annotate the data. We propose a Residual Dilate Gated Convolutional Neural Network(RDGCNN) based encoder to extract semantic features from the annotated data. Then, we put forward an entity relationship based decoder to generate the characteristic triplet (type, brand, model) of IoT devices by decoding extracted features. We have implemented the prototype of the system and conducted real-world experiments to evaluate the performance. Results show that our approach achieves 92.16% precision and 86.79% recall. In addition, we apply our proposed method to characterize 15 millions IoT devices on the internet.

Gaming video streaming services have grown tremendously in the past few years, with higher resolutions, higher frame rates and HDR gaming videos getting increasingly adopted among the gaming community. Since gaming content as such is different from non-gaming content, it is imperative to evaluate the performance of the existing encoders to help understand the bandwidth requirements of such services, as well as further improve the compression efficiency of such encoders. Towards this end, we present in this paper GamingHDRVideoSET, a dataset consisting of eighteen 10-bit UHD-HDR gaming videos and encoded video sequences using four different codecs, together with their objective evaluation results. The dataset is available online at [to be added after paper acceptance]. Additionally, the paper discusses the codec compression efficiency of most widely used practical encoders, i.e., x264 (H.264/AVC), x265 (H.265/HEVC) and libvpx (VP9), as well the recently proposed encoder libaom (AV1), on 10-bit, UHD-HDR content gaming content. Our results show that the latest compression standard AV1 results in the best compression efficiency, followed by HEVC, H.264, and VP9., 14 pages, 8 figures, submitted to IEEE journal

Contexts play an important role in salient object detection (SOD). High-level contexts describe the relations between different parts/objects and thus are helpful for discovering the specific locations of salient objects while low-level contexts could provide the fine detail information for delineating the boundary of the salient objects. However, the way of perceiving/leveraging rich contexts has not been fully investigated by existing SOD works. The common context extraction strategies (e.g., leveraging convolutions with large kernels or atrous convolutions with large dilation rates) do not consider the effectiveness and efficiency simultaneously and may cause sub-optimal solutions. In this paper, we devote to exploring an effective and efficient way to learn rich contexts for accurate SOD. Specifically, we first build a dense context exploration (DCE) module to capture dense multi-scale contexts and further leverage the learned contexts to enhance the features discriminability. Then, we embed multiple DCE modules in an encoder-decoder architecture to harvest dense contexts of different levels. Furthermore, we propose an attentive skip-connection to transmit useful features from the encoder part to the decoder part for better dense context exploration. Finally, extensive experiments demonstrate that the proposed method achieves more superior detection results on the six benchmark datasets than 18 state-of-the-art SOD methods.

Named Entity Recognition(NER), one of the most fundamental problems in natural language processing, seeks to identify the boundaries and types of entities with specific meanings in natural language text. As an important international language, Chinese has uniqueness in many aspects, and Chinese NER (CNER) is receiving increasing attention. In this paper, we give a comprehensive survey of recent advances in CNER. We first introduce some preliminary knowledge, including the common datasets, tag schemes, evaluation metrics and difficulties of CNER. Then, we separately describe recent advances in traditional research and deep learning research of CNER, in which the CNER with deep learning is our focus. We summarize related works in a basic three-layer architecture, including character representation, context encoder, and context encoder and tag decoder. Meanwhile, the attention mechanism and adversarial-transfer learning methods based on this architecture are introduced. Finally, we present the future research trends and challenges of CNER.

Multimodal Sentiment Analysis (MSA) is a challenging research area that investigates sentiment expressed from multiple heterogeneous sources of information. To integrate multimodal information including text, visual and audio modalities, state-of-the-art models focus on developing various fusion strategies, such as attention and outer product. However, the inferior quality of visual and audio features that is commonly observed in this area has not aroused much attention. We argue that this issue will obstruct the performance of the fusion strategies to a considerable extent. Therefore, in this paper, we propose Multimodal Translation for Sentiment Analysis (MTSA), a multimodal framework that improves the quality of visual and audio features by translating them to text features extracted by Bidirectional Encoder Representations from Transformers (BERT). Experiments on two benchmark datasets CMU-MOSI and CMU-MOSEI show that our model performs better than the state-of-the-art methods on both datasets across all the metrics, which illustrates the effectiveness of our method.

In this paper, the design and implementation of low power and high-efficiency filterbank for MPEG-2/4 AAC system is presented. Since filterbank represents the most computation-intensive kernel of AAC codec, we design it with algorithm and architecture aspects. We supply the dedicated algorithm for filterbank. The derived algorithm and the hardware shared engine (HSE) we proposed for AAC can reduce the computation power and implement as a codec used in encoder and decoder. We also optimize the performance, hardware resources, and power consumption thoroughly. It is designed as an intellectual property (IP) to construct the overall decoder in an embedded system. The hardware cost is with 16.1 k logic gates, 2 k-word local memory, and 1 K-word coefficient ROM. The proposed design has a real-time operation at only 1.25 MHz with a sampling rate of 48 kHz. It can achieve 0.70 mW power consumption in TSMC 0.18 μm CMOS technology. Furthermore, we use a programmable chip (SOPC) platform which includes the software and hardware engine. Throughout the computation analysis, the bitstream parser and lower complexity part are performed by a software solution, and the higher complexity part is computed by a hardware solution. Several design techniques are needed including the wrapper design, embedded CPU, and IP to construct the system. Based on this co-design approach, a whole AAC audio decoder is also established.

Synthesizing photo-realistic images based on text descriptions is a challenging image generation problem. Although many recent approaches have significantly advanced the performance of text-to-image generation, to guarantee semantic matchings between the text description and synthesized image remains very challenging. In this paper, we propose a new model, Cross-modal Semantic Matching Generative Adversarial Networks (CSM-GAN), to improve the semantic consistency between text description and synthesized image for a fine-grained text-to-image generation. Two new modules are proposed in CSM-GAN: Text Encoder Module (TEM) and Textual-Visual Semantic Matching Module (TVSMM). TVSMM is aimed at making \textcolor{red}{the distance of the pairs of synthesized image and its corresponding text description closer}, in global semantic embedding space, than those of mismatched pairs. This improves the semantic consistency and consequently, the generalizability of CSM-GAN. In TEM, we introduce Text Convolutional Neural Networks (Text\_CNNs) to capture and highlight local visual features in textual descriptions. Thorough experiments on two public benchmark datasets demonstrated the superiority of CSM-GAN over other representative state-of-the-art methods.

Existing studies for multi-source neural machine translation (NMT) either separately model different source sentences or resort to the conventional single-source NMT by simply concatenating all source sentences. However, there exist two drawbacks in these approaches. First, they ignore the explicit word-level semantic interactions between source sentences, which have been shown effective in the embeddings of multilingual texts. Second, multiple source sentences are simultaneously encoded by an NMT model, which is unable to fully exploit the semantic information of each source sentence. In this paper, we explore multi-stage information interactions for multi-source NMT. Specifically, we first propose a multi-source NMT model that performs information interactions at the encoding stage. Its encoder contains multiple semantic interaction layers, each of which sequentially consists of (1) monolingual semantic interaction sub-layer, which is based on the self-attention mechanism and used to learn word-level monolingual contextual representations of source sentences, and (2) cross-lingual semantic interaction sub-layer, which leverages word alignments to perform fine-grained semantic transitions among hidden states of different source sentences. Furthermore, at the training stage, we introduce a mutual distillation based training framework, where single-source models and ours perform information interactions. Such framework can fully exploit the semantic information of each source sentence to enhance our model. Extensive experimental results on the WMT14 English-German-French dataset show our model exhibits significant improvements upon competitive baselines.

Generative Adversarial Networks (GANs) are capable of effectively synthesising new realistic images and estimating the potential distribution of samples utilising adversarial learning. Nevertheless, conventional GANs require a large amount of training data samples to produce plausible results. Inspired by the capacity for humans to quickly learn new concepts from a small number of examples, several meta-learning approaches for the few-shot datasets are presented. However, most of meta-learning algorithms are designed to tackle few-shot classification and reinforcement learning tasks. Moreover, the existing meta-learning models for image generation are complex, thereby affecting the length of training time required. Fast Adaptive Meta-Learning (FAML) based on GAN and the encoder network is proposed in this study for few-shot image generation. This model demonstrates the capability to generate new realistic images from previously unseen target classes with only a small number of examples required. With 10 times faster convergence, FAML requires only one-fourth of the trainable parameters in comparison baseline models by training a simpler network with conditional feature vectors from the encoder, while increasing the number of generator iterations. The visualisation results are demonstrated in the paper. This model is able to improve few-shot image generation with the lowest FID score, highest IS, and comparable LPIPS to MNIST, Omniglot, VGG-Faces, and miniImageNet datasets.

Compared with traditional handcrafted features, deep learning has greatly improved the performance of scene parsing. However, it remains challenging under various environmental conditions caused by imaging limitations. Thermal imaging cameras have several advantages over cameras for the visible spectrum, such as operation in total darkness, robustness to shadow effects, insensitivity to illumination variations, and strong ability to penetrate smog and haze. These advantages of thermal imaging cameras make them ideal for the scene parsing of semantic objects in daytime and nighttime. In this paper, we propose a novel multiscale feature fusion and enhancement network (MFFENet) for accurate parsing of RGBthermal urban road scenes even when the quality of the available RGB data is compromised. The proposed MFFENet consists of two encoders, a feature fusion layer, and a multi-labelsupervision layer. We concatenate the multi-scale features with the features that contain global semantic information. Furthermore, we explore the cross-modal fusion of RGB and thermal features at multiple stages, rather than fusing them once at the low or high stage. Then, we propose a spatial attention mechanism module that provides a higher weight to (focuses more on) the foreground area, allowing MFFENet to emphasize foreground objects. Finally, multi-labelsupervision is introduced to optimize parameters of the proposed MFFENet. Experimental results confirm that the proposed MFFENet outperforms similar high-performing methods.

Automated pancreatic cancer segmentation is crucial for successful clinical aid diagnosis and surgical planning. However, the tiny size and inconspicuous boundaries of pancreatic cancer lesions lead to poor segmentation performance with single-modality imaging. The commonly used registration-based multi-modal fusion strategies may not only introduce uncertainties arising from registration, but also underutilize the complementary information between different modalities. Thus, to achieve different modality-based tumor segmentation, we propose for the first time a registration-free multi-modal and multi-scale adversarial segmentation network (MMSA-Net), which consist of a shared encoder and a dual decoder. Specifically, MMSA-Net combine two complementary modules, inter-modality adversarial learning and intra-modality multi-scale adversarial supervision, to obtain mode-specific segmentation results while facilitating multi-modal fusion. The inter-modality adversarial learning module facilitates the fusion of modality-shared features among different modalities by strengthening the similarity of features extracted by the shared encoder. The intra-modality multi-scale adversarial supervision emphasizes modality-specific features in different decoding paths, inherits and fuses modality-shared features while preserving the feature specificity of each modality, thus outputting competitive modality-specific segmentation results for each modality. Quantitative and qualitative experimental results on multi-parametric MRI pancreatic cancer data show that our method can effectively improve the performance of multi-modal segmentation. The method proposed in this work is expected to be another potential paradigm for addressing multi-modal segmentation tasks in addition to registration. Our source codes will be released at https://github.com/SJTUBME-QianLab/PancreaticCancer_Seg_AdvLearning , once the manuscript is accepted for publication.

Non-parallel data voice conversion is a popular and challenging research area. The main task is to build acoustic mappings from the source speaker to the target speaker in different units (e.g., frame, phoneme, cluster, sentence). With the help of the recent high-quality speech synthesis techniques, it is possible to directly produce parallel speech using non-parallel data. This paper proposes ParaGen: a data augmentation based technique for non-parallel data voice conversion. The system consists of a speaker disentangler based text-to-speech model and a simple frame-to-frame spectrogram conversion model. The text-to-speech model takes the text and reference audio as input to produce the speech in the target speaker identity with the time-aligned local speaking style from the reference audio. The spectrogram conversion model directly converts the source spectrogram to the target speaker framewisely. The local speaking style is extracted by an acoustic encoder while the speaker identity is eliminated by a conditional convolutional disentangler. The local style encodings are time-aligned with the text encodings by the attention mechanism. The attention contexts are decoded by a conditional recurrent decoder. The experiment shows that the speaker identity of the source speech is converted to the target speaker while the local speaking style (e.g., prosody) is preserved after the augmentation. The method is compared to the augmentation model with typical statistical parameter speech synthesis (SPSS) with pre-aligned phoneme duration. The result shows that the converted speech has better speech naturalness than the SPSS system, while the speaker similarities of the converted speech are close.

Code summarization (aka comment generation) provides a high-level natural language description of the function performed by code, which can benefit the software maintenance, code categorization and retrieval. To the best of our knowledge, the state-of-the-art approaches follow an encoder-decoder framework which encodes source code into a hidden space and later decodes it into a natural language space. Such approaches suffer from the following drawbacks: (a) they are mainly input by representing code as a sequence of tokens while ignoring code hierarchy; (b) most of the encoders only input simple features (e.g., tokens) while ignoring the features that can help capture the correlations between comments and code; (c) the decoders are typically trained to predict subsequent words by maximizing the likelihood of subsequent ground truth words, while in real world, they are excepted to generate the entire word sequence from scratch. As a result, such drawbacks lead to inferior and inconsistent comment generation accuracy. To address the above limitations, this paper presents a new code summarization approach using hierarchical attention network by incorporating multiple code features, including type-augmented abstract syntax trees and program control flows. Such features, along with plain code sequences, are injected into a deep reinforcement learning (DRL) framework (e.g., actor-critic network) for comment generation. Our approach assigns weights (pays “attention”) to tokens and statements when constructing the code representation to reflect the hierarchical code structure under different contexts regarding code features (e.g., control flows and abstract syntax trees). Our reinforcement learning mechanism further strengthens the prediction results through the actor network and the critic network, where the actor network provides the confidence of predicting subsequent words based on the current state, and the critic network computes the reward values of all the possible extensions of the current state to provide global guidance for explorations. Eventually, we employ an advantage reward to train both networks and conduct a set of experiments on a real-world dataset. The experimental results demonstrate that our approach outperforms the baselines by around 22% to 45% in BLEU-1 and outperforms the state-of-the-art approaches by around 5% to 60% in terms of S-BLEU and C-BLEU.

Multi-temporal high spatial resolution earth observation makes it possible to detect complex urban land surface changes, which is a significant and challenging task in remote sensing communities. Previous works mainly focus on binary change detection (BCD) based on modern technologies, e.g., deep fully convolutional network (FCN), whereas the deep network architecture for semantic change detection (SCD) is insufficiently explored in current literature. In this paper, we propose a deep multi-task encoder-transformer-decoder architecture (ChangeMask) designed by exploring two important inductive biases: sematic-change causal relationship and temporal symmetry. ChangeMask decouples the SCD into a temporal-wise semantic segmentation and a BCD, and then integrates these two tasks into a general encoder-transformer-decoder framework. In the encoder part, we design a semantic-aware encoder to model the semantic-change causal relationship. This encoder is only used to learn semantic representation and then learn change representation from semantic representation via a later transformer module. In this way, change representation can constrain semantic representation during training, which introduces a regularization to reduce the risk of overfitting. To learn a robust change representation from semantic representation, we propose a temporal-symmetric transformer (TST) to guarantee temporal symmetry for change representation and keep it discriminative. Based on the above semantic representation and change representation, we adopt simple multi-task decoders to output semantic change map. Benefiting from the differentiable building blocks, ChangeMask can be trained by a multi-task loss function, which significantly simplifies the whole pipeline of applying ChangeMask. The comprehensive experimental results on two large-scale SCD datasets confirm the effectiveness and superiority of ChangeMask in SCD. Besides, to demonstrate the potential value in real-world applications, e.g., automatic urban analysis and decision-making, we deploy the ChangeMask to map a large geographic area covering 30 km2 with 300 million pixels. Code will be made available.

Dynamic texture refers to the content in video sequences that is characterized by spatial repetition and temporal variation, such as swaying foliage and flowing water. It is a great challenge to compress the dynamic textures efficiently in the current prediction/transform hybrid video coding framework. However, these textures have little information for machine vision, and human visual perception is less sensitive to the textures than to the structures. Thus, we propose a spatiotemporal generative adversarial network (GAN) based dynamic texture synthesis method for surveillance video coding. We detect and remove the dynamic texture content at encoder side, which is irrelevant to machine vision. We generate the dynamic texture content using the proposed GAN at decoder side, so that the reconstructed videos can be observed by human without deteriorating perceptual quality. Specifically, we design a GAN network to synthesize dynamic textures by exploiting the correlation between spatial and temporal neighbors; we present a surveillance video coding scheme with the dynamic texture detection/synthesis method; we build a high-quality dynamic texture dataset, and we collect a dynamic texture testing dataset that goes beyond the existing video coding test datasets by focusing on surveillance scenes. The proposed video coding scheme has been implemented on top of the High Efficiency Video Coding (HEVC) reference software. Experiments have been conducted to evaluate the quantitative and qualitative performance of the proposed coding scheme. Our method achieves 7.4% and 7.6% bit-rate savings in low-delay-B and low-delay-P settings, respectively, at similar visual quality levels in comparison with HEVC.

In the engineering practice, lacking of data especially labeled data typically hinders the wide application of deep learning in mechanical fault diagnosis. However, collecting and labeling data is often expensive and time-consuming. To address this problem, a kind of semi-supervised meta-learning networks (SSMN) with squeeze-and-excitation attention is proposed for few-shot fault diagnosis in this paper. SSMN consists of a parameterized encoder, a non-parameterized prototype refinement process and a distance function. Based on attention mechanism, the encoder is able to extract distinct features to generate prototypes and enhance the identification accuracy. With semi-supervised few-shot learning, SSMN utilizes unlabeled data to refine original prototypes for better fault recognition. A combinatorial learning optimizer is designed to optimize SSMN efficiently. The effectiveness of the proposed method is demonstrated through three bearing vibration datasets and the results indicate the outstanding adaptability in different situations. Comparison with other approaches is also made under the same setup and the experimental results prove the superiority of the proposed method for few-shot fault diagnosis.

The computational prediction of novel drug-target interactions (DTIs) may effectively speed up the process of drug repositioning and reduce its costs. Most previous methods integrated multiple kinds of connections about drugs and targets by constructing shallow prediction models. These methods failed to deeply learn the low-dimension feature vectors for drugs and targets and ignored the distribution of these feature vectors. We proposed a graph convolutional autoencoder and generative adversarial network (GAN)-based method, GANDTI, to predict DTIs. We constructed a drug-target heterogeneous network to integrate various connections related to drugs and targets, i.e., the similarities and interactions between drugs or between targets and the interactions between drugs and targets. A graph convolutional autoencoder was established to learn the network embeddings of the drug and target nodes in a low-dimensional feature space, and the autoencoder deeply integrated different kinds of connections within the network. A GAN was introduced to regularize the feature vectors of nodes into a Gaussian distribution. Severe class imbalance exists between known and unknown DTIs. Thus, we constructed a classifier based on an ensemble learning model, LightGBM, to estimate the interaction propensities of drugs and targets. This classifier completely exploited all unknown DTIs and counteracted the negative effect of class imbalance. The experimental results indicated that GANDTI outperforms several state-of-the-art methods for DTI prediction. Additionally, case studies of five drugs demonstrated the ability of GANDTI to discover the potential targets for drugs.

Polarimetric synthetic aperture radar (PolSAR) image semantic segmentation is currently of great importance for synthetic aperture radar (SAR) image interpretation, especially in agricultural applications. Several convolutional neural networks (CNNs) have been implemented for SAR image semantic segmentation in urban or land cover applications. However, existing CNNs often break one semantic area into several pieces or confuse the adjacent semantic area in a PolSAR image. To address these issues uniformly, a refined pyramid scene parsing network (PSPNet) is proposed for PolSAR image semantic segmentation in an agricultural areas. Compared to conventional PSPNet architecture, the refined PSPNet adopts a multilevel feature fusion design in its decoder to effectively exploit the features learned from its different encoder branches. Besides, a polarimetric channel attention module is incorporated into the network to capture rich polarimetric features in a PolSAR image. Furthermore, an edge-aware loss function is devised to guide the network to refine pixel-level edge information directly from semantic segmentation prediction, separating easily confused agriculture area with sharp contours. Experimental results on one airborne millimeter-wave PolSAR dataset verify that the proposed network achieves promising semantic segmentation accuracy and preferable spatial consistency.

Named entity recognition and relation extraction are crucial tasks in natural language processing . As the traditional pipelined manners may suffer from the error propagation issue and ignore underlying interactions, joint extraction of entities and relations has become the dominant trend. However, the performance of existing joint extraction models needs improvement. This paper presents a two-stage tagging scheme that separately labels candidate head entities and multiple tail entities in specific relations. Next, it proposes a novel lightweight joint extraction neural model based on the entity-first labeling strategy. In the proposed model, the BiLSTM-based encoder combines the hidden state and global context features and feeds them as input for the next two entity labeling tasks. Further, with the input of the mixed context representation, the candidate-head-entity recognition module is adopted to identify the candidate head entity, while the multiple-tail-entities recognition module is equipped with an entity-correlated attention mechanism to identify the corresponding tail entity under a specific head entity. Comprehensive experiments on two widely used English datasets and one self-constructed Chinese dataset were performed. The experimental results showed that the proposed model outperformed the baseline approaches in the relation extraction task and achieved a competitive entity recognition effect via a lightweight architecture.

Traffic is very important to route planning and people's daily lives. Traffic prediction is still very challenging as it is affected by many complex factors including dynamic spatio-temporal dependencies and external factors (e.g., road types and nearby points of interest) in the road network. Dynamic spatio-temporal dependencies simultaneously contain spatial and temporal dependencies. Existing models for predicting traffic of links only consider the spatial dependencies from the perspective of links or the whole road network by ignoring the spatial dependencies among regions. To this end, this paper proposes a new Spatio-Temporal Neural Network (STNN) with the encoder-decoder architecture to improve the accuracy of traffic predictions by additionally taking into account the region-based spatial dependencies and external factors. Specifically, STNN learns dynamic spatio-temporal dependencies from historical traffic time series via an encoder in the perspective of the road network, with two spatial models, i.e., region-based spatial model and link-based spatial attention model in the perspectives of regions and links, respectively. Further, STNN decodes the output from the encoder via a decoder with a temporal attention model for recording long-term dependencies and fuses external factors in the road network, to improve network-wide traffic predictions. We conduct extensive experiments to evaluate the performance of STNN on three real-world traffic datasets, which shows that STNN is significantly better than the state-of-the-art models.

In this paper, we focus on subjective and objective Point Cloud Quality Assessment (PCQA) in an immersive environment and study the effect of geometry and texture attributes in compression distortion. Using a Head-Mounted Display (HMD) with six degrees of freedom, we establish a subjective PCQA database, named SIAT Point Cloud Quality Database (SIAT-PCQD). Our database consists of 340 distorted point clouds compressed by the MPEG point cloud encoder with the combination of 20 sequences and 17 pairs of geometry and texture quantization parameters. The impact of distorted geometry and texture attributes is further discussed in this paper. Then, we propose two projection-based objective quality evaluation methods, i.e., a weighted view projection based model and a patch projection based model. Our subjective database and findings can be used in point cloud processing, transmission, and coding, especially for virtual reality applications. The subjective dataset has been released in the public repository., Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Remaining useful life (RUL) prediction is of fundamental importance in reliability analysis and health diagnosis of complex industrial systems. Aiming at improving the prediction accuracy, this article proposes a normalizing flow embedded sequence-to-sequence (seq2seq) learning method to predict the RUL of an asset or a system. This method introduces a block of normalizing flow into the middle area of the familiar encoder–decoder structure of the seq2seq model. This normalizing flow enjoys the remarkable representation ability for the nonlinearity between input sequential data and outputs and enables the original seq2seq model to be more suitable for vibration signals of engines. The encoder and the decoder, which fall before and after the normalizing flow, are both built by gated recurrent units. Besides, a one-hot coding of clustering is concatenated with measurement data to indicate the frequently shifting vibration state, and a sensor selection method is designed to drop some weakly related and ineffective variables. Our method is tested and further analyzed by 2008 IEEE PHM challenge data (PHM08), of which many practical preprocessing methods are conducted. Numerous tests verify that our method outperforms other related deep learning methods for RUL estimation.

The standard approach for abstractive text summarization is to use an encoder-decoder architecture. The encoder is responsible for capturing the general meaning from the source text, and the decoder is in charge of generating the final text summary. While this approach can compose summaries that resemble human writing, some may contain unrelated or unfaithful information. This problem is called “hallucination” and it represents a serious issue in legal texts as legal practitioners rely on these summaries when looking for precedents, used to support legal arguments. Another concern is that legal documents tend to be very long and may not be fed entirely to the encoder. We propose our method called LegalSumm for addressing these issues by creating different “views” over the source text, training summarization models to generate independent versions of summaries, and applying entailment module to judge how faithful these candidate summaries are with respect to the source text. We show that the proposed approach can select candidate summaries that improve ROUGE scores in all metrics evaluated.

Non-autoregressive machine translation aims to speed up the decoding procedure by discarding the autoregressive model and generating the target words independently. Because non-autoregressive machine translation fails to exploit target-side information, the ability to accurately model source representations is critical. In this paper, we propose an approach to enhance the encoder’s modeling ability by using a pre-trained BERT model as an extra encoder. With a different tokenization method, the BERT encoder and the Raw encoder can model the source input from different aspects. Furthermore, having a gate mechanism, the decoder can dynamically determine which representations contribute to the decoding process. Experimental results on three translation tasks show that our method can significantly improve the performance of non-autoregressive MT, and surpass the baseline non-autoregressive models. On the WMT14 EN $$\rightarrow$$ DE translation task, our method achieves 27.87 BLEU with a single decoding step. This is a comparable result with the baseline autoregressive Transformer model which obtains a score of 27.8 BLEU.

An end-to-end text-to-speech system generates acoustic features directly from input text to synthesize speech from it. The challenges of using these models for Persian language are lack of a proper data, and also detection of exceptions and Ezafe between words inherently (without grapheme-to-phoneme). In this paper, we propose to use an special end-to-end tts system named Tacotron2, and suggest solutions for the mentioned problems. For the lack of data problem, we collect a dataset proper for end-to-end text-to-speech including 21 hours of Persian speech and corresponding text. We use multi-resolution convolution and part of speech embedding layers in the encoder part of Tacotron2, to overcome the exceptions and Ezafe detection problem. In addition, in the case of Tacotron2, Mel-spectrogram generation process is unstable due to high dropout rate at inference time. To handle this problem, we propose to use a convex optimization method, named Net-Trim. Experimental results show that our proposed method increases Tacotron2 mean opinion score from 3.01 to 3.97. Furthermore, the proposed method decreases Mel cepstral distortion in comparison with Tacotron2.

In this article, we propose FLAG for flexible, accurate, and long-time user load prediction in a large-scale WiFi system. FLAG enables prediction customization in both time granularity and prediction length. Under an operating WiFi system with more than 7000 APs, a reference implementation of FLAG is developed, which consists of three major components. For data acquisition , we process 25 074 733 association records contributed by 55 809 users, to extract the ground truth of AP-level user load. For feature extraction , we perform a comprehensive data analytics to mine vital features to label each AP, which are extracted and classified into three categories, i.e., individual features, spatial features, and temporal features. For the model design , we design a deep recurrent neural network (RNN) model, which contains two separate RNNs, i.e., the encoder RNN and decoder RNN. Particularly, the sequential feature vectors are injected into the encoder RNN to learn the “semantic” information, based on which the decoder RNN conducts sequential AP-level predictions. As the semantic vector is injected for each time step prediction, it can effectively reduce the accumulated prediction errors, which enable long period of time predictions. Real data set-based experiments corroborate the efficacy of FLAG .

In this study, we investigate the process of generating single-sentence representations for the purpose of Dialogue Act (DA) classification, including several aspects of text pre-processing and input representation which are often overlooked or underreported within the literature, for example, the number of words to keep in the vocabulary or input sequences. We assess each of these with respect to two DA-labelled corpora, using a range of supervised models, which represent those most frequently applied to the task. Additionally, we compare context-free word embedding models with that of transfer learning via pre-trained language models, including several based on the transformer architecture, such as Bidirectional Encoder Representations from Transformers (BERT) and XLNET, which have thus far not been widely explored for the DA classification task. Our findings indicate that these text pre-processing considerations do have a statistically significant effect on classification accuracy. Notably, we found that viable input sequence lengths, and vocabulary sizes, can be much smaller than is typically used in DA classification experiments, yielding no significant improvements beyond certain thresholds. We also show that in some cases the contextual sentence representations generated by language models do not reliably outperform supervised methods. Though BERT, and its derivative models, do represent a significant improvement over supervised approaches, and much of the previous work on DA classification.

Most state-of-the-art neural machine translation (NMT) models progressively encode feature representation in a bottom-up feed-forward fashion. This traditional encoding mechanism has no guidance from external signals. In computer vision tasks, the feedback connection plays a crucial role, particularly for understanding tasks. In this paper, we propose a simple but effective approach to learn latent feature representations explicitly from input sentences via a latent feature encoder (LFE), which are fed back to an NMT encoder via a top-down feedback mechanism. Through the feedback mechanism, the representations in one layer are influenced by representations of both lower and higher layers, resulting in a more effective encoding mechanism. Besides, to enhance the capability of the LFE in better capturing latent features from the source sentences, we pre-train the LFE via a Denoising Auto-Encoder (DAE) strategy. Experimentation on the large-scale WMT 2014 English-to-German and WMT 2017 Chinese-to-English translation tasks demonstrates that our proposed LFE, either pre-trained with the DAE or not, significantly outperforms the strong baseline.

The novel Coronavirus disease (COVID-19) is a highly contagious virus and has spread all over the world, posing an extremely serious threat to all countries. Automatic lung infection segmentation from computed tomography (CT) plays an important role in the quantitative analysis of COVID-19. However, the major challenge lies in the inadequacy of annotated COVID-19 datasets. Currently, there are several public non-COVID lung lesion segmentation datasets, providing the potential for generalizing useful information to the related COVID-19 segmentation task. In this paper, we propose a novel relation-driven collaborative learning model to exploit shared knowledge from non-COVID lesions for annotation-efficient COVID-19 CT lung infection segmentation. The model consists of a general encoder to capture general lung lesion features based on multiple non-COVID lesions, and a target encoder to focus on task-specific features based on COVID-19 infections. Features extracted from the two parallel encoders are concatenated for the subsequent decoder part. We develop a collaborative learning scheme to regularize feature-level relation consistency of given input and encourage the model to learn more general and discriminative representation of COVID-19 infections. Extensive experiments demonstrate that trained with limited COVID-19 data, exploiting shared knowledge from non-COVID lesions can further improve state-of-the-art performance with up to 3.0% in dice similarity coefficient and 4.2% in normalized surface dice. Our proposed method promotes new insights into annotation-efficient deep learning for COVID-19 infection segmentation and illustrates strong potential for real-world applications in the global fight against COVID-19 in the absence of sufficient high-quality annotations., 12 pages

Text summarization aims to reduce text by removing less useful information to obtain information quickly and precisely. In Indonesian abstractive text summarization, the research mostly focuses on multi-document summarization which methods will not work optimally in single-document summarization. As the public summarization datasets and works in English are focusing on single-document summarization, this study emphasized on Indonesian single-document summarization. Abstractive text summarization studies in English frequently use Bidirectional Encoder Representations from Transformers (BERT), and since Indonesian BERT checkpoint is available, it was employed in this study. This study investigated the use of Indonesian BERT in abstractive text summarization on the IndoSum dataset using the BERTSum model. The investigation proceeded by using various combinations of model encoders, model embedding sizes, and model decoders. Evaluation results showed that models with more embedding size and used Generative Pre-Training (GPT)-like decoder could improve the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) score and BERTScore of the model results.

Recently, atrous spatial pyramid module and encoder-decoder structure are widely investigated for many computer vision tasks to address the scale variation challenge in deep convolutional networks. The atrous spatial pyramid module aims at capturing local and global cues together with various sampling rates in convolutional or pooling layers. At the same time, encoder-decoder structure propagates context features from low-resolution, strong-semantic features to high-resolution, weak-semantic ones, while maintaining the detail object boundaries. However, the aforementioned two strategies have their own drawbacks, and the previous object detectors only employ one of them to handle scale variation. In this work, we propose to cooperate atrous spatial pyramid convolution (ASPC) with encoder-decoder structure (ED) for object detection, termed ASPC-ED, which combines the complementary advantages from both modules in an end-to-endfashion. Specifically, the proposed method is consist of three components: encoder, ASPC and decoder. The extensive experiments on PASCAL VOC and MS COCO benchmarks demonstrate that our method with various backbone achieves the state-of-the-art results for object detection, instance segmentation and panoptic segmentation.

We present a benchmark comparison of several deep learning models including Convolutional Neural Networks, Recurrent Neural Network and Bi-directional Long Short Term Memory, assessed based on various word embedding approaches, including the Bi-directional Encoder Representations from Transformers (BERT) and its variants, FastText and Word2Vec. Data augmentation was administered using the Easy Data Augmentation approach resulting in two datasets (original versus augmented). All the models were assessed in two setups, namely 5-class versus 3-class (i.e., compressed version). Findings show the best prediction models were Neural Network-based using Word2Vec, with CNN-RNN-Bi-LSTM producing the highest accuracy (96%) and F-score (91.1%). Individually, RNN was the best model with an accuracy of 87.5% and F-score of 83.5%, while RoBERTa had the best F-score of 73.1%. The study shows that deep learning is better for analyzing the sentiments within the text compared to supervised machine learning and provides a direction for future work and research.

Machine reading comprehension (MRC) is a challenging task in natural language processing (NLP), which requires machine to determine the corresponding answer to a given passage and question. Whereas, there always exist unanswerable questions in the real world, which poses a new challenge to MRC tasks. Abundant research work has been carried out on the verification mechanism for the answerability of passage-question pairs. However, these researches only focus on its design and implementation, which has limitations in real-world scenarios. Thus, the method proposed in this paper not only verifies the answerability, but also validates and adjusts the predicted answer to obtain an elaborate answer span. Using powerful pre-trained model as encoder block, this paper explores a more comprehensive verification mechanism. Similar to how humans read passages and give answers, we propose a three-stage mechanism called ”Verification for an Elaborate Span” (V4ES): 1) sketchy reading that the model briefly browses the overall information of the passage and question, and then generates an initial answer; 2) intensive reading that it reads the passage and question again, judges the answerability of the question and gives an answer at this stage; 3) verification that it verifies these two answers produced at the previous two stages, and then gives the final prediction. Moreover, the proposed model is evaluated on two MRC challenge datasets: SQuAD2.0 and CMRC2018, and the experiment results show that our model has achieved great improvement compared with the ALBERT and BERT baselines. In conclusion, our proposed verification mechanism has demonstrated its effectiveness through a series of experiments and analysis.

Accurate estimation of remaining useful life (RUL) of industrial equipment can enable advanced maintenance schedules, increase equipment availability and reduce operational costs. However, existing deep learning methods for RUL prediction are not completely successful due to the following two reasons. First, relying on a single objective function to estimate the RUL will limit the learned representations and thus affect the prediction accuracy. Second, while longer sequences are more informative for modelling the sensor dynamics of equipment, existing methods are less effective to deal with very long sequences, as they mainly focus on the latest information. To address these two problems, we develop a novel attention-based sequence to sequence with auxiliary task (ATS2S) model. In particular, our model jointly optimizes both reconstruction loss to empower our model with predictive capabilities (by predicting next input sequence given current input sequence) and RUL prediction loss to minimize the difference between the predicted RUL and actual RUL. Furthermore, to better handle longer sequences, we employ the attention mechanism to focus on all the important input information during the training process. Finally, we propose a new dual-latent feature representation to integrate the encoder features and decoder hidden states, to capture rich semantic information in data. We conduct extensive experiments on four real datasets to evaluate the efficacy of the proposed method. Experimental results show that our proposed method can achieve superior performance over 13 state-of-the-art methods consistently.