Search

Data selection is a widely used and effective solution to domain adaptation in statistical machine translation (SMT). The dominant methods are perplexity-based ones, which do not consider the mutual translations of sentence pairs and tend to select short sentences. In this paper, to address these problems, we propose bilingual semi-supervised recursive neural network data selection methods to differentiate domain-relevant data from out-domain data. The proposed methods are evaluated in the task of building domain-adapted SMT systems. We present extensive comparisons and show that the proposed methods outperform the state-of-the-art data selection approaches. [ABSTRACT FROM AUTHOR]

The ordinary feature selection methods select only the explicit relevant attributes by filtering the irrelevant ones. They trade the selection accuracy for the execution time and complexity. In which, the hidden supportive information possessed by the irrelevant attributes may be lost, so that they may miss some good combinations. We believe that attributes are useless regarding the classification task by themselves, sometimes may provide potentially useful supportive information to other attributes and thus benefit the classification task. Such a strategy can minimize the information lost, therefore is able to maximize the classification accuracy. Especially for the dataset contains hidden interactions among attributes. This paper proposes a feature selection methodology from a new angle that selects not only the relevant features, but also targeting at the potentially useful false irrelevant attributes by measuring their supportive importance to other attributes. The empirical results validate the hypothesis by demonstrating that the proposed approach outperforms most of the state-of-the-art filter based feature selection methods. [ABSTRACT FROM AUTHOR]

Biosignals such as electrocardiograms (ECG), electroencephalograms (EEG), and electromyograms (EMG), are important noninvasive measurements useful for making diagnostic decisions. Recently, considerable research has been conducted in order to potentially automate signal classification for assisting in disease diagnosis. However, the biosignal type (ECG, EEG, EMG or other) needs to be known prior to the classification process. If the given biosignal is of an unknown type, none of the existing methodologies can be utilized. In this paper, a blind biosignal classification model (B²SC Model) is proposed in order to identify the source biosignal type automatically, and thus ultimately benefit the diagnostic decision. The approach employs time series algorithms for constructing the model. It uses a dynamic time warping (DTW) algorithm with clustering to discover the similarity between two biosignals, and consequently classifies disease without prior knowledge of the source signal type. The empirical experiments presented in this paper demonstrate the effectiveness of the method as well as the scalability of the approach. [ABSTRACT FROM AUTHOR]

Highlights: [•] We proposed a lexicon expansion approach to improve latent variable grammars. [•] The lexicon expansion is based on transductive graph propagation technique. [•] We constructed word-level k-NN similarity graph over labeled and unlabeled data. [•] We used an unnormalized propagation algorithm to infer emission probabilities. [•] Lexicon expansion with self-training can further improve latent variable grammars. [Copyright &y& Elsevier]

Discretization of a continuous-valued symptom (attribute) in medical data set is a crucial preprocessing step for the medical classification task. This paper proposes a supportive attribute - assisted discretization (SAAD) model for medical diagnostic problems. The intent of this approach is to discover the best supportive symptom that correlates closely with the continuous-valued symptom being discretized and to conduct the discretization process using the significant supportive information that is provided by the best supportive symptom, because we hypothesize that a good discretization scheme should rely heavily on the interaction between a continuous-valued attribute and both its supportive attribute and the class attribute. SAAD can consider each continuous-valued symptom differently and intelligently, which allows it to be capable of minimizing the information lost and the data uncertainty. Hence, SAAD results in higher classification accuracy. Empirical experiments using ten real-life datasets from the UCI repository were conducted to compare the classification accuracy achieved by several prestigious classifiers with SAAD and other state-of-the-art discretization approaches. The experimental results demonstrate the effectiveness and usefulness of the proposed approach in enhancing the diagnostic accuracy. [ABSTRACT FROM AUTHOR]

Discretization of a continuous-valued symptom (attribute) in medical data set is a crucial preprocessing step for the medical classification task. This paper proposes a supportive attribute - assisted discretization (SAAD) model for medical diagnostic problems. The intent of this approach is to discover the best supportive symptom that correlates closely with the continuous-valued symptom being discretized and to conduct the discretization process using the significant supportive information that is provided by the best supportive symptom, because we hypothesize that a good discretization scheme should rely heavily on the interaction between a continuous-valued attribute and both its supportive attribute and the class attribute. SAAD can consider each continuous-valued symptom differently and intelligently, which allows it to be capable of minimizing the information lost and the data uncertainty. Hence, SAAD results in higher classification accuracy. Empirical experiments using ten real-life datasets from the UCI repository were conducted to compare the classification accuracy achieved by several prestigious classifiers with SAAD and other state-of-the-art discretization approaches. The experimental results demonstrate the effectiveness and usefulness of the proposed approach in enhancing the diagnostic accuracy. [ABSTRACT FROM AUTHOR]

Measuring learner cognition based on their problem-solving performance is a joint discipline of cognitive psychology and machine learning. In the case of learner problem-solving, the interaction between learner and knowledge forms a typical type of signed interaction graph. Interaction graphs are a widely used and effective solution to model the relationships between interacting entities. However, most of previous interaction graph methods are inclined to the observed interactions as positive links but they often fail to consider unobserved and negative links, which leads to an insufficiency in capturing the complete cognition/mis-cognition proximity information. To address this limitation, we propose a knowledge graph representation learning method that is based on signed knowledge interaction network (SKIN). We explicitly model the correct/incorrect cognitive performance as the positively + /negatively − signed links in the graph. The model simultaneously measures the nodes' local and global proximity , and then preserves them in the learned knowledge embedding. We architect a pairwise neural network that is based on a tri-sampling strategy and a sign-driven distance measuring objective function. The network generates knowledge representations by maximizing the knowledge distance between oppositely-signed pairs and minimizing the distance between identically-signed pairs. Our experimental results show the learned knowledge embedding demonstrates a desired Euclidean property and can be visualized with clear classification boundaries. We also show it can power downstream tasks such as learner-performance-prediction. The learned embeddings generate promising prediction scores on this task when compared to several methods in network sign prediction and learner-performance-prediction. [ABSTRACT FROM AUTHOR]

Autonomous learning diagnostics, where the students' strengths and weaknesses are disclosed from their observed performance data, is a challenging task in e-learning systems. Current student knowledge models can alleviate some of the problems in learning (i.e. predicting student performance) but they neglect learning diagnostics, which is based on causal reasoning. To this end, we propose a novel heterogeneous attention interpreter with a maximum entropy regularizer on top of a student knowledge model to achieve explainable learning diagnostics. Our model segregates the impact of the homogeneous knowledge points, while promoting the heterogeneous relatives by maximizing their chance to contribute to the prediction. We also propose a multi-spatial knowledge representation that is readily generalizable to other data-driven educational tasks. Extensive experiments on real-world datasets reveal that the proposed method is able to enhance the model's explanatory power, hence increases the trustworthiness towards learning diagnostics. It also brings notable improvement in accuracy in the student performance prediction task. The findings in this paper are adoptable to various types of e-learning systems to assist teachers to gain insights into student learning states and diagnose learning problems. • Autonomously diagnosing learning problems in e-learning systems can be challenging due to the lack of teacher resources (teachers) in e-learning. This work proposes to tackle the explainable learning diagnostics problem using attention-based explanation mechanisms by performing target-source relation prediction. The findings are adoptable to various types of e-learning systems to gain insights into their learner states and diagnose their learning problems. • This paper identifies the importance to use the 'close relatives of knowledge to generate prediction decisions in a knowledge tracing model. We propose a Heterogeneous Attention Relative Detector and a Maximum Entropy Regularizer to detect those relatives, i.e. knowledge relation discovery. • We propose a multi-spatial representation of knowledge in expressing knowledge relations in finer-granularity and low-dimensionality, which can be readily generalized to other data-driven educational tasks. • We provide a different perspective on knowledge graph completion and/or construction. In this perspective, we exploit the interaction data to uncover inner knowledge relations (links) where there is no existing relation data to learn. Our method is effective in settings where the knowledge space is relatively small while the interaction space is very large. [ABSTRACT FROM AUTHOR]

Tree-to-sequence neural machine translation models have proven to be effective in learning the semantic representations from the exploited syntactic structure. Despite their success, tree-to-sequence models have two major issues: (1) the embeddings of constituents at the higher tree levels tend to contribute less in translation; and (2) using a single set of model parameters is difficult to fully capture the syntactic and semantic richness of linguistic phrases. To address the first problem, we proposed a lexicalized dependency model, in which the source-side lexical representations are learned in a head-dependent fashion following a dependency graph. Since the number of dependents is variable, we proposed a variant recurrent neural network (RNN) to jointly consider the long-distance dependencies and the sequential information of words. Concerning the second problem, we adopt a latent vector to dynamically condition the parameters for the composition of each node representation. Experimental results reveal that the proposed model significantly outperforms the recently proposed tree-based methods in English–Chinese and English–German translation tasks with even far fewer parameters. [ABSTRACT FROM AUTHOR]

Curriculum learning (CL) has demonstrated its potential to fine-tune pre-trained models for the response selection task. However, existing CL works exhibit a significant reliance on fixed data buckets and a pre-ordained learning schedule. This paper seeks to transcend these constraints by proposing a novel dynamic curriculum learning (DCL) algorithm that employs: (1) a new difficulty measurement that amalgamates linguistic features with model confidence, and (2) an automatic uncertainty-based learning scheduler with a dynamic data sampling policy. This unique approach enables the model to autonomously assemble rational data batches and adjust the curriculum stage by itself. Empirical experiments show that our DCL method reveals its superior performance over three strong baselines and the competitive method on three multi-turn dialogue datasets. The in-depth analysis demonstrates our approach dynamically adapts the CL process, which in turn makes effective use of existing data and alleviates data scarcity. [ABSTRACT FROM AUTHOR]

Recently, Self-Attention Networks (SANs) have shown its flexibility in parallel computation and effectiveness of modeling both short- and long-term dependencies. However, SANs face two problems: 1) the weighted averaging inhibits relations among neighboring words (i.e., local context); and 2) it calculates dependencies between representations without considering contextual information (i.e., global context). Both local and global contexts have proven useful for modeling dependencies among neural representations in a variety of natural language processing tasks. Accordingly, we augment SANs with the ability of capturing usefully local and global context, and meanwhile maintain their simplicity and flexibility. Firstly, we cast local context modeling as a learnable Gaussian bias, which indicates the central and scope of the local region to be paid more attention. The bias is then incorporated into the original attention distribution to form a revised version. Secondly, we leverage the internal representations that embed sentence-level information as the global context. Specifically, we propose to contextualize the transformations of query and key layers, which are used to calculate the relevance between elements. Since the two approaches are potentially complementary to each other, we propose to combine them to further improve the performance. Empirical results on machine translation and linguistics probing tasks demonstrate the effectiveness and universality of the proposed approaches. Further analyses confirm that our approaches successfully capture contextual information as expected. [ABSTRACT FROM AUTHOR]

The phenomenon of zero pronoun (ZP) has attracted increasing interest in the machine translation community due to its importance and difficulty. However, previous studies generally evaluate the quality of translating ZPs with BLEU score on MT testsets, which is not expressive or sensitive enough for accurate assessment. To bridge the data and evaluation gaps, we propose a benchmark testset and evaluation metric for target evaluation on Chinese ZP translation. The human-annotated testset covers five challenging genres, which reveal different characteristics of ZPs for comprehensive evaluation. We systematically revisit advanced models on ZP translation and identify current challenges for future exploration. We release data, code, and trained models, which we hope can significantly promote research in this field. [ABSTRACT FROM AUTHOR]

Self-attention networks (SANs) have attracted an amount of research attention for their outstanding performance under the machine translation community. Recent studies proved that SANs can be further improved by exploiting different inductive biases, each of which guides SANs to learn a specific view of the input sentence, e.g., short-term dependencies, forward and backward views, as well as phrasal patterns. However, less studies investigate how these inductive techniques complementarily improve the capability of SANs and this would be an interesting question to be answered. In this paper we selected five inductive biases which are simple and not over parameterized to investigate their complementarily. We further propose multi-view self-attention networks, which jointly learn different linguistic aspects of the input sentence under a unified framework. Specifically, we propose and exploit a variety of inductive biases to regularize the conventional attention distribution. Different views are then aggregated by a hybrid attention mechanism to quantify and leverage the specific views and their associated representation conveniently. Experiments on various translation tasks demonstrate that different views are able to progressively improve the performance of SANs, and the proposed approach outperforms both the strong Transformer baseline and related models on Transformer-base and Transformer-big settings. Extensive analyses on 10 linguistic probing tasks verify that different views indeed tend to extract distinct linguistic features and our method gives highly effective improvements in their integration. 1 1 We release the code of this work in https://github.com/NLP2CT/MV-Attn. [ABSTRACT FROM AUTHOR]