Search

This article presents Cadlaws, a new English–French corpus built from Canadian legal documents, and describes the corpus construction process and preliminary statistics obtained from it. The corpus contains over 16 million words in each language and includes unique features since it is composed of documents that are legally equivalent in both languages but not the result of a translation. The corpus is built upon enactments co-drafted by two jurists to ensure legal equality of each version and to re­flect the concepts, terms and institutions of two legal traditions. In this article the corpus definition as a parallel corpus instead of a comparable one is also discussed. Cadlaws has been pre-processed for machine translation and baseline Bilingual Evaluation Understudy (bleu), a score for comparing a candidate translation of text to a gold-standard translation of a neural machine translation system. To the best of our knowledge, this is the largest parallel corpus of texts which convey the same meaning in this language pair and is freely available for non-commercial use.

In Electronic Health Record (EHR) systems, key patient information is often captured in the form of unstructured clinical notes. The information from these notes can be extracted using Clinical Natural Language Processing (NLP). Training corpus is a key factor in development of efficient clinical NLP models. Clinical NLP corpus construction is complex and multifaceted. There are several challenges in corpus construction, but one challenge often not researched well is cohort selection aspect. In this study, we present methods employed and challenges encountered in cohort selection for construction of a clinical NLP corpus. In specific we present our methods in selection of cancer pathology reports to construct a corpus for automatic deidentification. 2100 pathology reports were extracted from 1833 (518 male and 1313 female) cancer patients using Health Level-7 (HL7) message standard. In terms of the age group distribution, the age group 60–70 years was highest with 872 patients. Our findings suggest deciphering the segment information from HL7 messages that are collected from different hospitals is a challenging task. The quality of HL7 messages also varied significantly with inconsistent tags making it difficult to identify reports that meet criteria set a priori. One of key lessons learned is linking the HL7 reports data with additional EMR data such as admissions, would help in identifying high quality reports and resolve duplicates. Also, our findings suggest that, in general the EHR data quality is poor with varying clinical coding and metadata standards between different hospitals. It is vital to identify and address these challenges for development of a high-quality corpus.

Background In this study, we focus on building a fine-grained entity annotation corpus with the corresponding annotation guideline of traditional Chinese medicine (TCM) clinical records. Our aim is to provide a basis for the fine-grained corpus construction of TCM clinical records in future. Methods We developed a four-step approach that is suitable for the construction of TCM medical records in our corpus. First, we determined the entity types included in this study through sample annotation. Then, we drafted a fine-grained annotation guideline by summarizing the characteristics of the dataset and referring to some existing guidelines. We iteratively updated the guidelines until the inter-annotator agreement (IAA) exceeded a Cohen’s kappa value of 0.9. Comprehensive annotations were performed while keeping the IAA value above 0.9. Results We annotated the 10,197 clinical records in five rounds. Four entity categories involving 13 entity types were employed. The final fine-grained annotated entity corpus consists of 1104 entities and 67,799 tokens. The final IAAs are 0.936 on average (for three annotators), indicating that the fine-grained entity recognition corpus is of high quality. Conclusions These results will provide a foundation for future research on corpus construction and named entity recognition tasks in the TCM clinical domain.

Cardiovascular disease (CVD) has become the leading cause of death in China, and most of the cases can be prevented by controlling risk factors. The goal of this study was to build a corpus of CVD risk factor annotations based on Chinese electronic medical records (CEMRs). This corpus is intended to be used to develop a risk factor information extraction system that, in turn, can be applied as a foundation for the further study of the progress of risk factors and CVD. We designed a light annotation task to capture CVD risk factors with indicators, temporal attributes and assertions that were explicitly or implicitly displayed in the records. The task included: 1) preparing data; 2) creating guidelines for capturing annotations (these were created with the help of clinicians); 3) proposing an annotation method including building the guidelines draft, training the annotators and updating the guidelines, and corpus construction. Then, a risk factor annotated corpus based on de-identified discharge summaries and progress notes from 600 patients was developed. Built with the help of clinicians, this corpus has an inter-annotator agreement (IAA) F1-measure of 0.968, indicating a high reliability. To the best of our knowledge, this is the first annotated corpus concerning CVD risk factors in CEMRs and the guidelines for capturing CVD risk factor annotations from CEMRs were proposed. The obtained document-level annotations can be applied in future studies to monitor risk factors and CVD over the long term., Comment: 32 pages, 3 figures, 3 tables

Working with corpus construction becomes an interesting alternative to different applications of natural language processing, such as, question-answering, machine translation, information retrieval, etc. Similarly, with the heterogeneous data and the user demands for the accurate information, many studies have accentuated the need of the Web to highlight the corpus construction. As well as, Arabic doesn’t have an equivalent number of linguistic corpuses as compared to other languages like English. In this paper, we focus on building our corpus of Arab questions-texts. We present a method for recovering text passages. This method is based on a real automatic interrogation of Google, in order to generate passages of texts and answer the factual questions. The first part of this paper describes the formal details about this method; the second part presents some experiments and results that validate our method.

While it is already possible to construct a large-scale English corpus easily and inexpensively using web documents, such documents have problems of reliability with respect to their English quality. Thus, we have developed a system that automatically gathers English technical papers from the web, sorts them into those written by native speakers and those written by non-native speakers, and then uses them to construct a native speaker and non-native speaker corpus. We discuss a method of using the corpus for providing alternative candidates for appropriate adjectives against unnatural English adjective-noun co-occurrence constructions a , n >. The appropriateness of adjective a ’ is evaluated based on the similarity of the occurring environments between a and a ’.

In order to provide sufficient training data for statistical machine(-aided) translation in medical field, a large scale English-Chinese parallel corpus of medical works is constructed. Eighteen English medical printed books with Chinese translation are selected as raw materials. With the help of an OCR scanner, all texts are recognized, manually proofread and stored in electrical form. Within a rigid scheme of corpus construction and with the help of a self- coded Python program, English and Chinese texts are separated, sentence aligned and XML marked. After careful manual proofreading, an Internet-based corpus retrieval platform is constructed. The present parallel corpus contains 54,522 sentence pairs and more than 2,500,000 English words / Chinese characters, which can be preliminarily applied in the training and testing of statistical machine(-aided) translation researches in medical field.

In this paper, we report our tree based statistical translation study from English to Turkish. We describe our data generation process and report the initial results of tree-based translation under a simple model. For corpus construction, we used the Penn Treebank in the English side. We manually translated about 5K trees from English to Turkish under grammar constraints with adaptations to accommodate the agglutinative nature of Turkish morphology. We used a permutation model for subtrees together with a word to word mapping. We report BLEU scores under simple choices of inference algorithms. Publisher's Version

International audience; In this paper, we address the problem of automatically constructing a relevant corpus of scientific articles about food-drug interactions. There is a growing number of scientific publications that describe food-drug interactions but currently building a high-coverage corpus that can be used for information extraction purposes is not trivial. We investigate several methods for automating the query selection process using an expert-curated corpus of food-drug interactions. Our experiments show that index term features along with a decision tree classifier are the best approach for this task and that feature selection approaches and in particular gain ratio outperform frequency-based methods for query selection.