Search

Background: In the healthcare domain today, despite the substantial adoption of electronic health information systems, a significant proportion of medical reports still exist in paper-based formats. As a result, there is a significant demand for the digitization of information from these paper-based reports. However, the digitization of paper-based laboratory reports into a structured data format can be challenging due to their non-standard layouts, which includes various data types such as text, numeric values, reference ranges, and units. Therefore, it is crucial to develop a highly scalable and lightweight technique that can effectively identify and extract information from laboratory test reports and convert them into a structured data format for downstream tasks., Methods: We developed an end-to-end Natural Language Processing (NLP)-based pipeline for extracting information from paper-based laboratory test reports. Our pipeline consists of two main modules: an optical character recognition (OCR) module and an information extraction (IE) module. The OCR module is applied to locate and identify text from scanned laboratory test reports using state-of-the-art OCR algorithms. The IE module is then used to extract meaningful information from the OCR results to form digitalized tables of the test reports. The IE module consists of five sub-modules, which are time detection, headline position, line normalization, Named Entity Recognition (NER) with a Conditional Random Fields (CRF)-based method, and step detection for multi-column. Finally, we evaluated the performance of the proposed pipeline on 153 laboratory test reports collected from Peking University First Hospital (PKU1)., Results: In the OCR module, we evaluate the accuracy of text detection and recognition results at three different levels and achieved an averaged accuracy of 0.93. In the IE module, we extracted four laboratory test entities, including test item name, test result, test unit, and reference value range. The overall F1 score is 0.86 on the 153 laboratory test reports collected from PKU1. With a single CPU, the average inference time of each report is only 0.78 s., Conclusion: In this study, we developed a practical lightweight pipeline to digitalize and extract information from paper-based laboratory test reports in diverse types and with different layouts that can be adopted in real clinical environments with the lowest possible computing resources requirements. The high evaluation performance on the real-world hospital dataset validated the feasibility of the proposed pipeline., (© 2023. The Author(s).)

Named entities are the main carriers of relevant medical knowledge in Electronic Medical Records (EMR). Clinical electronic medical records lead to problems such as word segmentation ambiguity and polysemy due to the specificity of Chinese language structure, so a Clinical Named Entity Recognition (CNER) model based on multi-head self-attention combined with BILSTM neural network and Conditional Random Fields is proposed. Firstly, the pre-trained language model organically combines char vectors and word vectors for the text sequences of the original dataset. The sequences are then fed into the parallel structure of the multi-head self-attention module and the BILSTM neural network module, respectively. By splicing the output of the neural network module to obtain multi-level information such as contextual information and feature association weights. Finally, entity annotation is performed by CRF. The results of the multiple comparison experiments show that the structure of the proposed model is very reasonable and robust, and it can effectively improve the Chinese CNER model. The model can extract multi-level and more comprehensive text features, compensate for the defect of long-distance dependency loss, with better applicability and recognition performance.

Objective: To comparatively evaluate a range of Natural Language Processing (NLP) approaches for Information Extraction (IE) of low-prevalence concepts in clinical notes on the example of decline of insulin therapy recommendation by patients., Materials and Methods: We evaluated the accuracy of detection of documentation of decline of insulin therapy by patients using sentence-level naïve Bayes, logistic regression and support vector machine (SVM)-based classification (with and without SMOTE oversampling), token-level sequence labelling using conditional random fields (CRFs), uni- and bi-directional recurrent neural network (RNN) models with GRU and LSTM cells, and rule-based detection using Canary platform. All models were trained using the same manually annotated 50,046-document training set and evaluated on the same 1501-document held-out set. Hyperparameter optimization was performed using 10-fold cross-validation., Results: At the sentence level, prevalence of documentation of decline of insulin therapy by patients was 0.02% in both training and held-out sets. Naïve Bayes and logistic regression models did not achieve F 1 score ≥ 0.5 on the training set and were not further evaluated. Among the other models, evaluation against the held-out test set showed that SVM identified decline of insulin therapy by patients with F 1 score of 0.61, CRF with F 1 of 0.51, RNN with F 1 of 0.67 and Canary rule-based model with F 1 of 0.97., Conclusions: Identification of low-prevalence concepts can present challenges in medical language processing. Rule-based systems that include the designer's background knowledge of language may be able to achieve higher accuracy under these circumstances., (Copyright © 2019 Elsevier Inc. All rights reserved.)

Radiological measurements are reported in free text reports, and it is challenging to extract such measures for treatment planning such as lesion summarization and cancer response assessment. The purpose of this work is to develop and evaluate a natural language processing (NLP) pipeline that can extract measurements and their core descriptors, such as temporality, anatomical entity, imaging observation, RadLex descriptors, series number, image number, and segment from a wide variety of radiology reports (MR, CT, and mammogram). We created a hybrid NLP pipeline that integrates rule-based feature extraction modules and conditional random field (CRF) model for extraction of the measurements from the radiology reports and links them with clinically relevant features such as anatomical entities or imaging observations. The pipeline was trained on 1117 CT/MR reports, and performance of the system was evaluated on an independent set of 100 expert-annotated CT/MR reports and also tested on 25 mammography reports. The system detected 813 out of 806 measurements in the CT/MR reports; 784 were true positives, 29 were false positives, and 0 were false negatives. Similarly, from the mammography reports, 96% of the measurements with their modifiers were extracted correctly. Our approach could enable the development of computerized applications that can utilize summarized lesion measurements from radiology report of varying modalities and improve practice by tracking the same lesions along multiple radiologic encounters.

Background: This paper presents a conditional random fields (CRF) method that enables the capture of specific high-order label transition factors to improve clinical named entity recognition performance. Consecutive clinical entities in a sentence are usually separated from each other, and the textual descriptions in clinical narrative documents frequently indicate causal or posterior relationships that can be used to facilitate clinical named entity recognition. However, the CRF that is generally used for named entity recognition is a first-order model that constrains label transition dependency of adjoining labels under the Markov assumption., Methods: Based on the first-order structure, our proposed model utilizes non-entity tokens between separated entities as an information transmission medium by applying a label induction method. The model is referred to as precursor-induced CRF because its non-entity state memorizes precursor entity information, and the model's structure allows the precursor entity information to propagate forward through the label sequence., Results: We compared the proposed model with both first- and second-order CRFs in terms of their F 1 -scores, using two clinical named entity recognition corpora (the i2b2 2012 challenge and the Seoul National University Hospital electronic health record). The proposed model demonstrated better entity recognition performance than both the first- and second-order CRFs and was also more efficient than the higher-order model., Conclusion: The proposed precursor-induced CRF which uses non-entity labels as label transition information improves entity recognition F 1 score by exploiting long-distance transition factors without exponentially increasing the computational time. In contrast, a conventional second-order CRF model that uses longer distance transition factors showed even worse results than the first-order model and required the longest computation time. Thus, the proposed model could offer a considerable performance improvement over current clinical named entity recognition methods based on the CRF models.

Background: Constructing standard and computable clinical diagnostic criteria is an important but challenging research field in the clinical informatics community. The Quality Data Model (QDM) is emerging as a promising information model for standardizing clinical diagnostic criteria., Objective: To develop and evaluate automated methods for converting textual clinical diagnostic criteria in a structured format using QDM., Methods: We used a clinical Natural Language Processing (NLP) tool known as cTAKES to detect sentences and annotate events in diagnostic criteria. We developed a rule-based approach for assigning the QDM datatype(s) to an individual criterion, whereas we invoked a machine learning algorithm based on the Conditional Random Fields (CRFs) for annotating attributes belonging to each particular QDM datatype. We manually developed an annotated corpus as the gold standard and used standard measures (precision, recall and f-measure) for the performance evaluation., Results: We harvested 267 individual criteria with the datatypes of Symptom and Laboratory Test from 63 textual diagnostic criteria. We manually annotated attributes and values in 142 individual Laboratory Test criteria. The average performance of our rule-based approach was 0.84 of precision, 0.86 of recall, and 0.85 of f-measure; the performance of CRFs-based classification was 0.95 of precision, 0.88 of recall and 0.91 of f-measure. We also implemented a web-based tool that automatically translates textual Laboratory Test criteria into the QDM XML template format. The results indicated that our approaches leveraging cTAKES and CRFs are effective in facilitating diagnostic criteria annotation and classification., Conclusion: Our NLP-based computational framework is a feasible and useful solution in developing diagnostic criteria representation and computerization., Competing Interests: The authors declare that they have no competing interests., (Copyright © 2016 Elsevier Inc. All rights reserved.)

De-identification is a shared task of the 2014 i2b2/UTHealth challenge. The purpose of this task is to remove protected health information (PHI) from medical records. In this paper, we propose a novel de-identifier, WI-deId, based on conditional random fields (CRFs). A preprocessing module, which tokenizes the medical records using regular expressions and an off-the-shelf tokenizer, is introduced, and three groups of features are extracted to train the de-identifier model. The experiment shows that our system is effective in the de-identification of medical records, achieving a micro-F1 of 0.9232 at the i2b2 strict entity evaluation level., (Copyright © 2015 Elsevier Inc. All rights reserved.)

Extracting information from unstructured clinical narratives is valuable for many clinical applications. Although natural Language Processing (NLP) methods have been profoundly studied in electronic medical records (EMR), few studies have explored NLP in extracting information from Chinese clinical narratives. In this study, we report the development and evaluation of extracting tumor-related information from operation notes of hepatic carcinomas which were written in Chinese. Using 86 operation notes manually annotated by physicians as the training set, we explored both rule-based and supervised machine-learning approaches. Evaluating on unseen 29 operation notes, our best approach yielded 69.6% in precision, 58.3% in recall and 63.5% F-score., (Copyright © 2014 Elsevier Inc. All rights reserved.)

Objective: In this paper, we focus on three aspects: (1) to annotate a set of standard corpus in Chinese discharge summaries; (2) to perform word segmentation and named entity recognition in the above corpus; (3) to build a joint model that performs word segmentation and named entity recognition., Design: Two independent systems of word segmentation and named entity recognition were built based on conditional random field models. In the field of natural language processing, while most approaches use a single model to predict outputs, many works have proved that performance of many tasks can be improved by exploiting combined techniques. Therefore, in this paper, we proposed a joint model using dual decomposition to perform both the two tasks in order to exploit correlations between the two tasks. Three sets of features were designed to demonstrate the advantage of the joint model we proposed, compared with independent models, incremental models and a joint model trained on combined labels., Measurements: Micro-averaged precision (P), recall (R), and F-measure (F) were used to evaluate results., Results: The gold standard corpus is created using 336 Chinese discharge summaries of 71 355 words. The framework using dual decomposition achieved 0.2% improvement for segmentation and 1% improvement for recognition, compared with each of the two tasks alone., Conclusions: The joint model is efficient and effective in both segmentation and recognition compared with the two individual tasks. The model achieved encouraging results, demonstrating the feasibility of the two tasks.

In Indonesia, where the majority of the population is Muslim, one of the obligations of a Muslim is zakat. To reduce illiteracy about zakat among Muslims, they need to have access to basic information about it. In order to facilitate the acquisition of this information, this study utilized named entity recognition (NER) and defined 12 named entity classes for the zakat domain, including the pillars of Islam, various types of zakat, and zakat management institutions. The Conditional Random Fields method was used for testing Indonesian-NER in three scenarios. In the specific context of the Zakat domain, NER can extract information about organizations, individuals, and locations involved in collecting and distributing Zakat funds. This information can improve the Zakat system’s efficiency and transparency and support research and analysis on Zakat-related topics. The average performance evaluation of the Indonesian-NER model showed a precision of 0.902, recall of 0.834, and an F1-score of 0.867.

In the field of automobile production equipment fault, the entity category of Chinese named entity is complicated, and the traditional word vector can not solve the polysemy of one word. In view of these problems, this study proposes a named entity recognition model in the field of automobile production equipment fault based on BERT(Bidirectional Encoder Representations From Transformer). First, the semantic information and syntactic features are extracted by BERT pretraining model to generate dynamic word vectors. Then, the word vector is input into bidirectional long-short term memory for bidirectional encoding to obtain the semantic features of long sequences. Finally, the conditional random field is used for sequence decoding to learn the dependency relationship between labels and obtain the optimal label sequence. Experiments are carried out on the self-built real automobile production equipment fault data set, and the accuracy, recall rate and F1 value are 87.9 %, 89.6 % and 88.7 %, respectively. [ABSTRACT FROM AUTHOR]

Grammar error correction (GEC) is a crucial task in the field of Natural Language Processing (NLP). Its objective is to automatically detect and rectify grammatical mistakes in sentences, which possesses immense application research value. Currently, mainstream grammar-correction methods primarily rely on sequence labeling and text generation, which are two kinds of end-to-end methods. These methods have shown exemplary performance in areas with low error density but often fail to deliver satisfactory results in high-error density situations where multiple errors exist in a single sentence. Consequently, these methods tend to overcorrect correct words, leading to a high rate of false positives. To address this issue, we researched the specific characteristics of the Chinese grammar error correction (CGEC) task in high-error density situations. We proposed a grammar-correction method based on prompt templates. Firstly, we proposed a strategy for constructing prompt templates suitable for CGEC. This strategy transforms the CGEC task into a masked fill-in-the-blank task compatible with the masked language model BERT. Secondly, we proposed a method for dynamically updating templates, which incorporates already corrected errors into the template through dynamic updates to improve the template quality. Moreover, we used the phonetic and graphical resemblance knowledge from the confusion set as guiding information. By combining this with BERT's prediction results, the model can more accurately select the correct characters, significantly enhancing the accuracy of the model's prediction correction results. Our methods were validated through experiments on a public grammar-correction dataset. The results indicate that our method achieves higher correction performance and lower false correction rates in high-error density scenarios. [ABSTRACT FROM AUTHOR]

Text-to-scene conversion systems map natural language text to formal representations required for visual scenes. The difficulty involved in this mapping is one of the most critical challenges for developing these systems. The current study mapped Persian natural language text as the headmost system to a conceptual scene model. This conceptual scene model is an intermediate semantic representation between natural language and the visual scene and contains descriptions of visual elements of the scene. It will be used to produce meaningful animation based on an input story in this ongoing study. The mapping task was modeled as a sequential labeling problem, and a conditional random field (CRF) model was trained and tested for sequential labeling of scene model elements. To the best of the authors' knowledge, no dataset for this task exists; thus, the required dataset was collected for this task. The lack of required off-the-shelf natural language processing modules and a significant error rate in the available corpora were important challenges to dataset collection. Some features of the dataset were manually annotated. The results were evaluated using standard text classification metrics, and an average accuracy of 85.7% was obtained, which is satisfactory. [ABSTRACT FROM AUTHOR]

To date, several POS taggers have been introduced to facilitate the success of semantic analysis for different languages. However, the task of POS tagging becomes a bit intricate in morphologically complex languages, like Amharic. In this paper, we evaluated different models such as bidirectional long short term memory, convolutional neural network in combination with bidirectional long short term memory, and conditional random field for Amharic POS tagging. Various features, both language-dependent and -independent, have been explored in a conditional random field model. Besides, word-level and character-level features are analyzed in deep neural network models. A convolutional neural network is utilized for encoding features at the word and character level. Each model's performance has evaluated on the dataset that contained 321 K tokens and manually tagged with 31 POS tags. Lastly, the best performance obtained by an end-to-end deep neural network model, convolutional neural network in combination with bidirectional long term short memory and conditional random field, is 97.23% accuracy. This is the highest accuracy for Amharic POS tagging task and is competent with contemporary taggers currently existing in different languages.

We consider two models for the sequence labeling (tagging) problem. The first one is a Pattern-Based Conditional Random Field (PB), in which the energy of a string (chain labeling) x = x 1 ⁢ ... ⁢ x n ∈ D n is a sum of terms over intervals [ i , j ] where each term is non-zero only if the substring x i ⁢ ... ⁢ x j equals a prespecified word w ∈ Λ. The second model is a Weighted Context-Free Grammar (WCFG) frequently used for natural language processing. PB and WCFG encode local and non-local interactions respectively, and thus can be viewed as complementary. We propose a Grammatical Pattern-Based CRF model (GPB) that combines the two in a natural way. We argue that it has certain advantages over existing approaches such as the Hybrid model of Benedí and Sanchez that combines N -grams and WCFGs. The focus of this paper is to analyze the complexity of inference tasks in a GPB such as computing MAP. We present a polynomial-time algorithm for general GPBs and a faster version for a special case that we call Interaction Grammars. [ABSTRACT FROM AUTHOR]

In the organization, they purchased goods or services from different suppliers and used invoice documents to confirm the payment. The invoice documents contained information that can be used for a business decision but, the process of information extraction required many resources to collect the data. The traditional way used template matching-based methods. This process identifies the parts on an image that match a predefined template and requires new manual annotation when processing the new image layout. Therefore, developing a system for robustly extracting entities from different layouts of invoices is necessary. Existing research applied deep learning and Name Entity Recognition (NER) for information extraction but, extracting invoice information was widely done in English and Chinese languages. In this study, we constructed a deep learning model using BiLSTM-CRF (Bidirectional Long Short-Term Memory-Conditional Random Fields) with word and character embedding for information extraction from different layouts of Thai invoice images. The model was evaluated by Semantic Evaluation at a full named-entity level. Our experimental results showed that this method can achieve a precision of 0.9557, recall of 0.9486, and F1-score of 0.9521 for the partial match; precision of 0.9329, recall of 0.9259, and F1-score of 0.9294 for the exact match and the result of the F1-score was significantly influenced by the quality of images and text result from Optical Character Recognition (OCR). Abbreviations: BERT: bidirectional encoder representations from transformers; BiLSTM: bidirectional long short-term memory; COR: correct; CRF: conditional random fields; CV: computer vision; ELMO: embeddings from language model; INC: incorrect; MIS: missing; MSE: mean squared error; MUC: message understanding conference; NER: named entity recognition; NLP: natural language processing; OCR: optical character recognition; PAR: partial; SemEval: semantic evaluation; SPU: spurius [ABSTRACT FROM AUTHOR]

The trade in antiquities and cultural objects has proven difficult to understand and yet is highly dynamic. Currently, there are few computational tools that allow researchers to systematically understand the nature of the legal market, which can also potentially provide insights into the illegal market such as types of objects traded and countries trading antiquities. Online sales in antiquities and cultural objects are often unstructured data; relevant cultural affiliations, types, and materials for objects are important for distinguishing what might sell, but these data are rarely organized in a format that makes the quantification of sales a simple process. Additionally, sale locations and the total value of sales are relevant to understanding the focus and size of the market. These data all provide potentially useful insights into how the market in antiquities and cultural objects is developing. Based on this, this work presents the results of a machine learning approach using natural language processing and dictionary-based searches that investigate relatively low-end but high sales volume objects sold on eBay's U.S. site, where sales are often international, between October 2018 and May 2019. The use of named entity recognition, using a conditional random field approach, classifies objects based on the cultures in which they come from, what type of objects they are, and what the objects are made of. The results indicate that objects from the United Kingdom, affiliated with the Roman period, mostly constituting jewelry, and made of metals sell the most. Metal and jewelry objects, in fact, sold more than other object types. Other important countries for selling ancient and cultural objects include the United States, Thailand, Germany, and Cyprus. Some countries appear to more greatly sellspecific types of objects, such as Egypt being a leader in selling Islamic, terracotta, stone, and wood artifacts and Germany selling Viking/early Medieval weapons. Overall, the approach and tool used demonstrate that it is possible to monitor the online antiquities and cultural objects market while potentially gaining useful insights into the market. The tool developed is provided as part of this work so that it can be applied for other cases and online sites, where it can be applied in real time or using historical data. [ABSTRACT FROM AUTHOR]

Appellate Court Modifications Extraction consists of, given an Appellate Court decision, identifying the proposed modifications by the upper Court of the lower Court judge's decision. In this work, we propose a system to extract Appellate Court Modifications for Portuguese. Information extraction for legal texts has been previously addressed using different techniques and for several languages. Our proposal differs from previous work in two ways: (1) our corpus is composed of Brazilian Appellate Court decisions, in which we look for a set of modifications provided by the Court; and (2) to automatically extract the modifications, we use a traditional Machine Learning approach and a Deep Learning approach, both as alternative solutions and as a combined solution. We tackle the Appellate Court Modifications Extraction task, experimenting with a wide variety of methods. In order to train and evaluate the system, we have built the KauaneJunior corpus, using public data disclosed by the Appellate State Court of Rio de Janeiro jurisprudence database. Our best method, which is a Bidirectional Long Short-Term Memory network combined with Conditional Random Fields, obtained an F β = 1 score of 94.79%. [ABSTRACT FROM AUTHOR]

The conditional random fields (CRFs) model plays an important role in the machine learning field. Driven by the development of the artificial intelligence, the CRF models have enjoyed great advancement. To analyze the recent development of the CRFs, this paper presents a comprehensive review of different versions of the CRF models and their applications. On the basis of elaborating on the background and definition of the CRFs, it analyzes three basic problems faced by the CRF models and reviews their latest improvements. Based on that, it presents the applications of the CRFs in the natural language processing, computer vision, biomedicine, Internet intelligence and other relevant fields. At last, specific analysis and future directions of the CRFs are discussed. [ABSTRACT FROM AUTHOR]

In this paper, our focus is the problem of automatic prediction of Parts of Speech tagging of words in Amharic language text. We conducted a comparison between statistical-based taggers. These are Conditional Random Field (CRF), an HMM-based Trigrams'n'Tags (TnT) Tagger, and Naive Bays (NB) based tagger. We compare the performances of all taggers with the same size of training and testing dataset. Also, various types of language-dependent and independent feature set have been formed, and on each tagger a combination of them are applied. As experimental result revealed that CRF based tagger has achieved best performance than others. The best accuracy obtained from our experiment using CRF is 94.08%. The study has also shown that linguistic features play a decisive role for minimizing or handling the challenges that stems from the morphological complexity of the language. [ABSTRACT FROM AUTHOR]

Named entity recognition (NER) continues to be an important task in natural language processing because it is featured as a subtask and/or subproblem in information extraction and machine translation. In Urdu language processing, it is a very difficult task. This paper proposes various deep recurrent neural network (DRNN) learning models with word embedding. Experimental results demonstrate that they improve upon current state‐of‐the‐art NER approaches for Urdu. The DRRN models evaluated include forward and bidirectional extensions of the long short‐term memory and back propagation through time approaches. The proposed models consider both language‐dependent features, such as part‐of‐speech tags, and language‐independent features, such as the "context windows" of words. The effectiveness of the DRNN models with word embedding for NER in Urdu is demonstrated using three datasets. The results reveal that the proposed approach significantly outperforms previous conditional random field and artificial neural network approaches. The best f‐measure values achieved on the three benchmark datasets using the proposed deep learning approaches are 81.1%, 79.94%, and 63.21%, respectively. [ABSTRACT FROM AUTHOR]

Negation and speculation scope detection is an important task in natural language processing. Previous studies all show that syntactic information is crucial to the task. However, these work mainly focuses on human-designed discrete features and local features extracted from dependency tree, limiting the performance of the task. In this paper, we propose a recursive neural network sequence labeling model, representing whole dependency tree globally and learning automatically syntactic features, for the task. Specifically, recursive neural network first learns a high-level representation for the words in the context of each sentence, and captures the cue word with its target scope through the global dependency structure. Then, CRFs layer takes the representation from recursive neural network as input to jointly decode labels for the whole sentence. Experimental results on English dataset BioScope and Chinese dataset CNeSp show that our model outperforms the state-of-the-art systems. [ABSTRACT FROM AUTHOR]

Part of speech (POS) tagging, the assignment of syntactic categories for words in running text, is significant to natural language processing as a preliminary task in applications such as speech processing, information extraction, and others. Urdu language processing presents a challenge due to the dual behaviour of various Urdu POS tags in differing situations (morphosyntactic ambiguity). This paper addresses this challenge by developing a novel tagging approach using linear-chain conditional random fields (CRF). Our work is the first instance of a CRF approach for Urdu POS tagging. The proposed model employs a strong, stable and balanced language-independent as well as language dependent feature set. The language-dependent feature considered includes part-of-speech tag of the previous word and suffix of the current word while the language-independent features includes the 'context words window'. Our approach was evaluated against support vector machine techniques for Urdu POS—considered as state of the art—on two benchmark datasets. The results show our CRF approach to improve upon the F-measure of prior attempts by 8.3–8.5%. [ABSTRACT FROM AUTHOR]

As a fundamental research in the field of natural language processing, the Uyghur morphological analysis is used mainly to determine the part of speech (POS) and segmental morphemes (stem and affix) of a word in a given sentence, as well as to automatically annotate the grammatical function of the morphemes based on the context. It is necessary to provide various information for other tasks of natural language processing including syntactic analysis, machine translation, automatic summarization, and semantic analysis, etc. In order to increase the morphological analysis efficiency, this paper puts forward a hybrid approach to create a statistical model for Uyghur morphological tagging through a small-scale corpus. Experimental results show that this plan can obtain an overall accuracy of 92.58 % with a limited training corpus. [ABSTRACT FROM AUTHOR]

Natural Language Processing (NLP) is very important in modern data processing taking into consideration different sources, forms and purpose of data as well as information in different areas our industry, administration, public and private life. Our studies concern Natural Language Processing techniques in biomedical field. The increasing volume of information stored in medical health record databases both in natural language and in structured forms is creating increasing challenges for information retrieval (IR) technologies. The paper presents the comparison study of chosen Named Entity Recognition techniques for biomedical field. [ABSTRACT FROM AUTHOR]

The rapid growth in the extraction of clinical events from unstructured clinical records has raised considerable challenges. In this paper, we propose the use of different features with a statical modeling method called conditional random fields, which is consider an algorithm for effectively solving problems of sequence tagging. Our goal is to determine which feature selection can affect the performance of four subtasks presented in SemEval Task-12: Clinical TempEval 2016. We applied a careful preprocessing, where the proposed method was tested on real clinical records from Task-12: Clinical TempEval 2016. The comparative analyses obtained indicate that our proposal achieves good results compared to the work presented in Task-12: Clinical TempEval 2016 challenges. [ABSTRACT FROM AUTHOR]

Named entity recognition (NER) is one of the fundamental problems in many natural language processing applications and the study on NER has great significance. Combining words segmentation and parts of speech analysis, the paper proposes a new NER method based on conditional random fields considering the graininess of candidate entities. The recognition granularity can be divided into two levels: word-based and character-based. We use segmented text to extract characteristics according to the characteristic templates which had been trained in the training phase, and then calculate P (y | x) to get the best result from the input sequence. The paper valuates the algorithm for different graininess on large-scale corpus experimentally, and the results show that this method has high research value and feasibility. [ABSTRACT FROM AUTHOR]

Record linkage is a frequent obstacle to unlocking the benefits of integrated (spatial) data sources. In the absence of unique identifiers to directly join records, practitioners often rely on text‐based approaches for resolving candidate pairs of records to a match. In geographic information science, spatial record linkage is a form of geocoding that pertains to the resolution of text‐based linkage between pairs of addresses into matches and non‐matches. These approaches link text‐based address sequences, integrating sources of data that would otherwise remain in isolation. While recent innovations in machine learning have been introduced in the wider record linkage literature, there is significant potential to apply machine learning to the address matching sub‐field of geographic information science. As a response, this paper introduces two recent developments in text‐based machine learning—conditional random fields and word2vec—that have not been applied to address matching, evaluating their comparative strengths and drawbacks. [ABSTRACT FROM AUTHOR]

We consider the task of reference mining: the detection, extraction and classification of references within the full text of scholarly publications. Reference mining brings forward specific challenges, such as the need to capture the morphology of highly abbreviated words and the dependence among the elements of a reference, both following codified reference styles. This task is particularly difficult, and little explored, with respect to the literature in the arts and humanities, where references are mostly given in footnotes. We apply a deep learning architecture for reference mining from the full text of scholarly publications. We explore and discuss three architectural components: word and character-level word embeddings, different prediction layers (Softmax and Conditional Random Fields) and multi-task over single-task learning. Our best model uses both pre-trained word embeddings and characters embeddings, and a BiLSTM-CRF architecture. We test our solution on a dataset of annotated references from the historiography on Venice and, using a linear-chain CRF classifier as a baseline, we show that this deep learning architecture improves by a considerable margin. Furthermore, multi-task learning performs almost on par with a single-task approach. We thus confirm that there are important gains to be had by adopting deep learning for the task of reference mining.

Abstract Question processing is a natural language processing task which is more complex and requires proper understanding of the question. In this work, we aimed to carry out a deep analysis to determine what types of information can be extracted from various question types. The extracted information is of two types: directly available information like entity types and indirectly inferred information like question type, intent, and domain. Named Entity Recognition (NER) methods can be applied for extracting direct entities. There are predominantly two approaches for NER, one is based on Conditional Random Field (CRF) and other is based on Bi-directional Long Short-Term Memory (Bi-LSTM) network. For identifying intent, domain and type, a Convolutional Neural Network (CNN) based model and a Bi-LSTM based model were used. Global Vector Representation (GloVe) is uesd for word embedding. In addition, the initial GloVe embeddings are enriched with synonyms and antonyms by using the counter fitting method. Experiments were conducted over Quora question collection dataset which is annotated manually. Results are promising for both tasks and the word embedding enrichment using counter-fitting method indicate significant importance in capturing semantics. [ABSTRACT FROM AUTHOR]

Urdu is the national language of Pakistan, also the most widely spoken and understandable language of the globe. In order to accomplish successful Urdu NLP a robust and high-performance NLP tools and resources are utmost necessary. Word segmentation takes on an authoritative role for morphologically rich languages such as Urdu for diverse NLP domains such as named entity recognition, sentiment analysis, part of speech tagging, information retrieval etc. The morphological richness property of Urdu adds to the challenges of the word segmentation task, because a single word can be composed of null or a few prefixes, a stem and null or a few suffixes. In this paper we present supervised Urdu word segmentation scheme based on part of speech (POS) information of the corresponding words. For experiments conditional random fields (CRF) with contextual feature is used. The performance of the proposed system is evaluated on 300K words, results shows evidential improvements on baseline approach. [ABSTRACT FROM AUTHOR]

Personal webpages of researchers or faculty members make up a percentage of the academic web. These webpages contain semi-structured or plain text information, and research has shown the importance of combining information extracted from multiple academic websites to create a unified database that can help in expert finding, and thus improve information retrieval for end users. This research identifies the kind of named entities that could be present in academic biographies by manually examining the biographies extracted from ORCID public profiles, and describes a method that uses natural language processing techniques and supervised machine learning to automatically extract these named entities from the plain text biographies. Up to 86% accuracy was achieved with support vector machines, demonstrating that the method used in this research can be suitable for creating a reusable trained model that extracts useful academic information from researchers’ personal profiles in webpages or other data sources. [ABSTRACT FROM AUTHOR]

Part-of-speech (POS) tagging is a fundamental task in natural language processing (NLP). A robust POS tagger plays an important role in most NLP problems and applications, including syntactic parsing, semantic parsing, machine translation, and question answering. Although a lot of efficient POS taggers has been developed for general, conventional text, little work has been done for social media text. In this paper, we present an empirical study on POS tagging for Vietnamese social media text, which shows several challenges compared with tagging for general text. Social media text does not always conform to formal grammars and correct spelling. It also uses abbreviations, foreign words, and emoticons frequently. A POS tagger developed for conventional text would perform poorly on such noisy data. We address this problem by proposing a tagging model based on Conditional Random Fields (CRFs) with various kinds of features for Vietnamese social media text. We also investigate the effect of features extracted from word clusters under the Brown and canonical correlation analysis (CCA) based clustering in semi-supervised settings. We introduce an annotated corpus for POS tagging, which consists of more than four thousand sentences from Facebook, the most popular social network in Vietnam. Using this corpus, we performed a series of experiments to evaluate the proposed model. Our model achieved 88.26% and 88.92% tagging accuracy in supervised and semi-supervised scenarios, respectively, which are nearly 12% improvement over vnTagger, a state-of-the-art and most widely used Vietnamese POS tagger developed for general, conventional text. In addition, the semi-supervised model outperformed, in terms of accuracy, the version of vnTagger trained on the same Facebook dataset, showing the usefulness of word cluster features. 1 1 This paper is an improved and extended version of Bach et al. (2016) [ABSTRACT FROM AUTHOR]

Recently, the extraction of clinical events from unstructured medical texts has attracted much attention of the research community. Machine learning approaches are popular for this task, due to their ability to solve the problem of sequence tagging effectively. It has been suggested previously that simple features, such as word unigrams, part-of-speech tags, chunk tags, among others, are sufficient for this task. We show that more careful preprocessing and feature selection can significantly improve the results. We used conditional random field classifier with more linguistically oriented features and outperformed the current state-of-the-art approaches. We also show that the popular and much simpler Viterbi algorithm (hidden Markov model-based classification algorithm) can produce competitive results, when its parameters are tuned using specific optimization techniques. We evaluate these algorithms for the task of extraction of medical events from the corpus developed for SemEval shared Task 12: Clinical TempEval (Temporal Evaluation) 2016, namely, for its two subtasks: (i) event detection and (ii) event classification based on contextual modality. [ABSTRACT FROM AUTHOR]

In this paper, we introduce a simple unsupervised framework for pre-training hidden-unit conditional random fields (HUCRFs), i.e., learning initial parameter estimates for HUCRFs prior to supervised training.Our framework exploits the model structure of HUCRFs to make effective use of unlabeled data from the same domain or labeled data from a different domain. The key idea is to use the separation of HUCRF parameters between observations and labels: this allows us to pre-train observation parameters independently of label parameters. Pre-training is achieved by creating pseudo-labels from such resources. In the case of unlabeled data, we cluster observations and use the resulting clusters as pseudo-labels. Observation parameters can be trained on these resources and then transferred to initialize the supervised training process on the target labeled data. Experiments on various sequence labeling tasks demonstrate that the proposed pre-training method consistently yields significant improvement in performance. The core idea could be extended to other learning techniques including deep learning. We applied the proposed technique to recurrent neural networks (RNN) with long short term memory (LSTM) architecture and obtained similar gains. [ABSTRACT FROM AUTHOR]

In recent years, many applications in natural language processing (NLP) have been developed using the machine learning approach. Annotating data is an important task in applying machine learning to NLP applications. A common approach to improve the system performance is to train on a large and high-quality set of training data that is annotated by experts. Besides, active learning (AL) and self-learning can be utilized to reduce the annotation costs. The self-learning method discovers highly reliable instances based on a trained classifier, while AL queries the most informative instances based on active query algorithms. This paper proposes a method that combines AL and self-learning to reduce the labeling effort for the named entity recognition task from tweet streams by using both machine-labeled and manually-labeled data. We employ AL queries based on the diversity of the context and content of instances to select the most informative instances. The conditional random fields are also chosen as an underlying model to train a classifier for selecting highly reliable instances. The experiments using Twitter data show that the proposed method achieves good results in reducing the human labeling effort, and it can significantly improve the performance of the systems. [ABSTRACT FROM AUTHOR]