Search

Extractive text summarization selects asmall subset of sentences from a document, which gives good “coverage” of a document. When given a set of term weights indicating the importance of the terms, the concept of coverage may be formalized into a combinatorial optimization problem known as the budgeted maximum coverage problem. Extractive methods in this class are known to beamong the best of classic extractive summarization systems. This paper gives a synopsis of thesoftware package occams, which is a multilingual extractive single and multi-document summarization package based on an algorithm giving an optimal approximation to the budgeted maximum coverage problem. The occams package is written in Python and provides an easy-to-use modular interface, allowing it to work in conjunction with popular Python NLP packages, such as nltk, stanza or spacy.

This paper presents an evaluation and comparison of three state-of-the-art models for text summarization, and proposes a new digital reading interface designed for neophyte users to exploit these models, as well as automatic keyword extraction, with little or no programming experience.

ion is a strategy that gives the essential points of a document in a short period of time. The video abstraction approach proposed in this research is based on multi-modal video data, which comprises both audio and visual data. Segmenting the input video into scenes and obtaining a textual and visual summary for each scene are the major video abstraction procedures to summarize the video events into a static document. To recognize the shot and scene boundary from a video sequence, a hybrid features method was employed, which improves detection shot performance by selecting strong and flexible features. The most informative keyframes from each scene are then incorporated into the visual summary. A hybrid deep learning model was used for abstractive text summarization. The BBC archive provided the testing videos, which comprised BBC Learning English and BBC News. In addition, a news summary dataset was used to train a deep model. The performance of the proposed approaches was assessed using metrics like Rouge for textual summary, which achieved a 40.49% accuracy rate. While precision, recall, and F-score used for visual summary have achieved (94.9%) accuracy, which performed better than the other methods, according to the findings of the experiments.

The tremendous increase in documents available on the Web has turned finding the relevant pieces of information into a challenging, tedious, and time-consuming activity. Text summarization is an important natural language processing (NLP) task used to reduce the reading requirements of text. Automatic text summarization is an NLP task that consists of creating a shorter version of a text document which is coherent and maintains the most relevant information of the original text. In recent years, automatic text summarization has received significant attention, as it can be applied to a wide range of applications such as the extraction of highlights from scientific papers or the generation of summaries of news articles. In this research project, we are focused mainly on abstractive text summarization that extracts the most important contents from a text in a rephrased form. The main purpose of this project is to summarize texts in German. Unfortunately, most pretrained models are only available for English. We therefore focused on the German BERT multilingual model and the BART monolingual model for English, with a consideration of translation possibilities. As the source of the experiment setup, took the German Wikipedia article dataset and compared how well the multilingual model performed for German text summarization when compared to using machine-translated text summaries from monolingual English language models. We used the ROUGE-1 metric to analyze the quality of the text summarization.

—In this paper we present the study on the potential of Natural Language Processing (NLP) to enhance the online bookstore customer experience. NLP is a subfield of computer science that studies computer-human language interaction. With the rise of virtual data, NLP has grown in importance, and it is now vital to develop NLP software capable of analyzing huge amounts of textual data. The proposed system integrates various NLP techniques. The ultimate goal is to offer an innovative and user-centric solution for online bookstores, showcasing the potential of NLP to revolutionize the virtual shopping experience. Additionally, a seminar on the latest advancements in NLP for the book industry is presented, covering the three key areas of text summarization, sentiment analysis of book reviews, and book search using image recognition and NLP. The seminar gives useful insight on the most recent advancements in NLP and offers an opportunity for stimulating discussion between academics, practitioners, and business leaders.

Natural language processing (NLP) is a field in machine learning that analyses and manipulate huge amounts of data and generates human language. There are a variety of applications of NLP such as sentiment analysis, text summarization, spam filtering, language translation, etc. Since privacy documents are important and legal, they play a vital part in any agreement. These documents are very long, but the important points still have to be read thoroughly. Customers might not have the necessary time or the knowledge to understand all the complexities of a privacy policy document. In this context, this paper proposes an optimal model to summarize the privacy policy in the best possible way. The methodology of text summarization is the process where the summaries from the original huge text are extracted without losing any vital information. Using the proposed idea of a common word reduction process combined with natural language processing algorithms, this paper extracts the sentences in the privacy policy document that hold high weightage and displays them to the customer, and it can save the customer’s time from reading through the entire policy while also providing the customers with only the important lines that they need to know before signing the document. The proposed method uses two different extractive text summarization algorithms, namely LexRank and Kullback Leibler (KL) Summarizer, to summarize the obtained text. According to the results, the summarized sentences obtained via the common word reduction process and text summarization algorithms were more significant than the raw privacy policy text. The introduction of this novel methodology helps to find certain important common words used in a particular sector to a greater depth, thus allowing more in-depth study of a privacy policy. Using the common word reduction process, the sentences were reduced by 14.63%, and by applying extractive NLP algorithms, significant sentences were obtained. The results after applying NLP algorithms showed a 191.52% increase in the repetition of common words in each sentence using the KL summarizer algorithm, while the LexRank algorithm showed a 361.01% increase in the repetition of common words. This implies that common words play a large role in determining a sector’s privacy policies, making our proposed method a real-world solution for environmental sustainability.

Summarizing a document has become a necessity as, because so much information is produced every day. Document summary makes it simpler to understand the text document than it would be to read through a collection of documents. A foundation for creating an condensed version of one or more text documents is provided by text summary. It is a crucial method for finding pertinent information on the Internet or in sizable text libraries. Additionally, it is essential to extract data in a way that the user would find the information interesting. Extractive summarization and abstract summarization are the two basic approaches used for text summarizing. In order to create the summary, the extractive summarization method chooses the sentences from a Word document and arranges them according to their weight. Abstractive summarizing is a technique that takes the key ideas from a document's content and expresses them abstractly in plain English. Numerous summary methods have been created on the foundation of these two approaches. There are numerous techniques that are language-specific exclusively. In this paper, we used extractive summarization methods and got good results.

In the last decade, the techniques of news aggregation and summarization have been increasingly gaining relevance for providing users on the web with condensed and unbiased information. Indeed, the recent development of successful machine learning algorithms, such as those based on the transformers architecture, have made it possible to create effective tools for capturing and elaborating news from the Internet. In this regard, this work proposes, for the first time in the literature to the best of the authors’ knowledge, a methodology for the application of such techniques in news related to cryptocurrencies and the blockchain, whose quick reading can be deemed as extremely useful to operators in the financial sector. Specifically, cutting-edge solutions in the field of natural language processing were employed to cluster news by topic and summarize the corresponding articles published by different newspapers. The results achieved on 22,282 news articles show the effectiveness of the proposed methodology in most of the cases, with 86.8% of the examined summaries being considered as coherent and 95.7% of the corresponding articles correctly aggregated. This methodology was implemented in a freely accessible web application.

Tez çalışmasında çıkarım tabanlı metin özetleme yönteminin Türkçe metinlerde kullanımına ilişkin cümle derecelendirme yöntemleri deneysel çalışma olarak yürütülmüş, literatürde bulunan yöntemlere ek olarak yeni yöntemler önerilmiştir. Ulusal haber sitelerinde bulunan içeriklerden elde edilen haber metinleri veri seti olarak kullanılmış, bu veri setinde üç farklı kullanıcının özet çalışması yapılmış ve bu özet çalışmaları daha sonra tek bir sonuca indirgenmiştir. Kontrol veri seti olarak tanımlanan bu kısım oluşturulacak özet ile başarımın değerlendirilmesi için kullanılmıştır. Literatürde mevcut olan cümle derecelendirme yöntemlerine ek olarak büyük ünlü uyumu, küçük ünlü uyumu ve bu iki kuralın birlikte denendiği hibrit bir model önerilmiştir. Önerilen üç yöntemin değerlendirilmesi ROUGE ve BLEU metrikleri ile ölçümlenmiş ve sonucunda geleneksel yöntemlerden daha iyi sonuç verdiği gözlemlenmiştir. İki metrik ile değerlendirmenin yanı sıra özetleme çalışması bir sınıflandırma problemi olarak ele alınmış, literatürde bulunan dört farklı sınıflandırma algoritması bu çalışmada kullanılmıştır. Çalışma önerilen yöntemlerin salt kullanımı, geleneksel yöntemlerin salt kullanımı ve her iki yöntemin birlikte kullanıldığı hibrit yöntem üç ayrı başlıkta değerlendirilmiş ve önerilen yöntemlerin üç algoritmada en iyi sonucu, bir algoritmada ise hibrit yöntemin en iyi sonucu verdiği gözlemlenmiştir., In this thesis study, sentence grading methods related to the use of extractive text summarization method in Turkish texts were carried out as an experimental study, and new methods were proposed in addition to the methods found in the literature. The news texts obtained from the contents of the national news sites were used as a data set, a summary study of three different users was carried out in this data set, and these summary studies were later reduced to a single result. This part, defined as the control data set, was used to evaluate the performance with the summary to be created. In addition to the sentence grading methods available in the literature, major vowel harmony, minor vowel harmony, and a hybrid model in which these two rules are tested together were proposed. Evaluation of the three proposed methods was measured with ROUGE and BLEU metrics, and it was observed that it gave better results than the traditional methods. The summarization study was handled as a classification problem in addition to the evaluation with two metrics, and four different classification algorithms in the literature were used in this study. In the study, the use of only the proposed methods, the use of only the traditional methods and the hybrid method in which both methods are used together were evaluated in three different sections and it was observed that the proposed methods gave the best result in three algorithms and the hybrid method gave the best result in one algorithm.

Ο όγκος των δεδομένων που διατίθενται στο Διαδίκτυο αυξάνεται καθημερινά με ραγδαίους ρυθμούς. Ως αποτέλεσμα πολύτιμες πληροφορίες βρίσκονται ελεύθερα διαθέσιμες στο Διαδίκτυο χωρίς ωστόσο να είναι πάντα δυνατή η εύρεση, η επεξεργασία και η αξιοποίησή τους από τον άνθρωπο. Στο πλαίσιο αυτό παρουσιάζεται η ανάγκη για σύμπτυξη της διαθέσιμης πληροφορίας. Στην παρούσα εργασία μελετάται η αυτόματη παραγωγή περιλήψεων που αποτελεί μέθοδο σύμπτυξης της πληροφορίας κειμένου. Πιο συγκεκριμένα, διερευνάται η παραγωγή περιλήψεων κειμένων που προέρχονται από το κοινωνικό δίκτυο Reddit με χρήση προ-εκπαιδευμένων νευρωνικών δικτύων τύπου κωδικοποιητή-αποκωδικοποιητή, αρχιτεκτονικής Transformer. Αρχικά εξετάζονται οι επιδόσεις που πετυχαίνουν τα νευρωνικά μοντέλα BART, PEGASUS και ProphetNet στο πρόβλημα της εξαγωγής abstractive περιλήψεων στο θορυβώδες σύνολο δεδομένων Webis-TLDR-17 του Reddit. Ακολούθως πραγματοποιείται νέο fine-tuning των ίδιων μοντέλων σε υποσύνολο του Webis-TLDR-17 και διερευνώνται οι νέες επιδόσεις τους. Τα αποτελέσματα των πειραμάτων υποδεικνύουν ότι στο πρόβλημα της παραγωγής των περιλήψεων του Webis-TLDR-17 επιτυγχάνεται μεταφορά μάθησης τόσο από το PEGASUS που έχει προ-εκπαιδευτεί σε δεδομένα του ίδιου τομέα, όσο και από τα BART και ProphetNet που έχουν προ-εκπαιδευτεί σε ειδησεογραφικά δεδομένα. Το περαιτέρω fine-tuning στο ίδιο το Webis-TLDR-17 βελτιώνει τις επιδόσεις όλων των μοντέλων που εξετάστηκαν με όρους μετρικών ROUGE, ωστόσο αυτό δε συνεπάγεται απαραίτητα τη βελτίωση της ποιότητας των περιλήψεων., The volume of the data that is available on the Internet is growing rapidly on a daily basis. Although this results in valuable information being freely available online, humans are not always capable of finding, processing, and leveraging this information. In this context the need for information compression arises. In this work we study the automatic summarization of texts originating from the social network Reddit, with the use of pre-trained Transformer-based encoder-decoder neural networks. Initially we examine the performance of the neural models BART, PEGASUS, and ProphetNet on the noisy Webis-TLDR-17 Reddit dataset in the task of abstractive summarization. Subsequently we fine-tune the same models one more time using a subset of Webis-TLDR-17 and re-examine their performance. The experimental results suggest that successful transfer-learning from both PEGASUS, that has been pre-trained on same-domain data, and BART and ProphetNet, that have been pre-trained on news domain data, is achieved in the task of Webis-TLDR-17 summarization. Further fine-tuning of the models on Webis-TLDR-17 enhances their performance in terms of ROUGE scores, however, this does not directly translate to higher summary quality., Σε συνεργασία με: Λογιστικής και Χρηματοοικονομικής (Πανεπιστήμιο Μακεδονίας)

Along with the increasing number of scientific publications, many scientific communities must read the entire text to get the essence of information from a journal article. This will be quite inconvenient if the scientific journal article is quite long and there are more than one journals. Motivated by this problem, encourages the need for a method of text summarization that can automatically, concisely, and accurately summarize a scientific article document. The purpose of this research is to create an extractive text summarization by doing feature engineering to extract the semantic information from the original text. Comparing the long short-term memory algorithm and gated recurrent units and were used to get the most relevant sentences to be served as a summary. The results showed that both algorithms yielded relatively similar accuracy results, with gated recurrent units at 98.40% and long short-term memory at 98.68%. The evaluation method with matrix recall-oriented understudy for gisting evaluation (ROUGE) is used to evaluate the summary results. The summary results produced by the LSTM model compared to the summary results using the latent semantic analysis (LSA) method were then obtained recall values at ROUGE-1, ROUGE-2, and ROUGE-L respectively were 76.25%, 59.49%, and 72.72%.

The emergence of attention-based architectures has led to significant improvements in the performance of neural sequence-to-sequence models for text summarization. Although these models have proved to be effective in summarizing English-written documents, their portability to other languages is limited thus leaving plenty of room for improvement. In this paper, we present BART-IT, a sequence-to-sequence model, based on the BART architecture that is specifically tailored to the Italian language. The model is pre-trained on a large corpus of Italian-written pieces of text to learn language-specific features and then fine-tuned on several benchmark datasets established for abstractive summarization. The experimental results show that BART-IT outperforms other state-of-the-art models in terms of ROUGE scores in spite of a significantly smaller number of parameters. The use of BART-IT can foster the development of interesting NLP applications for the Italian language. Beyond releasing the model to the research community to foster further research and applications, we also discuss the ethical implications behind the use of abstractive summarization models.

Despite recent progress in abstractive summarization, models often generate summaries with factual errors. Numerous approaches to detect these errors have been proposed, the most popular of which are question answering (QA)-based factuality metrics. These have been shown to work well at predicting summary-level factuality and have potential to localize errors within summaries, but this latter capability has not been systematically evaluated in past research. In this paper, we conduct the first such analysis and find that, contrary to our expectations, QA-based frameworks fail to correctly identify error spans in generated summaries and are outperformed by trivial exact match baselines. Our analysis reveals a major reason for such poor localization: questions generated by the QG module often inherit errors from non-factual summaries which are then propagated further into downstream modules. Moreover, even human-in-the-loop question generation cannot easily offset these problems. Our experiments conclusively show that there exist fundamental issues with localization using the QA framework which cannot be fixed solely by stronger QA and QG models.

With the rapid advancement in healthcare, there has been exponential growth in the healthcare records stored in large databases to help researchers, clinicians, and medical practitioner’s for optimal patient care, research, and trials. Since these studies and records are lengthy and time consuming for clinicians and medical practitioners, there is a demand for new, fast, and intelligent medical information retrieval methods. The present study is a part of the project which aims to design an intelligent medical information retrieval and summarization system. The whole system comprises three main modules, namely adverse drug event classification (ADEC), medical named entity recognition (MNER), and multi-model text summarization (MMTS). In the current study, we are presenting the design of the ADEC module for classification tasks, where basic machine learning (ML) and deep learning (DL) techniques, such as logistic regression (LR), decision tree (DT), and text-based convolutional neural network (TextCNN) are employed. In order to perform the extraction of features from the text data, TF-IDF and Word2Vec models are employed. To achieve the best performance of the overall system for efficient information retrieval and summarization, an ensemble strategy is employed, where predictions of the selected base models are integrated to boost the robustness of one model. The performance results of all the models are recorded as promising. TextCNN, with an accuracy of 89%, performs better than the conventional machine learning approaches, i.e., LR and DT with accuracies of 85% and 77%, respectively. Furthermore, the proposed TextCNN outperforms the existing adverse drug event classification approaches, achieving precision, recall, and an F1 score of 87%, 91%, and 89%, respectively.

Submitted by Daniella Santos (daniella.santos@ufma.br) on 2022-11-29T16:53:08Z No. of bitstreams: 1 Leandro_Massetti.pdf: 1387963 bytes, checksum: 008d7727d567705ea9c079f7106ab33c (MD5) Made available in DSpace on 2022-11-29T16:53:08Z (GMT). No. of bitstreams: 1 Leandro_Massetti.pdf: 1387963 bytes, checksum: 008d7727d567705ea9c079f7106ab33c (MD5) Previous issue date: 2022-09-16 FAPEMA A Learning Object (LO) is a digital resource that can be used and reused or referenced during a process of technological support for teaching and learning. Despite being mostly multimedia, with audio, video, text and images synchronized with each other, some digital education resources have texts as one of their main elements in the teaching process, such as websites, texts, video classes, seminars, and the summarization of these texts can be a way of composing multimedia LOs. However, text summarization is a costly process in time and effort, creating the need to seek new ways to generate this content. The present work show a solution for the composition of multimedia LOs through automatic text summarizers based on Deep Learning Transformers models from two experiments: The first one composing LOs from educational texts in Portuguese using translators and text summarizers, in this experiment the results presented were positive and allow comparing the performance of summaries as generators of LO in text format; The second experiment presents an educational video summarization solution using the same Deep Learning models for subtitle summarization, the tests were performed using the EDUVSUM dataset in which it was possible to improve the results of the original article reaching 26.53% accuracy in a multi-class problem and average absolute error of 1.49 per video frame and 1.45 per video segment. Um Objeto de Aprendizagem (OA) é um recurso digital, que pode ser utilizado e reutilizado ou referenciado durante um processo de suporte tecnológico ao ensino e aprendizagem. Apesar de serem principalmente multimídia, com áudio, vídeo, texto e imagens sincronizados entre si, alguns recursos digitais de educação possuem textos como um de seus elementos principais no processo de ensino, como sites, textos, vídeo aulas, seminários, e a sumarização desses textos podem ser uma forma de composição de OAs multimídia. No entanto, a sumarização de textos é um processo oneroso em tempo e esforço, gerando a necessidade de buscar novas formas de gerar esse conteúdo. Este trabalho apresenta uma solução para a composição de OAs multimídia através de sumarizadores automáticos de texto baseados em modelos Deep Learning Transformers a partir de dois experimentos: O primeiro fazendo a composição de OAs a partir de textos educacionais na língua portuguesa utilizando tradutores e sumarizadores de texto, neste experimento os resultados apresentados foram positivos e permitem comparar o desempenho dos resumos como geradores de OA em formato de texto; O segundo experimento apresenta uma solução de sumarização de vídeos educacionais utilizando os mesmos modelos de Deep Learning para a sumarização da legenda, os testes foram realizados utilizando o dataset EDUVSUM no qual foi possível melhorar os resultados do artigo original alcançando 26,53% de acurácia em um problema multi-classe e erro absoluto médio de 1,49 por frame do vídeo e 1,45 por segmento de vídeo.

This data is the basic information of books from dangdang.

The progress of communications over internet media such as social media and messengers has led to the production of large amount of textual data. This kind of information contains a lot of valuable knowledge and can be used to improve the performance of other NLP[1] tasks. There are several ways to use such information, one of which is text summarization Summarizing textual information can extract the main content of text within a short time. In this paper, we propose an approach for extractive summarization on Persian texts by using sentences embedding and a sparse coding framework. Most previous works focuses on text’s sentences individually which may not consider the hidden structure patterns between them. In this paper, our proposed approach can consider the relations between the text’s sentences by using three main criteria, namely coverage, diversity and sparsity, when selecting the summary sentences. By considering these criteria, we select sentences that can reconstruct the whole text with least reconstruction error. The proposed approach is evaluated on Persian dataset Pasokh and achieved 10.02% and 8.65% improvement compared to the state-of-the-art methods in rouge-1 and rouge-2 f-scores, respectively. We show that considering semantic relations among the text’s sentences can lead us to better sentence summarization. [1] Natural language processing

Neural headline generation models have recently shown great results since neural network methods have been applied to text summarization. In this paper, we focus on news headline generation. We propose a news headline generation model based on a generative pre-training model. In our model, we propose a rich features input module. The headline generation model we propose only contains a decoder incorporating the pointer mechanism and the n-gram language features, while other generation models use the encoder-decoder architecture. Experiments on news datasets show that our model achieves comparable results in the field of news headline generation.

Text summarization is an information condensation technique that abbreviates a source document to a few representative sentences with the intention to create a coherent summary containing relevant information of source corpora. This promising subject has been rapidly developed since the advent of deep learning. However, summarization models based on deep neural network have several critical shortcomings. First, a large amount of labeled training data is necessary. This problem is standard for low-resource languages in which publicly available labeled data do not exist. In addition, a significant amount of computational ability is required to train neural models with enormous network parameters. In this study, we propose a model called Learning Free Integer Programming Summarizer (LFIP-SUM), which is an unsupervised extractive summarization model. The advantage of our approach is that parameter training is unnecessary because the model does not require any labeled training data. To achieve this, we formulate an integer programming problem based on pre-trained sentence embedding vectors. We also use principal component analysis to automatically determine the number of sentences to be extracted and to evaluate the importance of each sentence. Experimental results demonstrate that the proposed model exhibits generally acceptable performance compared with deep learning summarization models although it does not learn any parameters during the model construction process.

Precision education is a new challenge in leveraging artificial intelligence, machine learning, and learning analytics to enhance teaching quality and learning performance. To facilitate precision education, text marking skills can be used to determine students’ learning process. Text marking is an essential learning skill in reading. In this study, we proposed a model that leverages the state-of-the-art text summarization technique, Bidirectional Encoder Representations from Transformers (BERT), to calculate the marking score for 130 graduate students enrolled in an accounting course. Then, we applied learning analytics to analyze the correlation between their marking scores and learning performance. We measured students’ self-regulated learning (SRL) and clustered them into four groups based on their marking scores and marking frequencies to examine whether differences in reading skills and text marking influence students’ learning performance and awareness of self-regulation. Consistent with past research, our results did not indicate a strong relationship between marking scores and learning performance. However, high-skill readers who use more marking strategies perform better in learning performance, task strategies, and time management than high-skill readers who use fewer marking strategies. Furthermore, high-skill readers who actively employ marking strategies also achieve superior scores of environment structure, and task strategies in SRL than low-skill readers who are inactive in marking. The findings of this research provide evidence supporting the importance of monitoring and training students’ text marking skill and facilitating precision education.

When people debate, they want to familiarize themselves with a whole range of arguments about a given topic in order to deepen their knowledge and inspire new claims. However, the amount of differently phrased arguments is humongous, making the process of processing them time-consuming. In spite of many works on using arguments (e.g. counter-argument generation), there is only a few studies on argument aggregation. To address this problem, we propose a new task in argument mining – Argument Extraction, which gathers similar arguments into key points, usually single sentences describing a set of arguments for a given debate topic. Such a short summary of related arguments has been manually created in previous research, while in our research key point generation becomes fully automatic, saving time and cost. As the first step of key point generation we explore existing similarity calculation methods, i.e. Sentence-BERT and MoverScore to investigate their performance. Next, we propose a combination of argument similarity and Maximal Marginal Relevance (MMR) for extracting key phrases to be utilized in our novel task of Argument Extraction. Experimental results show that MoverScore-based MMR outperforms strong baselines covering 72.5% of arguments when eleven or more arguments are extracted. This percentage is almost identical with the cover rate of human-made key points.

Text summarization is an important task in natural language processing (NLP). Neural summary models summarize information by understanding and rewriting documents through the encoder-decoder structure. Recent studies have sought to overcome the bias that cross-entropy-based learning methods can have through reinforcement learning (RL)-based learning methods or the problem of failing to learn optimized for metrics. However, the ROUGE metric with only $n$ -gram matching is not a perfect solution. The purpose of this study is to improve the quality of the summary statement by proposing a reward function used in text summarization based on RL. We propose ROUGE-SIM and ROUGE-WMD, modified functions of the ROUGE function. ROUGE-SIM enables meaningfully similar words, in contrast to ROUGE-L. ROUGE-WMD is a function adding semantic similarity to ROUGE-L. The semantic similarity between articles and summary text was computed using Word Mover’s Distance (WMD) methodology. Our model with two proposed reward functions demonstrated superior performance on ROUGE-1, ROUGE-2, and ROUGE_L than on ROUGE-L as a reward function. Our two models, ROUGE-SIM and ROUGE-WMD, scored 0.418 and 0.406 for ROUGE-L, respectively, for the Gigaword dataset. The two reward functions outperformed ROUGE-L even in the abstractiveness and grammatical aspects.

Legal judgments are generally very long, and relevant information is often scattered throughout the text. To complete a legal judgment summarization, capturing important, relevant information comprehensively from a lengthy text is crucial. The existing abstractive-summarization models based on pre-trained language have restrictions on the length of an input text. Another concern is that the generated summaries have not been well integrated with the legal judgment’s technical terms and specific topics. In this paper, we used raw legal judgments as information of different granularities and proposed a two-stage text-summarization model to handle different granularities of information. Specifically, we treated the legal judgments as a sequence of sentences and selected key sentence sets from the full texts as an input corpus for summary generation. In addition, we extracted keywords related to technical terms and specific topics in the legal texts and introduced them into the summary-generation model as an attention mechanism. The experimental results on the CAIL2020 and the LCRD datasets showed that our model achieved an overall 0.19–0.41 improvement in its ROUGE score, as compared to the baseline models. Further analysis also showed that our method could comprehensively capture essential and relevant information from lengthy legal texts and generate better legal judgment summaries.

The quality of students’ highlights can be an indicator of their learning performance. While the most common approach to grade their highlights is by humans, human grading can be inconsistent, especially when the number of highlights are large or when graders have different background knowledge. In this research, we propose a model to automatically extract important concepts from class materials, analyze students’ highlights and find the correlation between highlight quality and students’ learning performance. We first compared different text summarization algorithms with different evaluations to see which of them generates the summarization that is closest to the reference answer generated by humans. Then we used the selected algorithm to summarize the text from learning materials as important concepts, and compared the summaries with students’ highlights to calculate their highlight scores. Finally, we considered the highlight score from the best method as the highlight quality and observed whether it has a correlation to students’ learning performance., 28th International Conference on Computers in Education, 23-27 November 2020, Web conference.

Attention-based encoding and decoding models have been widely used in text abstracts, machine translation and other sequence-to-sequence tasks. In deep learning framework, multi-layer neural network can obtain different feature representations of input data. Therefore, in conventional encoding and decoding model, the performance of the model is usually improved by stacking multi-layer decoders. However, the existing models only pay attention to the output of the last layer of the encoder when decoding, and ignore the information of other layers. In view of this, this paper proposes a novel abstractive text summarization model based on recurrent neural network and multi-layer interactive attention mechanism. The multi-layer interactive attention mechanism is introduced to extract contextual information from different levels of the encoder to guide the generation of abstracts. In order to deal with the problem of information redundancy caused by introducing different levels of context, the variational information bottleneck is adopted to compress data noise. Finally, this paper conducts experiments on Gigaword and DUC2004 datasets, and the results show that the proposed method achieves state of the art performance.

Velika količina javno dostupnih tekstnih podataka omogućila je brz razvoj dubokih neuronskih modela za obradu prirodnog jezika. Jedan od zadataka obrade prirodnog jezika koji je posebno profitirao od velika količina teksta, a koji istovremeno ima potencijal riješiti problem prekomjernosti podataka, jest strojno sažimanje teksta. Posebice, cilj ekstraktivnog sažimanja teksta jest stvaranje kratkog ali informacijskog bogatog i sažetog podskupa izvornog teksta. Tema ovog rada jest istražiti i (re)implementirati nov pristup unutar područja ekstraktivnog sažimanja, koji se fokusira na ekstrastivno sažimanje u kontekstu semantičkog podudaranja teksta. Do sada je većina neuronskih ekstraktivnih modela sažimanja slijedila istu paradigmu: izdvoji rečenice, ocijeni ih i odaberi one koje nose najviše informacija. Međutim, odabirom samo najistaknutijih rečenica često dobivamo sažetak u kojem je većina rečenica suvišna. Zhong i suradnici (2020.) predložili su nov pristup u kojem se, umjesto pojedinačnog biranja najistaknutijih rečenica (sažimanje na razini rečenice), naglasak stavlja na istovremeno generiranje i odabir najistaknutijih sažetaka (sažimanje na razini sažetka). Cilj rada jest nadograditi ovu novu paradigmu, istražiti nedostatke prethodnih i trenutnih modela te potencijalno poboljšati izvedbu ovog novog pristupa ekstrativnog sažimanja. Radu priložiti izvroni i izvršni kod modela, skupove podataka i programsku dokumentaciju te citirati korištenu literaturu. Large amounts of publicly available textual data made possible the rapid developments of neural natural language processing (NLP) models. One of the NLP tasks that particularly benefited from large amounts of text, but which at the same time holds promise for solving the problem of data overabundance, is automated text summarization. In particular, the goal of extractive text summarization is the production of a short but informationally rich and condensed subset of the original text. The topic of this thesis is to research and (re)implement a new approach within the extractive summarization field, which focuses on framing extractive summarization as a semantic text-matching problem. As of now, most of the neural extractive summarization models follow the same paradigm: extract sentences, score them and pick the most salient ones. However, by choosing the most salient sentences, we are often left with a summary where most sentences are redundant. Zhong et al. (2020) proposed a novel paradigm where instead of choosing the salient sentences individually (sentence-level summarization), the focus is on simultaneously generating and picking the most salient summaries (summary-level summarization). The objective of the thesis is to reimplement this novel paradigm, research the flaws of previous models, and potentially improve the capabilities of this new summarization approach. All references must be cited, and all source code, documentation, executables, and datasets must be provided with the thesis.

The Linz Center of Mechatronics’ SymSpace platform is designed to provide intelligent design and training for the traditional engineer training and industrial design approach in the field of motor design, which relies on the engineer’s own experience and manual work. This paper first analyzes and explores the usage patterns and possible improvement perspectives of motor design components using SymSpace user data. Then an attempt is made to summarize the motor design manual provided by LCM using a text summary model and use it for training engineers. Next, a question-and-answer system model was used to try to provide an aid system for engineers in design. The evaluation of text summaries and question and answer systems is difficult in the motor design domain because the amount of redundant textual information in this domain is small and key information is often presented in detail rather than in the main stem of the sentence. In this case, instead of evaluating the model using traditional machine scores, this paper refers to the feedback from LCM experts as future users. The final results show that, despite the problems of difficulty in explaining the reasons; the possibility of being misleading; and the loss of information details, both attempts are generally positive and the exploration in this direction is worthwhile. Symspace från Linz Center of Mechatronics är utformad för att tillhandahålla intelligent design och utbildning för den traditionella ingenjörsutbildningen och den industriella designmetoden inom motorkonstruktion, som bygger på ingenjörens egen erfarenhet och manuellt arbete. I den här artikeln analyseras och utforskas först användningsmönster och möjliga förbättringsperspektiv för komponenter för motorkonstruktion med hjälp av användaruppgifter från Symspace. Därefter görs ett försök att sammanfatta den motorkonstruktionshandbok som tillhandahålls av LCM med hjälp av en modell för textsammanfattningar och använda den för att utbilda ingenjörer. Därefter användes en modell för ett system med frågor och svar för att försöka tillhandahålla ett hjälpsystem för ingenjörer vid konstruktion. Utvärderingen av textsammanfattningar och fråga-och-svar-system är svår inom motorkonstruktionsområdet eftersom mängden överflödig textinformation inom detta område är liten och nyckelinformation ofta presenteras i detalj snarare än i huvudstammen av meningen. I det här fallet hänvisar den här artikeln i stället för att utvärdera modellen med hjälp av traditionella maskinpoäng till feedback från LCM-experter som framtida användare. De slutliga resultaten visar att trots problemen med svårigheten att förklara orsakerna, möjligheten att vara vilseledande och förlusten av informationsdetaljer är båda försöken generellt sett positiva och att utforskningen i denna riktning är värd att fortsätta.

Information gathering and information overload is an ever growing topic of concernfor Information Technology (IT) experts. The amount of information dealt withon an everyday basis is large enough to take up valuable time having to scatterthrough it all to find the relevant information. As for the application area of IT,time is directly related to money as having to waste valuable production time ininformation gathering and allocation of human resources is a direct loss of profitsfor any given company. Two issues are mainly addressed through this thesis: textsare too lengthy and the difficulty of finding relevant information. Through the useof Natural Language Processes (NLP) methods such as topic modelling and textsummarization, a proposed solution is constructed in the form of a technical basiswhich can be implemented in most business areas. An experiment along with anevaluation session is setup in order to evaluate the performance of the technical basisand enforce the focus of this paper, namely ”How effective is text summarizationcombined with individualized information recommendation in improving informationgathering of IT experts?”. Furthermore, the solution includes a construction of userprofiles in an attempt to individualize content and theoretically present more relevantinformation. The results for this project are affected by the substandard quality andmagnitude of data points, however positive trends are discovered. It is stated thatthe use of user profiles further enhances the amount of relevant articles presentedby the model along with the increasing recall and precision values per iteration andaccuracy per number of updates made per user. Not enough time is spent as for theextent of the evaluation process to confidently state the validity of the results morethan them being inconsistent and insufficient in magnitude. However, the positivetrends discovered creates further speculations on if the project is given enough timeand resources to reach its full potential. Essentially, one can theoretically improveinformation gathering by summarizing texts combined with individualization.

Financial reports include a lot of useful information for investors, but extracting this information is time-consuming. We think text summarization is a feasible method. In this thesis, we implement BERTSUM, a state-of-the-art language model for text summarization, and evaluate the results by ROUGE metrics. The experiment was carried out on a novel and large-scale financial dataset called EDGAR-CORPUS. The BERTSUM with a transformer achieves the best performance with a ROUGE-L F1 score of 9.26%. We also hand-picked some model-generated summaries that contained common errors and investigated the causes. The results were then compared to previous research. The ROUGE-L F1 value in the previous study was much higher than ours, we think this is due to the length of the financial reports.

A growing number of scholarly papers is being accepted for conferences and workshops in recent years. Poster presentations are prevalent at many scientific gatherings because they allow attendees to connect with one another. A scientific poster is a coherent written and visual depiction of a paper’s major results. For researchers who want to generate a poster, tools for automatic development of scientific posters would be quite useful, allowing them to convey their findings in an elegant manner with minimal effort. Other scholars and participants can also take advantage of the opportunity to read the posters rather than sifting through the massive volume of scholarly publications and save time. The purpose of this thesis is to develop an online platform that allows authors of scientific papers to design their own posters using a poster template and the key concepts from the paper that will be provided. At the same time, significant sub tasks such as picture and text extraction, section matching, and text summarizing are investigated. Also, some text summarization models are created based on PubMed dataset in order to extract some useful information about the evaluation metrics and about which models have the best performance., Με το πέρασμα των χρόνων, ένας ολοένα και μεγαλύτερος αριθμός ακαδημαϊκών εργασιών γίνεται δεκτός για παρουσίαση σε συνέδρια και ημερίδες. Οι ανηρτημένες ανακοινώσεις είναι πολύ συχνές σε πολλές επιστημονικές εκδηλώσεις, καθώς λειτουργούν ως μέσο αλληλεπίδρασης μεταξύ των συμμετεχόντων. Η ανηρτημένη ανακοίνωση είναι μια κειμενική και οπτική αναπαράσταση μιας επιστημονικής εργασίας και περιέχει όλα τα βασικά στοιχεία της, παρουσιάζοντας τα με πιο κατανοητό τρόπο. Η ύπαρξη κάποιων εργαλείων για την αυτόματη δημιουργία ανηρτημένων ανακοινώσεων θα ήταν πολύτιμη για ερευνητές που επιθυμούν να δημιουργήσουν μία, βοηθώντας τους να παρουσιάσουν το έργο τους με κομψό τρόπο και λιγότερη προσπάθεια. Επίσης, άλλοι ερευνητές και γενικότερα οι συμμετέχοντες ενός συνέδριου έχουν την δυνατότητα να εκμεταλλευτούν την ευκαιρία να διαβάσουν τις ανηρτημένες ανακοινώσεις αντί να μελετήσουν τον τεράστιο όγκο ακαδημαϊκών εργασιών και να εξοικονομήσουν χρήσιμο χρόνο. Στόχος της παρούσας διπλωματικής εργασίας είναι να δημιουργήσει μια ηλεκτρονική πλατφόρμα που θα μπορεί να χρησιμοποιηθεί από τους συγγραφείς μιας επιστημονικής εργασίας προκειμένου να δημιουργήσουν τη δική τους ανηρτημένη ανακοίνωση, βασισμένη σε ένα συγκεκριμένο πρότυπο αφίσας και στις βασικές ιδέες που περιέχει η εργασία. Ταυτόχρονα, εξετάζονται κάποιες σημαντικές επιμέρους έννοιες και τεχνικές, όπως η εξαγωγή εικόνας και κειμένου, η αντιστοίχιση ενοτήτων και η σύνοψη κειμένου. Επίσης, δημιουργούνται κάποια μοντέλα, με σκοπό την σύνοψη κειμένου, βασισμένα σε ένα σύνολο δεδομένων, με σκοπό την εύρεση του μοντέλου, το οποίο έχει την καλύτερη απόδοση για την δημιουργία περίληψης.

Тема выпускной квалификационной работы: «Создание чатбота Telegram для автоматизации аналитики новостей». Данная работа относится к сфере обработки естественного языка. Задача автоматизации аналитики новостей была сформулирована как задача реферирования текстов. В данной работе был выполнено исследование методов реферирования текста и реализован чат-бот Telegram. В первой главе и второй главе рассматриваются существующие методы реферирования и реализации. В третьей главе описываются используемые в исследовании метрики и набор Ð´Ð°Ð½Ð½Ñ‹Ñ . В четвертой главе производится исследование и анализ алгоритмов, Ð²Ñ‹Ð±Ñ€Ð°Ð½Ð½Ñ‹Ñ Ñ€Ð°Ð½ÐµÐµ. В пятой главе описывается процесс разработки чат-бота и его развертывания., The subject of the graduate qualification work is «Creating a Telegram chatbot to automate news analytics». This work belongs to the sphere of natural language processing. The task of automating news analytics was formulated as the task of summarizing texts. In this work, a study of text summarization methods was carried out and the Telegram chat bot was implemented. In the first chapter and the second chapter, the existing methods 5 of summarization and implementation are considered. The third chapter describes the metrics used in the study and the data set. In the fourth chapter, research and analysis of the algorithms chosen earlier is carried out. The fifth chapter describes the process of developing a chatbot and its deployment.

We present a concept of system which is aimed to create a literature review of scientific articles having a small sketch of statements as the input. Key elements of the system include transformer-based BERT encoder, deep LSTM decoder and a loss function which combines auto-encoder loss and forces generated summaries to be in the input text domain. We propose to use PMC open access subset for model learning.

Text Summarization is a process which efficiently retrieves the relevant information from documents. The objective of the proposed, unsupervised approach is to summarize bug reports (software artefacts) with complete content and diversified information. The proposed approach utilizes Rapid Automatic Keyword Extraction and term frequency-inverse document frequency method to extract meaningful keywords and key-phrases with a relevant score. For sentence extraction, fuzzy C-means clustering is used to extracts sentences having high degree of membership from each cluster above a set threshold value. A rule-engine is used for sentence selection. The rules are generated with the domain knowledge and based on the extracted information by the keywords and sentences selected by the clustering method. Cohesive and coherent summary is generated by the proposed method on apache bug reports. For redundancy removal and to re-rank generated summary, hierarchical clustering is presented to enrich the extracted summary. The proposed approach is evaluated on newly constructed Apache project Bug Report Corpus (APBRC) and existing Bug Report Corpus (BRC). The results are compared on the basis of performance metrics such as precision, recall, pyramid precision and F-score. The experimental results depict that our proposed approach attains significant improvement over other baseline approaches such as BRC and LRCA. It also attains significant improvement over existing state-of-art unsupervised approaches such as Hurried, centroid and others. It extracts significant keyword phrases and sentences from each cluster to achieve full coverage and coherent summary. The results evaluated on APBRC corpus attains an average value of 78.22%, 82.18%, 80.10% and 81.66% for precision, recall, f-score and pyramid precision respectively.

A good document summary should summarize the core content of the text. Research on automatic text summarization attempts to solve this problem. The encoder-decoder model is widely used in text summarization research. Soft attention is used to obtain the required contextual semantic information during decoding. However, due to the lack of access to the key features, the generated summary deviates from the core content. In this paper, we proposed an encoder-decoder model based on a double attention pointer network (DAPT). In DAPT, the self-attention mechanism collects key information from the encoder, the soft attention and the pointer network generate more coherent core content, and the fusion of both generates accurate and coherent summaries. In addition, the improved coverage mechanism is used to address the repetition problem and improve the quality of the generated summaries. Simultaneously, scheduled sampling and reinforcement learning (RL) are combined to generate new training methods to optimize the model. Experiments on the CNN/Daily Mail dataset and the LCSTS dataset show that our model performs as well as many state-of-the-art models. The experimental analysis shows that our model achieves higher summarization performance and reduces the occurrence of repetition.

Містить огляд літератури, присвяченої методам абстрактного реферування текстів. Розглянуто класифікацію методів абстрактного реферування. Із появою методів реферування текстів у 1950-х рр. техніки створення рефератів постійно покращувались, але оскільки абстрактне реферування потребує потужних технік для оброблення/генерації тексту, найбільший прогрес досягнуто в останні роки. Поточний швидкий розвиток у сфері як оброблення природної мови в цілому, так і автоматичного реферування зокрема робить особливо необхідним аналіз прогресу в цій сфері. Надано загальне уявлення як про попередні підходи, так і найновіші, включаючи пояснення методів і підходів. Додатково подано кількісні оцінки методів, запропонованих в оглянутих джерелах., The paper contains a literature review for automatic abstractive text summarization. The classification of abstractive text summarization methods was considered. Since the emergence of text summarization in the 1950s, techniques for summaries generation were constantly improving, but because the abstractive summarization require extensive language processing, the greatest progress was achieved only recently. Due to the current fast pace of development of both Natural Language Processing in general and Text Summarization in particular, it is essential to analyze the progress in these areas. The paper aims to give a general perspective on both the state-of-the-art and older approaches, while explaining the methods and approaches. Additionally, evaluation results of the research papers are presented., Содержит обзор литературы, посвященной методам абстрактного реферирования текстов. Рассмотрена классификация методов абстрактного реферирования. С появлением методов реферирования текстов в 1950-х гг. техники создания рефератов постоянно улучшались, но поскольку абстрактное реферирование требует мощных техник для обработки/генерации текста, наибольший прогресс был достигнут в последние годы. Текущее быстрое развитие в сфере как обработки естественного языка в целом, так и автоматического реферирования в частности делает особенно необходимым анализ прогресса в этой сфере. Дано общее представление как про более предыдущие подходы, так и про новые, включая обьяснение методов и подходов.Дополнительно представлены оценки методов, предложенных в рассмотреных работах.

Recently, an emerging application field through Twitter messages and algorithmic computation to detect real-time world events has become a new paradigm in the field of data science applications. During a high-impact event, people may want to know the latest information about the development of the event because they want to better understand the situation and possible trends of the event for making decisions. However, often in emergencies, the government or enterprises are usually unable to notify people in time for early warning and avoiding risks. A sensible solution is to integrate real-time event monitoring and intelligence gathering functions into their decision support system. Such a system can provide real-time event summaries, which are updated whenever important new events are detected. Therefore, in this work, we combine a developed Twitter-based real-time event detection algorithm with pre-trained language models for summarizing emergent events. We used an online text-stream clustering algorithm and self-adaptive method developed to gather the Twitter data for detection of emerging events. Subsequently we used the Xsum data set with a pre-trained language model, namely T5 model, to train the summarization model. The Rouge metrics were used to compare the summary performance of various models. Subsequently, we started to use the trained model to summarize the incoming Twitter data set for experimentation. In particular, in this work, we provide a real-world case study, namely the COVID-19 pandemic event, to verify the applicability of the proposed method. Finally, we conducted a survey on the example resulting summaries with human judges for quality assessment of generated summaries. From the case study and experimental results, we have demonstrated that our summarization method provides users with a feasible method to quickly understand the updates in the specific event intelligence based on the real-time summary of the event story.

Corpus of Academic Slovene (KAS) contains Slovene BSc/BA, MSc/MA, and PhD theses from 2000 - 2018. We present a cleaner version of the corpus with added text segmentation and updated POS-tagging. The updated corpus of abstracts contains fewer artefacts. Using machine learning classifiers, we filled in miss- ing research field information in the metadata. We used the full texts and corresponding abstracts to create several new datasets: monolingual and cross-lingual datasets for long text summariza- tion of academic texts and a dataset of aligned sentences from abstracts in English and Slovene, suitable for machine transla- tion. We release the corpora, datasets, and developed source code under a permissible licence.

With the increase in number of e-commerce sites, one finds it difficult to choose and buy a product. It is not possible for a person to read hundreds of product reviews from various sources. This problem can be solved by using text summarization. The issue with most text summarizers is that, they summarize the text but they do not tend to preserve the underlying meaning. The aim of this paper is to overcome this problem by developing a abstractive multi document text summarizer which summarizes reviews that are obtained from multiple sources like amazon, trip advisor, etc. and are stored in the form of multiple documents. The implementation is carried out by first cleaning the data and using a graph based approach that considers tagging parts of speech and building edges with weights and then using TF-IDF to find the most important set of words. The final summary is obtained by using maximum weight graph traversal. Evaluation metric is ROUGE.

Text summarization remains a challenging task in the natural language processing field despite the plethora of applications in enterprises and daily life. One of the common use cases is the summarization of web pages which has the potential to provide an overview of web pages to devices with limited features. In fact, despite the increasing penetration rate of mobile devices in rural areas, the bulk of those devices offer limited features in addition to the fact that these areas are covered with limited connectivity such as the GSM network. Summarizing web pages into SMS becomes, therefore, an important task to provide information to limited devices. This work introduces WATS-SMS, a T5-based French Wikipedia Abstractive Text Summarizer for SMS. It is built through a transfer learning approach. The T5 English pre-trained model is used to generate a French text summarization model by retraining the model on 25,000 Wikipedia pages then compared with different approaches in the literature. The objective is twofold: (1) to check the assumption made in the literature that abstractive models provide better results compared to extractive ones, and (2) to evaluate the performance of our model compared to other existing abstractive models. A score based on ROUGE metrics gave us a value of 52% for articles with length up to 500 characters against 34.2% for transformer-ED and 12.7% for seq-2seq-attention, and a value of 77% for articles with larger size against 37% for transformers-DMCA. Moreover, an architecture including a software SMS-gateway has been developed to allow owners of mobile devices with limited features to send requests and to receive summaries through the GSM network.

U području automatiziranog sažimanja teksta razlikujemo dvije vrste pristupa sažimanju: ekstraktivno i apstraktivno. Arhitektura pokazivač-generator predstavlja kombinaciju tih pristupa jer neuronskoj mreži omogućuje odabir između preslikavanja izvornog ili stvaranja novog teksta. U ovom radu je predstavljen duboki model čiju osnovu čini koder-dekoder arhitektura. Na osnovni model dodani su i mehanizmi intra-temporalne i intra-dekoderske pažnje te mehanizam pokazivač-generator koji umanjuju nedostatke dosadašnjih apstraktivnih modela za sažimanje teksta. Kvalitativna analiza mjerom ROUGE pokazala je da ovaj model postiže znatno bolje rezultate od nekih apstraktivnih modela ispitanih na skupu podataka Gigaword. Izgrađeni je model izložen krajnjim korisnicima u obliku web primjenskog sustava. The pointer-generator architecture combines abstractive and extractive approaches to text summarization as it allows the neural network to choose between copying from the source or generating new words. In this thesis, a deep model based on an encoder-decoder architecture is presented. To reduce the shortcomings of the previous abstractive models, the mechanisms of intra-temporal and intra-decoder attention and the pointer-generator mechanism have been added to the baseline model. Qualitative analysis using the ROUGE measure showed that this model achieves considerably better results than some of the comparable models. The trained model is exposed to end users as a web application.

U ovom radu dan je pregled nad područjem obrade prirodnog jezika kroz opis odabranih modela dubokog učenja korištenih kroz povijest. Opisani su postojeći pristupi za semantičko sažimanja teksta, a naglasak je stavljen na ekstraktivno sažimanje. Kao praktični ishod rada, ostvaren je web sustav za ekstraktivno sažimanje teksta. Dan je opis arhitekture i programskog ostvarenja sustava. Sustav se zasniva na modelu BERT i algoritmu grupiranja k-sredina. Ekstraktivni sažetak se stvara slijedom operacija. Ulazni tekst se prvo dijeli na rečenice. Pomoću modela BERT se računaju vektorski prikazi rečenica. Vektorski prikazi rečenica se grupiraju algoritmom k-sredina, a zatim se odabiru rečenice koje ulaze u sažetak. Ostvareni sustav vrednovan je mjerama ROUGE na skupovima podataka CNN/Daily Mail i Newsroom. This paper provides an overview of the natural language processing field through a description of selected deep learning models used throughout history. Existing approaches to semantic text summarization are described, with an emphasis on extractive text summarization. As a practical outcome of the paper, a web system for extractive text summarization is realized. A description of the architecture and software implementation of the system is given. The system is based on the BERT model and the k-means clustering algorithm. An extractive summary is created using a sequence of operations. The input text is first divided into sentences. Using the BERT model, vector representations of sentences are calculated. Vector representations of sentences are clustered using the k-means algorithm, followed by the selection of the sentences that enter the summary. The realized system is evaluated using the ROUGE metric on CNN/Daily Mail and Newsroom datasets.

Automatsko sažimanje teksta je područje obrade prirodnog jezika i ima sve veću primjenu i korisnost. Cilj automatskog sažimanja teksta je stvaranje kratke ljudski čitljive verzije izvornog teksta uz izvlačenje i očuvanje najbitnijih informacija. S obzirom na način stvaranja, sažetak može biti ekstraktivan i apstraktivan. Ovaj diplomski rad se prvenstveno bavi algoritmima dubokog učenja koji se koriste za stvaranje apstraktivnih sažetaka. Navedeni su i opisani osnovni načini obrade prirodnog jezika uz usmjeravanje na načine stvaranja sažetaka. U nastavku je detaljno opisan konkretan duboki model za stvaranje apstraktivnog sažetka koji se zasniva na mehanizmu pažnje. Opisani model je programski ostvaren i ugrađen u web primjenski sustav za stvaranje sažetaka ulaznih rečenica. Model je također vrednovan na javno dostupnim skupovima podataka koristeći prikladne mjere te su prikazani dobiveni rezultati. Text summarization is part of the natural language processing field which has wide application and usability. The main goal of text summarization is making shorter human-readable version of the original text while extracting and preserving key information. Given the method of creating, summary can be extractive or abstractive. This thesis is focusing on the deep learning algorithms which are used to make abstractive summaries. The main methods for natural language processing are listed and described while focusing on the methods for text summarization. In continuation, the attention-based deep learning model for generating abstractive summaries is described and implemented. Implemented model is embedded in web application for generating summaries of the input sentences. The model is also tested using online available datasets and appropriate metrics and obtained results are presented.

In the era of internet, the ability to quickly extract useful information out of big amounts of data has become an important capability. This includes text summarization, a Natural Language Processing task of compressing a given text into a shorter one in such a way that it is consistent with the original text, concise, correct and as informative as possible. The leading solutions of this problem use various Deep Neural Networks. Designing an optimal DNN's architecture is a difficult task requiring a lot of expertise, time and work. In this work I attempt to facilitate this process using coevolution of neural networks. I use Pytorch-dnnevo framework to find networks capable of solving NLP tasks, including text summarization using coevolution. I implement architectures based on RNN, LSTM and Seq2seq with attention mechanism. Metrics like ROUGE-N, BLEU and F1 as well as datasets like IMDb Movie Reviews and Amazon Fine Food Reviews are used. The results show that, given suitable layer types, coevolution is capable of constructing networks that can solve NLP tasks. It can help engineers find the optimal architecture and hyperparameters for a given dataset.

Despite the availability of online educational resources about human papillomavirus (HPV), many women around the world may be prevented from obtaining the necessary knowledge about HPV. One way to mitigate the lack of HPV knowledge is the use of auto-generated text summarization tools. This study compares the level of HPV knowledge between women who read an auto-generated summary of HPV made using the BERT deep learning model and women who read a long-form text of HPV. We randomly assigned 386 women to two conditions: half read an auto-generated summary text about HPV (n = 193) and half read an original text about HPV (n = 193). We administrated measures of HPV knowledge that consisted of 29 questions. As a result, women who read the original text were more likely to correctly answer two questions on the general HPV knowledge subscale than women who read the summarized text. For the HPV testing knowledge subscale, there was a statistically significant difference in favor of women who read the original text for only one question. The final subscale, HPV vaccination knowledge questions, did not significantly differ across groups. Using BERT for text summarization has shown promising effectiveness in increasing women’s knowledge and awareness about HPV while saving their time.

The corpus contains 136,996 (argumentative text, conclusion) pairs. There are four variations of thiscorpus: base:(argumentative text, conclusion) pairs topic: (topic encoded argumentative text, conclusion) pairs targets:(topic and target encoded argumentative text, conclusion) pairs aspects:(topic and aspectencoded argumentative text, conclusion) pairs Each variation is split into train, val, and test files. Additionally, the test set used for automatic evaluation is provided (automatic_evaluation.csv) which must be used for any quantitative evaluation.&nbsp

We present an extractive summarization model based on the Bert and dynamic memory network. The model based on Bert uses the transformer to extract text features and uses the pre-trained model to construct the sentence embeddings. The model based on Bert labels the sentences automatically without using any hand-crafted features and the datasets are symmetry labeled. We also present a dynamic memory network method for extractive summarization. Experiments are conducted on several summarization benchmark datasets. Our model shows comparable performance compared with other extractive summarization methods.

Automatic text summarization attempts to provide an effective solution to today’s unprecedented growth of textual data. This paper proposes an innovative graph-based text summarization framework for generic single and multi document summarization. The summarizer benefits from two well-established text semantic representation techniques; Semantic Role Labelling (SRL) and Explicit Semantic Analysis (ESA) as well as the constantly evolving collective human knowledge in Wikipedia. The SRL is used to achieve sentence semantic parsing whose word tokens are represented as a vector of weighted Wikipedia concepts using ESA method. The essence of the developed framework is to construct a unique concept graph representation underpinned by semantic role-based multi-node (under sentence level) vertices for summarization. We have empirically evaluated the summarization system using the standard publicly available dataset from Document Understanding Conference 2002 (DUC 2002). Experimental results indicate that the proposed summarizer outperforms all state-of-the-art related comparators in the single document summarization based on the ROUGE-1 and ROUGE-2 measures, while also ranking second in the ROUGE-1 and ROUGE-SU4 scores for the multi-document summarization. On the other hand, the testing also demonstrates the scalability of the system, i.e., varying the evaluation data size is shown to have little impact on the summarizer performance, particularly for the single document summarization task. In a nutshell, the findings demonstrate the power of the role-based and vectorial semantic representation when combined with the crowd-sourced knowledge base in Wikipedia.

In order to use lexical features (including n-gram and part of speech information) to identify more key vocabulary content in the summarization generation process to further improve the quality of the summarization, an algorithm based on sequence-to-sequence (Seq2Seq) structure and attention mechanism and combining lexical features is proposed. The input layer of the algorithm combines the part of speech vector with the word vector, which is the input of the encoder layer. The encoder layer is composed of bi-directional LSTM, and the context vector is composed of the output of the encoder and the lexical feature vector extracted from the convolution neural network. The convolutional neural network layer in the model controls the lexical information, the bi-directional LSTM controls the sentence information, and the decoder layer uses unidirectional LSTM to decode the context vector and generates the summarization. The experiments on public dataset and the self-collected dataset show that the performance of the summarization generation model considering lexical feature is better than that of the contrast model. The ROUGE-1, ROUGE-2 and ROUGE-L scores on the public dataset are improved by 0.024, 0.033 and 0.030, respectively. Therefore, the generation of summarization is not only related to the semantics and themes of the article, but also to the lexical features.The proposed model provides a certain reference value in the research of generating summarization of integrating key infromation.

In recent times, due to the excessiveness of research paper in the web, there is a need of automatic document summarization for papers. And, text summarization is depend on keyword extraction. In this paper, we propose keyword extraction method combining linguistics and graph theory proposed in related work. By inputting the co-occurrence, position, the term frequency, and inverse document frequency of word, this method provide words feature score.