Search

The Customer Success department at Visma handles about 200 000 customer chats each year, the chat dialogues are stored and contain both questions and answers. In order to get an idea of what customers ask about, the Customer Success department has to read a random sample of the chat dialogues manually. This thesis develops and investigates an analysis tool for the chat data, using the approach of clustering and summarization. The approach aims to decrease the time spent and increase the quality of the analysis. Models for clustering (K-means, DBSCAN and HDBSCAN) and extractive summarization (K-means, LSA and TextRank) are compared. Each algorithm is combined with three different text representations (TFIDF, S-BERT and FastText) to create models for evaluation. These models are evaluated against a test set, created for the purpose of this thesis. Silhouette Index and Adjusted Rand Index are used to evaluate the clustering models. ROUGE measure together with a qualitative evaluation are used to evaluate the extractive summarization models. In addition to this, the best clustering model is further evaluated to understand how different data sizes impact performance. TFIDF Unigram together with HDBSCAN or K-means obtained the best results for clustering, whereas FastText together with TextRank obtained the best results for extractive summarization. This thesis applies known models on a textual domain of customer chat dialogues, something that, to our knowledge, has previously not been done in literature.

MY REACH es una startup nacida en 2018 la cual se dedica a establecer un servicio de almacenamiento centralizado de los diferentes archivos y datos de los usuarios. El principal objetivo que se plantea en REACH es el hecho de poder reducir de manera drástica el tiempo de búsqueda de los datos de cada usuario. El elemento diferenciador que supone REACH delante de las plataformas que estan actualmente en el mercado es el hecho de poder guardar los diferentes archivos y datos del usuario (actualmente páginas web, documentos y categorías) en un sistema de grafo. Esta estructura proporciona al usuario un contexto más entendedor en sus datos que la estructura tradicional de carpetas. Con este potencial proporcionara al usuario poder crear relaciones entre los diferentes elementos haciendo así de sus datos un conjunto con más significado y siendo mucho más compresible. El sistema de recomendaciones se posiciona como uno de los elementos más importantes a tener en cuenta en esta plataforma, ya que el usuario cuando tiene una zona de trabajo con mucha información, un buen sistema de recomendaciones le permite conectar su grafo de manera sencilla creando relaciones con los otros datos y archivos. De esta manera, el usuario podrá utilizar de manera más eficiente la búsqueda y las ventajas que ofrece la plataforma. El objetivo de este proyecto es mejorar el actual sistema de recomendaciones implementado actualmente en la plataforma, ya que la implementación actual recomienda enlaces con otros archivos y datos que no son parecidos. La propuesta que se plantea en este proyecto es la mejora de la extracción de palabras claves de un archivo o dato del usuario y, además, la mejora en las recomendaciones dadas entre datos. En un futuro, también se plantea implementar la creación de relaciones automáticas. Después de un estado previo del estado del arte de este tipo de aplicaciones y de las diferentes técnicas que puedan ser utilizadas para llevar a cabo el trabajo. El procedimiento p, MY REACH is a startup born in 2018 which dedicates to establish a centralized storage service with files and data from different users. The main aim of REACH is to improve highly the time of a user's search. Currently, the most notorious difference between REACH and the rest of the companies, that exist on the market, is the way that how files and data of users (currently web pages, documents and subjects) are saved on a graph system. This structure brings to the user a better and more understable context of his data than on the traditional file system. With this potential tool will give to the user the possibility to make his data a more comprehensive set. The recommendation system is positioned as one of the most important elements to take into account in this platform, because when the user has a workspace with a lot of information, a good recommendation system allows him to connect his network in a simple way by creating relationships with other data and files. In this way, the user will be able to make a more efficient use of the search and the advantages offered by the software. The goal of this project is to improve the current recommender system which is currently implemented in the software, as the current implementation recommends links to other files and data that are not similar. The proposal put forward in this project is to improve the extraction of keywords/keyphrases from a file or user data and, in addition, to improve the links given among data. In the future, it is also proposed to implement the creation of automatic relations. After a previous state of the art of this type of applications and the different techniques that can be used to carry out the work. The procedure proposed in this project is based on the creation of two microservices. The first one is in charge of extracting keyphrases from a file or user data using natural language processing. The second microservice is in charge of recommending relationships also using natural language pr

The Customer Success department at Visma handles about 200 000 customer chats each year, the chat dialogues are stored and contain both questions and answers. In order to get an idea of what customers ask about, the Customer Success department has to read a random sample of the chat dialogues manually. This thesis develops and investigates an analysis tool for the chat data, using the approach of clustering and summarization. The approach aims to decrease the time spent and increase the quality of the analysis. Models for clustering (K-means, DBSCAN and HDBSCAN) and extractive summarization (K-means, LSA and TextRank) are compared. Each algorithm is combined with three different text representations (TFIDF, S-BERT and FastText) to create models for evaluation. These models are evaluated against a test set, created for the purpose of this thesis. Silhouette Index and Adjusted Rand Index are used to evaluate the clustering models. ROUGE measure together with a qualitative evaluation are used to evaluate the extractive summarization models. In addition to this, the best clustering model is further evaluated to understand how different data sizes impact performance. TFIDF Unigram together with HDBSCAN or K-means obtained the best results for clustering, whereas FastText together with TextRank obtained the best results for extractive summarization. This thesis applies known models on a textual domain of customer chat dialogues, something that, to our knowledge, has previously not been done in literature.

Магістерська дисертація: 96 с., 10 рис., 11 табл., 5 додатків, 38 джерел. Актуальність роботи заключається в тому, що кількість застосунків, що вирішують задачу автоматичного реферування текстів, написаних українською чи російською невелика, та результати їх роботи не можна назвати дуже ефективними з точки зору математичних показників лексичної близькості текстів. Але задача реферування текстів є досить важливою зважаючи на кількість потоків текстових даних, які стають більшими кожен день і для багатьох професій – від законотворців, журналістів та, навіть, до військових критично ці потоки ефективно обробляти використовуючи менше часу. Аналогічно, реферування є важливим і для машинної обробки текстів – це значно пришвидшує обмін даними та має дуже багато потенційних аплікацій. Мета дослідження є синтез нового математичного та програмного рішення для автоматичного реферування текстів, написаних українською та російською мовами. Для реалізації поставленої мети були сформульовані наступні завдання: - Аналіз існуючого теоретичного апарату реферування текстів - Огляд існуючого програмного забезпечення - Розробка нового застосунку та моделі для реферування текстів українською та російською мовами - Виконати аналіз отриманих результатів Об’єкт дослідження: є математичні моделі автоматичного реферування тексту. Предметом дослідження є розробка застосунків, що будуть використовувати існуючі методи або нові методи автоматичного реферування текстів, написаних українською та російською мовами. Методи досліджень є методи екстрактивного реферування текстів, що базуються на векторному представленні тексту Наукова новизна: Найбільш суттєвими науковими результатами магістерської дисертації є розробка власного алгоритму реферування текстів та його імплементація для вирішення задачі реферування текстів, написаних українською та російською мовами. Практичне значення отриманих результатів визначається тим, що запропонований алгоритм та розроблений програмний застосунок у результаті експе, Master dissertation: 96 pp., 10 figures, 11 tables, 5 applications, 38 sources. Topicality is concluded with that the number of applications that solve the problem of automatic abstract generation for the texts written in Ukrainian or Russian is extremely small, and the results of their work can not be called very effective in terms of mathematical indicators of the lexical proximity of texts. But the task of text summarization is quite important, especially given the number of textual data streams that are growing every day and for many professions - from lawmakers, journalists, and even the military – it’s critical to process these streams faster. Similarly, abstracting is important for word processing - it greatly speeds up data exchange and has many potential applications. The aim of the research is synthesis of a new mathematical and software solution for automatic abstracting of texts written in Ukrainian and Russian. To achieve this goal which were formulated following tasks: - Existing theoretical methods of text summarization analysis - Review of existing text summarization software - Development of a new application and mathematical model for Russian and Ukrainian texts summarization - Perform the analysis of the obtained results The object of research is the mathematical models for automatic text summarization The subject of research is the development of applications that will use existing methods or new methods of automatic text summarization of texts written in Ukrainian and Russian. Research methods are the extraction methods of the text summarization based on the vector representation of the text Scientific novelty of the obtained results. The most significant scientific results of the master's thesis are the development of its own algorithm for abstracting texts and its implementation to solve the problem of abstracting texts written in Ukrainian and Russian. Practical consequences of the results are determined by the fact that the proposed algorith

Publicly available data grows exponentially through web services and technological advancements. To comprehend large data-streams multi-document summarization (MDS) can be used. In this research, the area of multi-document summarization is investigated. Multiple systems for extractive multi-document summarization are implemented using modern techniques, in the form of the pre-trained BERT language model for word embeddings and sentence classification. This is combined with well proven techniques, in the form of the TextRank ranking algorithm, the Waterfall architecture and anti-redundancy filtering. The systems are evaluated on the DUC-2002, 2006 and 2007 datasets using the ROUGE metric. Where the results show that the BM25 sentence representation implemented in the TextRank model using the Waterfall architecture and an anti-redundancy technique outperforms the other implementations, providing competitive results with other state-of-the-art systems. A cohesive model is derived from the leading system and tried in a user study using a real-world application. The user study is conducted using a real-time news detection application with users from the news-domain. The study shows a clear favour for cohesive summaries in the case of extractive multi-document summarization. Where the cohesive summary is preferred in the majority of cases.

Publicly available data grows exponentially through web services and technological advancements. To comprehend large data-streams multi-document summarization (MDS) can be used. In this research, the area of multi-document summarization is investigated. Multiple systems for extractive multi-document summarization are implemented using modern techniques, in the form of the pre-trained BERT language model for word embeddings and sentence classification. This is combined with well proven techniques, in the form of the TextRank ranking algorithm, the Waterfall architecture and anti-redundancy filtering. The systems are evaluated on the DUC-2002, 2006 and 2007 datasets using the ROUGE metric. Where the results show that the BM25 sentence representation implemented in the TextRank model using the Waterfall architecture and an anti-redundancy technique outperforms the other implementations, providing competitive results with other state-of-the-art systems. A cohesive model is derived from the leading system and tried in a user study using a real-world application. The user study is conducted using a real-time news detection application with users from the news-domain. The study shows a clear favour for cohesive summaries in the case of extractive multi-document summarization. Where the cohesive summary is preferred in the majority of cases.

Automatic text summarization, a process of distilling the most important information from a text document, is to create an abridged summary with software. Basically, in this task, we can regard the "summarization" as a function which takes a single document or multiple documents as an input and has the summary as an output. There are two ways that we can manage to create a summary: extractive and abstractive. The extractive summarization means that we select the most relevant sentences from the input and concatenate them to form a summary. Graph-based algorithm like TextRank, Feature-based models like TextTeaser, Topic-based models like Latent Semantic Analysis (LSA), and Grammar-based models could be viewed as approaches to extractive summarization. Abstractive summarization aims to create a summary similar to humans. It keeps the original intent, but uses new phrases and words not found in the original text. One of the most commonly used models is the encoder-decoder model, a neural network model that is mainly used in machine translation tasks. Recently, there is another combination approach that combines both extractive and abstractive summarization, like Pointer-Generator Network, and the Extract then Abstract model. In this course, we're given both a small dataset (about 500 documents) and a big dataset (about 11,300 documents) that mainly consist of web archives about a specific event. Our group is focusing on reports about a terrorist event -- Attack Westminster. It occurred outside the Palace of Westminster in London on March 22, 2017. The attacker, 52 year-old Briton Khalid Masood, drove a car into pedestrians on the pavement, injuring more than 50 people, 5 of them fatally. The attack was treated as "Islamist-related terrorism". We first created a Solr index for both the small dataset and the big dataset, which helped us to perform various queries to know more about the data. Additionally, the index aided another team to create a gold standard summary of

Natural language processing (NLP) has advanced in recent years. Accordingly, we present progressively more complex generated text summaries on the topic Hurricane Harvey. We utilized TextRank, which is an unsupervised extractive summarization algorithm. TextRank is computationally expensive, and the sentences generated by the algorithm aren’t always directly related or essential to the topic at hand. When evaluating TextRank, we found that a single sentence interjected and ruined the flow of the summary. We also found that ROUGE evaluation for our TextRank summary was quite low compared to a golden standard that was prepared for us. However, the TextRank summary had high marks for ROUGE evaluation compared to the Wikipedia article lead for Hurricane Harvey. To improve upon the TextRank algorithm, we utilized template summarization with named entities. Template summarization takes less time to run than TextRank but is supervised by the author of the template and script to choose valuable named entities. Thus, it is highly dependent on human intervention to produce reasonable and readable summaries that aren’t error-prone. As expected, the template summary evaluated well compared to the Gold Standard and the Wikipedia article lead. This result is mainly due to our ability to include named entities we thought were pertinent to the summary. Beyond extractive summaries like TextRank and template summarization, we pursued abstractive summarization using pointer-generator networks and multi-document summarization with pointer-generator networks and maximal marginal relevance. The benefit of using abstractive summarization is that it is more in-line with how humans summarize documents. Pointer-generator networks, however, require GPUs to run properly and a large amount of training data. Luckily, we were able to use a pre-trained network to generate summaries. The pointer-generator network is the centerpiece of our abstractive methods and allowed us to create summaries in th

This master thesis deals with automatic summarization of text and how semantic folding can be used as a similarity measure between sentences in the TextRank algorithm. The method was implemented and compared with two common similarity measures. These two similarity measures were cosine similarity of tf-idf vectors and the number of overlapping terms in two sentences. The three methods were implemented and the linguistic features used in the construction were stop words, part-of-speech filtering and stemming. Five different part-of-speech filters were used, with different mixtures of nouns, verbs, and adjectives. The three methods were evaluated by summarizing documents from the Document Understanding Conference and comparing them to gold-standard summarization created by human judges. Comparison between the system summaries and gold-standard summaries was made with the ROUGE-1 measure. The algorithm with semantic folding performed worst of the three methods, but only 0.0096 worse in F-score than cosine similarity of tf-idf vectors that performed best. For semantic folding, the average precision was 46.2% and recall 45.7% for the best-performing part-of-speech filter., Det här examensarbetet behandlar automatisk textsammanfattning och hur semantisk vikning kan användas som likhetsmått mellan meningar i algoritmen TextRank. Metoden implementerades och jämfördes med två vanliga likhetsmått. Dessa två likhetsmått var cosinus-likhet mellan tf-idf-vektorer samt antal överlappande termer i två meningar. De tre metoderna implementerades och de lingvistiska särdragen som användes vid konstruktionen var stoppord, filtrering av ordklasser samt en avstämmare. Fem olika filter för ordklasser användes, med olika blandningar av substantiv, verb och adjektiv. De tre metoderna utvärderades genom att sammanfatta dokument från DUC och jämföra dessa mot guldsammanfattningar skapade av mänskliga domare. Jämförelse mellan systemsammanfattningar och guldsammanfattningar gjordes med måttet ROUGE-1. Algoritmen med semantisk vikning presterade sämst av de tre jämförda metoderna, dock bara 0.0096 sämre i F-score än cosinus-likhet mellan tf-idf-vektorer som presterade bäst. För semantisk vikning var den genomsnittliga precisionen 46.2% och recall 45.7% för det ordklassfiltret som presterade bäst.

This master thesis deals with automatic summarization of text and how semantic folding can be used as a similarity measure between sentences in the TextRank algorithm. The method was implemented and compared with two common similarity measures. These two similarity measures were cosine similarity of tf-idf vectors and the number of overlapping terms in two sentences. The three methods were implemented and the linguistic features used in the construction were stop words, part-of-speech filtering and stemming. Five different part-of-speech filters were used, with different mixtures of nouns, verbs, and adjectives. The three methods were evaluated by summarizing documents from the Document Understanding Conference and comparing them to gold-standard summarization created by human judges. Comparison between the system summaries and gold-standard summaries was made with the ROUGE-1 measure. The algorithm with semantic folding performed worst of the three methods, but only 0.0096 worse in F-score than cosine similarity of tf-idf vectors that performed best. For semantic folding, the average precision was 46.2% and recall 45.7% for the best-performing part-of-speech filter., Det här examensarbetet behandlar automatisk textsammanfattning och hur semantisk vikning kan användas som likhetsmått mellan meningar i algoritmen TextRank. Metoden implementerades och jämfördes med två vanliga likhetsmått. Dessa två likhetsmått var cosinus-likhet mellan tf-idf-vektorer samt antal överlappande termer i två meningar. De tre metoderna implementerades och de lingvistiska särdragen som användes vid konstruktionen var stoppord, filtrering av ordklasser samt en avstämmare. Fem olika filter för ordklasser användes, med olika blandningar av substantiv, verb och adjektiv. De tre metoderna utvärderades genom att sammanfatta dokument från DUC och jämföra dessa mot guldsammanfattningar skapade av mänskliga domare. Jämförelse mellan systemsammanfattningar och guldsammanfattningar gjordes med måttet ROUGE-1. Algoritmen med semantisk vikning presterade sämst av de tre jämförda metoderna, dock bara 0.0096 sämre i F-score än cosinus-likhet mellan tf-idf-vektorer som presterade bäst. För semantisk vikning var den genomsnittliga precisionen 46.2% och recall 45.7% för det ordklassfiltret som presterade bäst.

V této práci jsem se zaměřil na implementaci webové aplikace, která slouží jako prostředek pro automatickou tvorbu souhrnů v anglickém jazyce. Automatická tvorba souhrnů je v řešení prováděna pomocí metody TextRank a Latentní sémantické analýzy. Obě tyto metody jsou vylepšeny o rozpoznávání pojmenovaných entit. Přínosem této práce je zjištění, že využití rozpoznávání pojmenovaných entit u Latentní sémantické analýzy a především u metody TextRank, vede k vytváření kvalitnějších souhrnů. Tato kvalita souhrnů byla ověřena pomocí metrik ROUGE., This work is focused on an implementation a web application, which is a tool for automatic English text summarization. In result, automatic text summarization is made by TextRank and Latent semantic analysis method. Both of these methods are improved by named entity recognition. The main benefit of this work is proving that using the named entity recognition with Latent semantic analysis and especially with TextRank method leads to creation of higher quality summaries. This quality of the summaries was verified by ROUGE metrics.

V této práci jsem se zaměřil na implementaci webové aplikace, která slouží jako prostředek pro automatickou tvorbu souhrnů v anglickém jazyce. Automatická tvorba souhrnů je v řešení prováděna pomocí metody TextRank a Latentní sémantické analýzy. Obě tyto metody jsou vylepšeny o rozpoznávání pojmenovaných entit. Přínosem této práce je zjištění, že využití rozpoznávání pojmenovaných entit u Latentní sémantické analýzy a především u metody TextRank, vede k vytváření kvalitnějších souhrnů. Tato kvalita souhrnů byla ověřena pomocí metrik ROUGE., This work is focused on an implementation a web application, which is a tool for automatic English text summarization. In result, automatic text summarization is made by TextRank and Latent semantic analysis method. Both of these methods are improved by named entity recognition. The main benefit of this work is proving that using the named entity recognition with Latent semantic analysis and especially with TextRank method leads to creation of higher quality summaries. This quality of the summaries was verified by ROUGE metrics.

Tato bakalářská práce se zabývá prozkoumáním aktuální metodiky a možností textového dolování a následné aplikace některých metod. V rámci práce byly popsány metody pro předzpracování, metody pro převedení textu do vektorového prostoru a metody pro analýzu textu a diskutováno jejich možné použití. Na text byly použity jednotlivé metody pro předzpracování a následně bylo demonstrováno převedení do vektorového prostoru jednoduchými metodami jako jsou BOW, Bag of n-grams, TF-IDF nebo metodami se strojovým učením které jsou FastText a GloVe. Na získané vektory byly použity metody LSA, LDA, TextRank, kosinová podobnost, pro získání informací z textu., This bachelor thesis explores the current methodology and possibilities of text mining and the subsequent application of some methods. The thesis described methods for preprocessing, methods for converting text to vector space and methods for text analysis and discusses their possible applications. The different preprocessing methods were applied to the text and then the conversion to vector space was demonstrated using simple methods such as BOW, Bag of n-grams, TF-IDF or with machine learning methods which are FastText and GloVe. LSA, LDA, TextRank and cosine similarity methods were applied to the extracted vectors to extract information from the text.

Tato bakalářská práce se zabývá prozkoumáním aktuální metodiky a možností textového dolování a následné aplikace některých metod. V rámci práce byly popsány metody pro předzpracování, metody pro převedení textu do vektorového prostoru a metody pro analýzu textu a diskutováno jejich možné použití. Na text byly použity jednotlivé metody pro předzpracování a následně bylo demonstrováno převedení do vektorového prostoru jednoduchými metodami jako jsou BOW, Bag of n-grams, TF-IDF nebo metodami se strojovým učením které jsou FastText a GloVe. Na získané vektory byly použity metody LSA, LDA, TextRank, kosinová podobnost, pro získání informací z textu., This bachelor thesis explores the current methodology and possibilities of text mining and the subsequent application of some methods. The thesis described methods for preprocessing, methods for converting text to vector space and methods for text analysis and discusses their possible applications. The different preprocessing methods were applied to the text and then the conversion to vector space was demonstrated using simple methods such as BOW, Bag of n-grams, TF-IDF or with machine learning methods which are FastText and GloVe. LSA, LDA, TextRank and cosine similarity methods were applied to the extracted vectors to extract information from the text.

Tato bakalářská práce se zabývá prozkoumáním aktuální metodiky a možností textového dolování a následné aplikace některých metod. V rámci práce byly popsány metody pro předzpracování, metody pro převedení textu do vektorového prostoru a metody pro analýzu textu a diskutováno jejich možné použití. Na text byly použity jednotlivé metody pro předzpracování a následně bylo demonstrováno převedení do vektorového prostoru jednoduchými metodami jako jsou BOW, Bag of n-grams, TF-IDF nebo metodami se strojovým učením které jsou FastText a GloVe. Na získané vektory byly použity metody LSA, LDA, TextRank, kosinová podobnost, pro získání informací z textu., This bachelor thesis explores the current methodology and possibilities of text mining and the subsequent application of some methods. The thesis described methods for preprocessing, methods for converting text to vector space and methods for text analysis and discusses their possible applications. The different preprocessing methods were applied to the text and then the conversion to vector space was demonstrated using simple methods such as BOW, Bag of n-grams, TF-IDF or with machine learning methods which are FastText and GloVe. LSA, LDA, TextRank and cosine similarity methods were applied to the extracted vectors to extract information from the text.

V této práci jsem se zaměřil na implementaci webové aplikace, která slouží jako prostředek pro automatickou tvorbu souhrnů v anglickém jazyce. Automatická tvorba souhrnů je v řešení prováděna pomocí metody TextRank a Latentní sémantické analýzy. Obě tyto metody jsou vylepšeny o rozpoznávání pojmenovaných entit. Přínosem této práce je zjištění, že využití rozpoznávání pojmenovaných entit u Latentní sémantické analýzy a především u metody TextRank, vede k vytváření kvalitnějších souhrnů. Tato kvalita souhrnů byla ověřena pomocí metrik ROUGE., This work is focused on an implementation a web application, which is a tool for automatic English text summarization. In result, automatic text summarization is made by TextRank and Latent semantic analysis method. Both of these methods are improved by named entity recognition. The main benefit of this work is proving that using the named entity recognition with Latent semantic analysis and especially with TextRank method leads to creation of higher quality summaries. This quality of the summaries was verified by ROUGE metrics.

V této práci jsem se zaměřil na implementaci webové aplikace, která slouží jako prostředek pro automatickou tvorbu souhrnů v anglickém jazyce. Automatická tvorba souhrnů je v řešení prováděna pomocí metody TextRank a Latentní sémantické analýzy. Obě tyto metody jsou vylepšeny o rozpoznávání pojmenovaných entit. Přínosem této práce je zjištění, že využití rozpoznávání pojmenovaných entit u Latentní sémantické analýzy a především u metody TextRank, vede k vytváření kvalitnějších souhrnů. Tato kvalita souhrnů byla ověřena pomocí metrik ROUGE., This work is focused on an implementation a web application, which is a tool for automatic English text summarization. In result, automatic text summarization is made by TextRank and Latent semantic analysis method. Both of these methods are improved by named entity recognition. The main benefit of this work is proving that using the named entity recognition with Latent semantic analysis and especially with TextRank method leads to creation of higher quality summaries. This quality of the summaries was verified by ROUGE metrics.

Tato bakalářská práce se zabývá prozkoumáním aktuální metodiky a možností textového dolování a následné aplikace některých metod. V rámci práce byly popsány metody pro předzpracování, metody pro převedení textu do vektorového prostoru a metody pro analýzu textu a diskutováno jejich možné použití. Na text byly použity jednotlivé metody pro předzpracování a následně bylo demonstrováno převedení do vektorového prostoru jednoduchými metodami jako jsou BOW, Bag of n-grams, TF-IDF nebo metodami se strojovým učením které jsou FastText a GloVe. Na získané vektory byly použity metody LSA, LDA, TextRank, kosinová podobnost, pro získání informací z textu., This bachelor thesis explores the current methodology and possibilities of text mining and the subsequent application of some methods. The thesis described methods for preprocessing, methods for converting text to vector space and methods for text analysis and discusses their possible applications. The different preprocessing methods were applied to the text and then the conversion to vector space was demonstrated using simple methods such as BOW, Bag of n-grams, TF-IDF or with machine learning methods which are FastText and GloVe. LSA, LDA, TextRank and cosine similarity methods were applied to the extracted vectors to extract information from the text.

V této práci jsem se zaměřil na implementaci webové aplikace, která slouží jako prostředek pro automatickou tvorbu souhrnů v anglickém jazyce. Automatická tvorba souhrnů je v řešení prováděna pomocí metody TextRank a Latentní sémantické analýzy. Obě tyto metody jsou vylepšeny o rozpoznávání pojmenovaných entit. Přínosem této práce je zjištění, že využití rozpoznávání pojmenovaných entit u Latentní sémantické analýzy a především u metody TextRank, vede k vytváření kvalitnějších souhrnů. Tato kvalita souhrnů byla ověřena pomocí metrik ROUGE., This work is focused on an implementation a web application, which is a tool for automatic English text summarization. In result, automatic text summarization is made by TextRank and Latent semantic analysis method. Both of these methods are improved by named entity recognition. The main benefit of this work is proving that using the named entity recognition with Latent semantic analysis and especially with TextRank method leads to creation of higher quality summaries. This quality of the summaries was verified by ROUGE metrics.

V této práci jsem se zaměřil na implementaci webové aplikace, která slouží jako prostředek pro automatickou tvorbu souhrnů v anglickém jazyce. Automatická tvorba souhrnů je v řešení prováděna pomocí metody TextRank a Latentní sémantické analýzy. Obě tyto metody jsou vylepšeny o rozpoznávání pojmenovaných entit. Přínosem této práce je zjištění, že využití rozpoznávání pojmenovaných entit u Latentní sémantické analýzy a především u metody TextRank, vede k vytváření kvalitnějších souhrnů. Tato kvalita souhrnů byla ověřena pomocí metrik ROUGE., This work is focused on an implementation a web application, which is a tool for automatic English text summarization. In result, automatic text summarization is made by TextRank and Latent semantic analysis method. Both of these methods are improved by named entity recognition. The main benefit of this work is proving that using the named entity recognition with Latent semantic analysis and especially with TextRank method leads to creation of higher quality summaries. This quality of the summaries was verified by ROUGE metrics.

Tato bakalářská práce se zabývá prozkoumáním aktuální metodiky a možností textového dolování a následné aplikace některých metod. V rámci práce byly popsány metody pro předzpracování, metody pro převedení textu do vektorového prostoru a metody pro analýzu textu a diskutováno jejich možné použití. Na text byly použity jednotlivé metody pro předzpracování a následně bylo demonstrováno převedení do vektorového prostoru jednoduchými metodami jako jsou BOW, Bag of n-grams, TF-IDF nebo metodami se strojovým učením které jsou FastText a GloVe. Na získané vektory byly použity metody LSA, LDA, TextRank, kosinová podobnost, pro získání informací z textu., This bachelor thesis explores the current methodology and possibilities of text mining and the subsequent application of some methods. The thesis described methods for preprocessing, methods for converting text to vector space and methods for text analysis and discusses their possible applications. The different preprocessing methods were applied to the text and then the conversion to vector space was demonstrated using simple methods such as BOW, Bag of n-grams, TF-IDF or with machine learning methods which are FastText and GloVe. LSA, LDA, TextRank and cosine similarity methods were applied to the extracted vectors to extract information from the text.