Search

In this paper, we are proposing extensions to the multinomial principal component analysis (MPCA) framework, which is a Dirichlet (Dir)-based model widely used in text document analysis. The MPCA is a discrete analogue to the standard PCA (it operates on continuous data using Gaussian distributions). With the extensive use of count data in modeling nowadays, the current limitations of the Dir prior (independent assumption within its components and very restricted covariance structure) tend to prevent efficient processing. As a result, we are proposing some alternatives with flexible priors such as generalized Dirichlet (GD) and Beta-Liouville (BL), leading to GDMPCA and BLMPCA models, respectively. Besides using these priors as they generalize the Dir, importantly, we also implement a deterministic method that uses variational Bayesian inference for the fast convergence of the proposed algorithms. Additionally, we use collapsed Gibbs sampling to estimate the model parameters, providing a computationally efficient method for inference. These two variational models offer higher flexibility while assigning each observation to a distinct cluster. We create several multitopic models and evaluate their strengths and weaknesses using real-world applications such as text classification and sentiment analysis. [ABSTRACT FROM AUTHOR]

Text clustering system is a proper technique which mainly segments large measure of textual documents into clusters. The size of the material influences the clustering of text by reducing its performance. In this manner, the textual document comprises sparse and uninformative features, and thus raises the computational time and decreases the execution of primary clustering process. Feature selection is a crucial system to choose another subset of instructive text feature to enhance text clustering execution and diminish computational time. The implemented model proposes a 2logmean-particle swarm optimization algorithm for the unstructured text clustering. In this newly proposed technique, all the texts are initially converted into ASCII value, and then by using the particle swarm optimization, the document text is clustered. The outcomes display that clustering accuracy of the implemented method is high compared to the existing K-means algorithm. Furthermore, performances of newly implemented techniques are evaluated concerning scalability, less computation speed with colossal dimensionality reduction. [ABSTRACT FROM AUTHOR]

In the development of a task-oriented dialogue system, defining the dialogue structure is a time-consuming task. Hence, several works have looked into automatically inferring it from data, e.g., actual conversations between a customer and a support agent. To recover such dialogue structure, recent methods based on discrete variational models learn to jointly encode and cluster utterances in dialogue states, but (i) represent utterances by only considering preceding dialogue context, and (ii) are slow to train since they are optimized with a compute-expensive decoding objective. We revisit and improve upon an existing efficient pipeline approach, commonly adopted as a baseline, that first encodes utterances and then clusters them with k-means to induce the dialogue structure. However, the existing approach represents utterances as bag-of-words or skip-thought vectors, which have been shown to perform poorly in semantic similarity tasks, and without considering dialogue context. We therefore first investigate the use of more powerful transformer-based encoders for encoding utterances. Next, we propose ellodar, a method for learning representations that capture both preceding and subsequent dialogue context, inspired by word-to-vec training strategies. ellodar is efficient since representations are learned directly in the encoding space by finetuning just a single linear layer on top of a frozen sentence encoder with a vector-to-vector regression training objective. Extensive experiments on representative datasets for dialogue structure induction (SimDial, Schema Guided Dialogues, DSTC2, and CamRest676) demonstrate that in terms of effectiveness to induce the correct dialogue structure, (i) clustering utterances represented by transformed-based encoders improves recent joint models by 13%–32% on standard cluster metrics, and (ii) clustering ellodar's representations yields additional improvements ranging from +20% to +26%, with speedups of × 10 – 10 4 compared to the recent joint models. [ABSTRACT FROM AUTHOR]

Large Language Models (LLMs), one of the most advanced representatives of neural networks, have revolutionized the field of natural language processing. Among the many applications of these models, text clustering is gaining increasing interest. In particular, the fact that LLMs digitize text more semantically and contextually than existing methods in the literature has led LLMs to produce more successful results with clustering algorithms. However, since these models are not specifically designed for text clustering, they can lead to processing times that exceed acceptable runtime thresholds. To address this challenge, the Sentence-BERT (SBERT) model has been proposed as a solution, offering the ability to accurately measure text similarity by transforming entire texts into dense, fixed-size vectors. SBERT has been integrated into various LLMs, resulting in the creation of diverse SBERT model variants. This study aims to assess the transfer learning capabilities of SBERT models in the context of text clustering. Furthermore, it investigates the influence of CLS (classification token), mean, and max pooling techniques on the performance of these models. In this direction, we applied these pooling techniques to DistilBERT, DistilRoBERTa, ALBERT, and MPNET based SBERT models and compared their performance on different corpora. The results show that there is no clear superiority among the SBERT models. However, the mean pooling emerged as the most effective method in 13 out of 16 text clustering tasks. This finding underscores the high compatibility of the mean pooling technique with SBERT models.

In an era of exponential growth in online news sources, the need for intelligent digital solutions capable of efficiently analyzing and organizing large amounts of news content has become crucial. This paper presents a graph-based methodology designed to enhance Topic Detection and Tracking (TDT) tasks in natural language processing by efficiently clustering news events into coherent stories. The proposed approach leverages a novel event graph model that captures not only the characteristics of individual news events but also their collective narrative context. Using Named Entity Centred Frequent Subgraphs, the model excels in identifying recurring patterns of events and thus provides a framework for learning a robust, language-independent, and structured representation for structuring news stories, which represents a significant advance in the refinement of traditional clustering algorithms. Empirical experiments using a multilingual benchmark dataset, the News Clustering Dataset, highlight the superior clustering performance of our approach compared to state-of-the-art monolingual document clustering techniques, particularly in English and the competitive results in Spanish. To underline the adaptability of the methodology to low-resource languages, the Turkish ‘Story-Based News Dataset’ developed specifically for this study also promises to serve as an important resource for a wide range of natural language processing tasks.

The frequent occurrence of extreme weather makes people pay more attention to environmental protection. To cope with the global climate problem, various countries re-plan social development through the concept of low-carbon. As greatly popularized by the Internet, the topic of low carbon concept is spread more through online social media, so it is urgent to understand the user’s attention to low carbon topics in a more intelligent way for subsequent relevant publicity and policy guidance. This paper studies the low-carbon topic of attention in the context of social media. First, the BERT (Bidirectional Encoder Representation from Transformers) model is used to complete the word vector feature extraction of acquired data; Secondly, the FCM method was used to complete the clustering analysis of the main topics in the low-carbon concept, and the PSO method was used to optimize the model. After optimization, the accuracy of clustering for various topics was higher than 80%. For the Esse index of cluster center variance, the method proposed in this article is also close to 10% due to other classic methods; Finally, this paper carried out an application test of low-carbon topics in the region, achieved good results, and made a detailed analysis of the distribution of various topics. It can be predicted that this method will provide more public opinion references for low-carbon development paths in various countries and regions in the future, and provide technical support for information dissemination and analysis under social media.

This research paper aims to analyze and predict fraud patterns among failed companies in Malaysia. The approach involves utilizing textual analysis on the management discussion and analysis (MD&A) section within the annual reports. The dataset is subjected to text clustering to group companies based on similar financial characteristics. This clustering process entails several steps, including data conversion, collation, and summarization into a structured format, followed by text pre-processing to cleanse the dataset. Notably, RapidMiner Studio software was utilized to extract data for the study. Subsequently, the documents are clustered using both the K-means and latent dirichlet allocation (LDA) methods. Upon examining a sample of 22 failed companies in the year 2020, the study reveals that financially distressed companies exhibit prominent financial negativity and utilize litigious financial terms within their MD&A sections. These linguistic traits are found to be closely associated with seven distinct characteristics of fraudulent firms. This preliminary findings provide compelling evidence that financial pressure may serve as a triggering factor for fraudulent activities within companies. [ABSTRACT FROM AUTHOR]

Traditional text clustering based on distance struggles to distinguish between overlapping representations in medical data. By incorporating contrastive learning, the feature space can be optimized and applies mixup implicitly during the data augmentation phase to reduce computational burden. Medical case text is prevalent in everyday life, and clustering is a fundamental method of identifying major categories of conditions within vast amounts of unlabeled text. Learning meaningful clustering scores in data relating to rare diseases is difficult due to their unique sparsity. To address this issue, we propose a contrastive clustering method based on mixup, which involves selecting a small batch of data to simulate the experimental environment of rare diseases. The contrastive learning module optimizes the feature space based on the fact that positive pairs share negative samples, and clustering is employed to group data with comparable semantic features. The module mitigates the issue of overlap in data, whilst mixup generates cost-effective virtual features, resulting in superior experiment scores even when using small batch data and reducing resource usage and time overhead. Our suggested technique has acquired cutting-edge outcomes and embodies a favorable strategy for unmonitored text clustering. [ABSTRACT FROM AUTHOR]

Text clustering is the task of grouping text data based on similarity, and it holds particular importance in the medical field. sIn healthcare, medical data clustering is a highly active and e ective research area. It not only provides strong support for making correct medical decisions from medical datasets but also aids in patient record management and medical information retrieval. With the development of the healthcare industry, a large amount of medical data is being generated, and traditional medical data clustering faces significant challenges. Many existing text clustering algorithms are primarily based on the bag-of-words model, which has issues such as high dimensionality, sparsity, and the neglect of word positions and context. Pre-trained models are a deep learning-based approach that treats text as a sequence to accurately captureword positions and context information. Moreover, compared to traditional K-means and fuzzy C-means clustering models, deep learning-based clustering algorithms are better at handling high-dimensional, complex, and nonlinear data. In particular, clustering algorithms based on autoencoders can learn data representations and clustering information, e ectively reducing noise interference and errors during the clustering process. This paper combines pre-trained language models with deep embedding clustering models. Experimental results demonstrate that our model performs exceptionally well on four public datasets, outperforming most existing text clustering algorithms, and can be applied to medical data clustering. [ABSTRACT FROM AUTHOR]

In this paper, we formulate a hierarchical Bayesian version of the Mixture of Unigrams model for text clustering and approach its posterior inference through variational inference. We compute the explicit expression of the variational objective function for our hierarchical model under a mean-field approximation. We then derive the update equations of a suitable algorithm based on coordinate ascent to find local maxima of the variational target, and estimate the model parameters through the optimized variational hyperparameters. The advantages of variational algorithms over traditional Markov Chain Monte Carlo methods based on iterative posterior sampling are also discussed in detail. [ABSTRACT FROM AUTHOR]

Al-Quran is Muslims' main book of belief and behaviour. The Al-Quran is used as a reference book by millions of Muslims worldwide, and as such, it is useful for Muslims in general and Muslim academics to gain knowledge from it. Many translators have worked on the Quran's translation into many different languages around the world, including English. Thus, every translator has his/her own perspectives, statements, and opinions when translating verses acquired from the (Tafseer) of the Quran. However, this work aims to cluster these variations among translations of the Tafseer by utilising text clustering. As a part of the text mining approach, text clustering includes clustering documents according to how similar they are. This study adapted the (k-means) clustering technique algorithm (unsupervised learning) to illustrate and discover the relationships between keywords called features or concepts for five different translators on the 286 verses of the Al-Baqarah chapter. The datasets have been preprocessed, and features extracted by applying TF-IDF (Term Frequency-Inverse Document Frequency). The findings show two/three-dimensional clustering plotting for the first two/three most frequent features assigned to seven cluster categories (k = 7) for each of five translated Tafseer. The features 'allah/god', 'believ', and 'said' are the three most features shared by the five Tafseer. [ABSTRACT FROM AUTHOR]

Extracting knowledge from text data is a complex task that is usually performed by first structuring the texts and then applying machine learning algorithms, or by using specific deep architectures capable of dealing directly with the raw text data. The traditional approach to structure texts is called Bag of Words (BoW) and consists of transforming each word in a document into a dimension (variable) in the structured data. Another approach uses grammatical classes to categorize the words and, thus, limit the dimension of the structured data to the number of grammatical categories. Another form of structuring text data for analysis is by using a distributed representation of words, sentences, or documents with methods like Word2Vec, Doc2Vec, and SBERT. This paper investigates four classes of text structuring methods to prepare documents for being clustered by an artificial immune system called aiNet. The goal is to assess the influence of each structuring method in the quality of the clustering obtained by the system and how methods that belong to the same type of representation differ from each other, for example both LIWC and MRC are considered grammarbased models but each one of them uses completely different dictionaries to generate its representation. By using internal clustering measures, our results showed that vector space models, on average, presented the best results for the datasets chosen, followed closely by the state of the art SBERT model, and MRC had the overall worst performance. We could also observe a consistency in the number of clusters generated by each representation and for each dataset, having SBERT as the model that presented a number of clusters closer to the original number of classes in the data.

The use of technology in line with the increasing number of internet users has caused a shift in the product sales ecosystem to the realm of electronic commerce (electronic commerce). A total of 73.23 customers made purchase transactions using e-commerce and the most purchased products were products classified as Fast Moving Consumer Goods (FMCGs). The increasingly varied FMCGs data coupled with the increasing number of marketplaces is felt to need to be broken down into specific groups. The process is carried out by analyzing e-commerce product information, especially product names, and descriptions. In this study, we propose an automatic categorization of multiple marketplaces using data from multiple marketplaces. Data text is converted into structured data with a series of preprocessing, and comprehensive experiments are carried out to see the extraction performance of variables including TF-IDF, BOW, and N-Gram. All three methods are used to validate text data sets with K-Means grouping results used with the help of PCA to reduce data dimensions. The results show that the performance of the TF-IDF algorithm with a dimension reduction value of 70 and the use of Python can provide optimal results for the percentage of grouping data.

AbstractThis paper examines the relationship between Federal Reserve policy and the Taylor rule, a commonly used model for guiding monetary policy. The study analyzes the deviation of the actual Federal Funds Rate from the Taylor Rule model during distinct structural changes, using real-time macroeconomic data available to the Fed at the time of their interest rate decision. The research focuses on whether former Fed chair Alan Greenspan’s policies from 2003 to 2006, which have been linked to the housing bubble, were deviant from the Taylor Rule. The findings show that there isn’t sufficient statistical evidence to support this claim, and a machine learning text analysis of the Federal Open Market Committee transcripts confirms the presence of only one regime during this period. These results contribute to the existing literature on monetary policy and its impact on the economy, providing valuable insights into the relationship between Federal Reserve policy and the Taylor Rule.

Arabic text clustering is an essential topic in Arabic Natural Language Processing (ANLP). Its significance resides in various applications, such as document indexing, categorization, user review analysis, and others. After inspecting the current work on clustering Arabic text, it is observed that most researchers focus on applying K-Means clustering while hindering other clustering techniques. Our evaluation shows that K-Means has a weakness of inconsistent clustering results and weak clustering performance when the data dimensionality increases. Unlike K-Means clustering, Artificial Neural Networks (ANN) models such as Self-Organizing Maps (SOM) demonstrated higher accuracy and efficiency in clustering even with high dimensional datasets. In this paper, we introduce a new clustering model based on an optimization technique called Grey Wolf Optimization (GWO) used conjointly with SOM clustering to enhance its clustering performance and accuracy. The evaluation results of our proposed technique show an improvement in the effectiveness and efficiency in comparison with state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Chongqing University of Technology (Natural Science) is the property of Chongqing University of Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of South-Central Minzu University (Natural Science Edition) is the property of Journal of South-Central Minzu University (Natural Science Edition) Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Extracting knowledge from text data is a complex task that is usually performed by first structuring the texts and then applying machine learning algorithms, or by using specific deep architectures capable of dealing directly with the raw text data. The traditional approach to structure texts is called Bag of Words (BoW) and consists of transforming each word in a document into a dimension (variable) in the structured data. Another approach uses grammatical classes to categorize the words and, thus, limit the dimension of the structured data to the number of grammatical categories. Another form of structuring text data for analysis is by using a distributed representation of words, sentences, or documents with methods like Word2Vec, Doc2Vec, and SBERT. This paper investigates four classes of text structuring methods to prepare documents for being clustered by an artificial immune system called aiNet. The goal is to assess the influence of each structuring method in the quality of the clustering obtained by the system and how methods that belong to the same type of representation differ from each other, for example both LIWC and MRC are considered grammarbased models but each one of them uses completely different dictionaries to generate its representation. By using internal clustering measures, our results showed that vector space models, on average, presented the best results for the datasets chosen, followed closely by the state of the art SBERT model, and MRC had the overall worst performance. We could also observe a consistency in the number of clusters generated by each representation and for each dataset, having SBERT as the model that presented a number of clusters closer to the original number of classes in the data. [ABSTRACT FROM AUTHOR]

The text clustering model becomes an essential process to sort the unstructured text data in an appropriate format. But, it does not give the pave for extracting the information to facilitate the document representation. In today's date, it becomes crucial to retrieve the relevant text data. Mostly, the data comprises an unstructured text format that it is difficult to categorize the data. The major intention of this work is to implement a new text clustering model of unstructured data using classifier approaches. At first, the unstructured data is taken from standard benchmark datasets focusing on both English and Telugu languages. The collected text data is then given to the pre-processing stage. The pre-processed data is fed into the model of the feature extraction stage 1, in which the GloVe embedding technique is used for extracting text features. Similarly, in the feature extraction stage 2, the pre-processed data is used to extract the deep text features using Text Convolutional Neural Network (Text CNN). Then, the text features from Stage 1 and deep features from Stage 2 are all together and employed for optimal feature selection using the Hybrid Sea Lion Grasshopper Optimization (HSLnGO), where the traditional SLnO is superimposed with GOA. Finally, the text clustering is processed with the help of Deep CNN-assisted hierarchical clustering, where the parameter optimization is done to improve the clustering performance using HSLnGO. Thus, the simulation findings illustrate that the framework yields impressive performance of text classification in contrast with other techniques while implementing the unstructured text data using different quantitative measures. [ABSTRACT FROM AUTHOR]

Text clustering has been widely used in data mining, document management, search engines, and other fields. The K-means algorithm is a representative algorithm of text clustering. However, traditional K-means algorithm often uses Euclidean distance or cosine distance to measure the similarity between texts, which is not effective in face of high-dimensional data and cannot retain enough semantic information. In response to the above problems, we combine word rotator's distance with the K-means algorithm, and propose the WRDK-means algorithm, which use word rotator's distance to calculate the similarity between texts and preserve more text features. Furthermore, we define a new cluster center initialization method that improves cluster instability during random initial cluster center selection. And, to solve the problem of inconsistent length between texts, we propose a new iterative approximation method of cluster centers. We selected three suitable datasets and five evaluation indicators to verify the feasibility of the proposed algorithm. Among them, the RI value of our algorithm exceeds 90%. And for Marco_F1, our scheme was about 37.77%, 23.2%, 13.06% and 20.12% better than other four methods, respectively. [ABSTRACT FROM AUTHOR]

This paper describes an application dedicated to collecting and mining reports of software safety vulnerabilities and exposures in physical systems. This work focuses on the clustering problem of such reports, which means grouping them through automated computing process. The clustering is carried out in two stages. In the first stage, potential similarities between the reports, together with the number of clusters are detected through automated text analysis. In the second stage, a hierarchical clustering is conducted to reduce the number of these clusters to provide potential number of appropriate clusters of the group of reports. The clustering of the second stage provides the user with greater flexibility in viewing individual reports. This paper focuses mainly on the first stage of described clustering method. Two selected clustering algorithms have been compared with the aim to show how to detect the most appropriate number of groups between scraped documents. The computational experiment results are presented and discussed in the experimental section of this work. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Railway Science & Engineering is the property of Journal of Railway Science & Engineering Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Jilin University (Science Edition) / Jilin Daxue Xuebao (Lixue Ban) is the property of Zhongguo Xue shu qi Kan (Guang Pan Ban) Dian zi Za zhi She and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In this research, an effort has been put to develop an integrated predictive modeling framework to automatically estimate the rental price of Airbnb units based on listed descriptions and several accommodation-related utilities. This paper considers approximately 0.2 million listings of Airbnb units across seven European cities, Amsterdam, Barcelona, Brussels, Geneva, Istanbul, London, and Milan, after the COVID-19 pandemic for predictive analysis. RoBERTa, a transfer learning framework in conjunction with K -means-based unsupervised text clustering, was used to form a homogeneous grouping of Airbnb units across the cities. Subsequently, particle swarm optimization (PSO) driven advanced ensemble machine learning frameworks have been utilized for predicting rental prices across the formed clusters of respective cities using 32 offer-related features. Additionally, explainable artificial intelligence (AI), an emerging field of AI, has been utilized to interpret the high-end predictive modeling to infer deeper insights into the nature and direction of influence of explanatory features on rental prices at respective locations. The rental prices of Airbnb units in Geneva and Brussels have appeared to be highly predictable, while the units in London and Milan have been found to be less predictable. Different types of amenity offerings largely explain the variation in rental prices across the cities. [ABSTRACT FROM AUTHOR]

The multi-semantic problem of words, together with the problem of independence between the traditional text representation and the clustering process, leads to the low precision of text clustering. A multi-semantic representation based adaptive fuzzy C-means (MSR-AFCM) clustering algorithm was proposed to solve these problems. To be specific, multiple semantic spaces were constructed by using the soft clustering of words. Then, the number of semantic spaces was taken as the initial number of text clustering. Next, the semantic membership degree of each text belonging to the text space was calculated by utilizing the semantic members-hip degree of the words, and then the algorithm degree was initialized accordingly. During the running of the algorithm, the redundant text spaces were gradually eliminated based on the updated text semantic membership degree and text distribution, so as to achieve the goal of optimizing the number of text clustering. Compared with other text clustering algorithms, MSR-AFCM algorithm can improve the clustering performance by about 2. 66% ~ 12. 5%,it has higher accuracy and rand coefficient. [ABSTRACT FROM AUTHOR]

The purpose of this study is to develop a text clustering–based analysis of COVID-19 research articles. Owing to the proliferation of published COVID-19 research articles, researchers need a method for reducing the number of articles they have to search through to find material relevant to their expertise. The study analyzes 83,264 abstracts from research articles related to COVID-19. The textual data are analysed using singular value decomposition (SVD) and the expectation–maximisation (EM) algorithm. Results suggest that text clustering can both reveal hidden research themes in the published literature related to COVID-19, and reduce the number of articles that researchers need to search through to find material relevant to their field of interest. [ABSTRACT FROM AUTHOR]

Copyright of Food Science & Technology & Economy is the property of Grain Science & Technology & Economy Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)