Search

Sustainable agriculture is becoming increasingly important in the face of growing environmental challenges. One key aspect of sustainable agriculture is managing soil resources effectively. In this context, digital soil mapping (DSM) has emerged as a powerful tool to understand soil variability better and inform land management decisions. This paper proposes a comprehensive data warehouse for DSM that supports climate change analysis. Our architecture integrates frequent itemset mining (FMI) and association rules mining (ARM) to extract insights from large‐scale soil data. We review related studies in soil data warehousing and ARM, identify gaps, and propose a data warehouse architecture leveraging the galaxy multidimensional model for DSM based on the SCORPAN model, which incorporates all relevant soil forming factors. We employ and compare A‐priori, FP‐growth, and ECLAT algorithms to efficiently mine frequent itemsets and generate association rules. Our intensive experiments evaluation demonstrates that FP‐growth outperforms the other algorithms in accuracy, scalability, and speed and requires less memory. Additionally, we utilized correlation metrics for ARM, such as lift, cosine, kulc, and Imbalance ratio, to obtain the most significant and relevant association rules. These rules provide valuable insights into the complex relationships between soil properties and environmental factors, which can inform land management decisions and improve sustainable agriculture practices. This work contributes to the growing body of research on DSM and data‐driven approaches to sustainable agriculture. [ABSTRACT FROM AUTHOR]

In order to explore potential value of explosively growing data in thermal power unit, this paper proposes a big data mining method based on Hadoop-based Spark cluster-computing framework and algorithms. Firstly, positive and negative balance methods are used to accurately obtain actual net coal consumption, and maximum information coefficient method is used to select all parameters related to optimization objectives. Then, Spark-based Mini-Batch K-means algorithm and Elbow method are constructed to divide whole operating modes. After that, all data are discretized and mapped to corresponding intervals by using Spark-based Elbow method and Mini-Batch K-means algorithm. Finally, Spark-based parallel FP-growth algorithm is used to deeply mine the potential relationships and laws. To verify the proposed method, a 350-MW thermal power unit is taken as a study case. The important conclusions are as follows: (1) the proposed Spark-based Mini-Batch K-means algorithm reduces the calculation time by 57.11% compared with Mini-Batch K-means algorithm, and 85.61% calculation time compared with K-means algorithm. The proposed Spark-based FP-growth algorithm reduces computational time by 32.8% compared with FP-growth algorithm. (2) Strong association rules of whole operating modes are mined, and operating optimization guidance schemes for important parameters are obtained. Take operating mode 1 as an example: if the optimal result can be reasonably applied, it can save 2.942 g coal per kilowatt hour. (3) Besides, we have found out some other potential relationships among parameters, which have important reference value for on-site operators to analyze economy of the thermal power unit. [ABSTRACT FROM AUTHOR]

For decades, changes in fish catch composition and the marine environment have been monitored worldwide and recorded in databases like FAO FishStatJ and the European Union Copernicus Marine Service. However, the complexity and high variability in the dataset makes it challenging to find meaningful information through conventional data analytical methods. Therefore, in this pilot data mining study, we employed association rule mining algorithms (Apriori, ECLAT, and FP-Growth) to find frequently occurring itemsets in the fish-catch composition and marine environment data of the west and east coasts of India during the past decade (2011-2020). Firstly, the inherent spatial and temporal variations in fish-catch composition and marine environment (sea surface temperature and chlorophyll) on the west and east coasts of India were statistically analyzed and described. Then, the data were preprocessed, selected, and transformed into categorical attributes. By applying the association rule mining algorithms written in the Python language in the Google Colab workspace, we obtained frequent itemsets of fish catch and marine environment with different levels of minimum support and confidence. The preliminary results showed linear and inverse associations between changes in the sea surface temperature, chlorophyll concentration, and major catch groups, such as anchovies, Indian oil sardine, Indian mackerel, hairtails, butterfish-pomfrets, Bombay duck, flatfish, tunas, giant tiger prawn, crabs, lobsters, and cephalopods. Among the tested data mining algorithms, FP-Growth was found to be more efficient and reliable in finding associations between the spatiotemporal dynamics of the marine environment and fish distribution and abundance. Therefore, it can be potentially used to support marine fisheries' resource assessment and management strategies after refinement. [ABSTRACT FROM AUTHOR]

Schools and universities shuttered as a result of the worldwide COVID-19 pandemic lockdown, and student screen time skyrocketed. Since the programs are delivered online, a spike in social media use during lockdown resulted in many pupils becoming victims of cyberbullying, which includes criticizing one another, posting sexual comments on images of young ladies, and using fake accounts to bully others. Machine Learning (ML) and Natural Language Processing (NLP) techniques are being used in a growing body of work on automated cyberbullying detection. Different machine learning methods, however, are unable to converge to the requisite accuracy. Thus, numerous classifier systems known as "ensemble learning" are proposed in order to improve predictive performance by aggregating the predictions from various models. In our proposed system, we use a novel method of detecting online harassment (cyberbullying) on the Instagram dataset. The attributes of abusive words are initially analyzed from feature selection and pre-trained word embedding language models like Bidirectional Encoder Representations from Transformers (BERT) and Embeddings from Language Models (ELMO). A knowledge-based frequent pattern method is used to find the intention of the harasser and is created by the Knowledge-BERT (K-BERT). The unsupervised approaches such as Latent Semantic Analysis (LSA), Frequent pattern growth (FP-Growth), and a clustering technique K-Means. The results from the detection models are ensembled using Extreme Gradient Boosting (XGBoost) for classifying the categories of online harassment. The performance of the ensemble model is then cross-validated using machine learning metrics and compared with various existing techniques. An ensemble model performs better with a higher F1 score of 92.04% with less error rate in the classification of harassment categories. [ABSTRACT FROM AUTHOR]

The safety and stability of power networks have become increasingly important in the field of industrial control. Traditional information analysis for power networks overly relies on expert knowledge, and existing analysis models suffer from problems such as high algorithm complexity and rule redundancy. To address these issues, this paper proposes an advanced alarm information correlation analysis method that takes into account the characteristics of power networks. The method first eliminates noisy parts in the original alarm logs through a pre-processing module, then generates alarm transaction sets using a proposed method based on dynamic sliding time window, and subsequently applies the FP-Growth algorithm to mine alarm association rules for power networks. Finally, a time-based alarm rule filtering algorithm is proposed to eliminate invalid rules. Experiments conducted on alarm data collected from a situation awareness platform deployed in a power grid company show that this method reduces the redundancy of alarm rules by an average of about 30% compared to other similar association analysis method, and can effectively extract key alarm rules in power networks to guide fault warning. [ABSTRACT FROM AUTHOR]

It presents an improved FP-Growth frequent itemset mining algorithm for big data based on Hadoop framework and MapReduce programming model.Firstly, the transaction database is extracted according to each frequent 1-items to generate corresponding projection databases which are distributed to several node machines.Then each node machine divides the projection database into several smaller sub-databases, and uses the improved algorithm to mine partial frequent itemsets in parallel.Finally, all the frequent itemsets are obtained by merging all the partial frequent itemsets.This algorithm does not need to generate a huge FP tree for transaction database as the traditional FP-Growth algorithm and some other improved algorithm, so effectively solves the problem of FP-Growth algorithm and other improved algorithm failure because of insufficient memory for storage of huge FP tree in a single machine.At the same time, because the size of these sub-databases are close to each other, the load distributed to each node is more balanced, which makes the algorithm more efficient. [ABSTRACT FROM AUTHOR]

Efficient methods of analysis readily available for clinicians continue to be limited within neuropsychological assessment at the raw data level. Here, a novel approach for generating predictive response patterns and analysing neuropsychological raw data is offered. In order to assess the usefulness of association rule learning as an analysis tool for neuropsychological raw data, Frequent Pattern Growth (FP‐Growth) was used to mine patterns from the Consortium to Establish a Registry for Alzheimer's Disease Neuropsychological Battery (CERAD‐NB) database. Complete assessment data for 84 post‐mortem confirmed Alzheimer's disease (AD) cases and 294 age, race, and education matched controls were analysed across baseline and one‐year follow‐up using FP‐Growth, for the purpose of assessing the clinical utility of a finer analysis at the raw data level and the feasibility of predicting response patterns for clinical/control groups. Output from FP‐Growth, in terms of the number of frequent itemsets retained across both evaluation timepoints, was discernable between controls, mild, and moderate to severe Alzheimer's disease cases (p <.001, and η2 =.488). Patterns within raw data scores, both in terms of frequent itemsets and predictive association rules, appear to be differentiable across groups within neuropsychological analysis, which indicates that FP‐Growth is applicable as a supplementary analysis tool for neuropsychological assessment by means of offering an additional level of data analysis, predictive item response capabilities, and aiding in clinical decision making. [ABSTRACT FROM AUTHOR]

Formal concept analysis (FCA) is an unsupervised machine learning technique used for knowledge discovery and representation. A major task in FCA is the enumeration of the implications to construct the implication base. Even though there are many efficient classical and parallel algorithms proposed for constructing the implication base, the existing algorithms are not well suited for large formal contexts because of their architectural complexity. All the existing works use either stem-base or proper-premise approach to find the implication base in exponential time. Hence, we introduce a distributed algorithm to find the implication base quickly in larger datasets in polynomial time. In this paper, we propose a scalable algorithm to find the implication base using machine learning technique FP-growth, big data processing framework Apache Spark and executed on large formal contexts. Extensive experiments on the real-world datasets show that the proposed algorithm has an improved gain in performance metrics such as execution time, CPU and memory usage. The statistical validations on the experimental results prove that the proposed algorithm has the better potential to find the implication base. [ABSTRACT FROM AUTHOR]

In order to recommend an efficient drawing inspecting expert combination, an expert combination is selected by an expert recommendation algorithm based on Pearson's correlation coefficient and FP-growth. By introducing the Pearson correlation coefficient and the FP-growth association rule algorithm, the expert recommendation algorithm can accurately select the participating experts in the historical project similar to the scale of the project to be reviewed, and combine the experts to calculate and obtain the expert group with the highest fit, namely, the expert combination of project to be reviewed. This expert recommendation algorithm based on Pearson correlation coefficient and FP-growth can effectively recommend a kind of expert group with the highest efficiency of collaborative review, which solves the problem of how to recommend efficient expert combination accurately for drawing inspecting system. [ABSTRACT FROM AUTHOR]

According to the complexity of ocean data, this paper adopts a parallel mining algorithm of association rules to explore the correlation and regularity of oxygen, temperature, phosphate, nitrate and silicate in the ocean. After the marine data is interpolated, this paper utilizes the parallel FP-growth algorithm to mine the data and then briefly analyzes the mining results of the frequent itemsets and association rules. The relationship between the parallel efficiency and the core number of CPU is analyzed through datasets with different scales. The experimental results indicate that the acceleration effect is ideal when each thread scored 200,000-300,000 data, which leads to more than 1.2 times of performance improvement. [ABSTRACT FROM AUTHOR]

In this research we extract knowledge from human resources data, accumulated in IT companies for the right selection of teams to work on software projects. We are looking for interesting and unknown dependencies and connections in the data, based on which managers can form more cohesive and professionally working project teams. The proposed approach to improve the selection of teams working on IT projects is based on association rule mining and can be used by IT managers to select the members of the teams. The approbation of the proposed approach is made using the software product RapidMiner. [ABSTRACT FROM AUTHOR]

The paper proposes a new method to mine multiple-level fuzzy association rules among items at a concept hierarchy, using fuzzy partition and FP-growth algorithm. The method primarily consists of two phases: one is to find fuzzy grids at each level and the other is to generate multiple-level fuzzy association rules from those frequent patterns. As demonstrate with realistic database and different thresholds, the experimental results illustrate the proposed method are perceived more efficient. [ABSTRACT FROM AUTHOR]

The Apriori and FP-Growth are classical algorithms in frequent pattern mining. Since the Apriori has more flaws, the FP-Growth is a more efficient algorithm in stand-alone computing environment. Aiming at the bottlenecks of non-parallel computing in the era of big data, we propose a balanced parallel frequent pattern (BPFT) growth algorithm based on the connect-weight (CW) matrix of items in each transaction, called CWBPFP, which achieves parallel computing based on Spark framework. We use the load balance strategy to group data, and the corresponding code of each frequent item is stored in the relevant group during grouping. The connect information of items in each transaction of each grouped data is stored into a lower triangular connect-weight matrix by each working node. We use the restricted sub-tree to accelerate the speed of producing conditional FP-tree, and employ the connect-weight matrix to avoid the first scanning for the conditional patterns during mining frequent patterns of grouped data. The performance of the parallel mining FP-tree is improved due to the combination of the CW matrix and the restricted sub-tree applied to FP-tree mining process of each node. Experiments show that the CWBPFP has high performance and scalability on big data sets. [ABSTRACT FROM AUTHOR]

Association rule data mining is an important technique for finding important relationships in large datasets. Several frequent itemsets mining techniques have been proposed using a prefix-tree structure, FP-tree, a compressed data structure for database representation. The DIFFset data structure has also been shown to significantly reduce the run time and memory utilization of some data mining algorithms. Experimental results have demonstrated the efficiency of the two data structures in frequent itemsets mining. This work proposes FDM, a new algorithm based on FP-tree and DIFFset data structures for efficiently discovering frequent patterns in data. FDM can adapt its characteristics to efficiently mine long and short patterns from both dense and sparse datasets. Several optimization techniques are also outlined to increase the efficiency of FDM. An evaluation of FDM against three frequent itemset data mining algorithms, dEclat, FP-growth, and FDM* (FDM without optimization), was performed using datasets having both long and short frequent patterns. The experimental results show significant improvement in performance compared to the FP-growth, dEclat, and FDM* algorithms. [ABSTRACT FROM AUTHOR]

With the booming of social media and health informatics, there exists a pressing need for a powerful tool to sustain comprehensive analysis of public and personal health information. In particular, it should be able (1) to maximize the discovery of association rules amongst data items and (2) to handle the rapid growing data scale. The FP-Growth algorithm is a salient association rule learning method in exploring potential relation in database possibly with a lack of priori knowledge. It has the merits of low time & space complexity, whereas it cannot handle negative association rules which is necessary in comprehensive mining of health data. In order to enable comprehensive discovery of association rules, this study extends the FP-Growth algorithm to mine both positive and negative frequent patterns, namely the PNFP-Growth framework. The extended approach also adopts a prune strategy to filter out misleading patterns to the most by correlating the negative data items and the positive ones. Experiments had been performed to evaluate the performance of the PNFP-Growth over a public data set and a database consisting of thousands of people's real health examination information (collected within 5 years from the date of this publication). The results indicate that (1) the PNFP-Growth can excavate more patterns than the traditional counterpart does while it is still highly efficient, and (2) the analysis upon the health examination data is informative and well complies with the clinical practices, e.g., more than 30 % people suffering from hypertension are having high systolic pressure and liver problems. [ABSTRACT FROM AUTHOR]

FP-Growth algorithm is an algorithm of association rules that does not generate a set of candidate, so it has very high practical value in face of the rapid growth of data volume in wisdom medical treatment. Because FP-Growth is a memory-resident algorithm, it will appear to be powerless when it is used for massive data sets. The paper combines Hadoop and FP-Growth algorithm and through the actual analysis of traditional Chinese medicine (TCM) data compares the performance in two different environments of stand-alone and distributed. The experimental results show that FP-Growth algorithm has a great advantage in the processing and execution of massive data after the MapReduce parallel model, so that it will have better development prospects for intelligent medical treatment. [ABSTRACT FROM AUTHOR]

• To solve the problem how to activate rules that have high confidence but low support, the information gain (IG) concept is introduced in the model building process. • The proposed model consists of a set of decision rules that formally represent behavioral scripts, for predicting individuals' transportation mode choice. • A 10-fold cross validation test using a grid search parameter optimization method is conducted to validate the proposed approach. Predicting transportation mode choice is a classic challenge of travel behavior research. Over the years, different theoretical concepts and modeling approaches have been applied. This paper elaborates the application of class association rules (CARs) and examines their predictive performance using data extracted from the 2015 National Dutch Travel Survey. To solve the problem how to activate rules that have high confidence but low support, the information gain (IG) concept is introduced in the model building process. The modeling process in this study first involves extracting frequent items from the data using the FP-Growth algorithm and deriving CARs from these frequent items. Next, the IG statistic is used to construct a novel model (named CARIG), which consists of a set of decision rules that formally represent behavioral scripts, for predicting individuals' transportation mode choice. The performance of CARIG is compared with the performance of conventional class-based association rules (CBA), decision trees (DT), a convolutional neural network (CNN) and a logistic regression (LR) model. In addition, a 10-fold cross validation test using a grid search parameter optimization method is conducted to validate the proposed approach. The results show that the proposed method is promising in predicting transportation mode choices observed in the national travel survey data. [ABSTRACT FROM AUTHOR]

Automation in pharmacies has achieved innovative levels of effectiveness and savings. In the present day, automated pharmacies are facing extremely large demands of prescription orders specifically at the central fill pharmacies that distribute drugs to retail pharmacies. As a result, improvements are necessary to the Robotic Prescription Dispensing System (RPDS) and RPDS planogram to increase the throughput of prescriptions. RPDS planogram defines where to allocate the dispensers inside the robotic unit and how to distribute them among the multiple robotic units. This research utilizes the pharmacy prescriptions database to extract useful knowledge to improve different strategies in pharmacy automation by using a data mining approach. In this study, a data mining tool is proposed to enhance pharmacy automation. Frequent Pattern Growth (FP-growth) approach, which is one of the algorithms of Association Rule Mining (ARM), is applied to an actual prescriptions database of a central fill pharmacy to study the associations within the prescribed drug regime. The FP-growth application in a prescriptions database is novel; thus, FP-growth is tested on both sequential mode, and parallel mode by using a distributed platform Hadoop and MapReduce paradigm. Two types of association rules are extracted: 1) associations among different drugs that involve their different dosage strengths and manufacturers; and 2) associations that include only information about different drug generic and brand names. The importance of the extracted association rules is evaluated by the use of different measures, including the support, confidence, lift and conviction. The discovered rules disclose strong associations among the purchased drugs that improve the allocation and distribution of dispensers among the robotic units, in addition to enhancements in other pharmacy managerial strategies. [ABSTRACT FROM AUTHOR]

In this paper, we present a sparse memory allocation data structure for sequential and parallel data mining. We explored three algorithms utilizing the proposed data structure: MASP-tree, apriori-TID, and FP-growth. We modified the data structure of apriori-TID and FP-growth algorithms to reduce memory allocation cost. Five data sets are used for comparison. The results show that the modified apriori-TID has a higher speed-up than the modified FP-growth when the proposed data structure is used. A maximum speed-up of 3.42 is observed when MASP algorithm is tested. [ABSTRACT FROM AUTHOR]

For solving users' disorientation problems when using Web-based systems, there is an important issue to understand why they cause such problems. To this end, there is a need to investigate the relationship between users' characteristics and their disorientation problems. However, when facing this challenge, it is difficult to identify which users' characteristics may play important factors or how their characteristics interact with each other to influence their disorientation problems. Thus, this paper tends to propose an automatic architecture for solving this issue. More specifically, this study proposes a multiple-strategy evolutionary neural fuzzy network (MSE-NFN) to not only provides an efficiency way to automatically identify users' disorientation problems but also investigate which users' characteristics greatly influence their disorientation problems. The results indicate that users' experience of using Internet, experience of using navigation tools and different levels of prior knowledge are influential factors to affect their disorientation problems. Moreover, it also demonstrates that the proposed architecture (MSE-NFN) outperform than other existing evolutionary methods. Based on the results, a framework is conducted, which can be used to automatically identify users' disorientation problems when developing the personalized Web-based systems. [ABSTRACT FROM AUTHOR]

In this paper, a recurrent wavelet-based neuro-fuzzy identifier (RWNFI) with a self-organization hybrid evolution learning algorithm (SOHELA) is proposed for solving various identification problems. In the proposed SOHELA, the group-based symbiotic evolution (GSE) is adopted such that each group in the GSE represents a collection of only one fuzzy rule. The proposed SOHELA consists of structure learning and parameter learning. In structure learning, the proposed SOHELA uses the self-organization algorithm (SOA) to determine a suitable rule number in the RWNFI. In parameter learning, the proposed SOHELA uses the data mining-based selection method (DMSM) and the data mining-based crossover method (DMCM) to determine groups and parent groups using the data mining method called the frequent pattern growth (FP-Growth) method. Based on identification simulations, the excellent performance of the proposed SOHELA compares with other various existing models. [ABSTRACT FROM AUTHOR]

Traditional data mining approaches look for patterns in a single table, while multi-relational data mining aims for identifying patterns that involve multiple tables. In recent years, the most common mining techniques have been extended to the multi-relational context, but there are few dedicated to deal with data stored following the multi-dimensional model, in particular the star schema. These schemas are composed of a central huge fact table linking a set of small dimension tables. Joining all the tables before mining may not be a feasible solution due to the usual massive number of records. This work proposes a method for mining frequent patterns on data following a star schema that does not materialize the join between the tables. As it extends the algorithm FP-Growth, it constructs an FP-Tree for each dimension and then combines them through the records in the fact table to form a super FP-Tree. This tree is then mined with FP-growth to find all frequent patterns. The paper presents a case study on bibliographic data, comparing efficiency and scalability of our algorithm against FP-Growth. [ABSTRACT FROM AUTHOR]

Most of the existing methods to solve the problem of association rules mining (ARM) rely on special data structures to project the database (either totally or partially) in the primary memory. Traditionally, these data structures reside in the main memory and rely on the existing paging mechanism of the virtual memory manager (VMM) to handle the storage problem when they go out of the primary memory. Typically, VMM stores the overloaded data into the secondary memory based on some preassumed memory usage criteria. However, this direct and unplanned use of virtual memory results in an unpredictable behavior or thrashing, as depicted by some of the works described in the literature. This problem is tackled in this paper by presenting an ARM model capable of mining a transactional database, regardless of its size and without relying on the underlying VMM; the proposed approach could use only a bounded portion of the primary memory and this gives the opportunity to assign other parts of the main memory to other tasks with different priority. In other words, we propose a specialized memory management system which caters to the needs of the ARM model in such a way that the proposed data structure is constructed in the available allocated primary memory first. If at any point the structure grows out of the allocated memory quota, it is forced to be partially saved on secondary memory. The secondary memory version of the structure is accessed in a block-by-block basis so that both the spatial and temporal localities of the I/O access are optimized. Thus, the proposed framework takes control of the virtual memory access and hence manages the required virtual memory in an optimal way to the best benefit of the mining process to be served. Several clever data structures are used to facilitate these optimizations. Our method has the additional advantage that other tasks of different priorities may run concurrently with the main mining task with as little interference as possible because we do not rely on the default paging mechanism of the VMM. The reported test results demonstrate the applicability and effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

Due to the rise of e-Learning, more and more useful learning materials are open to public access. Therefore, an appropriate learning suggestion mechanism is an important tool to enable learners to work more efficiently. A smoother learning process increases the learning effect, avoiding unnecessarily difficult concepts and disorientation during learning. However, many suggestion demands come from different abstraction levels, and traditional single level frequent pattern mining is not sufficient. This paper proposes a methodology for mining frequent patterns of learners' behavior which connote a hierarchical scheme to provide cross-level learning suggestions for the next learning course. With this system, a learner can get multiple levels of abstract suggestions instead of merely single level frequent pattern mining results. Our study shows that the algorithms can mine considerable quantities of frequent patterns from real life learning data. The experimental data are collected from a Web learning system originating from National Cheng Kung University in Taiwan. The proposed methodology gives learners many suggestions to help them learn more effectively and efficiently. Finally, we collect some representative cases to realize different requirements which are extracted from a learners' access database. These cases are classified into three types; notably, type three generalized four meaningful external factors which are inferred by our observations from these cross-level frequent patterns. [ABSTRACT FROM AUTHOR]