Search

One of the most challenging tasks in association rule mining is that when a new incremental database is added to an original database, some existing frequent itemsets may become infrequent itemsets and vice versa. As a result, some previous association rules may become invalid and some new association rules may emerge. We designed a new, more efficient approach for incremental association rule mining using a Fast Incremental Updating Frequent Pattern growth algorithm (FIUFP-Growth), a new Incremental Conditional Pattern tree (ICP-tree), and a compact sub-tree suitable for incremental mining of frequent itemsets. This algorithm retrieves previous frequent itemsets that have already been mined from the original database and their support counts then use them to efficiently mine frequent itemsets from the updated database and ICP-tree, reducing the number of rescans of the original database. Our algorithm reduced usages of resource and time for unnecessary sub-tree construction compared to individual FP- Growth, FUFP-tree maintenance, Pre-FUFP, and FCFPIM algorithms. From the results, at 3% minimum support threshold, the average execution time for pattern growth mining of our algorithm performs 46% faster than FP- Growth, FUFP-tree, Pre-FUFP, and FCFPIM. This approach to incremental association rule mining and our experimental findings may directly benefit designers and developers of computer business intelligence methods.

Association rule data mining is an important technique for finding important relationships in large datasets. Several frequent itemsets mining techniques have been proposed using a prefix-tree structure, FP-tree, a compressed data structure for database representation. The DIFFset data structure has also been shown to significantly reduce the run time and memory utilization of some data mining algorithms. Experimental results have demonstrated the efficiency of the two data structures in frequent itemsets mining. This work proposes FDM, a new algorithm based on FP-tree and DIFFset data structures for efficiently discovering frequent patterns in data. FDM can adapt its characteristics to efficiently mine long and short patterns from both dense and sparse datasets. Several optimization techniques are also outlined to increase the efficiency of FDM. An evaluation of FDM against three frequent itemset data mining algorithms, dEclat, FP-growth, and FDM* (FDM without optimization), was performed using datasets having both long and short frequent patterns. The experimental results show significant improvement in performance compared to the FP-growth, dEclat, and FDM* algorithms. [ABSTRACT FROM AUTHOR]

In the present scenario of global economy and World Wide Web, large sets of evolving and distributed data can be handled efficiently by incremental data mining. Frequent patterns are very important in knowledge discovery and data mining process, such as mining of association rules, correlations. FP-tree is a very versatile data structure used for mining of frequent patterns in knowledge discovery and data mining process. FP-tree is a compact representation of transaction database that contains frequency information of all relevant frequent patterns (FP) of the database. All of the existing incremental frequent pattern mining algorithms, such as AFPIM, CATS, CanTree, CP-tree, and SPO-tree, perform incremental mining by processing one transaction of the incremental part of database at a time and updating it to the FP-tree of initial (original) database. Here, in this paper, we propose a novel method that takes advantage of FP-tree representation of incremental transaction database for incremental mining. We propose a batch incremental processing algorithm BIT_FPGrowth that restructures and merges two small consecutive duration FP-trees to obtain a FP-tree of the FP-Growth algorithm. Our BIT_FPGrowth uses FP-tree as preprocessed data repository to get transactions (i.e., item-sets), unlike other sequential incremental algorithms that read transactions from database. BIT_FPGrowth algorithm takes less time for constructing FP-tree. Our experimental results show that, as the size of the database increases, increase in runtime of BIT_FPGrowth is much less and is least of all the other algorithms. [ABSTRACT FROM AUTHOR]

Abstract: FP-Growth is a recursive mining algorithm for frequent patterns. With each recursion, a new FP-tree and the corresponding header table are built. This paper proposes another approach to improve the mining efficiency: only one FP-tree is constructed, and instead of building new trees in every recursion, builds new header tables; frequent patterns are generated along the way, so we call this algorithm “header table recursion (HTR)” method. Tests show that it will get better performance for relative small datasets because the compress of the FP-tree has no significant impact on the performance yet inflicts extra time cost in tree creation operations. However on larger datasets, the traversal of the original FP-tree will become more time-consuming and its overall performance needs further testing. [Copyright &y& Elsevier]

Most of the existing methods to solve the problem of association rules mining (ARM) rely on special data structures to project the database (either totally or partially) in the primary memory. Traditionally, these data structures reside in the main memory and rely on the existing paging mechanism of the virtual memory manager (VMM) to handle the storage problem when they go out of the primary memory. Typically, VMM stores the overloaded data into the secondary memory based on some preassumed memory usage criteria. However, this direct and unplanned use of virtual memory results in an unpredictable behavior or thrashing, as depicted by some of the works described in the literature. This problem is tackled in this paper by presenting an ARM model capable of mining a transactional database, regardless of its size and without relying on the underlying VMM; the proposed approach could use only a bounded portion of the primary memory and this gives the opportunity to assign other parts of the main memory to other tasks with different priority. In other words, we propose a specialized memory management system which caters to the needs of the ARM model in such a way that the proposed data structure is constructed in the available allocated primary memory first. If at any point the structure grows out of the allocated memory quota, it is forced to be partially saved on secondary memory. The secondary memory version of the structure is accessed in a block-by-block basis so that both the spatial and temporal localities of the I/O access are optimized. Thus, the proposed framework takes control of the virtual memory access and hence manages the required virtual memory in an optimal way to the best benefit of the mining process to be served. Several clever data structures are used to facilitate these optimizations. Our method has the additional advantage that other tasks of different priorities may run concurrently with the main mining task with as little interference as possible because we do not rely on the default paging mechanism of the VMM. The reported test results demonstrate the applicability and effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

Due to the rise of e-Learning, more and more useful learning materials are open to public access. Therefore, an appropriate learning suggestion mechanism is an important tool to enable learners to work more efficiently. A smoother learning process increases the learning effect, avoiding unnecessarily difficult concepts and disorientation during learning. However, many suggestion demands come from different abstraction levels, and traditional single level frequent pattern mining is not sufficient. This paper proposes a methodology for mining frequent patterns of learners' behavior which connote a hierarchical scheme to provide cross-level learning suggestions for the next learning course. With this system, a learner can get multiple levels of abstract suggestions instead of merely single level frequent pattern mining results. Our study shows that the algorithms can mine considerable quantities of frequent patterns from real life learning data. The experimental data are collected from a Web learning system originating from National Cheng Kung University in Taiwan. The proposed methodology gives learners many suggestions to help them learn more effectively and efficiently. Finally, we collect some representative cases to realize different requirements which are extracted from a learners' access database. These cases are classified into three types; notably, type three generalized four meaningful external factors which are inferred by our observations from these cross-level frequent patterns. [ABSTRACT FROM AUTHOR]