Search

This paper reports on conceptual development in the areas of database mining and knowledge discovery in databases (KDD). The authors' efforts have also led to a prototype implementation, called MOTC, for exploring hypothesis space in large and complex data sets. Their KDD conceptual development rests on two main principles. First, they use the crosstab representation for working with qualitative data. This is by now standard in on-line analytical processing (OLAP) applications, and the authors reaffirm it with additional reasons. Second, and innovatively, they use prediction analysis as a measure of goodness for hypotheses. Prediction analysis is an established statistical technique for analysis of associations among qualitative variables. It generalizes and subsumes a large number of other such measures of association, depending on specific assumptions the user is willing to make. As such, it provides a very useful framework for exploring hypothesis space in a KDD context. The paper illustrates these points with an extensive discussion of MOTC. [ABSTRACT FROM AUTHOR]

Data warehouse systems are used for decision-making purposes. The Online Analytical Processing (OLAP) tools are commonly used to query and analysis of results on such systems. It is complex task for non-technical users (executives, managers etc.,) to query the data warehouse using OLAP tool keeping in view the schema knowledge. For such data warehouse users, a natural language interface is a viable solution that transparently access data to fulfil their requirement. As data warehouse contain several times more data (that increase with incremental refreshes) than the operational systems. So keyword-based searching in such systems cannot be performed similar to database based natural language systems. Existing natural language interfaces to data warehouse commonly explore keywords in data instances directly that takes more than sufficient time in generating results. This paper proposes a Logical Schema-based Mapping (LSM) technique to reduce search space in the data warehouse data instances. It performs mapping of the natural language query keywords with logical schema of the data warehouse to identify the elements prior to search in the data instances. The retrieved matches for a keyword are ranked based on six criteria proposed in this paper. Further, an algorithm has been presented which is developed upon the proposed criteria. Targeted search in the data instances is then performed efficiently after the identification of schema elements. The in-depth experiments have been carried out on real dataset to evaluate the system with respect to completeness, accuracy and performance parameters. The results show that LSM technique outperforms the existing systems. [ABSTRACT FROM AUTHOR]

Myriad of graph-based algorithms in machine learning and data mining require parsing relational data iteratively. These algorithms are implemented in a large-scale distributed environment to scale to massive data sets. To accelerate these large-scale graph-based iterative computations, we propose delta-based accumulative iterative computation (DAIC). Different from traditional iterative computations, which iteratively update the result based on the result from the previous iteration, DAIC updates the result by accumulating the “changes” between iterations. By DAIC, we can process only the “changes” to avoid the negligible updates. Furthermore, we can perform DAIC asynchronously to bypass the high-cost synchronous barriers in heterogeneous distributed environments. Based on the DAIC model, we design and implement an asynchronous graph processing framework, Maiter. We evaluate Maiter on local cluster as well as on Amazon EC2 Cloud. The results show that Maiter achieves as much as 60\times speedup over Hadoop and outperforms other state-of-the-art frameworks. [ABSTRACT FROM PUBLISHER]

In this paper we describe possible interpretation and reduction of fuzzy attributes in Generalized One-sided Concept Lattices (GOSCL). This type of concept lattices represent generalization of Formal Concept Analysis (FCA) suitable for analysis of data tables with different types of at- tributes. FCA as well as generalized one-sided concept lattices represent conceptual data mining methods. With growing number of attributes the interpretation of fuzzy subsets may become unclear, hence another interpretation of this fuzzy attribute subsets can be valuable. The originality of the presented method is based on the usage of one-sided concept lattices derived from submodels of former object-attribute model by grouping attributes with the same truth value structure. This leads to new method for attribute reduction in GOSCL environment. [ABSTRACT FROM AUTHOR]

This paper describes a novel method for hurricane trajectory prediction based on data mining (HTPDM) according to the hurricane's motion characteristics. Firstly, all frequent trajectories in the historical hurricane trajectory database are mined by using association analysis technology and their corresponding association rules are generated as motion patterns. Then, the current hurricane trajectories are matched with the motion patterns for predicting. If no association rule is found for matching, a predicted result according to the hurricane current movement trend would be returned. All experiments are conducted with the Atlantic weather Hurricane/Tropical Data from 1900 to 2008. The experimental results show that if the matching failure part is contained, the prediction accuracy is 57.5 %. Whereas, the valve would be to 65% provided all matches are successful. [ABSTRACT FROM AUTHOR]

We investigate relations between bestselling artists in last decade on phonographic market and from perspective of listeners by using the Social Network Analyzes. Starting network is obtained from the matrix of correlations between the world's bestselling artists by considering the synchronous time evolution of weekly record sales. This method reveals the structure of phonographic market, but we claim that it has no impact on people who see relationship between artists and music genres. We compare "sale" (based on correlation of record sales) or "popularity" (based on data mining of the record charts) networks with "similarity" (obtained mainly from survey within music experts' opinion) and find no significant relations. We postulate that non-laminar phenomena on this specific market introduce turbulence to how people view relations of artists. [ABSTRACT FROM AUTHOR]

This paper describes how data mining is used in cloud computing. Data Mining is used for extracting potentially useful information from raw data. The integration of data mining techniques into normal day-to-day activities has become common place. Every day people are confronted with targeted advertising, and data mining techniques help businesses to become more efficient by reducing costs. Data mining techniques and applications are very much needed in the cloud computing paradigm. The implementation of data mining techniques through Cloud computing will allow the users to retrieve meaningful information from virtually integrated data warehouse that reduces the costs of infrastructure and storage. [ABSTRACT FROM AUTHOR]

We report on the panel discussion held at the ICDM'10 conference on the top 10 data mining case studies in order to provide a snapshot of where and how data mining techniques have made significant real-world impact. The tasks covered by 10 case studies range from the detection of anomalies such as cancer, fraud, and system failures to the optimization of organizational operations, and include the automated extraction of information from unstructured sources. From the 10 cases we find that supervised methods prevail while unsupervised techniques play a supporting role. Further, significant domain knowledge is generally required to achieve a completed solution. Finally, we find that successful applications are more commonly associated with continual improvement rather than by single "aha moments" of knowledge ("nugget") discovery. [ABSTRACT FROM AUTHOR]

As an active research field, database keyword search (KWS) has put much emphasis on the performance issues, due to its high computational cost. However, a closer examination on KWS reveals that there are some other interesting aspects worth noting. In this paper, we examine KWS from a broader perspective, analyzing its profound implications. Freed from syntax-related considerations, KWS users now have better opportunities to explore the data in the way as they wish, and such exploration may reveal useful hindsight for understanding the hidden nature of the data, thus benefiting the exploitation on the use of the data. In particular, we examine the potential of KWS to data mining from the behavior mining perspective. This examination also leads us to a discussion of viewing KWS in the "web of life" context. We further discuss connections of KWS with other related concepts, including dataspace. Based on our findings and critical examination, we also provide a brief overview of an integrated environment which facilitates the interplay of KWS and data mining. [ABSTRACT FROM AUTHOR]

Privacy preserving data mining (PPDM) algorithms attempt to reduce the injuries to privacy caused by malicious parties during the rule mining process. Usually, these algorithms are designed for the semi-honest model, where participants do not deviate from the protocol. However, in the real-world, malicious parties may attempt to obtain the secret values of other parties by probing attacks or collusion. In this study, the authors study how to preserve the privacy of participants in a collusion-free model of the frequent itemset mining process, where the protocol protects against probing attacks and collusion. The mining of frequent itemsets is the main step of association rule mining algorithms, and, in this study, the authors propose two privacy-preserving frequent itemset mining algorithms for both two-party and multi-party states in a collusion-free model for vertically partitioned (heterogeneous) data; in addition, a privacy measuring technique is proposed, which quantifies privacy based on the amount of disclosed sensitive information. [ABSTRACT FROM AUTHOR]

Optimizing the two computing resources of any computing system - time and space - has always been one of the priority objectives of any database. A current and effective solution in this respect is the computer database. Optimizing computer applications by means of database machines has been a steady preoccupation of researchers since the late seventies. Several information technologies have revolutionized the present information framework. Out of these, those which have brought a major contribution to the optimization of the databases are: efficient handling of large volumes of data (Data Warehouse, Data Mining, OLAP - On Line Analytical Processing), the improvement of DBMS - Database Management Systems facilities through the integration of the new technologies, the dramatic increase in computing power and the efficient use of it (computer networks, massive parallel computing, Grid Computing and so on). All these information technologies, and others, have favored the resumption of the research on database machines and the obtaining in the last few years of some very good practical results, as far as the optimization of the computing resources is concerned. [ABSTRACT FROM AUTHOR]

We propose a new method for mining frequent patterns in a language that combines both Semantic Web ontologies and rules. In particular we consider the setting of using a language that combines description logics with DL-safe rules. This setting is important for the practical application of data mining to the Semantic Web. We focus on the relation of the semantics of the representation formalism to the task of frequent pattern discovery, and for the core of our method, we propose an algorithm that exploits the semantics of the combined knowledge base. We have developed a proof-of-concept data mining implementation of this. Using this we have empirically shown that using the combined knowledge base to perform semantic tests can make data mining faster by pruning useless candidate patterns before their evaluation. We have also shown that the quality of the set of patterns produced may be improved: the patterns are more compact, and there are fewer patterns. We conclude that exploiting the semantics of a chosen representation formalism is key to the design and application of (onto-)relational frequent pattern discovery methods. [ABSTRACT FROM AUTHOR]

The summarization system takes a Database (DB) table as input and produces a reduced version of this table through both a rewriting and a generalization process. The resulting table provides records with less precision than the original but it is very informative of the actual DB content. This reduced form can be used as input for advanced Data Mining processes. Several approaches of DB summarization have been proposed in the literature. The most recent is the SaintEtiQ summarization model, proposed initially by Raschia.1 Based on a hierarchical conceptual clustering algorithm, SaintEtiQ builds a summary hierarchy from DB records. In this paper, we propose to extend this DB summarization model by introducing some optimization processes including: (i) minimization of the expert risks domain, (iii) building of the summary hierarchy from DB records, and (iv) cooperation with the user by giving him summaries in different hierarchy levels. [ABSTRACT FROM AUTHOR]

Several classes of scientific and commercial applications require the execution of a large number of independent tasks. One highly successful and low-cost mechanism for acquiring the necessary computing power for these applications is the ‘public-resource computing’, or ‘desktop Grid’ paradigm, which exploits the computational power of private computers. So far, this paradigm has not been applied to data mining applications for two main reasons. First, it is not straightforward to decompose a data mining algorithm into truly independent sub-tasks. Second, the large volume of the involved data makes it difficult to handle the communication costs of a parallel paradigm. This paper introduces a general framework for distributed data mining applications called Mining@home. In particular, we focus on one of the main data mining problems: the extraction of closed frequent itemsets from transactional databases. We show that it is possible to decompose this problem into independent tasks, which however need to share a large volume of the data. We thus introduce a data-intensive computing network, which adopts a P2P topology based on super peers with caching capabilities, aiming to support the dissemination of large amounts of information. Finally, we evaluate the execution of a pattern extraction task on such network. Copyright © 2009 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

The present paper deals with the problem of an assessment of symptoms in medical diagnosis. A unified interpretation of symptoms is necessary to estimate their significance in a diagnosis. Yet, even if they are properly defined, different evaluations of them based on experts' knowledge or statistical estimation are possible. The present study aims at combining evaluations that may originate from an expert or can be found from statistical features of the data, as well as those determined for ‘easy’ and ‘difficult’ diagnostic cases. A model of diagnostic inference is proposed in the framework of the Dempster–Shafer theory extended for fuzzy focal elements. The basic probability assignment defined in this theory estimates weights of symptoms. Two basic probability assignments can be created and then combined. In this way weights of symptoms represent knowledge common for two kinds of data or obtained from an expert and from data. Thus, a combination of heuristics and data mining results becomes possible. An algorithm of the basic probability assignment calculation is suggested and tested for medical data: a database from the internet and individually gathered data. [ABSTRACT FROM AUTHOR]

Abstract-A transaction database usually consists of a set of time-stamped transactions. Mining frequent patterns in transaction databases has been studied extensively in data mining research. However, most of the existing frequent pattern mining algorithms (such as Apriori and FP-growth) do not consider the time stamps associated with the transactions. In this paper, we extend the existing frequent pattern mining framework to take into account the time stamp of each transaction and discover patterns whose frequency dramatically changes over time. We define a new type of patterns, called transitional patterns, to capture the dynamic behavior of frequent patterns in a transaction database. Transitional patterns include both positive and negative transitional patterns. Their frequencies increase/decrease dramatically at some time points of a transaction database. We introduce the concept of significant milestones for a transitional pattern, which are time points at which the frequency of the pattern changes most significantly. Moreover, we develop an algorithm to mine from a transaction database the set of transitional patterns along with their significant milestones. Our experimental studies on real-world databases illustrate that mining positive and negative transitional patterns is highly promising as a practical and useful approach for discovering novel and interesting knowledge from large databases. [ABSTRACT FROM AUTHOR]

Recently, high utility pattern (HUP) mining is one of the most important research issues in data mining due to its ability to consider the nonbinary frequency values of items in transactions and different profit values for every item. On the other hand, incremental and interactive data mining provide the ability to use previous data structures and mining results in order to reduce unnecessary calculations when a database is updated, or when the minimum threshold is changed. In this paper, we propose three novel tree structures to efficiently perform incremental and interactive HUP mining. The first tree structure, Incremental HUP Lexicographic Tree (IHUPL-Tree), is arranged according to an item's lexicographic order. It can capture the incremental data without any restructuring operation. The second tree structure is the IHUP Transaction Frequency Tree (IHUPTF-Tree), which obtains a compact size by arranging items according to their transaction frequency (descending order). To reduce the mining time, the third tree, IHUP-Transaction-Weighted Utilization Tree (IHUPTWU-Tree) is designed based on the TWU value of items in descending order. Extensive performance analyses show that our tree structures are very efficient and scalable for incremental and interactive HUP mining. [ABSTRACT FROM AUTHOR]

In this paper, we consider the problem of unsupervised discrete feature selection/weighting. Indeed, discrete data are an important component in many data mining, machine learning, image processing, and computer vision applications. However, much of the published work on unsupervised feature selection has concentrated on continuous data. We propose a probabilistic approach that assigns relevance weights to discrete features that are considered as random variables modeled by finite discrete mixtures. The choice of finite mixture models is justified by its flexibility which has led to its widespread application in different domains. For the learning of the model, we consider both Bayesian and information-theoretic approaches through stochastic complexity. Experimental results are presented to illustrate the feasibility and merits of our approach on a difficult problem which is clustering and recognizing visual concepts in different image data. The proposed approach is successfully applied also for text clustering. [ABSTRACT FROM AUTHOR]

The sheer size of all frequent itemsets is one challenging problem in data mining research. Based on both closed itemset and maximal itemset, meta itemset which is a new concise representation of frequent itemset is proposed. It is proved that both closed itemset and maximal itemset are special cases of meta itemset. The set of all closed itemsets and the set of all maximal itemsets form the upper bound and the lower bound of the set of all meta itemsets. Then, property and pruning strategies of meta itemset are discussed. Finally, an efficient algorithm for mining meta itemset is proposed. Experimental results show that the proposed algorithm is effective and efficient. [ABSTRACT FROM AUTHOR]

The mining of frequent patterns is a basic problem in data mining applications. Frequent maximal and closed itemsets mining has become an important alternative of association rule mining. In this paper we present an effective algorithm which based on the blanket approach for mining all frequent maximal & closed itemsets. The performance of the proposed algorithm had been compared with recently developed algorithms. The results show how the proposed algorithm gives better performance. This is achieved by examining the performance and functionality of the proposed technique. [ABSTRACT FROM AUTHOR]

The paper presents problems pertaining to spatial data mining. Based on the existing solutions a new method of knowledge extraction in the form of spatial association rules and collocations has been worked out and is proposed herein. Delaunay diagram is used for determining neighborhoods. Based on the neighborhood notion, spatial association rules and collocations are defined. A novel algorithm for finding spatial rules and collocations has been presented. The approach allows eliminating the parameters defining neighborhood of objects, thus avoiding multiple “test and trial” repetitions of the process of mining for various parameter values. The presented method has been implemented and tested. The results of the experiments have been discussed. [ABSTRACT FROM AUTHOR]

Abstract: Airborne separation assurance is a key requirement for Free Flight operations. A variety of conflict detection (CD) and resolution algorithms have been developed for this task. A lack of rigorous evaluation and the existence of an infinite number of possible conflict geometries in Free Flight makes the choice of which algorithm to be placed in the cockpit a challenging task for the designers of future air traffic management (ATM) systems. In this paper, we propose an ensemble approach for CD in Free Flight. The ensemble consists of several CD algorithms, a rule set for each algorithm describing its learned behavior from its past performance and a switch mechanism to choose an appropriate CD algorithm given probe characteristics. A novel mechanism to evolve complex conflict scenarios, using genetic algorithms (GA), is developed and integrated in a fast time air traffic simulator to generate the performance data of CD algorithms. Data mining techniques are then employed to identify implicit patterns in the probe characteristics where the CD algorithms missed or falsely identified a conflict. These patterns are formulated as rule sets for each CD algorithm and are then used by a switch in the ensemble to route a probe for conflict prediction. Given probe characteristics, the CD algorithm, which is less likely to miss or falsely identify a conflict, is selected to evaluate the probe for potential conflict. The performance of the ensemble and of individual algorithms is evaluated by comparing the Pareto efficient set of solutions generated by them. The ensemble approach demonstrates a significant reduction in the number of missed detects and false alarms as compared to individual algorithms. The proposed methodology is capable of accommodating existing as well as new CD models and can be extended to other ATM concepts as well. [Copyright &y& Elsevier]

In recent years, a number of indirect data collection methodologies have led to the proliferation of uncertain data. Such databases are much more complex because of the additional challenges of representing the probabilistic information. In this paper, we provide a survey of uncertain data mining and management applications. We will explore the various models utilized for uncertain data representation. In the field of uncertain data management, we will examine traditional database management methods such as join processing, query processing, selectivity estimation, OLAP queries, and indexing. In the field of uncertain data mining, we will examine traditional mining problems such as frequent pattern mining, outlier detection, classification, and clustering. We discuss different methodologies to process and mine uncertain data in a variety of forms. [ABSTRACT FROM AUTHOR]

Abstract: Vulnerabilities in common security components such as firewalls are inevitable. Intrusion Detection Systems (IDS) are used as another wall to protect computer systems and to identify corresponding vulnerabilities. In this paper, a novel framework based on data mining techniques is proposed for designing an IDS. In this framework, the classification engine, which is actually the core of the IDS, uses Association Based Classification (ABC). The proposed classification algorithm uses fuzzy association rules for building classifiers. Particularly, the fuzzy association rulesets are exploited as descriptive models of different classes. The compatibility of any new sample (which is to be classified) with different class rulesets is assessed by the use of some matching measures and the class corresponding to the best matched ruleset is declared as the label of the sample. A new method is also proposed to speed up the rule induction algorithm via reducing items that may be included in extracted rules. KDD-99 dataset is used to evaluate the proposed framework. Although results on unseen attacks are not so promising, total detection rate and detection rate of known attacks is significant while false positive rate is kept low. Results are compared with some recent works in the literature using the same dataset. Generally, the proposed approach outperforms other methods, specially in terms of false positive rate. [Copyright &y& Elsevier]

Information is a key element of organizational processes. In recent years, the technology has had an accelerated growth, in line with the increase of what are regarded as useful and necessary tools to facilitate and expedite those processes giving an added value to productivity. Business intelligence has been defined as the transforming of data into knowledge, providing decision-making support at the strategic and tactical level where and when appropriate, providing a competitive advantage and increasing the effectiveness. One of the tools that has become useful for the exploration of data is On-line Analytical Processing (OLAP), which allows to obtain outstanding data among quantities of information, but it is faulty for the analysis of geographical data, for which SOLAP has arisen, which offers methods of special treatment for space data. Data mining has been adapted within companies, with the purpose of carrying out exploration and analysis of data focused on the discovery of knowledge. Because of the important place that space information is occupying nowadays, the spatial data mining has arisen. This process allows us to discover useful and unexpected patterns inside the data. The techniques of spatial data mining are applied to extract knowledge, starting from large volumes of data, which can be of space and non-space types. Among them are generalization, grouping, and space association. [ABSTRACT FROM AUTHOR]

Clustering is a widely used technique in data mining application for discovering patterns in underlying data. Most traditional clustering algorithms are limited in handling datasets that contain categorical attributes. However, datasets with categorical types of attributes are common in real life data mining problem. For these data sets, no inherent distance measure, like the Euclidean distance, would work to compute the distance between two categorical objects. In this article, we have described two algorithms based on genetic algorithm and simulated annealing in the field of crisp and fuzzy domain. The performance of the proposed algorithms has been compared with that of different well known categorical data clustering algorithms in crisp and fuzzy domain and demonstrated for a variety of artificial and real life categorical data sets. Also statistical significance tests have been performed to establish the superiority of the proposed algorithms. [ABSTRACT FROM AUTHOR]

Ontology has recently received considerable attention in tile know/edge management community. This article discusses the needs of ontology development data mining. Based on a domain analysis of knowledge representations in data mining, it proposes a generic structure of ontologies for data mining. Furthermore, this article specifies the unique ontology resources of the subdomain of innovative data mining with incomplete data. A project on an ontology-based data mining system for discovering knowledge from incomplete data demonstrates time effectiveness of ontology in knowledge management. [ABSTRACT FROM AUTHOR]

Abstract: The discovery of dependencies between attributes in databases is an important problem in data mining, and can be applied to facilitate future decision-making. In the present paper some properties of the branching dependencies are examined. We define a minimal branching dependency and we propose an algorithm for finding all minimal branching dependencies between a given set of attributes and a given attribute in a relation of a database. Our examination of the branching dependencies is motivated by their application in a database storing realized sales of products. For example, finding out that arbitrary p products have totally attracted at most q new users can prove to be crucial in supporting the decision making. In addition, we also consider the fractional and the fractional branching dependencies. Some properties of these dependencies are examined. An algorithm for finding all fractional dependencies between a given set of attributes and a given attribute in a database relation is proposed. We examine the general case of an arbitrary relation, as well as a particular case where the problem of discovering the fractional dependencies is considerably simplified. [Copyright &y& Elsevier]

Abstract-In this paper, we propose a distributed parallel support vector machine (DPSVM) training mechanism in a configurable network environment for distributed data mining. The basic idea is to exchange support vectors among a strongly connected network (SCN) so that multiple servers may work concurrently on distributed data set with limited communication cost and fast training speed. The percentage of servers that can work in parallel and the communication overhead may be adjusted through network configuration. The proposed algorithm further speeds up through online implementation and synchronization. We prove that the global optimal classifier can be achieved iteratively over an SCN. Experiments on a real-world data set show that the computing time scales well with the size of the training data for most networks. Numerical results show that a randomly generated SCN may achieve better performance than the state of the art method, cascade SVM, in terms of total training time. [ABSTRACT FROM AUTHOR]

A Bayesian network (BN) is a formalism for representing and reasoning about uncertain domains. In BNs the knowledge is represented by a combination of a graph-based structure and probability theory. A particular type of BN known as Bayesian Classifier (BC) aims at classifying a given instance into a discrete class. BCs have been intensively used for knowledge modeling in many different applications and have been the focus of many works related to data mining. Data mining tasks are usually applied to real domains having large number of variables. In such domains, the classifiers tend to be large and complex and consequently are not so easily understood by human beings. This paper proposes an approach for promoting the understandability of the knowledge represented by a BC, by translating it into a more convenient and easily understandable form of representation, that of classification rules. The proposed method named BayesRule uses the concept of Markov-Blanket to obtain a reduced set of rules in relation to both, the number of rules and the number of conditions in the antecedent of a rule. Experiments using seven knowledge domains show that the reduced set of rules extracted from a BC can be smaller and still maintain the BC classification accuracy. [ABSTRACT FROM AUTHOR]

The use of online analytical processing (OLAP) systems as data sources for data mining techniques has been widely studied and has resulted in what is known as online analytical mining (OLAM). As a result of both the use of OLAP technology in new fields of knowledge and the merging of data from different sources, it has become necessary for models to support imprecision. We, therefore, need OLAM methods which are able to deal with this imprecision. Association rules are one of the most used data mining techniques. There are several proposals that enable the extraction of association rules on DataCubes but few of these deal with imprecision in the process. The main problem observed in these proposals is the complexity of the rule set obtained. In this paper, we present a novel association rule extraction method that works over a fuzzy multidimensional model which is capable of representing and managing imprecise data. Our method deals with the problem of reducing the complexity of the result obtained by using fuzzy concepts and a hierarchical relation between them. [ABSTRACT FROM AUTHOR]

For one to infer the structures of a gene regulatory network (GRN), it is important to identify, for each gene in the GRN, which other genes can affect its expression and how they can affect it. For this purpose, many algorithms have been developed to generate hypotheses about the presence or absence of interactions between genes. These algorithms, however, cannot be used to determine if a gene activates or inhibits another. To obtain such information to better infer GRN structures, we propose a fuzzy data mining technique here. By transforming quantitative expression values into linguistic terms, it defines a measure of fuzzy dependency among genes. Using such a measure, the technique is able to discover interesting fuzzy dependency relationships in noisy, high dimensional time series expression data so that it can not only determine if a gene is dependent on another but also if a gene is supposed to be activated or inhibited. In addition, the technique can also predict how a gene in an unseen sample (i.e., expression data that are not in the original database) would be affected by other genes in it and this makes statistical verification of the reliability of the discovered gene interactions easier. For evaluation, the proposed technique has been tested using real expression data and experimental results show that the use of fuzzy-logic based technique in gene expression data analysis can be quite effective. [ABSTRACT FROM AUTHOR]

Abstract: Discovering patterns with great significance is an important problem in data mining discipline. An episode is defined to be a partially ordered set of events for consecutive and fixed-time intervals in a sequence. Most of previous studies on episodes consider only frequent episodes in a sequence of events (called simple sequence). In real world, we may find a set of events at each time slot in terms of various intervals (hours, days, weeks, etc.). We refer to such sequences as complex sequences. Mining frequent episodes in complex sequences has more extensive applications than that in simple sequences. In this paper, we discuss the problem on mining frequent episodes in a complex sequence. We extend previous algorithm MINEPI to for episode mining from complex sequences. Furthermore, a memory-anchored algorithm called EMMA is introduced for the mining task. Experimental evaluation on both real-world and synthetic data sets shows that EMMA is more efficient than . [Copyright &y& Elsevier]

Business Intelligence (BI) consists of tools and technologies to access and analyze enterprise information in a useful way to make better business decisions. BI technologies include query, reporting and analysis tools, On-Line Analytical Processing (OLAP), decision support systems, analytics, dashboards, data mining, enterprise performance management, balanced scorecards, predictive analytics. This paper describes BI, the benefits of BI, its applications in various industry verticals, and popular BI tools in the market and their capabilities. Some of the areas where BI finds application are customer segment analysis, compliance reporting and trend analysis. [ABSTRACT FROM AUTHOR]

Abstract: Data mining technique is capable of mining valuable knowledge from large and changeable database. This paper puts forward a data mining method combining attribute-oriented induction, information gain, and decision tree, which is suitable for preprocessing financial data and constructing decision tree model for financial distress prediction. On the base of financial ratios attributes and one class attribute, adopting entropy-based discretization method, a data mining model for listed companies’ financial distress prediction is designed. The empirical experiment with 35 financial ratios and 135 pairs of listed companies as initial samples got satisfying result, which testifies the feasibility and validity of the proposed data mining method for listed companies’ financial distress prediction. [Copyright &y& Elsevier]

Abstract: Processing and extracting meaningful knowledge from count data is an important problem in data mining. The volume of data is increasing dramatically as the data is generated by day-to-day activities such as market basket data, web clickstream data or network data. Most mining and analysis algorithms require multiple passes over the data, which requires extreme amounts of time. One solution to save time would be to use samples, since sampling is a good surrogate for the data and the same sample can be used to answer many kinds of queries. In this paper, we propose two deterministic sampling algorithms, Biased-L2 and DRS. Both produce samples vastly superior to the previous deterministic and random algorithms, both in sample quality and accuracy. Our algorithms also improve on the run-time and memory footprint of the existing deterministic algorithms. The new algorithms can be used to sample from a relational database as well as data streams, with the ability to examine each transaction only once, and maintain the sample on-the-fly in a streaming fashion. We further show how to engineer one of our algorithms (DRS) to adapt and recover from changes to the underlying data distribution, or sample size. We evaluate our algorithms on three different synthetic datasets, as well as on real-world clickstream data, and demonstrate the improvements over previous art. [Copyright &y& Elsevier]

Abstract: Many Web sites publish their data in a hierarchical structure. For instance, Amazon.com organizes its pages on books as a hierarchy, in which each leaf node corresponds to a collection of pages of books in the same class (e.g., books on Data Mining). Users can easily browse this class by following a path from the root to the corresponding leaf node, such as “Computers &Internet –Databases –Storage –Data Mining”. Business applications often require to submit aggregation queries on such data, such as “finding the average price of books on Data Mining”. On the other hand, it is computationally expensive to compute the exact answer to such a query due to the large amount of data, its dynamicity, and limited Web-access resources. In this paper, we study how to answer such aggregation queries approximately with quality guarantees using sampling. We study how to use adaptive-sampling techniques that allocate the resources adaptively based on partial samples retrieved from different nodes in the hierarchy. Based on statistical methods, we study how to estimate the quality of the answer using the sample. Our experimental study using real and synthetic data sets validates the proposed techniques. [Copyright &y& Elsevier]

Data mining is commonly used in attempts to induce association rules from transaction data. Most previous studies focused on binary-valued transaction data. Transactions in real-world applications, however, usually consist of quantitative values. In the past, we proposed a fuzzy-genetic data-mining algorithm for extracting both association rules and membership functions from quantitative transactions. It used a combination of large 1-itemsets and membership-function suitability to evaluate the fitness values of chromosomes. The calculation for large 1-itemsets could take a lot of time, especially when the database to be scanned could not totally fed into main memory. In this paper, an enhanced approach, called the cluster-based fuzzy-genetic mining algorithm, is thus proposed to speed up the evaluation process and keep nearly. the same quality of solutions as the previous one. It divides the chromosomes in a population into k clusters by the k-means clustering approach and evaluates each individual according to both cluster and their own information. Experimental results also show the effectiveness and efficiency of the proposed approach. [ABSTRACT FROM AUTHOR]

Abstract: Most of existing text clustering algorithms use the vector space model, which treats documents as bags of words. Thus, word sequences in the documents are ignored, while the meaning of natural languages strongly depends on them. In this paper, we propose two new text clustering algorithms, named Clustering based on Frequent Word Sequences (CFWS) and Clustering based on Frequent Word Meaning Sequences (CFWMS). A word is the word form showing in the document, and a word meaning is the concept expressed by synonymous word forms. A word (meaning) sequence is frequent if it occurs in more than certain percentage of the documents in the text database. The frequent word (meaning) sequences can provide compact and valuable information about those text documents. For experiments, we used the Reuters-21578 text collection, CISI documents of the Classic data set [Classic data set, ftp://ftp.cs.cornell.edu/pub/smart/], and a corpus of the Text Retrieval Conference (TREC) [High Accuracy Retrieval from Documents (HARD) Track of Text Retrieval Conference, 2004]. Our experimental results show that CFWS and CFWMS have much better clustering accuracy than Bisecting k-means (BKM) [M. Steinbach, G. Karypis, V. Kumar, A Comparison of Document Clustering Techniques, KDD-2000 Workshop on Text Mining, 2000], a modified bisecting k-means using background knowledge (BBK) [A. Hotho, S. Staab, G. Stumme, Ontologies improve text document clustering, in: Proceedings of the 3rd IEEE International Conference on Data Mining, 2003, pp. 541–544] and Frequent Itemset-based Hierarchical Clustering (FIHC) [B.C.M. Fung, K. Wang, M. Ester, Hierarchical document clustering using frequent itemsets, in: Proceedings of SIAM International Conference on Data Mining, 2003] algorithms. [Copyright &y& Elsevier]

Representation of uncertain knowledge by using a Bayesian network requires the acquisition of a conditional probability table (CPT) for each variable. The CPT can be acquired by data mining or elicitation. When data are insufficient for supporting mining, causal modeling such as the noisy-OR aids elicitation by reducing the number of probability parameters to be acquired from human experts. Multiple causes can reinforce each other in producing the effect or can undermine the impact of each other. Most existing causal models do not consider causal interactions from the perspective of reinforcement or undermining. Our analysis shows that none can represent both interactions. Except for the RNOR, other models also limit parameters to probabilities of single-cause events. We present the first general causal model, that is, the nonimpeding noisy-AND tree, that allows encoding of both reinforcement and undermining. It supports efficient CPT acquisition by elicitating a partial ordering of causes in terms of a tree topology, plus the necessary numerical parameters. It also allows the incorporation of probabilities for multicause events. [ABSTRACT FROM AUTHOR]

One of the tasks of data mining is classification, which provides a mapping from attributes (observations) to pre-specified classes. Classification models are built by using underlying data. In principle, the models built with more data yield better results. However, the relationship between the available data and the performance is not well understood, except that the accuracy of a classification model has diminishing improvements as a function of data size. In this paper, we present an approach for an early assessment of the extracted knowledge (classification models) in the terms of performance (accuracy), based on the amount of data used. The assessment is based on the observation of the performance on smaller sample sizes. The solution is formally defined and used in an experiment. In experiments we show the correctness and utility of the approach. [ABSTRACT FROM AUTHOR]

This article introduces a nested clustering technique and its application to the analysis of freeway operating condition. A clustering model is developed using the traffic data (flow, speed, occupancy) collected by the detectors and aggregated to 5-minute increments. An optimum fit of the statistical characteristics of the data set is provided by the model based on the Bayesian Information Criterion and the ratio of changes in dispersion measurement. This technique is flexible in determining the number of clusters based on the statistical characteristics of the data. Tests on multiple sites with varying operating conditions have attested to its effectiveness as a data mining tool for the analysis of freeway operating condition. [ABSTRACT FROM AUTHOR]

Frequent pattern mining has been a focused theme in datamining research for over a decade. Abundant literature has been dedicated to this research and tremendous progress has been made, ranging from efficient and scalable algorithms for frequent itemset mining in transaction databases to numerous research frontiers, such as sequential pattern mining, structured pattern mining, correlation mining, associative classification, and frequent pattern-based clustering, as well as their broad applications. In this article, we provide a brief overview of the current status of frequent pattern mining and discuss a few promising research directions. We believe that frequent pattern mining research has substantially broadened the scope of data analysis and will have deep impact on data mining methodologies and applications in the long run. However, there are still some challenging research issues that need to be solved before frequent pattern mining can claim a cornerstone approach in data mining applications. [ABSTRACT FROM AUTHOR]

In this paper a novel restoration algorithm based on iterative loops and data mining is proposed to restore the original object image from a few frames of aero-optics degraded images through the turbulence medium. Based on the iterative loops, the iterative mathematic models to estimate the random turbulent optical point spread functions and object image are built. A series of restoration experiments for both simulated and real aero-optics degraded images were performed to examine the proposed algorithm under a computerized simulation environment, which show that the proposed algorithm outperform previous algorithms and therefore is effective for practical applications. [ABSTRACT FROM AUTHOR]

Due to the ability of graphs to represent more generic and more complicated relationships among different objects, graph mining has played a significant role in data mining, attracting increasing attention in the data mining community. In addition, frequent coherent subgraphs can provide valuable knowledge about the underlying internal structure of a graph database, and mining frequently occurring coherent subgraphs from large dense graph databases has witnessed several applications and received considerable attention in the graph mining community recently. In this article, we study how to efficiently mine the complete set of coherent closed quasi-cliques from large dense graph databases, which is an especially challenging task due to the fact that the downward-closure property no longer holds. By fully exploring some properties of quasi-cliques, we propose several novel optimization techniques which can prune the unpromising and redundant subsearch spaces effectively. Meanwhile, we devise an efficient closure checking scheme to facilitate the discovery of closed quasi-cliques only. Since large databases-cannot be held in main memory, we also design an out-of-core solution with efficient index structures for mining coherent closed quasi-cliques from large dense databases. We call this Cocain. Thorough performance study shows that Cocain is very efficient and scalable for large dense graph databases. [ABSTRACT FROM AUTHOR]

The ability to approximately answer aggregation queries accurately and efficiently is of great benefit for decision support and data mining tools. In contrast to previous sampling-based studies, we treat the problem as an optimization problem where, given a workload of queries, we select a stratified random sample of the original data such that the error in answering the workload queries using the sample is minimized. A key novelty of our approach is that we can tailor the choice of samples to be robust, even for workloads that are "similar" but not necessarily identical to the given workload. Finally, our techniques recognize the importance of taking into account the variance in the data distribution in a principled manner. We show how our solution can be implemented on a database system, and present results of extensive experiments on Microsoft SQL Server that demonstrate the superior quality of our method compared to previous work. [ABSTRACT FROM AUTHOR]

Abstract: On account of the enormous amounts of rules that can be produced by data mining algorithms, knowledge post-processing is a difficult stage in an association rule discovery process. In order to find relevant knowledge, the user needs to rummage through the rules. To make this task easier, we propose a new interactive mining methodology based on well adapted dynamic visual representations. It allows the user to drive the discovery process by focusing his/her attention on limited subsets of rules. We have implemented our methodology with two complementary 2D and 3D visualization supports. These implementations exploit the user''s focus to guide the generation of the rules by means of a specific constraint-based rule-mining algorithm. [Copyright &y& Elsevier]