Search

Large repositories of data contain sensitive information that must be protected against unauthorized access. The protection of the confidentiality of this information has been a long-term goal for the database security research community and for the government statistical agencies. Recent advances in data mining and machine learning algorithms have increased the disclosure risks that one may encounter when releasing data to outside parties. A key problem, and still not sufficiently investigated, is the need to balance the confidentiality of the disclosed data with the legitimate needs of the data users. Every disclosure limitation method affects, in some way, and modifies true data values and relationships. In this paper, we investigate confidentiality issues of a broad category of rules, the association rules. In particular, we present three strategies and five algorithms for hiding a group of association rules, which is characterized as sensitive. One rule is characterized as sensitive if its disclosure risk is above a certain privacy threshold. Sometimes, sensitive rules should not be disclosed to the public since, among other things, they may be used for inferring sensitive data, or they may provide business competitors with an advantage. We also perform an evaluation study of the hiding algorithms in order to analyze their time complexity and the impact that they have in the original database. Index Terms--Privacy preserving data mining, association rule mining, sensitive rule hiding.

The traditional approach for estimating the performance of numerical methods is to combine an operation's count with an asymptotic error analysis. This analytic approach gives a general feel of the comparative efficiency of methods, but it rarely leads to very precise results. It is now recognized that accurate performance evaluation can be made only with actual measurements on working software. Given that such an approach requires an enormous amount of performance data related to actual measurements, the development of novel approaches and systems that intelligently and efficiently analyze these data is of great importance to scientists and engineers. This paper presents new intelligent knowledge acquisition approaches and an integrated prototype system, which enables the automatic and systematic analysis of performance data. The system analyzes the performance data which is usually stored in a database with statistical, and inductive learning techniques and generates knowledge which can be incorporated in a knowledge base incrementally. We demonstrate the use of the system in the context of a case study, covering the analysis of numerical algorithms for the pricing of American vanilla options in a Black and Scholes modeling framework. We also present a qualitative and quantitative comparison of two techniques used for the automated knowledge acquisition phase. Although the system is presented with a particular pricing library in mind, the analysis and evaluation methodology can be used to study algorithms available from other libraries, as long as, these libraries can provide the necessary performance data. Index Terms--Data mining, decision trees, inductive logic programming.

Attrition is one of the main concerns in distance learning due to the impact on the incomes and institutions reputation. Timely identification of students at risk has high practical value in effective students' retention services. Big Data mining and machine learning methods are applied to manipulate, analyze and predict students' failure, supporting self-directed learning. Despite the extensive application of data mining to education, the imbalance problem in minority classes of students' attrition is often overlooked in conventional models. This document proposes a large data frame using the Hadoop ecosystem and the application of machine learning techniques to different datasets of an academic year at the Hellenic Open University. Datasets were divided into thirty-five weeks. Thirty-two classifiers were created, compared and statistical analyzed to address the minority classes' imbalance of student's failure. The algorithms metacost-SMO, and C4.5 provide the most accurate performance for each target class. Early predictions of timeframes determine a remarkable performance, while the importance of written assignments and specific quizzes is noticeable. The models' performance in any week is exploited by developing a prediction tool for student attrition, contributing to timely and personalized intervention. [ABSTRACT FROM AUTHOR]

Data sharing has become an increasingly common process among health organizations, but any organization will most likely try to hide some sensitive patterns before sharing its data with others. Local Distortion Hiding (LDH), a recently proposed algorithm, has been evaluated only on the assumption of an opponent using the J48 (C4.5) classification algorithm. We now extend the basic approach, and we present a medical dataset hiding case study of a processed by LDH and attacked with the CART algorithm. [ABSTRACT FROM AUTHOR]

Data sharing among health organizations has become an increasingly common process, but any organization will most likely try to hide some sensitive patterns before it shares its data with others. This article focuses on the protection of sensitive patterns when we assume that decision trees will be the models to be induced. We apply a heuristic approach to hide any arbitrary rule from the derivation of a binary decision tree. The proposed hiding method is preferred over other heuristic solutions such as output disturbance or encryption methods that limit data usability, as the raw data itself can then more easily be offered for access by any third parties. [ABSTRACT FROM AUTHOR]

The process of integrating large volumes of data coming from disparate data sources, in order to detect records that refer to the same entities, has always been an important problem in both academia and industry. This problem becomes significantly more challenging when the integration involves a huge amount of records and needs to be conducted in a real-time fashion to address the requirements of critical applications. In this paper, we propose two novel schemes for online record linkage, which achieve very fast response times and high levels of recall and precision. Our proposed schemes embed the records into a Bloom filter space and employ the Hamming Locality-Sensitive Hashing technique for blocking. Each Bloom filter is hashed to a number of hash tables in order to amplify the probability of formulating similar Bloom filter pairs. The main theoretical premise behind our first scheme relies on the number of times a Bloom filter pair is formulated in the hash tables of the blocking mechanism. We prove that this number strongly depends on the distance of that Bloom filter pair. This correlation allows us to estimate in real-time the Hamming distances of Bloom filter pairs without performing the comparisons. The second scheme is progressive and achieves high recall, upfront during the linkage process, by continuously adjusting the sequence in which the hash tables are scanned, and also guarantees, with high probability, the identification of each similar Bloom filter pair. Our experimental evaluation, using four real-world data sets, shows that the proposed schemes outperform four state-of-the-art methods by achieving higher recall and precision, while being very efficient. [ABSTRACT FROM AUTHOR]