Search

Duplicate record is a common problem within data sets especially in huge volume databases. The accuracy of duplicate detection determines the efficiency of duplicate removal process. However, duplicate detection has become more challenging due to the presence of missing values within the records where during the clustering and matching process, missing values can cause records deemed similar to be inserted into the wrong group, hence, leading to undetected duplicates. In this paper, duplicate detection improvement was proposed despite the presence of missing values within a data set through Duplicate Detection within the Incomplete Data set (DDID) method. The missing values were hypothetically added to the key attributes of three data sets under study, using an arbitrary pattern to simulate both complete and incomplete data sets. The results were analyzed, then, the performance of duplicate detection was evaluated by using the Hot Deck method to compensate for the missing values in the key attributes. It was hypothesized that by using Hot Deck, duplicate detection performance would be improved. Furthermore, the DDID performance was compared to an early duplicate detection method namely DuDe, in terms of its accuracy and speed. The findings yielded that even though the data sets were incomplete, DDID was able to offer a better accuracy and faster duplicate detection as compared to DuDe. The results of this study offer insights into constraints of duplicate detection within incomplete data sets. [ABSTRACT FROM AUTHOR]

Partially missing data sets are a prevailing problem in clustering analysis. We propose a hybrid algorithm combining fuzzy clustering with particle swarm optimization (PSO) for incomplete data clustering, and missing attributes are represented as intervals. Furthermore, we develop a neighbor interval reconstruction (NIR) method based on pre-classification results that estimates the nearest-neighbor interval of missing attribute using the nearest-neighbor rule, which avoids endpoints of intervals determined by different species information, thereby improving the accuracy of missing attribute intervals and enhancing the robustness of missing attribute imputation. Then, the PSO and fuzzy c-means hybrid algorithm are used for clustering the interval-valued data set, and the global optimization ability of the PSO can improve the accuracy of clustering results compared with gradient-based optimization methods. The experimental results for several UCI data sets show the superiority of the proposed NIR hybrid algorithm. [ABSTRACT FROM AUTHOR]

In this article we assess the potential for developing mimics in situations where detailed data on the ecophysiology and ecosystem function of the native ecosystem are not available. We examine two ecosystems in south western Australia, and draw together available information on these. We discuss woodlands dominated by Eucalyptus salmonophloia, which formerly covered much of the valley bottom land in the Western Australian wheatbelt, and kwongan or heath which occurs on sandplain soils at the top of the landscape. From both of these we derive possible mimics which utilise aspects of the native vegetation which wehypothesise contributed to ecosystem function and resilience, but which still allow the production of a harvestable agricultural product.We conclude that situations for which we have adequate ecological knowledge from which to develop sound mimics are likely to be rare. We hence have to be prepared to make best-bet estimates and use these todevelop experimental mimic systems. [ABSTRACT FROM AUTHOR]