Search

Collaborating filtering (CF) has become one of the most powerful approaches in the recommender system. Neighborhood-based CF uses a similarity measure to identity neighbors of an active user, and these neighbors play an essential role in the personalized recommendation. Recently introduced new heuristic similarity measure (NHSM) based CF is found to be performing well compared to the CF approaches, which use traditional measures like Pearson correlation coefficient (PCC), proximity impact popularity (PIP), etc. However, NHSM is not appropriately normalized, and it may mislead in finding neighbors in specific scenarios. In this paper, we propose an improved NHSM similarity measure to excel in the recommendation by overcoming the shortfall of NHSM. We propose to utilize hyperbolic trigonometric function for the normalization of each component of NHSM. Relative difference (RD) is exploited to address the misleading problem of NSHM. Experimental results demonstrate that our improved NHSM (i-NHSM) based CF outperforms NHSM based CF.

The proliferation of GPS enabled devices has led people to share locations both consciously and unconsciously. Large spatio-temporal data comprising of shared locations and whereabouts are now being routinely collected for analysis. As user movements are generally driven by their interests, so mining these mobility patterns can reveal commonalities between a pair of users. In this paper, we present a framework for mining the published trajectories to identify patterns in user mobility. In this framework, we extract the locations where a user stays for a period of time popularly known as stay points. These stay points help to identify the interests of a user. The statistics of pattern and check-in distributions over the GPS data are used to formulate similarity measures for finding K-nearest neighbors of an active user. In this work, we categorize the neighbors into three groups namely strongly similar, closely similar and weakly similar. We introduce three similarity measures to determine them, one for each of the categories. We perform experiments on a real-world GPS log data to find the similarity scores between a pair of users and subsequently find the effective K-neighbors. Experimental results show that our proposed metric outperforms existing metrics in literature.

Outlier detection is an important task as it leads to the discovery of critical information in a variety of the application domains. The variants of k-nearest neighbor based outlier detection method have been successfully applied over decades. However, these approaches have high execution time as they compute a score (known as outlier score) for each data point. In this paper, we propose a method to reduce the execution time of k-nearest neighbor based algorithms. Proposed method quickly identifies the data points which are normal and therefore outlier score for such points need not be computed in further processing. The proposed method is generic and can be applied to improve the execution efficiency of many density-based and distance-based outlier detection methods. Proposed work is compared with other existing methods and the result shows that the proposed work outperforms other methods.

Cluster analysis in a large dataset is an interesting challenge in many fields of Science and Engineering. One important clustering approach is hierarchical clustering, which outputs hierarchical (nested) structures of a given dataset. The single-link is a distance-based hierarchical clustering method, which can find non-convex (arbitrary)-shaped clusters in a dataset. However, this method cannot be used for clustering large dataset as this method either keeps entire dataset in main memory or scans dataset multiple times from secondary memory of the machine. Both of them are potentially severe problems for cluster analysis in large datasets. One remedy for both problems is to create a summary of a given dataset efficiently, and the summary is subsequently used to speed up clustering methods in large datasets. In this paper, we propose a summarization scheme termed data sphere (ds) to speed up single-link clustering method in large datasets. The ds utilizes sequential leaders clustering method to collect important statistics of a given dataset. The single-link method is modified to work with ds. Modified clustering method is termed as summarized single-link (SSL). The SSL method is considerably faster than the single-link method applied directly to the dataset, and clustering results produced by SSL method are close to the clustering results produced by single-link method. The SSL method outperforms single-link using data bubble (summarization scheme) both in terms of clustering accuracy and computation time. To speed up proposed summarization scheme, a technique is introduced to reduce a large number of distance computations in leaders method. Experimental studies demonstrate effectiveness of the proposed summarization scheme for large datasets.

Popular non-parametric methods like k-nearest neighbor classifier and density based clustering method like DBSCAN show good performance when data set sizes are large. The time complexity to find a density at a point in the data set is O ( n ) where n is the size of the data set, hence these non-parametric methods are not scalable for large data sets. A two level rough fuzzy weighted leader based classifier has been developed which is a scalable and efficient method for classification. However, a generalized model does not exist to estimate density non-parametrically that can be used for density based classification and clustering. This paper presents a generalized model which proposes a single level rough fuzzy weighted leader clustering method to condense data set inorder to reduce computational burden and use these rough-fuzzy weighted leaders to estimate density at a point in the data set for classification and clustering. We show that the proposed rough fuzzy weighted leader based non-parametric methods are fast and efficient when compared with related existing methods interms of accuracy and computational time.

Recommender system (RS) is an important instrument in e-commerce, which provides personalized recommendations to individual user. Classical algorithms in recommender system mainly emphasize on recommendation accuracy in order to match individual user's past profile. However, recent study shows that novelty and diversity in recommendations are equally important factors from both user and business view points. In this paper, we introduce a knowledge reuse framework to increase novelty and diversity in the recommended items of individual users while compromising very little recommendation accuracy. The proposed framework uses features information which have already been extracted by an existing collaborative filtering. Experimental results with real datasets show that our approach outperforms state-of-the-art solutions in providing novel and diverse recommended items to individual users and aggregate diversity gain achieved by our approach is on par with recently proposed rank based approach.

Collaborative filtering (CF) is the most successful approach for personalized product or service recommendations. Neighborhood based collaborative filtering is an important class of CF, which is simple, intuitive and efficient product recommender system widely used in commercial domain. Typically, neighborhood-based CF uses a similarity measure for finding similar users to an active user or similar products on which she rated. Traditional similarity measures utilize ratings of only co-rated items while computing similarity between a pair of users. Therefore, these measures are not suitable in a sparse data. In this paper, we propose a similarity measure for neighborhood based CF, which uses all ratings made by a pair of users. Proposed measure finds importance of each pair of rated items by exploiting Bhattacharyya similarity. To show effectiveness of the measure, we compared performances of neighborhood based CFs using state-of-the-art similarity measures with the proposed measured based CF. Recommendation results on a set of real data show that proposed measure based CF outperforms existing measures based CFs in various evaluation metrics.

Clustering has been recognized as one of the important tasks in data mining. One important class of clustering is distance based method. To reduce the computational and storage burden of the classical clustering methods, many distance based hybrid clustering methods have been proposed. However, these methods are not suitable for cluster analysis in dynamic environment where underlying data distribution and subsequently clustering structures change over time. In this paper, we propose a distance based incremental clustering method, which can find arbitrary shaped clusters in fast changing dynamic scenarios. Our proposed method is based on recently proposed al-SL method, which can successfully be applied to large static datasets. In the incremental version of the al-SL (termed as IncrementalSL), we exploit important characteristics of al-SL method to handle frequent updates of patterns to the given dataset. The IncrementalSL method can produce exactly same clustering results as produced by the al-SL method. To show the effectiveness of the IncrementalSL in dynamically changing database, we experimented with one synthetic and one real world datasets.

Discovering clusters of arbitrary shape with variable densities is an interesting challenge in many fields of science and technology. There are few clustering methods, which can detect clusters of arbitrary shape and different densities. However, these methods are very sensitive with parameter settings and are not scalable with large datasets. In this paper, we propose a clustering method, which detects clusters of arbitrary shapes, sizes and different densities. We introduce a parameter termed Nearest Neighbor Factor (NNF) to determine relative position of an object in its neighborhood region. Based on relative position of a point, proposed method expands a cluster recursively or declares the point as outlier. Proposed method outperforms a classical method DBSCAN and recently proposed TI-k-Neighborhood-Index supported NBC method.

Finding clusters in large datasets is an interesting challenge in many fields of Science and Technology. Many clustering methods have been successfully developed over the years. However, most of the existing clustering methods need multiple data scans to get converged. Therefore, these methods cannot be applied for cluster analysis in large datasets. Data summarization can be used as a pre-processing step to speed up classical clustering methods for large datasets. In this paper, we propose a data summarization scheme based on tolerance rough set theory termed rough bubble. Rough bubble utilizes leaders clustering method to collect sufficient statistics of the dataset, which can be used to cluster the dataset. We show that proposed summarization scheme outperforms recently introduced data bubble as a summarization scheme when agglomerative hierarchical clustering (single-link) method is applied to it. We also introduce a technique to reduce the number of distance computations required in leaders clustering method. Experiments are conducted with synthetic and real world datasets which show effectiveness of our methods for large datasets.

The single-link (SL) clustering method is not scalable with the size of the dataset and needs many database scans. This is potentially a severe problem for large datasets. One way to speed up the SL method is to summarize the data efficiently and subsequently apply the SL method to the summary of the data. In this paper, we propose a summarization scheme based on a tolerance rough set theory called data-sphere (DS). The SL method is modified to work with data spheres. The proposed clustering method takes considerably less time compared to the classical single-link method which is applied to the dataset directly. The clustering results produced by the proposed method is very close to that of the SL method. We also show that proposed summarization scheme outperforms recently introduced data bubbles (DB) as a summarization scheme when single-link is applied to it at clustering quality. Experiments are conducted with two synthetic and two real world datasets to show effectiveness of the proposed method.

The hierarchical clustering methods are not scalable with the size of the dataset and need many database scans. This is potentially a severe problem for large datasets. One way to speed up the hierarchical methods is to summarize the data efficiently and subsequently apply the clustering methods to the summary of the data. In this paper, we propose a new scheme to summarize the dataset called data sphere (DS). data sphere (DS) collects sufficient statistics applying the leaders clustering method twice on the dataset. The single-link clustering method which is a well known hierarchical clustering method is modified to work with data spheres. The proposed method is called DS-SL method.The DS-SL takes considerably less time compared to the classical single-link method which is applied on the dataset directly. The clustering results produced by DS-SL is very close to the single-link method. We also show that DS-SL outperforms the single-link method using recently proposed data bubble (DB) as a summarization scheme, both at clustering quality and execution time. Experiments are conducted with two synthetic and three real world datasets which shows effectiveness of the proposed method for large datasets.

Discovering outliers in a collection of patterns is a very well known problem that has been studied in various application domains. Density based technique is a popular one for finding outliers in a dataset. This technique calculates outlierness of each pattern using statistics of neighborhood of the pattern. However, density based approaches do not work well with large datasets as these approaches need to compute a large number of distance computations inorder to find neighborhood statistics. In this paper, we propose to utilize triangle inequality based indexing approach to speed up the classical density based outlier detection method LOF. Proposed approach computes less number of distance computations compared to the LOF method. Experimental results demonstrate that our proposed method reduces a significant number of distance computations compared to the LOF method.