Search

Data quality issues have attracted widespread attentions due to the negative impacts of dirty data on data mining and machine learning results. The relationship between data quality and the accuracy of results could be applied on the selection of the appropriate model with the consideration of data quality and the determination of the data share to clean. However, rare research has focused on exploring such relationship. Motivated by this, this paper conducts an experimental comparison for the effects of missing, inconsistent, and conflicting data on classification and clustering models. From the experimental results, we observe that dirty-data impacts are related to the error type, the error rate, and the data size. Based on the findings, we suggest users leverage our proposed metrics, sensibility and data quality inflection point, for model selection and data cleaning.

In traditional facility location recommendations, the objective is to select the best locations which maximize the coverage or convenience of users. However, since users’ behavioral habits are often influenced by time, the temporal impacts should not be neglected in recommendation. In this paper, we study the problem of time-aware facility location recommendation problem, taking the time factor into account. To solve this problem, we develop a framework, TAILOR, which incorporates the temporal influence, user-coverage, and user-convenience. Based on TAILOR, we derive a greedy algorithm with (1- $$\frac{1}{e}$$ )-approximation and an online algorithm with ( $$\frac{1}{4}$$ )-competitive ratio. Extensive experimental evaluation and two case studies demonstrate the efficiency and effectiveness of the proposed approaches.

With the explosive growth of information, inconsistent data are increasingly common. However, traditional feature selection methods are lack of efficiency due to inconsistent data repairing beforehand. Therefore, it is necessary to take inconsistencies into consideration during feature selection to not only reduce time costs but also guarantee accuracy of machine learning models. To achieve this goal, we present FRIEND, a f eatu r e select i on approach on inconsistent data. Since features in consistency rules have higher correlation with each other, we aim to select a specific amount of features from these. We prove that the specific feature selection problem is NP-hard and develop an approximation algorithm for this problem. Extensive experimental results demonstrate the efficiency and effectiveness of our proposed approach.

Data quality issues have attracted widespread attentions due to the negative impacts of dirty data on regression model results. The relationship between data quality and the accuracy of results could be applied on the selection of appropriate regression model with the consideration of data quality and the determination of data share to clean. However, rare research has focused on exploring such relationship. Motivated by this, we design a generalized framework to evaluate dirty-data impacts on models. Using the framework, we conduct an experimental evaluation for the effects of missing, inconsistent, and conflicting data on regression models. Based on the experimental findings, we provide guidelines for regression model selection and data cleaning.

To effectively manage increasing knowledge graphs in various domains, a hot research topic, knowledge graph storage management, has emerged. Existing methods are classified to relational stores and native graph stores. Relational stores are able to store large-scale knowledge graphs and convenient in updating knowledge, but the query performance weakens obviously when the selectivity of a knowledge graph query is large. Native graph stores are efficient in processing complex knowledge graph queries due to its index-free adjacent property, but they are inapplicable to manage a large-scale knowledge graph due to limited storage budgets or inflexible updating process. Motivated by this, we propose a dual-store structure which leverages a graph store to accelerate the complex query process in the relational store. However, it is challenging to determine what data to transfer from relational store to graph store at what time. To address this problem, we formulate it as a Markov Decision Process and derive a physical design tuner DOTIL based on reinforcement learning. With DOTIL, the dual-store structure is adaptive to dynamic changing workloads. Experimental results on real knowledge graphs demonstrate that our proposed dual-store structure improves query performance up to average 43.72% compared with the most commonly used relational stores.

The great amount and complex structure of graph data bring a big challenge to graph data management. However, traditional management approaches cannot tackle the challenge. Fortunately, reinforcement learning provides a new approach to solve this problem due to its automation and adaptivity in decision making. Motivated by this, we develop April, an automatic graph data management system, which performs storage structure selection, index selection, and query optimization based on reinforcement learning. The system selects storage structure, indices effectively and automatically, and optimizes the SPARQL queries efficiently. April also offers a friendly interface for users, which allows users to interact with the system in a customized mode. We demonstrate the effectiveness and efficiency of April with two graph data benchmarks.

When a ship in need of assistance has requested a place of refuge with regards to the maritime administration, deciding on the most suitable form of refuge is often both a political and technical problem. To avoid opposition by local communities, a decision model responding to such ship requests is generally established to enable the refuge designation on a case-by-case basis. Within this framework, a step-by-step ranking process is proposed, first selecting indices for potential candidate refuge sites according to the purpose of requesting refuge, with all indices then ordered by pairwise comparison. The weight of each index is determined according to the tone word used for adjacent comparison, and the suitability score for each candidate port obtained. The model is then applied to a hypothetical ship accident in the Bohai Sea in China, which further illustrates the envisaged response process and provides a solution to contemporary maritime administration.

Data imputation is a basic step for data cleaning. Traditional data imputation approaches are lack of accuracy in the absence of knowledge. Involving knowledge base in imputation could overcome this shortcoming. A challenge is that the missing value could be hardly found directly in the knowledge bases (KBs). To use knowledge base sufficiently for missing value imputation, we present FROG, an in f erence algo r ithm from kn o wled g e bases. The inference not only makes full use of true facts in KBs, but also utilizes types to ensure the accuracy of captured missing values. Extensive experiments show that our proposed algorithm can capture missing values efficiently and effectively.

Missing value imputation with crowdsourcing is a novel method in data cleaning to capture missing values that could hardly be filled with automatic approaches. However, the time cost and overhead in crowdsourcing are high. Therefore, we have to reduce cost and guarantee the accuracy of crowdsourced imputation. To achieve the optimization goal, we present COSSET+, a crowdsourced framework optimized by knowledge base. We combine the advantages of both knowledge-based filter and crowdsourcing platform to capture missing values. Since the amount of crowd values will affect the cost of COSSET+, we aim to select partial missing values to be crowdsourced. We prove that the crowd value selection problem is an NP-hard problem and develop an approximation algorithm for this problem. Extensive experimental results demonstrate the efficiency and effectiveness of the proposed approaches.

Nowadays, in information systems such as web search engines and databases, diversity is becoming increasingly essential and getting more and more attention for improving users' satisfaction. In this sense, query result diversification is of vital importance and well worth researching. Some issues such as the definition of diversification and efficient diverse query processing are more challenging to handle in information systems. Many researchers have focused on various dimensions of diversify problem. In this survey, we aim to provide a thorough review of a wide range of result diversification techniques including various definitions of diversifications, corresponding algorithms, diversification technique specified for some applications including database, search engines, recommendation systems, graphs, time series and data streams as well as result diversification systems. We also propose some open research directions, which are challenging and have not been explored up till now, to improve the quality of query results.

This study proposes a new three-component method for timely detection of land cover changes using polarimetric synthetic aperture radar (PolSAR) images. The three components are object-oriented image analysis (OOIA), change vector analysis (CVA), and post-classification comparison (PCC). First, two PolSAR images acquired over the same area at different dates are segmented hierarchically to delineate land parcels (image objects). Then, parcel-based CVA is performed with the coherency matrices of the PolSAR data to detect changed parcels. Finally, PCC based on a parcel-based classification algorithm integrating polarimetric decomposition, decision tree algorithms, and support vector machines is used to determine the type of change for the changed parcels. Compared with conventional PCC based on the widely used Wishart supervised classification, the three-component method achieves much higher accuracy for land cover change detection with PolSAR images. The contribution of each component is evaluated by excluding it from the method. The integration of OOIA in the method greatly reduces the false alarms caused by speckle noise in PolSAR images as well as improves the accuracy of PolSAR image classification. CVA contributes to the method by significantly reducing the effect of the classification errors on the change detection. The use of PCC in the method not only identifies different types of land cover change but also reduces the false alarms introduced by the change in the environment. The three-component method is validated in land development detection, which is important to many developing countries that are confronting a growing problem of unauthorized construction land expansion. The results show that the three-component method is effective in detecting land developments with PolSAR images.

Data imputation is a basic step for data cleaning. Traditional data imputation approaches are lack of accuracy in the absence of knowledge. Involving knowledge base in imputation could overcome this shortcoming. A challenge is that the missing value could be hardly found directly in the knowledge bases (KBs). To use knowledge base sufficiently for imputation, we present FOKES, an inference algorithm on knowledge bases. The inference not only makes full use of true facts in KBs, but also utilizes types to ensure the accuracy of captured missing values. Extensive experiments show that our proposed algorithm can capture missing values efficiently and effectively.

Object-based image analysis (OBIA) has been widely used for land use and land cover (LULC) mapping using optical and synthetic aperture radar (SAR) images because it can utilize spatial information, reduce the effect of salt and pepper, and delineate LULC boundaries. With recent advances in machine learning, convolutional neural networks (CNNs) have become state-of-the-art algorithms. However, CNNs cannot be easily integrated with OBIA because the processing unit of CNNs is a rectangular image, whereas that of OBIA is an irregular image object. To obtain object-based thematic maps, this study developed a new method that integrates object-based post-classification refinement (OBPR) and CNNs for LULC mapping using Sentinel optical and SAR data. After producing the classification map by CNN, each image object was labeled with the most frequent land cover category of its pixels. The proposed method was tested on the optical-SAR Sentinel Guangzhou dataset with 10 m spatial resolution, the optical-SAR Zhuhai-Macau local climate zones (LCZ) dataset with 100 m spatial resolution, and a hyperspectral benchmark the University of Pavia with 1.3 m spatial resolution. It outperformed OBIA support vector machine (SVM) and random forest (RF). SVM and RF could benefit more from the combined use of optical and SAR data compared with CNN, whereas spatial information learned by CNN was very effective for classification. With the ability to extract spatial features and maintain object boundaries, the proposed method considerably improved the classification accuracy of urban ground targets. It achieved overall accuracy (OA) of 95.33% for the Sentinel Guangzhou dataset, OA of 77.64% for the Zhuhai-Macau LCZ dataset, and OA of 95.70% for the University of Pavia dataset with only 10 labeled samples per class.

This study proposes a new four-component algorithm for land use and land cover (LULC) classification using RADARSAT-2 polarimetric SAR (PolSAR) data. These four components are polarimetric decomposition, PolSAR interferometry, object-oriented image analysis, and decision tree algorithms. First, polarimetric decomposition can be used to support the classification of PolSAR data. It is aimed at extracting polarimetric parameters related to the physical scattering mechanisms of the observed objects. Second, PolSAR interferometry is used to extract polarimetric interferometric information to support LULC classification. Third, the main purposes of object-oriented image analysis are delineating image objects, as well as extracting various textural and spatial features from image objects to improve classification accuracy. Finally, a decision tree algorithm provides an efficient way to select features and implement classification. A comparison between the proposed method and the Wishart supervised classification which is based on the coherency matrix was made to test the performance of the proposed method. The overall accuracy of the proposed method was 86.64%, whereas that of the Wishart supervised classification was 69.66%. The kappa value of the proposed method was 0.84, much higher than that of the Wishart supervised classification, which exhibited a kappa value of 0.65. The results indicate that the proposed method exhibits much better performance than the Wishart supervised classification for LULC classification. Further investigation was carried out on the respective contribution of the four components to LULC classification using RADARSAT-2 PolSAR data, and it indicates that all the four components have important contribution to the classification. Polarimetric information has significant implications for identifying different vegetation types and distinguishing between vegetation and urban/built-up. The polarimetric interferometric information extracted from repeat-pass RADARSAT-2 images is important in reducing the confusion between urban/built-up and vegetation and that between barren/sparsely vegetated land and vegetation. Object-oriented image analysis is very helpful in reducing the effect of speckle in PolSAR images by implementing classification based on image objects, and the textural information extracted from image objects is helpful in distinguishing between water and lawn. The decision tree algorithm can achieve higher classification accuracy than the nearest neighbor classification implemented using Definiens Developer 7.0, and the accuracy of the decision tree algorithm is similar with that of the support vector classification which is implemented based on the features selected using genetic algorithms. Compared with the nearest neighbor and support vector classification, the decision tree algorithm is more efficient to select features and implement classification. Furthermore, the decision tree algorithm can provide clear classification rules that can be easily interpreted based on the physical meaning of the features used in the classification. This can provide physical insight for LULC classification using PolSAR data.

Traditional pixel-based classification methods yield poor results when applied to SAR imagery because of the presence of speckle and limited information in backscatter coefficients. A novel classification method, integrating polarimetric target decomposition, object-oriented image analysis, and decision tree algorithms, is proposed for the classification of polarimetric SAR data (PolSAR). The polarimetric target decomposition is aimed at extracting physical information related to the scattering mechanism of targets for the classification of scattering data. The main purposes of the object-oriented image analysis are delineating objects and extracting various spatial and textural features. The decision tree algorithm provides an efficient way to select features and create a decision tree for the classification. A comparison between the proposed method and the Wishart supervised classification was made. The overall accuracies of these two methods were 89.34% and 79.36%, respectively. The results show that the proposed method is an effective method for the classification of PolSAR data.