Search

The electricity price mechanism based on game theory is one of the research focuses on microgrids energy trading. The complete information game is based on the certainty of the identity of roles of players and variables. However, there are many uncertain factors that cause the game in the state of incomplete information. In this paper, Microgrids Energy Trading Bayesian Game (METBG) model is proposed. The model was based on the Bayesian game, in which MGs make their decision as an agent of native users to tackle bidirectional energy trading between others. First, The Bayesian game modeled types of roles of players by the uncertainty of information including the stochastic characteristics of PEVs which result in hardness that the game participants determine whether they are sellers or buyers in the utility function that depends on the state of power surplus or lack. Moreover, the utility model of players was established by a Bayesian game with the game equilibrium derived rigorously by obligated to coordinate the sharing of energy with maximization of the players' profit. Finally, the solution of game equilibrium has been rigorously derived, and the effectiveness of the model is verified in terms of seller profit, the utilities of buyers, and the net energy usage in the microgrids. The results of the static pricing model and proposed model were compared to demonstrate the effectiveness.

Barrage is the commentary content that pops up when a video playing, which is short and concise and has strong domain characteristics. Because the sentiment analysis method in the field of barrage language is relatively single, this paper presents a method for constructing a library of barrage ontology based on SO-PMI algorithm and the word vector maximum similarity algorithm. The barrage ontology is based on professional knowledge and the barrage sentiment dictionary, which is added into the ontology as an example to construct the knowledge base. The sentiment classification of barrage can be realized and a new way for sentiment analysis of barrage language is provided by this method. The barrage ontology with tendency of sentiment can be better applied to video recommendation and opinion monitoring.

As a typical smart energy system, the integrated energy system can promote the in-depth integration of source, grid, and load, and increase the utilization rate of renewable energy. Therefore, the evaluation indicators of the smart energy system should simultaneously reflect the core characteristics of low carbon and high efficiency. This paper takes maximizing the comprehensive energy efficiency level and improving the energy consumption economy as the optimization goals, and proposes a multi-objective planning optimization model for the integrated energy system. Under the constraints of the physical constraints of the equipment and the balance of cooling, heating and power, the energy consumption economy and comprehensive planning and optimization strategies for energy efficiency levels. The analysis of calculation examples shows that the optimal scheduling strategy proposed in this paper can comprehensively improve the cleanliness and efficiency of system energy consumption.

Database exploration – the problem of finding and ranking desired tuples – is important for data discovery and analysis. Precisely specifying SQL queries is not always feasible in practice, such as "finding and ranking off-road cars based on a combination of Price, Make, Model, Age, and Mileage." – not only due to the query complexity (e.g., which may have many if-then-else, and, or and not logic), but also because the user typically does not have the knowledge of all data instances.We propose DExPlorer, a system for interactive database exploration. DExPlorer offers a simple and user-friendly interface which allows to: (1) confirm whether a tuple is desired or not, and (2) decide whether a tuple is more preferred than another. Behind the scenes, we jointly use multiple ML models to learn from the above two types of user feedback. Moreover, in order to effectively involve users, we carefully select the set of tuples for which we need to solicit feedback. Therefore, we devise question selection algorithms that consider not only the estimated benefit of each tuple, but also the possible partial orders between any two suggested tuples. Experiments on real-world datasets show that DExPlorer is more effective than existing approaches.

In this paper, we study how to jointly predict travel demands and traffic flows for all regions of a city at a future time interval. From an empirical analysis of traffic data, we outline three desired properties, namely region-level correlations, temporal periodicity and inter-traffic correlations. Then, we propose a comprehensive neural network based traffic prediction model, where various effective embeddings or encodings are designed to capture the aforementioned properties. First, we design effective region embeddings to capture two forms of region-level correlations: spatially close regions have similar embeddings, and regions with similar properties (e.g., the number of POIs and the number of roads in a region) other than locations have similar embeddings. Second, we extract the "day-in-week" and "time-in-day" and utilize the temporal periodicity in designing the embeddings for time intervals. Third, we propose an effective encoding for past traffic data which captures two forms of inter-traffic correlations - the correlation between past and future traffic, and the correlation between travel demands and traffic flows within past traffic data. Extensive experiments on two real datasets verify the high effectiveness of our model.

Computing the shortest path distances between two vertices on road networks is a core operation in many real-world applications, e.g., finding the closest taxi/hotel. However existing techniques have several limitations. First, traditional Dijkstra-based methods have long latency and cannot meet the high-performance requirement. Second, existing indexing-based methods either involve huge index sizes or have poor performance. To address these limitations, in this paper we propose a learning-based method which can efficiently compute an approximate shortest-path distance such that (1) the performance is super fast, e.g., taking 60-150 nanoseconds; (2) the error ratio of the approximate results is super small, e.g., below 0.7%; (3) scales well to large road networks, e.g., millions of nodes. The key idea is to first embed the road networks into a low dimensional space for capturing the distance relations between vertices, get an embedded vector for each vertex, and then perform a distance metric (L 1 metric) on the embedded vectors to approximate shortest-path distances. We propose a hierarchical model to represent the embedding, and design an effective method to train the model. We also design a fine-tuning method to judiciously select high-quality training data. Extensive experiments on real-world datasets show that our embedding based approach significantly outperforms the state-of-the-art methods.

For the data monitoring and fault early warning of new energy grid connection, the equipment status data, Supervisory Control And Data Acquisition (SCADA) system data, Wide Area Measurement System (WAMS) system data and other multi-source complex data were synchronized into the remote monitoring and early warning system of new energy station. The multi-source information fusion technology was used to analyze the data, and a new energy station operation parameter prediction method based on Multiple Extremum Learning Particle Swarm Optimization (MELPSO) algorithm was proposed. The correctness and effectiveness of the monitoring and early warning system was proved by comparing the predicted results of operational parameters with the measured results. The system can monitor the operation status of equipment in new energy station in real time, and has high prediction accuracy. It can judge the fault trend in advance and reduce the impact of new energy grid connection on the safe and stable operation of power grid.

Data labelling is very important in many database and machine learning applications. Traditional methods rely on humans (workers or experts) to acquire labels. However, the human cost is rather expensive for a large dataset. Active learning based methods only label a small set of data with large uncertainty, train a model on these labelled data, and use the trained model to label the remainder unlabelled data. However they have two limitations. First, they cannot judiciously select appropriate data (task selection) and assign the tasks to proper humans (task assignment). Moreover, they independently process task selection and task assignment, which cannot capture the correlation between them. Second, they simply infer the truth of a task based on the answers from humans and the trained model (truth inference) by independently modeling humans and models. In other words, they ignore the correlation between them (the labelled data may have noise caused by humans with biases, and the model trained by the noisy labels may bring additional biases), and thus lead to poor inference results.To address these limitations, in this paper, we propose CrowdRL, an end-to-end reinforcement learning (RL) based framework for data labelling. To the best of our knowledge, CrowdRL is the first RL framework designed for the data labelling workflow by seamlessly integrating task selection, task assignment and truth inference together. CrowdRL fully utilizes the power of heterogeneous annotators (experts and crowdsourcing workers) and machine learning models together to infer the truth, which highly improves the quality of data labelling. CrowdRL uses RL to model task assignment and task selection, and designs an agent to judiciously assign tasks to appropriate workers. CrowdRL jointly models the answers of workers, experts and models, and designs a joint inference model to infer the truths. Experimental results on real datasets show that CrowdRL outperforms state-of-the-art approaches with the same (even fewer) monetary cost while achieving 5%-20% higher accuracy.

Materialized views (MVs) can significantly optimize the query processing in databases. However, it is hard to generate MVs for ordinary users because it relies on background knowledge, and existing methods rely on DBAs to generate and maintain MVs. However, DBAs cannot handle large-scale databases, especially cloud databases that have millions of database instances and support millions of users. Thus it calls for an autonomous MV management system. In this paper, we propose an autonomous materialized view management system, AutoView. It analyzes query workloads, estimates the costs and benefits of materializing queries as views, and selects MVs to maximize the benefit within a space budget. We propose a deep reinforcement learning model to select high-quality MVs, which enriches the state representation with query and MVs’ embedding. Experimental results show that our method outperforms existing studies in terms of MV selection quality.

Knowledge bases (KBs), which store high-quality information, are crucial for many applications, such as enhancing search results and serving as external sources for data cleaning. Not surprisingly, there exist outdated facts in most KBs due to the rapid change of information. Naturally, it is important to keep KBs up-to-date. Traditional wisdom has investigated the problem of using reference data (such as new facts extracted from the news) to detect outdated facts in KBs. However, existing approaches can only cover a small percentage of facts in KBs. In this paper, we propose a novel human-in-the-loop approach for outdated fact detection in KBs. It trains a binary classifier using features such as historical update frequency and existence time of a fact to compute the likelihood of a fact in a KB to be outdated. Then, it interacts with humans to verify whether a fact with high likelihood is indeed outdated. In addition, it also uses logical rules to detect more outdated facts based on human feedback. The outdated facts detected by the logical rules will also be fed back to train the ML model further for data augmentation. Extensive experiments on real-world KBs, such as Yago and DBpedia, show the effectiveness of our solution.

In this paper, we propose a Deep Reinforcement Learning (RL) framework for task arrangement, which is a critical problem for the success of crowdsourcing platforms. Previous works conduct the personalized recommendation of tasks to workers via supervised learning methods. However, the majority of them only consider the benefit of either workers or requesters independently. In addition, they do not consider the real dynamic environments (e.g., dynamic tasks, dynamic workers), so they may produce sub-optimal results. To address these issues, we utilize Deep Q-Network (DQN), an RL-based method combined with a neural network to estimate the expected long-term return of recommending a task. DQN inherently considers the immediate and the future rewards and can be updated quickly to deal with evolving data and dynamic changes. Furthermore, we design two DQNs that capture the benefit of both workers and requesters and maximize the profit of the platform. To learn value functions in DQN effectively, we also propose novel state representations, carefully design the computation of Q values, and predict transition probabilities and future states. Experiments on synthetic and real datasets demonstrate the superior performance of our framework.

Visualization charts are widely utilized for presenting structured data. Under many circumstances, people want to explore the data in the charts collected from various sources, such as papers and websites, so as to further analyzing the data or creating new charts. However, the existing automatic and semi-automatic approaches are not always effective due to the variety of charts. In this paper, we introduce a crowdsourcing approach that leverages human ability to extract data from visualization charts. There are several challenges. The first one is how to avoid tedious human interaction with charts and design simple crowdsourcing tasks. Second, it is challenging to evaluate worker’s quality for truth inference, because workers may not only provide inaccurate values but also misalign values to wrong data series. To address the challenges, we design an effective crowdsourcing task scheme that splits a chart into simple micro-tasks. We introduce a novel worker quality model by considering worker’s accuracy and task difficulty. We also devise an effective early-stopping mechanisms to save the cost. We have conducted experiments on a real crowdsourcing platform, and the results show that our framework outperforms state-of-the-art approaches on both cost and quality.

Join order selection (JOS) – the problem of finding the optimal join order for an SQL query – is a primary focus of database query optimizers. The problem is hard due to its large solution space. Exhaustively traversing the solution space is prohibitively expensive, which is often combined with heuristic pruning. Despite decades-long effort, traditional optimizers still suffer from low scalability or low accuracy when handling complicated SQL queries. Recent attempts using deep reinforcement learning (DRL), by encoding join trees with fixed-length handtuned feature vectors, have shed some light on JOS. However, using fixed-length feature vectors cannot capture the structural information of a join tree, which may produce poor join plans. Moreover, it may also cause retraining the neural network when handling schema changes (e.g., adding tables/columns) or multialias table names that are common in SQL queries.In this paper, we present RTOS, a novel learned optimizer that uses Reinforcement learning with Tree-structured long short-term memory (LSTM) for join Order Selection. RTOS improves existing DRL-based approaches in two main aspects: (1) it adopts graph neural networks to capture the structures of join trees; and (2) it well supports the modification of database schema and multi-alias table names. Extensive experiments on Join Order Benchmark (JOB) and TPC-H show that RTOS outperforms traditional optimizers and existing DRL-based learned optimizers. In particular, the plan RTOS generated for JOB is 101% on (estimated) cost and 67% on latency (i.e., execution time) on average, compared with dynamic programming that is known to produce the state-of-the-art results on join plans.

Materializing views is an important method to reduce redundant computations in DBMS, especially for processing large scale analytical queries. However, many existing methods still need DBAs to manually generate materialized views, which are not scalable to a large number of database instances, especially on the cloud database. To address this problem, we propose an automatic view generation method which judiciously selects "highly beneficial" subqueries to generate materialized views. However, there are two challenges. (1) How to estimate the benefit of using a materialized view for a queryƒ (2) How to select optimal subqueries to generate materialized viewsƒ To address the first challenge, we propose a neural network based method to estimate the benefit of using a materialized view to answer a query. In particular, we extract significant features from different perspectives and design effective encoding models to transform these features into hidden representations. To address the second challenge, we model this problem to an ILP (Integer Linear Programming) problem, which aims to maximize the utility by selecting optimal subqueries to materialize. We design an iterative optimization method to select subqueries to materialize. However, this method cannot guarantee the convergence of the solution. To address this issue, we model the iterative optimization process as an MDP (Markov Decision Process) and use the deep reinforcement learning model to solve the problem. Extensive experiments show that our method outperforms existing solutions by 28.4%, 8.8% and 31.7% on three real-world datasets.

Medical image segmentation is a problem of fundamental importance in medical image processing. The accurate segmentation of a medical image can provide important information for the diagnosis and treatment of many diseases. Since a medical image often contains noises and the objects in it are inherently complex in general, methods that can accurately segment an arbitrary medical image are still unavailable. In this paper, a new approach that combines convolutional operators and an adaptive Hidden Markov Model is developed for the segmentation of medical images. Specifically, the features associated with each pixel in a medical image are obtained with a set of convolutional operators. The semantic and spatial correlations among pixels in the image are then progressively captured by an adaptive Hidden Markov Model. The labels of the pixels can be efficiently obtained with a dynamic programming algorithm in linear time. Our experimental results show that this approach can achieve segmentation results with improved accuracy on a set of brain medical images.

Many data management and analytics tasks, such as entity resolution, cannot be solely addressed by automated processes. Crowdsourcing is an effective way to harness the human cognitive ability to process these computer-hard tasks. Thanks to public crowdsourcing platforms, e.g., Amazon Mechanical Turk and CrowdFlower, we can easily involve hundreds of thousands of ordinary workers (i.e., the crowd) to address these computer-hard tasks. However it is rather inconvenient to interact with the crowdsourcing platforms, because the platforms require one to set parameters and even write codes. Inspired by traditional DBMS, crowdsourcing database systems have been proposed and widely studied to encapsulate the complexities of interacting with the crowd. In this tutorial, we will survey and synthesize a wide spectrum of existing studies on crowdsourcing database systems. We first give an overview of crowdsourcing, and then summarize the fundamental techniques in designing crowdsourcing databases, including task design, truth inference, task assignment, answer reasoning and latency reduction. Next we review the techniques on designing crowdsourced operators, including selection, join, sort, top-k, max/min, count, collect, and fill. Finally, we discuss the emerging challenges.

Huawei is a leading global provider of information and communication technologies (ICT) infrastructure and smart devices. With integrated solutions across four key domains: telecommunication networks, IT, smart devices, and cloud services, Huawei is committed to bringing digital transformation to every person, home and organization for a fully connected and intelligent world. Founded in 1987, Huawei currently has more than 180,000 employees, and operates in more than 170 countries and regions with revenue over 100 billion USD in 2018. Data management plays a key role in all of the four key domains above. We have developed innovative products and solutions to support rapid business growth driven by customer requirements. While many data management problems are common, each domain also has its own special requirements and challenges. In this paper, we will go through recent advancements in Huawei data management technologies including a petabyte scale enterprise analytics platform (FusionInsight MPPDB) and a highly available in-memory database for telecommunication networks (GMDB). In addition, we discuss data management challenges that we are facing in the areas of autonomous databases and device-edge-cloud collaboration data platforms.

Many applications, e.g., Uber, collect large-scale trajectory data from moving vehicles on road network. Trajectory data analytics can benefit many real-world applications, such as route planning and transportation optimizations. Two core operations in trajectory data analytics are trajectory similarity search and join, and both of them rely on a trajectory similarity function to measure the similarity between two trajectories. However, existing similarity functions focus on trajectory points distance and neglect the fact the trajectories should be on road network. Obviously aligning trajectories on road network can remove the noise points introduced by system errors. Toward this goal, we define a road-network-aware trajectory similarity function to measure trajectory similarity. To support trajectory similarity search and join, we propose a filtering-refine framework. In the filtering step, we compute a signature of each trajectory such that if two trajectories are similar, they must share a common signature. We utilize the signatures to prune a huge number of dissimilar pairs. In the refine step, we design effective algorithms to verify the candidates that are not pruned in the filtering step. To support large-scale trajectories, we develop a system DISON for Distributed In-Memory Trajectory Similarity Search and Join on Road Network. DISON splits trajectories into disjoint partitions by considering load balance and locality, and designs effective global index to prune irrelevant partitions. Extensive experiments on real datasets showed that our method achieved high effectiveness, efficiency, and scalability and outperformed existing solutions significantly.

Data outsourcing to cloud has been a common IT practice nowadays due to its significant benefits. Meanwhile, security and privacy concerns are critical obstacles to hinder the further adoption of cloud. Although data encryption can mitigate the problem, it reduces the functionality of query processing, e.g., disabling SQL queries. Several schemes have been proposed to enable one-dimensional query on encrypted data, but multi-dimensional range query has not been well addressed. In this paper, we propose a secure and scalable scheme that can support multi-dimensional range queries over encrypted data. The proposed scheme has three salient features: (1) Privacy: the server cannot learn the contents of queries and data records during query processing. (2) Efficiency: we utilize hierarchical cubes to encode multi-dimensional data records and construct a secure tree index on top of such encoding to achieve sublinear query time. (3) Verifiability: our scheme allows users to verify the correctness and completeness of the query results to address server's malicious behaviors. We perform formal security analysis and comprehensive experimental evaluations. The results on real datasets demonstrate that our scheme achieves practical performance while guaranteeing data privacy and result integrity.

The IEEE Std. 1687 facilitates flexible access to on-chip instruments through the JTAG test-access port. This flexibility enables the minimization of the overall access time (OAT), and a number of techniques have been proposed in the literature to achieve this goal. However, the OAT is still high for instruments that require a large amount of test data if this data is shifted through the scan chain serially. In order to further reduce the OAT, we present an efficient test-scheduling method that exploits broadcast and hardware parallelism for instrument access. A broadcast scheduling method is synergistically combined with three parallel IJTAG designs. We show that under different cost criteria, we can select the most efficient parallel IJTAG design such that the equivalent access time (EAT) is minimized. Two industry chip designs and three IJTAG benchmarks are used to evaluate the effectiveness of the proposed method.

Wafer-level integration of micro heater on an alkali vapor cell for chip-scale SERF magnetometers is presented. In order to obtain a suitable resistivity, polysilicon and aluminum are mixed as the target material. Magnetron sputtering is adopted to fabricate the heater on the surface of the alkali vapor cell. Various voltages are applied to the heater, of which the temperatures are measured by thermal sensitive sensors. Spin exchange relaxation free (SERF) atomic magnetometer with the heater integrated on the alkali vapor cell is characterized. Results show that the micro heater is successfully integrated on the alkali vapor cell. With the integrated heater, the rubidium vapor cell is heated up to 120°C by a 445kHz AC current. The sensitivity of the SERF magnetometer is 6.3pT/Hz1/2. Results also indicate that it provides an effective and low-cost solution for the miniaturization of atomic magnetometers.

Modern large system-on-chip (SoC) designs typically have hundreds of cores. Each core requires a certain number of input/output test channels. At the chip level, however, the total number of test pins is limited such that all core-level test channels cannot be accessed at the same time. Therefore, hierarchical pattern retargeting is required for SoC test. Test scheduling algorithms can be applied to reduce the total test time. In this paper, we add the EDT architectures as one additional dimension of parameter into the prior test scheduling algorithm. Experimental results based on real case studies show that with the proposed flow, the test time can be further reduced up to approximately 24%.

In this paper, the problem of delay dependent robust stability for a class of interval time-varying delay systems with nonlinear perturbations is studied. Based on the idea of delay-partitioning method (DPM), the whole delay interval is divided into two unequal subintervals at appropriate point and a new Lyapunov-Krasovskii functional (LKF) which contains some quadruple-integral terms and augment terms is introduced on each interval. Then, a new bounding technique based on a novel integral inequality approach together with reciprocally convex combination technique is used to deal with the cross term of the derivative. Hence, an enhanced delay-dependent stability criterion is derived in terms of linear matrix inequality (LMI). Finally, the numerical simulation results show that the proposed method is effective and less conservative.

This paper presents a novel scheme on the integration of chip-scale SERF magnetometer using a verticalcavity surface-emitting laser(VCSEL) as pumping and detecting laser. The fabrication and integration process of the whole system based on stacking PCB boards consist of VCSEL, optical components, rubidium vapor cell, photo-detector, three axis Helmholtz coils. By calculation and measurement, it was found that the module is designed with a minimum output power of 0.15mW and a competitive volume of 1.4×1.4×2.5cm3. Experiments show that the sensitivity of chip-scale SERF magnetometer based on VCSEL is around 1.87pT/Hz1/2 in magnetic shield environment, the integration and miniaturization design process of the magnetometer can be empolyed for magnetoencephalography(MEG) measurement.

With the continuous improvement of weak magnetic detection technology, there has been a growing trend towards more compact and integrated magnetometer. The SERF magnetometer and the Mz magnetometer have many common features in operation. Therefore, we design a new kind of miniaturized atomic sensors to integrate the SERF magnetometer and the Mz magnetometer based on micro-fabricated alkali vapor cell. The sensor structure consists of alkali vapor cell, three-axis magnetic field coils, heating module and optical module. This small size sensor has the potential to achieve a portable multi-function device.

The wide adoption of social networks has brought a new demand on influence analysis. This paper presents OCTOPUS that offers social network users and analysts valuable insights through topic-aware social influence analysis services. OCTOPUS has the following novel features. First, OCTOPUS provides a user-friendly interface that allows users to employ simple and easy-to-use keywords to perform influence analysis. Second, OCTOPUS provides three powerful keyword-based topic-aware influence analysis tools: keyword-based influential user discovery, personalized influential keywords suggestion, and interactive influential paths exploration. These tools can not only discover influential users, but also provide insights on how the users influence the network. Third, OCTOPUS enables online influence analysis, which provides end-users with instant results. We have implemented and deployed OCTOPUS, and demonstrate its usability and efficiency on two social networks.

Entity categorization – the process of grouping entities into categories for some specific purpose – is an important problem with a great many applications, such as Google Scholar and Amazon products. Unfortunately, many real-world categories contain mis-categorized entities, such as publications in one’s Google Scholar page that are published by the others. We have proposed a general framework for a new research problem – discovering mis-categorized entities. In this demonstration, we have developed a Google Chrome extension, namely GSCleaner, as one important application of our studied problem. The attendees will have the opportunity to experience the following features: (1) mis-categorized entity discovery – The attendee can check mis-categorized entities on anyone’s Google Scholar page; and (2) Cleaning onsite – Any attendee can login and clean his Google Scholar page using GSCleaner.We describe our novel rule-based framework to discover mis-categorized entities. We also propose effective optimization techniques to apply the rules. Some empirical results show the effectiveness of GSCleaner on discovering mis-categorized entities.

The IEEE Std. 1687 enables flexible access to on-chip instruments through the JTAG test-access port. This flexibility enables the minimization of the overall access time (OAT), and a number of techniques have been proposed to achieve this goal. However, these techniques do not utilize the broadcast feature in the 1687 network. In order to further reduce the OAT, we present an efficient test-scheduling method that exploits the broadcast feature for instrument access. Two optimization solutions are then proposed — the first solution minimizes the OAT without the retargeting time, while the second one reorders the configurations so as to minimize the overall retargeting time among configurations. Three industry test cases are used to evaluate the effectiveness of the proposed method.

Crowdsourcing is widely accepted as a means for resolving tasks that are hard for computers, e.g., entity resolution. Unfortunately, Crowdsourcing may yield relatively low-quality results if there is no proper quality control. Although previous studies attempt to eliminate workers by estimating workers' qualities via qualification tests or hidden tests, the qualities estimated may not be accurate, because workers may have diverse qualities across tasks. Thus, the quality of the results could be further improved by wisely assigning tasks to the workers who are specialized in the tasks and online task assignment is an effective way to achieve this goal. However, existing crowdsourcing platforms either do not support online task assignment (e.g., CrowdFlower) or are not user-friendly because they require to write complicated codes (e.g., Amazon MTurk). To address these limitations, we develop an online task assignment system, which can on-the-fly assign workers with appropriate tasks. We have deployed our system on top of MTurk. We demonstrate the following scenarios using our system. Firstly, requesters can easily utilize our system to enable online task assignment in order to improve answer quality. Moreover, requesters do not need to write any code. Secondly, our system can infer the quality of workers, and requesters can design and test their own task assignment algorithms using our proposed information. Thirdly, our system can monitor tasks and workers in real time, and the requesters can eliminate bad workers or terminate the crowdsourcing process to reduce the unnecessary cost.

Entity categorization – the process of grouping entities into categories for some specific purpose – is an important problem with a great many applications, such as Google Scholar and Amazon products. Unfortunately, in practice, many entities are mis-categorized. In this paper, we study the problem of discovering mis-categorized entities from a given group of entities. This problem is inherently hard: all entities within the same group have been "well" categorized by state-of-the-art solutions. Apparently, it is nontrivial to differentiate them. We propose a novel rule-based framework to solve this problem. It first uses positive rules to compute disjoint partitions of entities, where the partition with the largest size is taken as the correctly categorized partition, namely the pivot partition. It then uses negative rules to identify mis-categorized entities in other partitions that are dissimilar to the entities in the pivot partition. We describe optimizations on applying these rules, and discuss how to generate positive/negative rules. Extensive experimental results on two real-world datasets show the effectiveness of our solution.

Crowdsourced entity collection leverages human's ability to collect entities that are missing in a database, which has many real-world applications, such as knowledge base enrichment and enterprise data collection. There are several challenges. First, it is hard to evaluate the workers' quality because a worker's quality depends on not only the correctness of her provided entities but also the distinctness of these entities compared with the collected ones by other workers. Second, crowd workers are likely to provide popular entities and different workers will provide many duplicated entities, leading to a waste of money and low coverage. To address these challenges, we propose an incentive-based crowdsourced entity collection framework CrowdEC that encourages workers to provide more distinct items using an incentive strategy. CrowdEC has fundamental differences from existing crowdsourcing collection methods. One the one hand, CrowdEC proposes a worker model and evaluates a worker's quality based on cross validation and entity checking. CrowdEC devises a worker utility model that considers both worker's quality and entities' distinctness provided by workers. CrowdEC proposes a worker elimination method to block workers with a low utility, which solves the first challenge. On the other hand, CrowdEC proposes an incentive pricing technique that encourages each qualified (i.e., non-eliminated) worker to provide distinct entities rather than duplicates. CrowdEC provides two types of tasks and judiciously assigns workers with appropriate tasks to address the second challenge. We have conducted both real and simulated experiments, and the results show that CrowdEC outperforms existing state-of-the-art works on both cost and quality.