15 results on '"Christoforou, Evgenia"'
Search Results
2. Crowd Computing as a Cooperation Problem: An Evolutionary Approach
- Author
-
Christoforou, Evgenia, Fernández Anta, Antonio, Georgiou, Chryssis, Mosteiro, Miguel A., and Sánchez, Angel
- Published
- 2013
- Full Text
- View/download PDF
3. An experimental characterization of workers' behavior and accuracy in crowdsourced tasks.
- Author
-
Christoforou, Evgenia, Fernández Anta, Antonio, and Sánchez, Angel
- Subjects
- *
TASKS , *CROWDSOURCING , *DEDICATIONS , *LIFTING & carrying (Human mechanics) - Abstract
Crowdsourcing systems are evolving into a powerful tool of choice to deal with repetitive or lengthy human-based tasks. Prominent among those is Amazon Mechanical Turk, in which Human Intelligence Tasks, are posted by requesters, and afterwards selected and executed by subscribed (human) workers in the platform. Many times these HITs serve for research purposes. In this context, a very important question is how reliable the results obtained through these platforms are, in view of the limited control a requester has on the workers' actions. Various control techniques are currently proposed but they are not free from shortcomings, and their use must be accompanied by a deeper understanding of the workers' behavior. In this work, we attempt to interpret the workers' behavior and reliability level in the absence of control techniques. To do so, we perform a series of experiments with 600 distinct MTurk workers, specifically designed to elicit the worker's level of dedication to a task, according to the task's nature and difficulty. We show that the time required by a worker to carry out a task correlates with its difficulty, and also with the quality of the outcome. We find that there are different types of workers. While some of them are willing to invest a significant amount of time to arrive at the correct answer, at the same time we observe a significant fraction of workers that reply with a wrong answer. For the latter, the difficulty of the task and the very short time they took to reply suggest that they, intentionally, did not even attempt to solve the task. [ABSTRACT FROM AUTHOR]
- Published
- 2021
- Full Text
- View/download PDF
4. Achieving Reliability and Fairness in Online Task Computing Environments
- Author
-
Christoforou, Evgenia, Fernández Anta, Antonio, Sánchez, Angel, Universidad Carlos III de Madrid. Departamento de Matemáticas, IMDEA Networks Institute, and UC3M. Departamento de Matemáticas
- Subjects
Online task computing environment ,Telecomunicaciones ,Internet-based task computing system ,Matemáticas ,Reliability ,Masterworker task computing system - Abstract
Mención Internacional en el título de doctor We consider online task computing environments such as volunteer computing platforms running on BOINC (e.g., SETI@home) and crowdsourcing platforms such as Amazon Mechanical Turk. We model the computations as an Internet-based task computing system under the masterworker paradigm. A master entity sends tasks across the Internet, to worker entities willing to perform a computational task. Workers execute the tasks, and report back the results, completing the computational round. Unfortunately, workers are untrustworthy and might report an incorrect result. Thus, the first research question we answer in this work is how to design a reliable masterworker task computing system. We capture the workers’ behavior through two realistic models: (1) the “error probability model” which assumes the presence of altruistic workers willing to provide correct results and the presence of troll workers aiming at providing random incorrect results. Both types of workers suffer from an error probability altering their intended response. (2) The “rationality model” which assumes the presence of altruistic workers, always reporting a correct result, the presence of malicious workers always reporting an incorrect result, and the presence of rational workers following a strategy that will maximize their utility (benefit). The rational workers can choose among two strategies: either be honest and report a correct result, or cheat and report an incorrect result. Our two modeling assumptions on the workers’ behavior are supported by an experimental evaluation we have performed on Amazon Mechanical Turk. Given the error probability model, we evaluate two reliability techniques: (1) “voting” and (2) “auditing” in terms of task assignments required and time invested for computing correctly a set of tasks with high probability. Considering the rationality model, we take an evolutionary game theoretic approach and we design mechanisms that eventually achieve a reliable computational platform where the master receives the correct task result with probability one and with minimal auditing cost. The designed mechanisms provide incentives to the rational workers, reinforcing their strategy to a correct behavior, while they are complemented by four reputation schemes that cope with malice. Finally, we also design a mechanism that deals with unresponsive workers by keeping a reputation related to the workers’ response rate. The designed mechanism selects the most reliable and active workers in each computational round. Simulations, among other, depict the trade-off between the master’s cost and the time the system needs to reach a state where the master always receives the correct task result. The second research question we answer in this work concerns the fair and efficient distribution of workers among the masters over multiple computational rounds. Masters with similar tasks are competing for the same set of workers at each computational round. Workers must be assigned to the masters in a fair manner; when the master values a worker’s contribution the most. We consider that a master might have a strategic behavior, declaring a dishonest valuation on a worker in each round, in an attempt to increase its benefit. This strategic behavior from the side of the masters might lead to unfair and inefficient assignments of workers. Applying renown auction mechanisms to solve the problem at hand can be infeasible since monetary payments are required on the side of the masters. Hence, we present an alternative mechanism for fair and efficient distribution of the workers in the presence of strategic masters, without the use of monetary incentives. We show analytically that our designed mechanism guarantees fairness, is socially efficient, and is truthful. Simulations favourably compare our designed mechanism with two benchmark auction mechanisms. This work has been supported by IMDEA Networks Institute and the Spanish Ministry of Education grant FPU2013-03792. Programa Oficial de Doctorado en Ingeniería Matemática Presidente: Alberto Tarable.- Secretario: José Antonio Cuesta Ruiz.- Vocal: Juan Julián Merelo Guervós
- Published
- 2017
5. Internet Computing: Using Reputation to Select Workers from a Pool
- Author
-
Christoforou, Evgenia, Anta, Antonio Fern��ndez, Georgiou, Chryssis, and Mosteiro, Miguel A.
- Subjects
FOS: Computer and information sciences ,reinforcement learning ,Computer Science - Distributed, Parallel, and Cluster Computing ,Computer Science - Computer Science and Game Theory ,worker reliability ,Volunteer computing ,reputation ,Distributed, Parallel, and Cluster Computing (cs.DC) ,pool of workers ,task computing ,worker unresponsiveness ,Computer Science and Game Theory (cs.GT) - Abstract
The assignment and execution of tasks over the Internet is an inexpensive solution in contrast with supercomputers. We consider an Internet-based Master-Worker task computing approach, such as SETI@home. A master process sends tasks, across the Internet, to worker processors. Workers execute, and report back a result. Unfortunately, the disadvantage of this approach is the unreliable nature of the worker processes. Through different studies, workers have been categorized as either malicious (always report an incorrect result), altruistic (always report a correct result), or rational (report whatever result maximizes their benefit). We develop a reputation-based mechanism that guarantees that, eventually, the master will always be receiving the correct task result. We model the behavior of the rational workers through reinforcement learning, and we present three different reputation types to choose, for each computational round, the most reputable from a pool of workers. As workers are not always available, we enhance our reputation scheme to select the most responsive workers. We prove sufficient conditions for eventual correctness under the different reputation types. Our analysis is complemented by simulations exploring various scenarios. Our simulation results expose interesting trade-offs among the different reputation types, workers availability, and cost. TRUE pub
- Published
- 2016
6. Reputation-Based Mechanisms for Reliable Crowdsourcing Computation
- Author
-
Christoforou, Evgenia, Fernández Anta, Antonio, Georgiou, Chryssis, Mosteiro, Miguel A., and Sánchez, Ángel
- Abstract
We consider an Internet-based Master-Worker framework, for machine-oriented computing tasks (i.e. SETI@home) or human intelligence tasks (i.e. Amazon’s Mechanical Turk). In this framework a master sends tasks to unreliable workers, and the workers execute and report back the result. We model such computations using evolutionary dynamics and consider three type of workers: altruistic, malicious and rational. Altruistic workers always return the correct result, malicious workers always return an incorrect result, and rational (selfish) workers decide whether to be truthful depending on what increases their benefit. The goal of the master is reaching eventual correctness, that is, a stable state of the system in which it always obtains the correct results. To this respect, we propose a mechanism that uses reinforcement learning to induce a correct behavior to rational workers; coping with malice leveraging reputation schemes. We analyze our system as a Markov chain and we give provable guarantees under which truthful behavior can be ensured. Simulation results, obtained using parameter values similar to the values observed in real systems, reveal interesting trade-offs between various metrics and parameters, such as cost, time of convergence to a truthful behavior, tolerance to cheaters and the type of reputation metric employed. pub
- Published
- 2014
7. Applying the dynamics of evolution to achieve reliability in master–worker computing
- Author
-
Christoforou, Evgenia, Fernández Anta, Antonio|||0000-0001-6501-2377, Georgiou, Chryssis, Mosteiro, Miguel A., and Sánchez, Ángel
- Subjects
Performing tasks ,Reinforcement learning ,Evolutionary dynamics ,Internet-based computing ,Algorithmic mechanism design - Abstract
We consider Internet-based master–worker task computations, such as SETI@home, where a master process sends tasks, across the Internet, to worker processes; workers execute and report back some result. However, these workers are not trustworthy, and it might be at their best interest to report incorrect results. In such master–worker computations, the behavior and the best interest of the workers might change over time. We model such computations using evolutionary dynamics, and we study the conditions under which the master can reliably obtain task results. In particular, we develop and analyze an algorithmic mechanism based on reinforcement learning to provide workers with the necessary incentives to eventually become truthful. Our analysis identifies the conditions under which truthful behavior can be ensured and bounds the expected convergence time to that behavior. The analysis is complemented with illustrative simulations. pub
- Published
- 2013
8. Algorithm mechanisms for reliable master - worker internet - based computing under communication uncertainty
- Author
-
Christoforou, Evgenia, Georgiou, Chryssis, Philippou, Anna, Pallis, George, Πανεπιστήμιο Κύπρου, Σχολή Θετικών και Εφαρμοσμένων Επιστημών, Τμήμα Πληροφορικής, University of Cyprus, Faculty of Pure and Applied Sciences, Department of Computer Science, and Georgiou, Chryssis [0000-0003-4360-0260]
- Abstract
Thesis (Master) -- University of Cyprus, Faculty of Pure and Applied Sciences, Department of Computer Science, 2012. We consider Internet-based master-worker computations, where a master processor assigns, across the Internet, a computational task to a set of untrusted worker processors and collects their responses. Examples of such computations are the “@home” projects such as SETI. Building on prior work we consider a framework where altruistic, malicious and rational workers co-exist. Altruistic workers always return the correct result of the task, malicious workers always return an incorrect result, and rational workers act based on their self-interest.The master must obtain the correct task result while maximizing its benefit. Adding on that work, we consider the possibility that the communication between the master and the workers is not reliable, and that workers could be unavailable; assumptions that are very realistic for Internetbased master-worker computations. Within this framework we design and analyze two algorithmic mechanisms to provide appropriate incentives to rational workers to act correctly, despite the malicious’ workers actions and the unreliability of the network. Only when necessary, the incentives are used to force the rational players to a certain equilibrium (which forces the workers to be truthful) that overcomes the attempt of the malicious workers to deceive the master. Finally, the mechanisms are analyzed in two realistic Internet-based master-worker settings, a SETI-like one and a contractor-based one, such as Amazon’s Mechanical Turk. This analysis identifies trade-offs between reliability and cost, under different system parameters.
- Published
- 2012
9. Algorithmic Mechanisms for Internet Supercomputing under Unreliable Communication
- Author
-
Christoforou, Evgenia, Fernández Anta, Antonio, Georgiou, Chryssis, Mosteiro, Miguel A., Georgiou, Chryssis [0000-0003-4360-0260], and Fernández Anta, Antonio [0000-0001-6501-2377]
- Subjects
QA Mathematics::QA75 Electronic computers. Computer science [Q Science] ,Computer science ,Reliability (computer networking) ,Distributed computing ,T Technology (General) [T Technology] ,Computer security ,computer.software_genre ,Task (project management) ,Q Science (General) [Q Science] ,malicious ,Set (psychology) ,Game theory ,TA Engineering (General). Civil engineering (General) [T Technology] ,altruistic ,Mechanism design ,Internet ,business.industry ,Internetbased computing ,Telecommunication networks ,unreliable communication ,Internet-based computing ,mechanism design ,Supercomputer ,task performance ,Internet based computing ,TK Electrical engineering. Electronics Nuclear engineering [T Technology] ,Machine design ,The Internet ,Algorithm design ,rational workers ,business ,computer ,Algorithms - Abstract
This work, using a game-theoretic approach, considers Internet-based computations, where a master processor assigns, over the Internet, a computational task to a set of untrusted worker processors, and collects their responses. The master must obtain the correct task result, while maximizing its benefit. Building on prior work, we consider a framework where altruistic, malicious, and rational workers co-exist. In addition, we consider the possibility that the communication between the master and the workers is not reliable, and that workers could be unavailable assumptions that are very realistic for Internet-based master-worker computations. Within this framework, we design and analyze two algorithmic mechanisms that provide, when necessary, appropriate incentives to rational workers to act correctly, despite the malicious' workers actions and the unreliability of the network. These mechanisms are then applied to two realistic Internet-based master-worker settings, a SETI-like one and a contractor-based one, such as Amazon's mechanical turk. © 2011 IEEE. 275 280 Sponsors: Technical Committee on Distributed Processing IEEE Computer Society Akamai Irianc Conference code: 87009 Cited By :3
- Published
- 2011
10. Evaluating reliability techniques in the master-worker paradigm.
- Author
-
Christoforou, Evgenia, Anta, Antonio Fernandez, Konwar, Kishori M., and Nicolaou, Nicolas
- Published
- 2016
- Full Text
- View/download PDF
11. A Mechanism for Fair Distribution of Resources without Payments.
- Author
-
Christoforou, Evgenia, Anta, Antonio Fernández, and Santos, Agustín
- Subjects
- *
RESOURCE allocation , *MULTIAGENT systems , *PAYMENT , *BAYESIAN analysis , *QUANTITATIVE research - Abstract
We design a mechanism for Fair and Efficient Distribution of Resources (FEDoR) in the presence of strategic agents. We consider a multiple-instances, Bayesian setting, where in each round the preference of an agent over the set of resources is a private information. We assume that in each of r rounds n agents are competing for k non-identical indivisible goods, (n > k). In each round the strategic agents declare how much they value receiving any of the goods in the specific round. The agent declaring the highest valuation receives the good with the highest value, the agent with the second highest valuation receives the second highest valued good, etc. Hence we assume a decision function that assigns goods to agents based on their valuations. The novelty of the mechanism is that no payment scheme is required to achieve truthfulness in a setting with rational/strategic agents. The FEDoR mechanism takes advantage of the repeated nature of the framework, and through a statistical test is able to punish the misreporting agents and be fair, truthful, and socially efficient. FEDoR is fair in the sense that, in expectation over the course of the rounds, all agents will receive the same good the same amount of times. FEDoR is an eligible candidate for applications that require fair distribution of resources over time. For example, equal share of bandwidth for nodes through the same point of access. But further on, FEDoR can be applied in less trivial settings like sponsored search, where payment is necessary and can be given in the form of a flat participation fee. FEDoR can be a good candidate in a setting like that to solve the problem of starvation of publicity slots for some advertisers that have a difficult time determining their true valuations. To this extent we perform a comparison with traditional mechanisms applied to sponsored search, presenting the advantage of FEDoR. [ABSTRACT FROM AUTHOR]
- Published
- 2016
- Full Text
- View/download PDF
12. Reputation-Based Mechanisms for Evolutionary Master-Worker Computing.
- Author
-
Christoforou, Evgenia, Anta, Antonio Fernández, Georgiou, Chryssis, Mosteiro, Miguel A., and Sánchez, Angel (Anxo)
- Published
- 2013
- Full Text
- View/download PDF
13. Achieving Reliability in Master-Worker Computing via Evolutionary Dynamics.
- Author
-
Christoforou, Evgenia, Anta, Antonio Fernández, Georgiou, Chryssis, Mosteiro, Miguel A., and Sánchez, Angel (Anxo)
- Published
- 2012
- Full Text
- View/download PDF
14. Algorithmic Mechanisms for Reliable Master-Worker Internet-Based Computing.
- Author
-
Christoforou, Evgenia, Fernandez Anta, Antonio, Georgiou, Chryssis, and Mosteiro, Miguel A.
- Subjects
- *
COMPUTER algorithms , *TELECOMMUNICATION equipment , *COMPUTER network protocols , *INTERNET , *MALWARE , *CLOUD computing - Abstract
We consider Internet-based master-worker computations, where a master processor assigns, across the Internet, a computational task to a set of untrusted worker processors, and collects their responses. Examples of such computations are the "@homeâ' projects such as SETI. In this work, various worker behaviors are considered. Altruistic workers always return the correct result of the task, malicious workers always return an incorrect result, and rational workers act based on their self-interest. In a massive computation platform, such as the Internet, it is expected that all three type of workers coexist. Therefore, in this work, we study Internet-based master-worker computations in the presence of malicious, altruistic, and rational workers. A stochastic distribution of the workers over the three types is assumed. In addition, we consider the possibility that the communication between the master and the workers is not reliable, and that workers could be unavailable. Considering all the three types of workers renders a combination of game-theoretic and classical distributed computing approaches to the design of mechanisms for reliable Internet-based computing. Indeed, in this work, we design and analyze two algorithmic mechanisms to provide appropriate incentives to rational workers to act correctly, despite the malicious workers' actions and the unreliability of the communication. Only when necessary, the incentives are used to force the rational players to a certain equilibrium (which forces the workers to be truthful) that overcomes the attempt of the malicious workers to deceive the master. Finally, the mechanisms are analyzed in two realistic Internet-based master-worker settings, a SETI-like one and a contractor-based one, such as Amazon's mechanical turk. We also present plots that illustrate the tradeoffs between reliability and cost, under different system parameters. [ABSTRACT FROM AUTHOR]
- Published
- 2014
- Full Text
- View/download PDF
15. Brief announcement.
- Author
-
Christoforou, Evgenia, Fernández Anta, Antonio, Georgiou, Chryssis, Mosteiro, Miguel A., and Sanchez, Angel
- Published
- 2012
- Full Text
- View/download PDF
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.