Search

Label ranking is a machine learning task that deals with mapping an instance to a ranking of labels, representing the labels’ ordered relevance to the instance. Three recent studies have suggested the use of ensembles to improve the performance of simple label ranking models. However, none of them has explicitly inspected the question of how should the results obtained by simple models be aggregated into a single combined output. While classification tasks and regression tasks typically employ trivial aggregation techniques (i.e., majority voting and averaging, respectively), the case of label ranking tasks is not straightforward. The contribution of this paper is twofold. First, we propose to apply voting rules, typically used in the field of social choice, as the aggregation technique for label ranking ensembles. Our evaluation reveals that there is no single rule that consistently outperforms all other voting rules, and that under different settings different voting rules perform the best. Second, we propose a novel aggregation method for label ranking ensembles that learns the best voting rule to be used in a given setting. An extensive evaluation of the proposed method on semi-synthetic as well as real-world datasets shows that it obtains prediction performance that is significantly higher than that of the aggregation techniques currently used by state-of-the-art label ranking ensembles.

A voting center is in charge of collecting and aggregating voter preferences. In an iterative process, the center sends comparison queries to voters, requesting them to submit their preference between two items. Voters might discuss the candidates among themselves, figuring out during the elicitation process which candidates stand a chance of winning and which do not. Consequently, strategic voters might attempt to manipulate by deviating from their true preferences and instead submit a different response in order to attempt to maximize their profit. We provide a practical algorithm for strategic voters which computes the best manipulative vote and maximizes the voter's selfish outcome when such a vote exists. We also provide a careful voting center which is aware of the possible manipulations and avoids manipulative queries when possible. In an empirical study on four real-world domains, we show that in practice manipulation occurs in a low percentage of settings and has a low impact on the final outcome. The careful voting center reduces manipulation even further, thus allowing for a non-distorted group decision process to take place. We thus provide a core technology study of a voting process that can be adopted in opinion or information aggregation systems and in crowdsourcing applications, e.g., peer grading in Massive Open Online Courses (MOOCs).