Search

Rankings and leaderboards are often used in crowdsourcing contests and online communities to motivate individual contributions but feedback based on social comparison can also have negative effects. Here, we study the unequal effects of such feedback on individual effort and performance for individuals of different ability. We hypothesize that the effects of social comparison differ for top performers and bottom performers in a way that the inequality between the two increases. We use a quasi-experimental design to test our predictions with data from Topcoder, a large online crowdsourcing platform that publishes computer programming contests. We find that in contests where the submitted code is evaluated against others' submissions, rather than using an absolute scale, top performers increase their effort while bottom performers decrease it. As a result, relative scoring leads to better outcomes for those at the top but lower engagement for bottom performers. Our findings expose an important but overlooked drawback from using gamified competitions, rankings, and relative evaluations, with potential implications for crowdsourcing markets, online learning environments, online communities, and organizations in general.

Ranking in contests and competitions is often used to motivate individual contributions but it can have negative impact on the group. Here, we study the effect of social comparison on the dispersion and predictability of individual performance. We hypothesize that the effects of social comparison on effort will differ for top-performers and bottom-performers in a way that the inequality between the two increases. We use a quasi-experimental design to test our predictions with data from Topcoder, an online crowdsourcing platform that publishes computer programming contests. We find that in contests where the submitted code is evaluated against others' submissions, rather than using an absolute scale, top-performers increase their effort while bottom-performers decrease it. As a result, relative scoring leads to a higher dispersion and somewhat higher predictability of performance, strengthening the status quo and precluding under-performers from engaging and learning. Our findings expose the drawbacks of using gamified competitions, rankings, and relative evaluations in learning environments and organizations.