Problems of Indicator Weights and Multicolinearity in World University Rankings: Comparisons of Three Systems

World university rankings (WUR) use the weight-and-sum approach to arrive at an overall measure which is then used to rank the participating universities of the world. Although the weight-and-sum procedure seems straightforward and accords with common sense, it has hidden methodological or statistical problems which render the meaning of the overall measure (and therefore the ranking results) suspect. The study investigates indicator weight inconsistency and multicolinearity. The 2013 ranking results of the ARWU, QSWUR, and THEWUR for the top 100 universities were re-analysed. Correlations, weight discrepancies, and tolerances were estimated and then compared across the three ranking systems. Weight discrepancy and multicolinearity were found in all three systems to different degrees. These can be attributed to the nature of the indicators and their inter-correlations. Each system has strengths and weaknesses. Rank users need to be aware of the interpretive problems, to avoid equating the rankings from the systems, and to make specific reference to the system concerned when citing rankings. Re-conceptualisation and possible modifications to the systems are discussed.