An Analysis of Mode Effects in the 2010 Course Experience Questionnaire

Historically, responses to the Course Experience Questionnaire (CEQ) were required to be collected by self-administered paper or online questionnaire to be eligible for official analysis. CEQ responses collected by telephone were excluded from the final analysis file to minimise the potential for bias due to mode effects: systematic variation in responses obtained using different data collection methods. For the 2010 CEQ, however, telephone data collection was permitted to maximise response rates, with responses collected in this manner included in the final analysis file for the first time. In all, nearly a tenth of all valid responses to the 2010 CEQ were collected by telephone, with institutional use of telephone data collection ranging from 18 to 56% of all responses received for that institution. Using regression and matching methods, this article seeks to identify mode effects in the 2010 CEQ data that cannot be attributed to compositional differences between the telephone and self-administered respondent samples. Implications for survey practice are also discussed. [This paper was first presented at the Annual Conference of the Australasian Association for Institutional Research, "Let the sunshine in," Gold Coast, 9-11 November 2011.]