From Likert to Forced Choice: Statement Parameter Invariance and Context Effects in Personality Assessment

Users of forced-choice questionnaires (FCQs) to measure personality commonly assume statement parameter invariance across contexts -- between Likert and forced-choice (FC) items and between different FC items that share a common statement. In this paper, an empirical study was designed to check these two assumptions for an FCQ assessment measuring interpersonal and intrapersonal skills. We compared parameters of common statements between two Likert forms and two FCQ forms with a block size of two (statement pairs) and among five FCQ pair forms. In three of the five FCQ forms, statements were paired only with a statement they had appeared with in a triplet block. In the other two FCQ forms, statements were paired with statements they had not been paired with in a triplet block. This design allows us to evaluate statement parameter changes due to changes in context. The results do not support the statement parameter invariance assumption between Likert and FC items or the assumption between FC items when recombining statements form new items. However, the assumption between FC items was generally held for pairs formed by dropping a statement from a triplet item. There were some suggestions for sources of context effects, but the analyses were not definitive. Implications of the findings for test practice are discussed.