Search

This investigation examined the effects of 3 item characteristics-the average number of words per item, within-scale variability in item length, and item "direction"-on internal consistency reliability and interitem correlation. In Study 1, we examined the effects of these variables on overall scale-level reliability using 444 subscales from 9 personality scales. In Study 2, we examined interitem correlation at the paired-item level using 477 nonredundant item pairs from 14 personality scales. Lower scale reliability was associated with more average words per item, greater within-scale variability in item length, and a greater percentage of reverse-keyed items. Similarly, smaller interitem correlations were associated with a greater degree of mismatch in item length between the paired items and with a mismatch (vs. match) in the items' respective "directions." The pattern of results across both studies supports our notion that lower internal consistency results from increased context switching; that is, from the confusion that occurs when respondents must switch back and forth between the interpretive frames pertaining to short versus long items, or between items pertaining to one pole of a personality dimension and its "opposite" pole. Suggestions for maximizing the internal consistency of personality scales are proposed.

This study tested implications of the context switching perspective proposed by Hamby, Ickes, and Babcock ( 2016 ). Using trained raters to assess the amount of reframing required to interpret the meaning of the subsequent (second) item within all adjacent item pairs, we first established that this process variable could be measured reliably. Then, in the data for 18 personality measures drawn from 3 individual-difference domains, we found that the amount of reframing (i.e., context switching) needed to interpret successive items predicted both lower interitem correlations and a greater percentage of misresponders. Similarly, item pairs that were mismatched in "directional" wording also predicted both lower interitem correlations and more misresponders. Finally, item pairs representing different factors predicted lower interitem correlations. Although the effects of direction switching and factor switching were partially mediated by the amount of reframing required, they remained significant even when the mediating effect of reframing was statistically controlled. These results indicate that interpreting the meaning of test items is a task for which the level of difficulty can vary with each successive item, as a function of how the current item compares to the previous item in aspects such as its context generality or specificity, directional wording, and content domain.