Search

Detecting participants not exerting sufficient effort (e.g., answering randomly; termed 'insufficient effort responding' or IER) in self-report surveys is crucial but intricate. Challenges of appropriately removing IER resemble a surgeon excising necrotic tissue without being inadequate, excessive, or incorrect. Current cutoff criteria for stopping the elimination of suspected IER responses are often arbitrary. This study proposes an external criterion strategy to identify the optimal cutoff values for various IER detection methods. We investigated the change in correlations between the IER-containing scale (e.g., motivation) and the external criterion (e.g., academic performance, socioeconomic status index, or another questionnaire scale) utilizing simulated and authentic international survey data. The findings revealed that the stopping rule and consequently, the optimal cutoff values for IER detection methods, can be accurately identified by locating the inflection point in the correlation plot with the external criterion. Practical strategies are recommended for applied researchers. [ABSTRACT FROM AUTHOR]

Short scales are time‐efficient for participants and cost‐effective in research. However, researchers often mistakenly expect short scales to have the same reliability as long ones without considering the effect of scale length. We argue that applying a universal benchmark for alpha is problematic as the impact of low‐quality items is greater on shorter scales. In this study, we proposed simple guidelines for item reduction using the "alpha‐if‐item‐deleted" procedure in scale construction. An item can be removed if alpha increases or decreases by less than.02, especially for short scales. Conversely, an item should be retained if alpha decreases by more than.04 upon its removal. For reliability benchmarks,.80 is relatively safe in most conditions, but higher benchmarks are recommended for longer scales and smaller sample sizes. Supplementary analyses, including item content, face validity, and content coverage, are critical to ensure scale quality. [ABSTRACT FROM AUTHOR]