51. Investigating Lay Perceptions of Psychological Measures: A Registered Report.
- Author
-
Mason, Joseph, Pownall, Madeleine, Palmer, Amy, and Azevedo, Flavio
- Subjects
PROTOCOL analysis (Cognition) ,PSYCHOMETRICS ,COGNITIVE interviewing ,PSYCHOLOGICAL research ,CONTEXT effects (Psychology) ,PUBLIC opinion - Abstract
In recent years, the reliability and validity of psychology measurement practices has been called into question, as part of an ongoing reappraisal of the robustness, reproducibility, and transparency of psychological research. While useful progress has been made, to date, the majority of discussions surrounding psychology's measurement crisis have involved technical, quantitative investigations into the validity, reliability, and statistical robustness of psychological measures. This registered report offers a seldom-heard qualitative perspective on these ongoing debates, critically exploring members of the general public's (i.e., non-experts) lay perceptions of widely used measures in psychology. Using a combination of cognitive interviews and a think aloud study protocol, participants (n = 23) completed one of three popular psychology measures. Participants reflected on each of the measures, discussed the contents, and provided perceptions of what the measures are designed to test. Coding of the think aloud protocols showed that participants across the measures had issues in interpreting and responding to items. Thematic analysis of the cognitive interviews identified three dominant themes that each relate to lay perceptions of psychology measurements. These were: (1) participants' grappling with attempting to 'capture their multiple selves' in the questionnaires, (2) participants perceiving the questionnaire method as generally 'missing nuance and richness' and (3) exposing the 'hidden labour of questionnaires'. These findings are discussed in the context of psychology's measurement reform. Highlights: In recent years, there have been concerns surrounding the robustness, reliability, and validity of psychological measurement practices. This study uses cognitive interviews and a think-aloud protocol with lay (i.e., non-expert) people to explore public perceptions of popular psychology measures. Generally, lay people experience issues with completing some of psychology's most popular measures and find them to lack nuance. These results demonstrate the value of adopting qualitative methods in the reappraisal of psychology's measurement crisis. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF