Search

During the 2020-2022 COVID-19 global pandemic many researchers had to adopt or switch to online testing, which emphasized the need for ensuring measurement equivalence across different administration methods. The aim of this study was to determine whether gender stereotype measure (VGSA scale; Vocational Gender Stereotype Attitudes Scale) is equivalent in two different computer-based administration modes: supervised in-class and unsupervised at-home web survey. In-class web survey participants were 219 primary (11 to 16 years old; M = 12.97; SD = 1.16) and 195 secondary (15 to 19 years old; M = 16.78; SD = 0.96) school students that were surveyed in the group setting during a regular school day in their computer classrooms while the researcher was present in the room. At-home web survey participants were 144 primary (10 to 15 years old; M = 11.60; SD = 1.43) and 212 secondary (14 to 19 years old; M = 16.78; SD = 1.22) school students that were surveyed individually through the web link in their homes that was administered to them via the school administration. Results confirmed VGSA scale’s high internal reliability (ɑin-class = .89; ɑat-home = .92), expected bi-factor structure (χ2in-class (12) = 22.56, p = .032; χ2at-home(12) = 17.24, p = .141), and scalar measurement invariance across administration methods (Δχ2(21) = 49.05, p < .001; ΔCFI = -.003; ΔTLI = -.001; ΔRMSEA = .009; ΔSRMR = .013).

The effects of verbal and/or numerical labeling and number of categories on the measurement of latent variables in web surveys are addressed. Data were collected online in a quota sample of the German adult population (N = 741). A randomized 2x2x2 experimental design was applied, with variation of the number of categories, as well as of verbal and numerical labeling, using an abbreviated version of the Positive and Negative Affect Schedule (PANAS). Experimental manipulation of the rating scale formats resulted in an effect on measurement model testing and reliability, as well as on factorial and convergent validity. In addition, measurement invariance between several rating scale formats was limited. With the five category end verbalized and fully labeled seven category formats, acceptable results for all measurement quality metrics could be obtained.

In cross-cultural comparative studies it is essential to establish equivalent measurement of relevant constructs across cultures. If this equivalence is not confirmed it is difficult if not impossible to make meaningful comparison of results across countries. This work presents concept of measurement equivalence, its relationship with other related concepts, different equivalence levels and causes of inequivalence in cross-cultural research. It also reviews three main approaches to the analysis of measurement equivalence – multigroup confirmatory factor analysis, differential item functioning, and multigroup latent class analysis – with special emphasis on their similarities and differences, as well as comparative advantages.