Recalled childhood gender role/identity is a construct that is related to sexual orientation, abuse, and psychological health. The purpose of this study was to assess the factorial validity of a short version of Zucker et al.'s (2006) "Recalled Childhood Gender Identity/Gender Role Questionnaire" using confirmatory factor analysis and to test the stability of the factor structure across groups (measurement invariance). Six items of the questionnaire were completed online by 1929 participants from a variety of gender identity and sexual orientation groups. Models of the six items loading onto one factor had poor fit for the data. Items were removed for having a large proportion of error variance. Among birth-assigned females, a five-item model had good fit for the data, but there was evidence for differences in scale's factor structure across gender identity, age, level of education, and country groups. Among birth-assigned males, the resulting four-item model did not account for all of the relationship between variables, and modeling for this resulted in a model that was almost saturated. This model also had evidence of measurement variance across gender identity and sexual orientation groups. The models had good reliability and factor score determinacy. These findings suggest that results of previous studies that have assessed recalled childhood gender role/identity may have been susceptible to construct bias due to measurement variance across these groups. Future studies should assess measurement invariance between groups they are comparing, and if it is not found the issue can be addressed by removing variant indicators and/or applying a partial invariance model. [ABSTRACT FROM AUTHOR]