Search

This study explores stereotype threat on low-stakes tests used in a large-scale assessment, math and reading tests in the Education Longitudinal Study of 2002 (ELS). Issues identified in laboratory research (though not observed in studies of high-stakes tests) were assessed: whether inquiring about their race and gender is related to the performance of Black and female test takers and, secondarily, whether this association is greater for test takers most identified with math and reading. After high school sophomores completed a questionnaire that included inquiries about their race and gender, only one change in test performance was consistent with expectations from stereotype-threat theory: Black test takers' math scores decreased. Their reading scores and young women's math scores did not decrease, and identification with math and reading did not moderate score decreases for Black test takers or women.

This is an account of a portion of the research on cognitive, personality, and social psychology at ETS since the organization's inception. The topics in cognitive psychology are the structure of abilities; in personality psychology, response styles and social and emotional intelligence; and in social psychology, prosocial behavior and stereotype threat. Research on motivation is also covered.

This study assessed the invariance in the factor structure of the "Test of English as a Foreign Language"™ Internet-based test (TOEFL® iBT) across subgroups of test takers who differed in native language and exposure to the English language. The subgroups were defined by (a) Indo-European and Non-Indo-European language family, (b) Kachru's classification of outer and expanding circles of countries (based on prevalence of English use in educational and business contexts), and (c) years of classroom instruction in the English language. The same factor structure (four first-order factors corresponding to the test sections and a single higher-order factor encompassing these factors) was identified in each subgroup. The results support the present scoring scheme for the TOEFL iBT assessment and suggest that the test functions the same way for diverse subgroups of test takers.

This study assessed the factor structure of the LanguEdge™ test and the invariance of its factors across language groups. Confirmatory factor analyses of individual tasks and subsets of items in the four sections of the test, Listening, Reading, Speaking, and Writing, was carried out for Arabic-, Chinese-, and Spanish-speaking test takers. Two factors were identified, Speaking and a fusion of the other sections of the test. The number of factors, the factor loadings, and the factors' error variances were invariant in the three samples, although the correlations between the factors differed. The failure to find separate factors for each section of the LanguEdge test necessarily raises questions about the test's functioning that need to be resolved.

A biographical inventory has been used in the selection of students for naval aviation training since World War II, and its validity in predicting their retention-attrition in this training has been well established. This study investigated the constructs underlying the inventory and their relations to students' retention-attrition. A factor analysis of the items on the inventory for student pilots identified five factors. One factor, being a commissioned officer, appeared to account for the inventory's validity.

This construct validation study investigated the factor structure of the Test of English as a Foreign Language[TM] Internet-based test (TOEFL[R] iBT). An item-level confirmatory factor analysis was conducted for a test form completed by participants in a field study. A higher-order factor model was identified, with a higher-order general factor (ESL/EFL ability) and four first-order factors for reading, listening, speaking and writing. Integrated Speaking and Writing tasks, which require language processing in multiple modalities, defined the target modalities (speaking and writing). These results broadly supported the current practice of reporting a total score and four scores corresponding to the modalities for the test, as well as the test design that permits the integrated tasks to contribute only to the scores of the target modalities. (Contains 1 footnote, 4 tables, and 5 figures.)

The aim of this study was to assess test takers' attitudes and beliefs about an admissions test used extensively in graduate schools of business in the United States, the Graduate Management Admission Test (GMAT), and the relationships of these attitudes and beliefs to test performance. A set of attitude and belief items was administered by computer to GMAT test takers after they finished the computer-based test. On average, test takers reported moderately negative attitudes about whether the test was valid and moderately positive attitudes about whether the test was unbiased. Few test takers reported that they believed their test performance reflected innate ability. The results were similar for men and women, and for different ethnic groups, though White test takers' attitudes about whether the test was unbiased were generally more positive, and Black test takers' attitudes were generally less positive than those of other ethnic groups. Test takers' reported attitudes and beliefs were slightly related to their test performance. These relationships were similar for men and women, but varied for some ethnic groups.

This study investigated the extent and nature of preparation for the Pre-Professional Skills Tests (PPST), and the reasons for preparing or not preparing. The PPST, a high-stakes test with a high failure rate, is used for admission to teacher education programs and for teacher licensing. Recent test takers were surveyed. Preparation was limited and mainly involved activities that were free or inexpensive, such as taking a sample test. The reported reasons for not preparing and the empirical correlates of measures of preparation were primarily attitudinal.

Studied sex and ethnic group differences on 6 scales that measure accomplishments for 739 male and 1,746 female graduate school applicants. With the exception of the scale measuring mechanical accomplishment, men and women did not differ in performance, and ethnic groups did not differ on any scale. (SLD)

A study investigated differences in the length of time taken to earn a doctoral degree in three disciplines--chemistry, English, and psychology. The clearest findings emerged for psychology, where institutional variables affecting time to degree concerned faculty accessibility and department size. (Author/MSE)

To study whether different forms of the Scholastic Aptitude Test (SAT) used since the mid-1970s varied in their correlations with academic performance criteria, 1975 and 1985 forms were administered to 1,554 and 1,753 high school juniors, respectively. The 1975 form did not have greater validity than the 1985 form. (SLD)

A recent multidimensional scaling analysis of the Test of English-as-a-Foreign-Language (TOEFL) item response data identified clusters of items in the test sections that, being more homogeneous than their parent sections, might be better for diagnostic use. The analysis was repeated using different scoring techniques. Results diverged only for high- and low-scoring examinees. (Author/JL)

To purchase or authenticate to the full-text of this article, please visit this link: http://dx.doi.org/10.1111/j.1559-1816.2008.00363.x Byline: Lawrence J. Stricker (a2), William C. Ward (a) Abstract: This is a rebuttal to Danaher & Crandall's (2008) rejoinder to Stricker & Ward's (2004) article. The article reports 2 field experiments concerning the effects of inquiring about gender and ethnicity on the performance of women and Black students taking operational tests. The rejoinder's conclusion (after reanalyzing our data) that several thousand more young women would pass the AP calculus test if they were not asked about their gender, thereby reducing stereotype threat, is unwarranted because of flawed estimates and extrapolations. Our findings are selectively described, distorting their overall thrust. Criticisms of our statistical analyses are debatable or incorrect. Our original conclusion appears justified: Inquiring about gender and ethnicity did not affect test performance of women or Black students. Author Affiliation: (a)Educational Testing ServicePrinceton, NJ Article note: (2) Lawrence J. Stricker, Educational Testing Service, Princeton, NJ 08541. E-mail: lstricker@ets.org

Steele and Aronson (1995) found that the performance of Black research participants on ability test items portrayed as a problem-solving task, in laboratory experiments, was affected adversely when they were asked about their ethnicity. This outcome was attributed to stereotype threat: Performance was disrupted by participants' concerns about fulfilling the negative stereotype concerning Black people's intellectual ability. The present field experiments extended that research to other ethnic groups and to males and females taking operational tests. The experiments evaluated the effects of inquiring about ethnicity and gender on the performance of students taking 2 standardized tests--the Advanced Placement Calculus AB Examination, and the Computerized Placement Tests--in actual test administrations. This inquiry did not have any effects on the test performance of Black, female, or other subgroups of students that were both statistically and practically significant.

This study investigated (a) the ability to minimize or eliminate stereotype threat by reducing the difficulty of items administered via a computer-adaptive version of the Graduate Record Examinations General Test; and (b) the generalizability of these findings for Black students as well as women, and for verbal as well as quantitative sections of the test. Standard and easier versions of the test, and measures of stereotype threat and possible symptoms or sequelae of stereotype threat were administered to students bound for graduate school or already there. Reducing test difficulty did not have any differential effects on test performance or on explicit indexes of stereotype threat for White and Black students, and for men and women, that were statistically and practically significant. However, such effects did occur for some symptoms or sequelae, mainly for White students and women.

College-related characteristics and undergraduate performance are strongly related to Graduate Record Examination (GRE) performance compared to the initial characteristics of the students. Gender, ethnicity, and geographic regions of residence, have moderate relations with test performance. The association is more consistent and stronger with parental education. Socioeconomic status is an important correlate of performance. The weakest association is found with age.

The authors compared the effectiveness of several methods for statistically adjusting college grade point average (GPA) criteria for course and departmental differences in grading standards, using first-semester grades from an entire entering class at a large state university. Most of the adjusted GPAs produced by these methods functioned similarly and, despite high correlations with actual GPA, had greater internal-consistency reliability than actual GPA and were more predictable from admission measures, the standard predictors of GPA criteria. These methods are potentially useful in dealing with the lack of comparability of other criterion measures in applied settings that rely on subjective evaluations by different raters or on objective indexes of behavior in different organizational units. (Reprinted by permission of the publisher.)

In this study we appraised 2 explanations for sex differences in over- and underprediction of college grades by the Scholastic Aptitude Test: sex-related differences in (a) the nature of the grade criterion and (b) the variables associated with academic performance. An entire freshman class at a large state university was studied. Women's grade point average (GPA) was underpredicted (and men's GPA was overpredicted), but only by a small amount. When we adjusted the GPA for differences in grading standards for individual courses, over- and underprediction were not affected, but when we controlled for sex differences in individual-differences variables concerned with academic preparation, studiousness, and attitudes about mathematics, over- and underprediction were reduced.

Multidimensional scaling was used to analyze item response data for the Test of English as a Foreign Language (TOEFL) to uncover the dimensions underlying the test. Four dimensions were identified for samples varying in native language and level of English proficiency: 3 corresponded to the test's sections and 1 was an end-of-test phenomenon. Dimensions were predominantly defined by easy items and were most important for low-scoring examinees. The dimensions' importance did not differ across language groups, except for the end-of-test dimension. Major conclusions were that (a) the TOEFL measures the intended constructs; (b) the test assesses the same constructs in each language group, but the constructs are more differentiated for low-scorers; and (c) easy and difficult items differ in what they measure. Multidimensional scaling appears to be a useful for item-level analyses of test structure. (Reprinted by permission of the publisher.)

The stability was evaluated of a partial correlation index, comparisons of item characteristic curves, and comparisions of item difficulties in assessing race and sex differences in the performance of verbal items on the Graduate Record Examination Aptitude Test. All three indexes exhibited consistency in identifying the same items in different samples. (Author/BW)

Evaluated the extent to which the Graduate Record Examinations General Test measures the same constructs for older test takers that it does for younger examinees. Results suggest that the convergent validity of the test is similar across the age groups, but discriminant validity is somewhat different for older examinees. (Author/RWB)

The test takers' acceptance of the computer-based version of the Test of English as a Foreign Language (TOEFL) was accessed. A questionnaire concerning attitudes about the test, familiarity with computers and other relevant variables was administered to TOEFL test takers at large testing centers.

Previous research assessing Obama's effectiveness as a role model in alleviating the effects of stereotype threat on Black Americans' test performance yielded provocative though conflicting results. A field study with research participants observed that Black–White mean differences were not detectable at points in his 2008 presidential campaign when he clearly succeeded—his nomination and election, although they persisted at other points. But a laboratory experiment found that prompts to think positively about Obama had no effect. The present study extended this research to actual test-takers and an operational test (GRE General Test). Black–White mean differences just after the election in November 2008 were substantial and comparable to earlier differences, in November 2006; the level of Black test-takers' performance was also unchanged. [ABSTRACT FROM PUBLISHER]

The purpose of this study was to replicate previous research on the construct validity of the paper-based version of the TOEFL and extend it to the computer-based TOEFL. Two samples of Graduate Record Examination (GRE) General Test-takers were used: native speakers of English specially recruited to take the computer-based TOEFL, and ESL test-takers who routinely took the computer-based TOEFL recently. Native speakers performed well on the TOEFL, relative to ESL test-takers and to the maximum possible scores on the test, and varied less in their test performance than did ESL test-takers. TOEFL scores were moderately or highly correlated with the computer-based GRE General Test scores for both groups of test-takers. Regressions of the GRE General Test Verbal scores on the TOEFL scores for ESL test-takers were nonlinear, and the regressions of the other GRE General Test scores were linear. And trends in the variances of the GRE General Test Verbal scores associated with TOEFL scores were also non-linear, and the trends were either unsystematic or negatively linear for the other GRE General Test scores. All of the findings are consistent with previous results with the paper-based TOEFL, support the construct validity of the computer-based TOEFL, and underscore its usefulness in interpreting the performances of ESL test-takers on admissions tests for English-speaking test-takers. [ABSTRACT FROM AUTHOR]

Evaluates the utility of the Scholastic Aptitude Test (SAT) scores in combination with the grades in high school courses and the number and quality of these courses in predicting college grades in various fields of study. Assessment of possible impact of the evaluation on the flow of students into these fields; How predictions were made using SAT and collateral variables.

This study is concerned with the ability of standard indexes, of socioeconomic status and similar devices to measure major dimensions of social stratification for whites and blacks. Factor analyses of survey data for samples of both races uncovered five matching factors and two others unique to a single sample, all of which seem to represent important components of stratification. Most of the indexes were relevant to three matching factors, and some of these measures consistently loaded them in both samples. However, several of the pertinent indexes only loaded white versions of these factors, and some failed to load any. Few, if any, indexes were relevant to or loaded the remaining factors. Additionally, some indexes were not pertinent to and did not load any factor. In contrast to the indexes, other variables loaded all but one of the factors. The main conclusions are that: (1) the coverage of the indexes is restricted; (2) the devices are not uniformly applicable to blacks; and (3) the complexity of this domain requires separate measures of each factor, achievable by combining standard indexes and new variables. [ABSTRACT FROM AUTHOR]

The aim of this study was to demonstrate the feasibility of biographical inventories free of the limitations common to many current biographical measures by constructing and validating an inventory composed of homogeneous scales, with item content that is factual and fair, to assess personality traits predictive of leadership. The experimental inventory, with tentative scales for Dominance, Emotional Stability, Need for Achievement, Self-Confidence, and Sociability, was administered to incoming midshipmen at the US Naval Academy and item analyzed. The validity of the final scales was appraised against subsequent peer ratings of the traits and of leadership shortly after entering the Academy; military performance grades, and midshipman rank and leadership position at the Academy; and recommendations for promotion after graduating and serving as an officer. The Sociability, Dominance, and Need for Achievement scales demonstrated convergent and discriminant validity, and all of the scales except Self-Confidence correlated with the leadership criteria. The Sociability scale's consistent correlations with several of the leadership criteria, including a nontrivial correlation (r = 0.28) with the leadership peer rating, implies that this measure may be useful in assessing leadership potential. [ABSTRACT FROM AUTHOR]

The aim of this study was to determine the major dimensions of social stratification for whites as well as blacks. A survey was conducted with household heads in the Toledo, Ohio area of both races, using an interview that covered a comprehensive set of potentially important variables. Eighteen first-order factors were found for whites and 19 for blacks. Five factors matched in the two samples: social status, residence, organization activity, political activity and efficacy, and main support's social status. Other important factors, unique to each sample, were: for whites, self-employed and majority group membership, and for blacks, class consciousness and method variance. Second- and third-order factors were also obtained, but they were difficult to interpret. The findings indicate that stratification is more complex than anticipated by current conceptualizations and previous research. [ABSTRACT FROM AUTHOR]

The membership of the Society for Personality and Social Psychology was surveyed in order to determine its opinions about a variety of matters concerning the Society, including the relations of the Society and its members with the American Psychological Association, the Society's structure, its publications, and its convention program. How the survey was carried out is described, and its main findings are summarized. [ABSTRACT FROM PUBLISHER]

The Tzeng and Tzeng (1982) criticisms of the assumptions underlying the Jackson, Chan, and Stricker (1979) study of Implicit Personality Theory fail to take account of the relevant empirical data. For example, the key contention of Tzeng and Tzeng—that the measures of judged and empirical trait-co-occurrence used by Jackson et al. were not comparable—is contradicted by replicated findings which establish that the two are substantially and consistently related, results for which Tzeng and Tzeng offer no alternative explanation. In short, Tzeng and Tzeng have not demonstrated that their criticisms have any real substance. Hence, none of the Jackson et al. results and conclusions, including those questioning the conditional probability index as a measure of empirical co-occurrence and those supporting the validity of Implicit Personality Theory, are affected or changed. [ABSTRACT FROM AUTHOR]