Search

Ratings from an operational assessment centre (AC) were examined as a function of the order that candidates participated in the assessments. Assessment scheduling had a significant multivariate effect on the preconsensus exercise-based ratings. The effect was small, however, accounting for at most 1% of the rating variance. Differences in exercise order were seen as being unlikely to result in serious unfairness to applicants, although programme designers should be sensitive to the possibility of early vs. late-day performance differences., The assessment centre (AC) method has been used for many years to help make a variety of employment decisions (Thornton, 1992). Use of the method typically requires that candidates participate [...]

This study investigates the degree to which subgroup (Black-White) mean differences on various assessment center exercises (e.g., in-basket, role play) may be a function of the type of exercise employed; and furthermore, begins to explore why these different types of exercises result in subgroup differences. The sample consisted of 633 participants who completed a managerial assessment center that evaluated them on 14 ability dimensions across 7 different types of assessment exercises. In addition, each participant completed a cognitive ability measure. The results suggest that subgroup differences varied by type of assessment exercise; and furthermore that the subgroup difference appeared to be a function of the cognitive component of the exercise. Lastly, preliminary support is found that the validity of some of the assessment center exercises in predicting supervisor ratings of job performance is based, in part, on their cognitive component; however, evidence of incremental validity does exist. (Reprinted by permission of the publisher.)

The effectiveness of training programs on the managerial skills of some 60 middle managers included in a management development program was assessed through the use of a 360 feedback method. Conventional methods to evaluate effectiveness of training, which rely on reactions of participants, are deemed inefficient, necessitating the use of the 360 method. Results show that training improved the success of 360 feedback in aiding managerial skill development at the general profile of skill level but not at the individual level.

This paper discusses the roles of validity, cut score choice, and adverse impact on selection system utility using data from two concurrent validation studies. We contrast an assessment center and published aptitude test on several metrics, including validity, testing costs, adverse impact, and utility. The assessment center produced slightly lower validity than the aptitude test while costing roughly 10 times as much per candidate. In spite of these advantages for the aptitude test, the assessment center produced so much less adverse impact its operational utility would be higher given cut scores likely to be chosen in this organization. Potential concerns with applying net utility models to this type of situation are discussed in comparison to gross utility models. (Reprinted by permission of the publisher.)

Two hundred fifteen organizations in the United States provided information about multiple aspects of their assessment centers, including design, usage, and their adherence to professional guidelines and research-based suggestions for the use of this method. Results reveal that centers are usually conducted for selection, promotion, and development purposes. Supervisor recommendation plays a sizable role in choosing center participants. Most often, line managers act as assessors; they typically arrive at participant ratings through a consensus process. In general, respondents indicate close adherence to recommendations for center design and assessor training. Recommendations involving other practices (e.g., informing participants, evaluating assessors, validating center results) are frequently not followed. Furthermore, methods thought to improve predictive validity of center ratings are underutilized. Variability in center practices according to industry and center purpose was revealed. We encourage practitioners to follow recommendations for center usage, and researchers to work to better understand moderators of center validity. (Reprinted by permission of the publisher.)

Overall previous research has found assessment centres to have low construct validity. Criticism of the analytical procedures used in such research have stimulated a re-evaluation of this issue. In the present study we have examined two variables whose influence on assessment centre construct validity has been neglected. These variables are assessor type (or profession, i.e. psychologist or manager) and the number of distinguishable dimension categories required to adequately represent the ratings. Using Marsh's (1989) approach to confirmatory factor analysis for the assessment centre ratings, we have found that two different models fit the psychologist and manager data. While all the predetermined dimensions were reflected in the psychologists' ratings (i.e. high construct validity), only two broad dimension categories: performance skills and interpersonal skills, accounted for the managers' ratings. Theoretical and practical implications of the findings are discussed., The increased popularity of the assessment centre (AC) in recent years stems partly from its high criterion related validity (Gaugler, Rosenthal, Thornton & Bentson, 1987; McEvoy & Beatty, 1989; Tziner, [...]

This study addresses the question of whether developmental assessment centers are effective interventions. Career outcomes of assessment center participants (n = 151) were evaluated with respect to center ratings, developmental activity, and acceptance of feedback. Although comparison between a subgroup of assessees (n = 113) and a naturally occurring control group of non-assessees (n = 167) showed no differences in terms of career advancement, findings within the assessee sample did show that developmental activity related to advancement. Findings also showed that assessors' ratings of career motivation, rather than ratings of knowledge or skill (KS) dimensions, predicted both developmental activity and career advancement. (Reprinted by permission of the publisher.)

Manufacturing applicants' perceptions of two selection devices were examined. In Study 1, applicants (n=3,984) completed cognitive ability tests and a survey of reactions. In Study 2, a subset of applicants from Study 1 (n=194) participated in an assessment center and completed the survey. Applicants reacted favorably to the procedures but viewed the assessment center as more face valid than the cognitive tests. Applicants who perceived the selection techniques more favorably were also more satisfied with the selection process, the job, and the organization. Although applicants' perceptions of the procedures were related to job acceptance intentions, applicants' liking of the job and organization explained the largest unique variance. In future studies, applicants' job acceptance intentions and attitudes toward the job and organization should be assessed before and after administration of selection devices; not controlling for prior impressions resulted in overestimation of the contribution of applicants' perceptions of selection procedures. (Reprinted by permission of the publisher.)

Assessors evaluated assessees' performances in one of three performance conditions in an assessment center simulation: a consistent performance condition, a within-assessee performance variation condition, or a between-assessee performance variation condition. Contrast effects influenced ratings of assessees when there was variation in performance within assessees. For each assessee, the low target performance was rated lower when the assessee's prior performance had been dissimilar (high) than when the assessee's prior performance had been similar (low). Contrast effects also influenced assessors' ratings of the target assessee when there was variation in performance among assessees. The low performing target assessee was rated significantly lower when he was evaluated with two high performers than when he was evaluated with two low performers. In addition, ratings obtained when there was performance variation among assessees were more accurate than those obtained when there was no performance variation, suggesting that under some circumstances contrast effects may not lead to inferior judgments. (Reprinted by permission of the publisher.)

The construct validity of assessment center final dimension ratings was examined within a nomological network of cognitive and personality measures. Four hundred forty-one employees of a large mid-western petroleum company were assessed on 11 dimensions in two broad categories and completed four tests. Results showed that several cognitive ability measures related more strongly to performance-style dimension ratings than to interpersonal-style dimension ratings, providing evidence for convergent and discriminant validity. Correlation analysis and factor analysis support the two a priori interpersonal- and performance-style categories. The results suggest that final dimension ratings posses construct validity and that assessors can differentiate between two broad categories of assessment dimensions. (Reprinted by permission of the publisher.)

Although research has established the criterion-related validity of assessment centers for selection purposes, the construct validity of dimension ratings has not been demonstrated. A quasi-experimental design was used to investigate the influence of retranslated behavior checklists on the construct validity of dimension ratings for two assessment center exercises. Assessor use of behavior checklists increased the average convergent (i.e., same dimension across exercise) validity from .24 to .43 while decreasing the average discriminant (i.e., different dimension within exercise) validity (.47 to .41). Behavior checklist sums were moderately correlated with corresponding dimension ratings and demonstrated a comparable level of construct validity. It is suggested that using behavior checklists may improve dimension construct validity by reducing the cognitive demands placed on raters. (Reprinted by permission of the publisher.)

Validity coefficients for a single assessment center implemented in multiple locations are presented. Correction for range restriction, criterion unreliability, and sampling error did not account for a large portion of the variability in these validity coefficients. The type of assessor used, the center's administrative arrangement, and prior assessor-assessee contact moderated these validities. It is suggested that the quality of the implementation of a selection procedure when there is local latitude in its implementation is an important factor in determining the procedure's effectiveness. (Reprinted by permission of the publisher.)

Individual assessment, defined as one psychologist making an assessment decision for a personnel-related purpose about one individual, was the focus of a survey of a segment of the Division of Industrial and Organizational Psychology of the American Psychological Association. A response rate of 31.6% was obtained. The responses of 163 individuals who conduct individual assessment to questions about their practices and of 153 individuals who do not conduct individual assessment to questions concerning their attitudes toward individual assessment were examined. Findings regarding what types of assessment are typically conducted, how methods are chosen, what methods are typically used, how findings are relayed to client organizations and assesses, and how practices are evaluated are reported. Implications for training and practice as well as areas for research are discussed. (Reprinted by permission of the publisher.)

The finding that assessment centers are successful in making valid predictions of managerial success is well established in the literature. Unfortunately, it is not clearly understood why assessment centers work. For example, construct validity of assessment center dimensions is rarely obtained. In this paper we affirm the evidence for predictive validity of assessment centers and conclude that assessment centers can work for a variety of purposes and in numerous contexts. But we also assert that we do not know why they work. The bulk of the paper goes on to raise possible explanations for the predictive validity observed in assessment centers and raises implications for practice and guidance of future research. (Reprinted by permission of the publisher.)

Employee assessment techniques used by supervisors to select, organize and recall data influence the manner in which the supervisors assess employee performance and their ratings of employees. Two methodologies were tested, with independent assessor groups each using one, to appraise candidates in an assessment center. Results of the tests verify that the assessment methodologies used influence the employee ratings. More research is necessary to learn the effect of varying assessment center methodologies on ratings of candidates.

Assessment center ratings of 52 state law enforcement agency managers predicted both a contaminated promotions criterion and two uncontaminated ratings criteria, but they did significantly better in predicting the former. Criteria data were gathered two, four, and seven years after the assessment center, and a modest upward trend in validity coefficients was observed. Ratings by subordinates demonstrated promise as a near term predictor, outperforming the assessment center on uncontaminated criteria in the two- and four-year time frames. Last, regression analyses using the Year Seven criteria revealed unique predictive power for both ratings by subordinates and assessment centers above and beyond that provided by supervisor ratings. (Reprinted with permission of the publisher.)

Literature on the use of content validity to establish the job-relatedness of assessment centers is reviewed. Most approaches to content validity are shown to focus on job-analysis-based exercise construction and dimension selection as the bases for determining the content validity of an assessment center. The article attempts to show that content validity requires more that the careful construction of stimulus materials (i.e., exercises); how these stimulus materials are presented to candidates and how responses to these stimuli are evaluated are also critical considerations in making judgments about content validity. Variability in the way these issues are dealt with is documented. (Reprinted by permission of the publisher.)