Search

This study investigated within-source interrater reliability of supervisor, peer, and subordinate feedback ratings made for managerial development. Raters provided 360-degree feedback ratings on a sample of 153 managers. Using generalizability theory, results indicated that little within-source agreement exists; a large portion of the error variance is attributable to the combined rater main effect and Rater x Ratee effect; more raters are needed than currently used to reach acceptable levels of reliability; supervisors are the most reliable with trivial differences between peers and subordinates when the numbers of raters and items are held constant; and peers are the most reliable, followed by subordinates, followed by supervisors, under conditions commonly encountered in practice. Implications for the validity, design, and maintenance of 360-degree feedback systems are discussed along with directions for future research in this area.

The authors investigated the relative validity of personality testing and the assessment center (AC) in order to evaluate the incremental validity of both assessment methods in the prediction of managerial performance. Managers (N = 68) from a forestry products organization were evaluated with an AC and selected traits on the Personality Research Form (D. N. Jackson, 1984). Performance and promotability served as criteria. Overall, personality was found to be at least equivalent to the AC in predicting performance. Personality testing resulted in significant incremental validity over that of the AC in the prediction of performance, but the converse was also true. This suggested that personality and the AC assess different domains, with each uniquely and significantly predicting performance. Promotability was not significantly predicted. Practical implications and future directions were discussed.

Videotaping of assessment center exercises has become an increasingly common practice, yet little is known about the impact of video technology on rating accuracy. This study compared ratings of a group discussion made after live observation (direct), after viewing a video (indirect), or after viewing a video with opportunities to pause and rewind (controlled). Results indicated some differences in observational accuracy but not in rating accuracy. Implications for the use of video technology in assessment centers are discussed. (Reprinted by permission of the publisher.)

The author investigates whether the extent to which participants recognize rating dimensions in assessment centers has an effect on performance. Results showed that people who more accurately identify dimensions perform better. Convergent validity of dimension ratings is greater when participants accurately perceive that the same dimension is being evaluated in 2 exercises. In the discussion, the author considers how the ability to adequately perceive rating dimensions (i.e. transparency) may influence criterion validity. (Reprinted by permission of the publisher.)

The importance of internal, cognitive or covert support mechanisms to ensure successful application of managerial skills is highlighted. This durability factor has often been left out of the managerial success equation. But consistent application of managerial qualities demands a supportive network to ensure durability. Thus, the introduction of cognitive behavior modification-based management consultations is seen to alter maladaptive thoughts, behaviors and affect which hinder job performance. A skillbuilding managerial framework is laid out for the effective implementation of such cognitive behavioral techniques, even in hard business environments.

Successful managers develop practical theories or frameworks to guide them in problem solving and decision making. A practical theory can be a key element in focusing and handling change and in managerial development. A managerial framework is most useful when it conceptualizes and outlines a vision for people based on the assumptions and values of managers and employees. The development of a managerial framework in a long-term care hospital is described, with focus on performance appraisal, training and career development.

Assessors evaluated assessees' performances in one of three performance conditions in an assessment center simulation: a consistent performance condition, a within-assessee performance variation condition, or a between-assessee performance variation condition. Contrast effects influenced ratings of assessees when there was variation in performance within assessees. For each assessee, the low target performance was rated lower when the assessee's prior performance had been dissimilar (high) than when the assessee's prior performance had been similar (low). Contrast effects also influenced assessors' ratings of the target assessee when there was variation in performance among assessees. The low performing target assessee was rated significantly lower when he was evaluated with two high performers than when he was evaluated with two low performers. In addition, ratings obtained when there was performance variation among assessees were more accurate than those obtained when there was no performance variation, suggesting that under some circumstances contrast effects may not lead to inferior judgments. (Reprinted by permission of the publisher.)

The effects of participation in an assessment center program was studied in relation to the self-assessments made by candidates just before attendance in the assessment center program, immediately after participation in the assessment center program, and a half year after attendance in the assessment center program. The self-assessments of the candidates at these three different time periods were compared to the ratings they received from assessors during the assessment program in terms of accuracy. Unsuccessful candidates were observed to have greatly misjudged their performance effectiveness both before and after participation in the assessment center program, indicating the lack of adequate feedback systems after assessment.

A robust finding in assessment center research is that ratings cluster according to exercises and not dimensions. Seeking further understanding of exercise effects, we proposed two exercise-based factors, exercise form and exercise content, as sources of variance in assessment center ratings. Exercise designs were manipulated so that two levels of form (leaderless group discussions and role play exercises) were crossed with two levels of content (cooperative and competitive task designs). Analysis of variance and confirmatory factor analysis procedures applied to multitrait-multimethod ratings of 89 high school student assessees revealed that most of the variance in the ratings was explained by exercises and not dimensions. Exercise form accounted for 16% of method variance, whereas exercise content had near zero effect. The form effect was primarily due to higher correlations of dimension ratings across role play exercises. Practical implications and future directions for exercise research design research are discussed. (Reprinted by permission of the publisher.)

The construct validity of final self- and peer evaluations in an assessment center was examined within a nomological network of conceptually related and unrelated variables. Data included self-, peer, and assessor evaluations, cognitive ability and personality measures, and job advancement. The evidence for construct validity was stronger for peer than for self-evaluations, and for more easily observable dimensions than for dimensions requiring greater inferential judgment. Self- and peer evaluations were associated with assessor ratings of management potential, whereas only peer evaluations predicted job advancement. Implications for the use of self- and peer evaluations in assessment centers and the need for further research are discussed. (Reprinted by permission of the publisher.)

Undergraduates (N = 131) were trained as assessors who, evaluated the performance of confederates in an assessment center simulation on 3, 6, or 9 dimensions. Number of dimensions significantly affected some assessment center judgments but not others. Ss who rated a small number of dimensions classified behaviors more accurately and made more accurate ratings than did Ss who rated a large number of dimensions. Number of dimensions did not affect the accuracy of assessors' observations nor the discriminant validity of their dimension ratings. Given these results and the findings of others (Hinrichs & Haanpera, 1976; Russell, 1985; Sackett & Hakel, 1979; Schmitt, 1977), developers of assessment centers should limit the cognitive demands placed on assessors by, for example, minimizing the number of dimensions assessors are required to process. (Reprinted by permission of the publisher.)

The finding that assessment centers are successful in making valid predictions of managerial success is well established in the literature. Unfortunately, it is not clearly understood why assessment centers work. For example, construct validity of assessment center dimensions is rarely obtained. In this paper we affirm the evidence for predictive validity of assessment centers and conclude that assessment centers can work for a variety of purposes and in numerous contexts. But we also assert that we do not know why they work. The bulk of the paper goes on to raise possible explanations for the predictive validity observed in assessment centers and raises implications for practice and guidance of future research. (Reprinted by permission of the publisher.)

Employee assessment techniques used by supervisors to select, organize and recall data influence the manner in which the supervisors assess employee performance and their ratings of employees. Two methodologies were tested, with independent assessor groups each using one, to appraise candidates in an assessment center. Results of the tests verify that the assessment methodologies used influence the employee ratings. More research is necessary to learn the effect of varying assessment center methodologies on ratings of candidates.

Assessment center ratings on 275 police recruits correlated .14 (p