Search

Supervisors' opportunity to observe incumbents' performance (i.e., how often a supervisor typically sees an employee's performance) has been suggested to be important for accurate performance rating and to be a moderator of criterion‐related validity. Here we test these suggestions and present empirical evidence for the effects of opportunity to observe. In Study 1, supervisors in a multi‐occupation/organization criterion‐related validation study for a biodata measure indicated the opportunity they had to observe incumbents. The data were split according to different levels of opportunity to observe. Higher validities were found when opportunity to observe was maximal. In Study 2, this finding was replicated using a cognitive ability test. These results suggest that psychologists should consider measuring opportunity to observe in criterion‐related validity studies.

Summary In recent years, there has been an increasing emphasis on the role of theory in organizational behavior (OB) research. Authors are strongly encouraged to develop “theory” in their manuscripts and to make “theoretical contributions.” This trend is in stark contrast to the process used in other fields of science. Our counterparts in those fields follow the scientific method and define theory as a concise and elegant hypothesis that has survived extensive empirical testing. Rather than being based on extensive empirical research, many of OB's theories are based on a limited number of primary studies (at best) or speculation and conjecture (at worst). OB researchers are discouraged from testing other researchers' theories or replicating previously published work. Consequently, many OB theories do not meet the criteria for a true scientific theory. We propose that OB researchers should re-embrace the scientific method and focus on creating a body of empirical research that could be used in the future to establish true scientific theories (through extensive hypothesis testing, empirical research, and conceptual replications) rather than concocting pseudotheories. Research in the personnel selection subfield of OB provides a reasonable exemplar of this model, yet it has been derided of late for being too atheoretical. Copyright © 2016 John Wiley & Sons, Ltd.

The positive relationship between complexity of work and the validity of general mental ability (GMA) measures across a variety of occupations is well supported by research and provides important practical and theoretical support for cognitive ability measures. However, there is currently no research demonstrating a systematic relationship between the size of the validities of any personality measure and the personality requirements of jobs, thus leaving open to question the predictive and construct validity of personality measures for applicant selection. We compared the validities of two biodata measures – one scored to measure social competence and one to measure GMA – across six job families that varied in social requirements. The validity of the GMA measure generalized at approximately the same level across the job families while the validity of the social competence measure decreased as social requirements diminished.

A conceptual rationale is presented for the use of a high-fidelity simulation: a video-based test (VBT), in which applicants view job-related scenarios, respond orally, and are later scored on five dimensions by trained raters. Using applicant data, we empirically investigate the criterion-related validity, reliability, and utility of the VBT. Overall, we found that the VBT predicted performance in training and on the job. In addition, different raters strongly agreed with one another on each applicant's scores. Despite somewhat higher development costs, the VBT demonstrated high utility. We suggest that organizations consider the use of VBTs as a selection instrument for jobs that require the use of applied social and interpersonal skills.

Five quasi-rational biodata scales were developed by empirically keying biodata items to predict scores on a measure of the Big Five dimensions of personality. The criterion-related validities of the quasi-rational scales were compared to empirical and rational biodata keying methods using supervisory ratings of job performance as the criterion. Empirical keying outperformed the quasi-rational and rational methods (which had similar validities).

The criterion-related validities of empirical, rational, and hybrid keying procedures for a biodata inventory were compared at different sample sizes. Rational keying yielded the lowest validities. Hybrid keying performed best at the smallest sample sizes studied, followed by empirical keying at moderate sizes, and stepwise regression weighting of items at the largest sample sizes.

This study investigated linear and curvilinear relationships between performance at a US law enforcement training academy and both facet and factor personality scales linking to conscientiousness (i.e. dependability and achievement) and emotional stability (i.e. stress resistance and frustration tolerance). For both conscientiousness and emotional stability, the use of separate facet scales provided the best prediction of training grades when linear and quadratic effects were considered. In all cases, the curvilinear relationships indicated that differences in scores at the low end of the distribution were more predictive of training grades than were differences in scores at the high end of the distribution. Results are discussed in terms of the value of using facet scales to predict performance and the importance of considering curvilinear relationships between personality and specific criteria, such as training grades.

This study investigated whether scores of item response accuracy (AC) and item response latencies (RLs) from a computer based artificial language test (ALT) uniquely predicted grades in a Spanish language course at a U.S. law enforcement training academy. As expected, ALT–RLs added to the prediction of course grades provided by ALT–AC. This finding held even after controlling for scores on a logical reasoning test. Results are discussed in terms of the potential benefits of using RLs to predict performance in training.

This article evaluated the validity of two traditional traitedness measures (i.e., standard deviation traitedness measures and global, single-item traitedness measures) and three new traitedness measures (i.e., multi-item, ranking, and facet traitedness measures). Overall, there was poor construct validity for both the traditional and new measures of traitedness. Furthermore, traitedness was a spurious moderator of the relationship between openness to experience and academic performance. Finally, evidence of nonlinear personality-academic performance relationships was discovered.