Search

This meta-analytic path analysis presents evidence that first-year academic performance (FYAP), measured by first-year grade point average (FYGPA) plays the major role in determining second-year student retention and that socioeconomic status (SES), measured by parental income, plays a negligible role. Based on large sample data used in a previous study, we conducted additional analyses that included corrections for measurement error and created path models using precollege academic achievement, measured by ACT Composite scores and high school GPA (HSGPA), and SES to predict FYAP and then second-year retention. Precollege academic performances had direct effects on FYAP, and FYAP fully mediated their effects on second-year retention. SES did not contribute to the prediction of FYAP, and it had only a trivial effect on second-year retention. The results of this study point to the criticality of FYAP, and supporting first-year student academic success may serve as the central vehicle for retention efforts.

In this article, I provide answers to the questions posed by Will Shadish about the history and development of the Schmidt-Hunter methods of meta-analysis. In the 1970s, I headed a research program on personnel selection at the US Office of Personnel Management (OPM). After our research showed that validity studies have low statistical power, OPM felt a need for a better way to demonstrate test validity, especially in light of court cases challenging selection methods. In response, we created our method of meta-analysis (initially called validity generalization). Results showed that most of the variability of validity estimates from study to study was because of sampling error and other research artifacts such as variations in range restriction and measurement error. Corrections for these artifacts in our research and in replications by others showed that the predictive validity of most tests was high and generalizable. This conclusion challenged long-standing beliefs and so provoked resistance, which over time was overcome. The 1982 book that we published extending these methods to research areas beyond personnel selection was positively received and was followed by expanded books in 1990, 2004, and 2014. Today, these methods are being applied in a wide variety of areas.

This meta-analysis examines the strength of the relationships of ACT® Composite scores, high school grades, and socioeconomic status (SES) with academic performance and persistence into the 2nd and 3rd years at 4-year colleges and universities. Based upon a sample of 189,612 students at 50 institutions, ACT Composite scores and high school grade point average (GPA) are highly correlated with 1st-year academic performance. First-year academic performance emerges as the best predictor of 2nd- and 3rd-year retention. SES is a weak predictor of both academic performance and retention. Moderator analyses of admission selectivity indicate that although the estimated mean validity coefficients for ACT Composite scores and high school GPA vary slightly, the credibility intervals indicate they are valid predictors across levels of admission selectivity. This longitudinal study demonstrates the importance of precollege academic preparation and how success in the 1st year of college strongly influences persistence toward completing a degree.

Construct empirical redundancy may be a major problem in organizational research today. In this paper, we explain and empirically illustrate a method for investigating this potential problem. We applied the method to examine the empirical redundancy of job satisfaction (JS) and organizational commitment (OC), two well-established organizational constructs. Analysis based on responses from a sample of 292 employees collected at two occasions showed that: (a) the construct-level correlation between JS and OC was very high (0.91) and (b) both JS and OC are similarly related to positive affectivity and negative affectivity. These results suggest that the constructs may be empirically indistinguishable, despite their well-established conceptual distinction. These findings illustrate the problem of empirical redundancy of organizational constructs and provide a basis for a possible movement towards parsimony in the realm of constructs that could open the way to more rapid advances in knowledge in organizational research. (Contains 1 figure and 4 tables.)

Using computer simulation, the authors assessed the accuracy of J. E. Hunter, F. L. Schmidt, and H. Le's (2006) procedure for correcting for indirect range restriction, the most common type of range restriction, in comparison with the conventional practice of applying the Thorndike Case II correction for direct range restriction. Hunter et al.'s procedure produced more accurate estimates of both the mean and standard deviation in meta-analysis than the conventional procedure. Even when its key assumption that the effect of selection on a 3rd variable is fully mediated by the independent variable was violated, Hunter et al.'s procedure was still relatively more accurate than the conventional procedure. When applied to data from a previously published meta-analysis, the new procedure yielded results that led to different substantive conclusions.

Presents simple and direct demonstrations showing why basic measurement principles require that biases in data created by measurement error be removed. Refutes common objections to the correction for these biases. Also describes substantive psychological processes responsible for some types of measurement error. (SLD)

The potential importance of suppressor variables in the personality domain was studied in classifying white-collar criminals with a validation sample of 435 prisoners and a cross-validation sample of 214. Results suggest that suppressor variables can increase prediction and may contribute to knowledge and theory development. (SLD)

To purchase or authenticate to the full-text of this article, please visit this link: http://dx.doi.org/10.1111/j.1744-6570.2008.00132.x Byline: FRANK L. SCHMIDT (1), JONATHAN A. SHAFFER (1), IN-SUE OH (1) Abstract: A recently developed procedure produces substantial improvements in the accuracy of corrections for range restriction and reveals that predictive validities of employment selection methods are higher than previously thought. This study applied this procedure to meta-analytic validities of 2 personality measures (Conscientiousness and Emotional Stability) and general cognitive ability measures. Results show that the increases in validity estimates are greater for cognitive ability than for personality, reducing the relative validity of personality. In addition, the incremental validity of these personality measures over that provided by cognitive ability is smaller than previously estimated. Path analyses based on the more accurate data reveal relatively smaller path coefficients from these personality traits to job and training performance. These findings have implications for theories of job performance and for the practical value in selection of personality measures relative to cognitive ability measures. Author Affiliation: (1)Department of Management and Organizations, Tippie College of Business University of Iowa Article note: Correspondence and requests for reprints should be addressed to Jonathan A. Shaffer, Department of Management and Organizations, Tippie College of Business, University of Iowa, W252 John Pappajohn Business Building, Iowa City, IA 52242; jonathan-shaffer@uiowa.edu.

The authors examine 3 methods of combining new studies into existing meta-analyses: (a) adding the new study or studies to the database and recalculating the meta-analysis (the medical model); (b) using the Bayesian procedure advocated by F. L. Schmidt and J. E. Hunter (1977) and F. L. Schmidt, J. E. Hunter, K. Pearlman, and G. S. Shane (1979) to update the meta-analysis; and (c) using the Bayesian methods advocated by these authors and M. T. Brannick (2001) and M. T. Brannick, S. M. Hall, and Y. Liu (2002) to estimate study-specific parameters. Method b was found to severely overweight new studies relative to the previous studies contained in the meta-analysis, and Method c was found to do the same while also requiring an assumption with a low prior probability of being correct, causing the method to violate Bayesian principles. The authors present an alternative Bayesian procedure that does not suffer from these drawbacks and yields meta-analytic results very similar to those obtained with the medical model. They recommend use of the medical model or this alternative Bayesian procedure. Keywords: meta-analysis, Bayesian methods, cumulative knowledge, research synthesis

A study to calibrate the under correction error by comparing meta-analysis results based on corrections for direct range restriction with more accurate results from a method of correcting individual range restriction was conducted. The results revealed that on an average correction for direct range restriction resulted in substantial underestimation of operational validities for both job performance measures by 21% and training performance measures by 28%.

Range restriction in most data sets is indirect, but the meta-analysis methods used to date have applied the correction for direct range restriction to data in which range restriction is indirect. The authors show that this results in substantial undercorrections for the effects of range restriction, and they present meta-analysis methods for making accurate corrections when range restriction is indirect. Applying these methods to a well-known large-sample empirical database, the authors estimate that previous meta-analyses have underestimated the correlation between general mental ability and job performance by about 25%, indicating that this is potentially an important methodological issue in meta-analysis in general. Keywords: range restriction, indirect range restriction, bias corrections, data analysis

A database integrating 90 years of empirical studies reporting intercorrelations among rated job performance dimensions was used to test the hypothesis of a general factor in job performance. After controlling for halo error and 3 other sources of measurement error, there remained a general factor in job performance ratings at the construct level accounting for 60% of total variance. Construct-level correlations among rated dimensions of job performance were substantially inflated by halo for both supervisory (33%) and peer (63%) intrarater correlations These findings have important implications for the measurement of job performance and for theories of job performance.

This study found mixed support for the hypothesis that the difference in criterion-related validity between unstructured and structured employment interviews is due solely to the greater reliability of structured interviews. Using data from prior meta-analyses, this hypothesis was tested in 4 data sets by using standard psychometric procedures to remove the effects of measurement error in interview scores from correlations with rated job performance and training performance. In the 1st data set, support was found for this hypothesis. However, in a 2nd data set structured interviews had higher true score correlations with performance ratings, and in 2 other data sets unstructured interviews had higher true score correlations. We also found that averaging across 3 to 4 independent unstructured interviews provides the same level of validity for predicting job performance as a structured interview administered by a single interviewer. Practical and theoretical implications are discussed.

The psychological construct of general mental ability (GMA), introduced by C. Spearman (1904) nearly 100 years ago, has enjoyed a resurgence of interest and attention in recent decades. This article presents the research evidence that GMA predicts both occupational level attained and performance within one's chosen occupation and does so better than any other ability, trait, or disposition and better than job experience. The sizes of these relationships with GMA are also larger than most found in psychological research. Evidence is presented that weighted combinations of specific aptitudes tailored to individual jobs do not predict job performance better than GMA alone, disconfirming specific aptitude theory. A theory of job performance is described that explicates the central role of GMA in the world of work. These findings support Spearman's proposition that GMA is of critical importance in human affairs.

Abstract Most theorizing on the relationship between corporate social/environmental performance (CSP) and corporate financial performance (CFP) assumes that the current evidence is too fractured or too variable to draw any [...]

Meta-analysis was used to cumulate the correlations between supervisor and peer ratings for different dimensions of job performance. The general pattern in the data suggested that raters from the same organizational level disagree as much as raters from different levels. Methods were used to separate the effects of rating difficulty and lack of construct-level convergence on the correlation between supervisor and peer ratings. The authors found complete construct-level convergence for ratings of overall job performance, productivity, effort, job knowledge, quality, and leadership but not for ratings of administrative competence, interpersonal competence, and compliance or acceptance of authority. Higher rating difficulty was more strongly associated with lower mean observed peer-supervisor correlations than were construct-level disagreements between peers and supervisors. Implications for research and practice are discussed.

Based on 7,939 business units in 36 companies, this study used meta-analysis to examine the relationship at the business-unit level between employee satisfaction-engagement and the business-unit outcomes of customer satisfaction, productivity, profit, employee turnover, and accidents. Generalizable relationships large enough to have substantial practical value were found between unit-level employee satisfaction-engagement and these business-unit outcomes. One implication is that changes in management practices that increase employee satisfaction may increase business-unit outcomes, including profit.

This article examines whether differences in the equations commonly used to calculate effect size for single group pretest-posttest (SGPP) designs versus those for control group designs can account for the finding that SGPP designs yield larger mean effect sizes (e.g., M. S. Lipsey & D. B. Wilson, 1993). It was found that the assumptions of no control group effect and the equivalence of pretraining and posttraining dependent variable standard deviations required for these equations to produce equivalent estimates of effect size were violated for some dependent variable types. Results indicate that control group effects and inflation in the standard deviation of the posttraining dependent variable measure account for most of the observed difference in effect size. The most severe violations occurred when the dependent variable was a knowledge assessment. Methods for including data from SGPP designs in meta-analyses that minimize potential biases are discussed.

This article summarizes the practical and theoretical implications of 85 years of research in personnel selection. On the basis of meta-analytic findings, this article presents the validity of 19 selection procedures for predicting job performance and training performance and the validity of paired combinations of general mental ability (GMA) and the 18 other selection procedures. Overall, the 3 combinations with the highest multivariate validity and utility for job performance were GMA plus a work sample test (mean validity of .63), GMA plus an integrity test (mean validity of .65), and GMA plus a structured interview (mean validity of .63). A further advantage of the latter 2 combinations is that they can be used for both entry level selection and selection of experienced employees. The practical utility implications of these summary findings are substantial. The implications of these research findings for the development of theories of job performance are discussed.

This meta-analytic review presents the findings of a project investigating the validity of the employment interview. Analyses are based on 245 coefficients derived from 86,311 individuals. Results show that interview validity depends on the content of the interview (situational, job related, or psychological), how the interview is conducted (structured vs. unstructured; board vs. individual), and the nature of the criterion (job performance, training performance, and tenure; research or administrative ratings). Situational interviews had higher validity than did job-related interviews, which, in turn, had higher validity than did psychologically based interviews. Structured interviews were found to have higher validity than unstructured interviews. Interviews showed similar validity for job performance and training performance criteria, but validity for the tenure criteria was lower. (Reprinted by permission of the publisher.)

The authors evaluated an improved procedure for range-restriction corrections in meta-analysis. When population correlations were approximately normally distributed, the new nonlinear range-correction procedure improved the accuracy of the Schmidt-Hunter (S-H) interactive method in estimating both the mean (Msubp) and the standard deviation (SDsubp) of population correlations, making it the most accurate of the procedures examined. In the homogeneous case (SDsubp=0), the nonlinear range corrction again improves accuracy of estimates of SDsubp. In this important case, Taylor Series Approximation (TSA) 1 is considerably less accurate that the S-H interactive and the TSA2 procedures, and N.S. Raju, M.J. Burke, J. Normand, and G.M. Langlois's (1991) procedure yields the least accurate estimates of SDsubp. Finally, the authors found that the nonlinear range-correction procedure produces improvements in the accuracy of the interactive method even under extreme violations of the normality assumption. (Reprinted by permission of the publisher.)