Evaluating the Economic Impact of a Valid Test: A Comparison of Supervisor and Incumbent Estimates of SDy.

To determine if individuals other than first-line supervisors could be used as expert judges in making estimates of job performance worth when conducting a utility analysis, this study compared the estimates of experienced job incumbents with those of supervisors. The job class investigated was that of eligibility technician, who performs varied technical work in determining clients' initial and continuing eligibility for one of several forms of public aid and is primarily found in the Department of Social Services in San Diego County (California). Participants were 120 eligibility supervisors and 203 eligibility technicians (incumbents). A modification of the global estimation technique of F. L. Schmidt and others was used to obtain estimates of job worth in dollars (SDy=job worth in dollars) at the 15th, 50th, and 85th percentiles of job performance. A questionnaire containing the modified Schmidt method was administered to the subjects. Results indicate that job incumbents provided estimates that were not substantially different from those of supervisors. The obtained estimates of worth were used in a utility analysis to compute costs and benefits associated with the eligibility technician selection process that was in use between December 1986 and January 1990; the process saved the county over one million dollars a year because it screened in high-performing individuals. One graph and 11 tables present study data, and a 35-item list of references is included. (SLD)