1. Validating Grant-Making Processes: Construct Validity of the 2013 Senior Corps RSVP Grant Review
- Author
-
Robin Ghertner, Vielka E. Garibaldi, Erwin J. Tan, Patricia J. Stengel, and Malcolm Coles
- Subjects
Public Administration ,Sociology and Political Science ,Operations research ,Process (engineering) ,Strategy and Management ,Geography, Planning and Development ,05 social sciences ,050401 social sciences methods ,Construct validity ,050109 social psychology ,Confirmatory factor analysis ,0504 sociology ,Convergent validity ,Accountability ,0501 psychology and cognitive sciences ,Measurement invariance ,Operations management ,Business and International Management ,Selection (genetic algorithm) ,Reliability (statistics) ,Earth-Surface Processes - Abstract
Accountability in grant-making requires a valid, fair and transparent selection process. This study proposes a four-step framework for validating such a process: determine standards for qualified applicants, assess inter-reviewer reliability, assess factorial validity, and assess reliability. This framework is applied to the Corporation for National and Community Service’s 2013 RSVP grant-making process. The standards were close to the highest points of reliability. Inter-reviewer reliability was above 0.90, a common threshold for high-stakes measurement. After conducting confirmatory factor analysis, the final model merged two of the original five domains of selection criteria, resulting in four domains. The final model was found to have strict measurement invariance, high convergent validity, and measurement reliability between 0.88 and 0.93 for all domains. The results validate the 2013 review process and indicated that the scores exhibited high degrees of reliability, giving public assurance that the process was sufficiently objective and accurately reflected program priorities.
- Published
- 2015