An Instrumental Case Study of Confirmative Evaluation in the Continuous Improvement of Online Remote Leaders' Work

Confirmative evaluation is often noted as an important element of models of Human Performance Technology and Performance Improvement, but there exists confusion around what is and what is not confirmative evaluation. A significant issue limiting the use of confirmative evaluation is the Performance Improvement field's inability to clearly classify confirmative evaluation in terms of its purpose and scope. Additionally, the performance improvement field lacks sufficient examples of confirmative evaluation in the literature necessary to adequately define confirmative evaluation and demonstrate its use. Without sufficient examples of confirmative evaluation, practitioners of performance improvement are left without a clear path of how to conduct confirmative evaluation in ways that are contextually relevant and cost effective. Additionally, those who train instructional designers and performance improvement practitioners lack the tools necessary to help students of the field learn about and use this vital aspect of evaluation. Through the conceptual framework of systems theory and change management, this instrumental case study reviewed the literature around confirmative evaluation to identify the confusion around confirmative evaluation and then presented a case of confirmative evaluation applied in a context of change management and continuous improvement in online higher education. This case used job descriptions of remote leaders' work to determine the degree to which an organizational restructuring had been implemented as it had been intended and had influenced remote leaders' actual work. Time study methodology and matrix sampling were employed to observe remote leaders' work. The data from the time study was compared against remote leaders' job descriptions and institutional surveys gauging satisfaction among those served by these remote leaders. The difference between the expected work and the observed work demonstrated the degree to which remote leaders had embraced the new duties given to them in the organizational restructuring and identified key gaps in their collective performance. Together, this instrumental case study demonstrated that confirmative evaluation plays a linking or bridging role in continuous improvement processes by moving the process from summative evaluation to a new phase of formative evaluation. The study then examined the results of the case to consider how it clarified the field's understanding of confirmative evaluation. Key findings included the recognition that confirmative evaluation is distinct from other forms of evaluation because of its purpose and that purpose and timing are intertwined and inform one another in evaluation efforts. It also identified where confirmative evaluation fits within and can improve performance improvement and change management models. The study ended with a discussion of challenges faced when conducting confirmative evaluation and implications for future studies of confirmative evaluation. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]