Realist-informed rapid-cycle evaluations for integrated care programs

Introduction: While integrated care programs are proliferating around the world, rigorous measurement and evaluation of the intended and unintended effects of these programs are rare outside of the context of specific research programs. There are a number of reasons for the lack of evaluation including implementation without regard to measurement and evaluation, a lack of funding for evaluation activity, and challenges associated with measurement and evaluation in complex interventions. As a result, most research on integrated care is descriptive including case studies that most often summarize what was implemented, in some cases how it was implemented, but far less often what outcomes were achieved. Approach: We describe an evaluation framework for integrated care. It includes multiple components of evaluation (implementation, reporting and feedback, methods and measures) and a host of supplementary tools for measuring successful implementation of integrated care. The broad approach is to use rapid-cycle realist-informed evaluation methodology inclusive of implementation science, person-centred evaluation and comparative-effectiveness methodology. The approach to evaluation adds value and must be adaptable to local variations in models and organizational approaches to integrating care. Evaluation is framed into four components: Implementation: The evaluation of integrated care must first assess the implementation of new models of delivery, including description of the policy and organizational context of providers. Drawing from approaches taken including the Consolidated Framework for Implementation Research, evaluation is best begun parallel with the implementation of new models of integrated care allowing for early successes and challenges or barriers to be identified. Program operational monitoring: Second, programs themselves need to set out key measure of performance. A logic model for program implementation may be a particularly useful way to ensure that the program design will enable desired outcomes and ensure the essential component of the logic model are measured. Patient-centred assessment: The third major component of evaluation is patient-centred measurement including patient reported outcome and patient reported experience measures. Ultimately, patients themselves are the best target for measurement of integrated care. We will address essential domains for measurement of patient and caregiver outcomes and patient and caregiver experience. Outcome evaluation: Finally outcome evaluation is an essential activity. While the most rigorous form of outcome evaluation might involve experimental designs, the focus here is primarily on quasi-experimental approaches. Conclusion: This presentation describes a comprehensive framework for realist-informed rapid-cycle evaluations for integrated care programs. The framework is used to evaluate integrated care. Requirements for Implementation of Evaluation will also be discussed