Search

The mission of the National Implementation Research Network (NIRN) is to contribute to the best practices and science of implementation, organizational change, and system reinvention to improve outcomes across the spectrum of human services. As such, it has been involved in advancing implementation practice through numerous initiatives over the past 17 years. Although recognized as an implementation practice leader, NIRN has not been as consistent with its implementation research and evaluation agenda (REA). Yet, it is uniquely positioned to advance implementation research and evaluation based on the wealth of data generated through its projects. By contributing innovative practice-based research insights, demonstrating the value of implementation practice, improving the rigor of the evidence generated through practice-based research, and increasing its scholarship and policy outputs, NIRN is primed to further establish itself as a national player in implementation science. This report highlights NIRN's research and evaluation agenda.

Context matters in implementation. The Lenawee ISD (LISD) in Michigan navigated implementation through major changes in its school district's administration and the COVID-19 pandemic. The district highlighted in the impact story experienced leadership change in the District Superintendent position and three school principal positions. The district and LISD credit the sustained use of MTSS to the accomplished system work. The initial cohort of districts reported their ability to respond quickly and proactively to the COVID-19 pandemic due to their teaming and systems being in place. As the LISD implementation team reflects on their MTSS implementation journey to date, several key learnings are generated. [This report was created with the Lenawee Intermediate School District.]

This brief explains how four important system design initiatives frequently promoted by state and local education agencies for school improvement can be implemented through an intentional systematic approach for a powerful, positive impact on student outcomes. As technical assistance providers with expertise in four specific frameworks for educational system design, and with great appreciation for one another, the contributors suggest how a coordinated Stage-based Implementation process with embedded, systems level critical Equity Inquiries at every stage can enable schools to establish Universal Design for Learning (UDL) principles and guidelines within a Multi-Tiered System of Support (MTSS) for a designed experience in instructional decision making.

In educational spaces, it is no longer aspirational but imperative that the community--with its richness and diversity--joins educators as key instructional partners to liberate the creativity, uniqueness, and potential of all students. As educators, we can miss the value of this collaboration. This brief defines critical perspectives, why engagement is necessary, how to select and engage critical perspectives, and evaluate the process.

The overall aim of the Effective Implementation Cohort (EIC) investment is to increase district capacity to implement a high-quality middle years math curriculum as part of a Coherent Instructional System (CIS) to accelerate learning for students who are experiencing poverty, Black, Latino/a, and/or English Learner (EL)-Designated ("priority students"). As a Learning Partner, NIRN seeks to support partnerships between Providers and Local Education Agencies (LEAs) in their implementation and measurement efforts, as well as to collect and study data to answer the investment's learning questions within the cohort wide learning agenda. Implementation support activities and data collection, analysis, and reporting activities for the cohort wide learning agenda will be guided by the Cohort's learning questions and theory of action grounded in implementation science.

Fidelity is defined as doing what is intended. The purpose of measuring fidelity is to assist with supporting implementation. Collecting fidelity data provides an opportunity to problem-solve throughout the process versus waiting for outcome data. More often than not, purchased programs and curriculum materials are missing a research-validated measurement to ensure it is implemented as intended. Educational institutions may be measuring fidelity through the use of a walkthrough tool or teacher self-report, but are the tools aligned with what they want to measure (Hill & Erickson, 2019). This brief examines the concept and provides real-world examples from the Florida Department of Education, the Kentucky Department of Education, and the Minnesota Department of Education.

The purpose of the study was to investigate whether the districts that received ongoing training and coaching to use the Active Implementation Frameworks improved mathematics outcomes. The Active Implementation Frameworks represent five essential approaches for effective implementation of any evidence-based program or practice. One of the frameworks is a clearly defined and operationalized usable innovation, or any evidence-based practice that is easily measured for adherence to the program design. The subject of Kentucky's usable innovation is mathematics and use of the 8 mathematical teaching practices developed by the National Council of Teachers of Mathematics (NCTM, 2014). The evaluation focused on priority populations, specifically students with disabilities and Black students, and compared their outcomes to a carefully matched district that did not receive the same level of support.

Regardless of sector, implementation science and improvement science are used to support and improve the implementation and scale-up of evidence-based programs and practices. This brief highlights similarities and differences between implementation science and improvement science, as well as how they can be used together to support programs and practices aiming to improve outcomes for children, students, families, communities, and other stakeholders. Included is a flow chart to assist teams in thinking through how to integrate improvement efforts into "Active Implementation."

Practice profiles are a tool used to ensure practices are well-defined so that they can be taught, learned, used, and measured. This brief highlights how the Michigan Department of Education (MDE) engaged systematically and purposefully in the practice profile development process to clearly define Multi-Tiered System of Supports (MTSS), a research-based framework that attends to the academic and behavioral needs of students.

This profile outlines the guiding principles and core competencies needed to build the capacity of practitioners and communities to effectively implement evidence in health and human services. Implementation support practitioners (ISP) are professionals who support organizations, leaders, and staff in their implementation of evidence-informed practices and policies. They identify, contextualize, and improve the use of evidenced-informed implementation strategies in a range of settings. They also build implementation capacity among teams, organizations, and systems. They can be referred to as coaches, improvement advisors, technical assistance providers, facilitators, consultants, mentors, and implementation specialists. Implementation support practitioners often reside outside of the service systems they work in, but may also reside within a service system when those systems have work units specifically designed to support implementation and scaling efforts. This ISP Profile presents "principles" that guide the work of ISPs and "core competencies" that they require to provide effective implementation support. Both were identified through an initial literature review, a document review, vetting with individuals providing implementation support, an integrative systematic review, and a content validation survey with international intermediary organizations. [This profile was prepared with the Centre for Effective Services and the European Implementation Collaborative. For "Implementation Support Practitioner Profile: Guiding Principles and Core Competencies for Implementation Practice, Version 3.0," see ED607181.]

The State Capacity Assessment (SCA) was developed to assess the impact and presence of efforts to build strong foundations needed to adopt, sustain, and scale effective practices so that they lead to positive outcomes. In use, the SCA assesses the extent to which an agency invests in and aligns system components to support use of best practices, which includes support and development of implementation teams within and across all levels of the system. In other health and human services fields, this State Capacity Assessment is referred to as the Agency Capacity Assessment (ACA). This document addresses: (1) the administration process and key roles; (2) focus of the SCA-ACA and scoring; (3) preparation for the assessment; (4) research basis; and (5) next steps after administration.

The purpose of the "Implementation Support Practitioner Profile" is to identify the skills and competencies needed by implementation support practitioners to support effective implementation and scaling of evidence-informed practices, programs and policies to improve outcomes for people and communities. The integration of implementation research and practice often relies on a specific set of actors who use evidence derived from implementation research to support leaders, practitioners, and community partners in change efforts. We refer to these actors as implementation support practitioners. They often reside outside the service systems they work in but may also reside within a service system when those systems have work units specifically designed to support implementation and scaling efforts. The "Implementation Support Practitioner Profile" outlines the skills and competencies needed to build the capacity of practitioners and communities to effectively use evidence to improve outcomes. The role of the implementation support practitioner is informed by philosophical principles that guide the work and defined by core competencies that describe a range of activities implementation support practitioners conduct. Skills and competencies were identified through a literature review, document review, vetting with individuals providing implementation support, and a content validation survey with international intermediary organizations. A systematic scoping review is underway and focus groups and usability testing will be conducted with implementation support practitioners working in a range of contexts. [This profile was prepared with the Centre for Effective Services.]

Practice Profiles provide structure to the process of making a practice teachable, learnable, and assessable. This Coaching Practice Profile synthesizes research on coaching best practices to guide development of learning modules, training supports, and infrastructure for effective and efficient coaching.

State agencies provide for MTSS [multi-tiered systems of support] initiatives with the purpose of producing meaningful outcomes. The impact of MTSS is dependent upon the ability of educators to implement the practices correctly and consistently. A lack of implementation fidelity can result in a practice or program being less effective, less efficient, or producing less-predictable responses (Sanetti Collier-Meek, Long, Kim, & Kratochwill, 2014; Wang et al., 2015). A focus on supporting fidelity of implementation increases the likelihood of student success. This document brief highlights four key actions SEA [State Education Agency] teams can take to help support implementation of MTSS with fidelity.

The purpose of the brief is to provide examples from two state education agencies in the United States who took a different path to systemic change through the use of the Active Implementation Frameworks (Fixsen et al. 2013a; Metz & Bartley, 2012). Both states received intensive implementation-informed support from the State Implementation and Scaling-up of Evidence-based Practices (SISEP) Center. The aim of the SISEP Center is to establish implementation and scaling capacity in a State Education Agency (SEA), its regional entities, its districts, and their schools to continuously support teacher practice and improve outcomes. Under optimal conditions, a simultaneous process of horizontal and vertical alignment occurs: horizontal alignment across the SEA to reduce duplication of efforts at the state level that often translates into burdensome duplication of efforts in districts and schools; vertical alignment as the state facilitates and resources a co-creative process with its regional entities, districts, schools, and stakeholders to design an implementation infrastructure. Everyone's efforts, horizontally and vertically, are focused on supporting state, regional, district, and school leaders, who in turn support continuous improvement of teacher skills to maximize academic and behavioral outcomes for all students. The benefits and challenges inherent in the separate approaches will be presented to offer the field a deeper understanding of organizational readiness for change and the importance of co-creation to engage benefactors from every level of the system to implement, sustain, and scale-up a practice with evidence (Metz, 2016; Tommeraas & Ogden, 2015).

Kentucky's educators are creating systemic change to produce noticeable improvements in learning for all students. Systemic change has vexed education systems for decades (Tyack & Cuban, 1999). In recent years, it has become clear that implementation science is the missing ingredient in efforts to purposefully improve student outcomes in state systems (Fixsen, Blase, Metz, & Van Dyke, 2013; Fixsen, Blase, Ward, Jackson, & Coffey, in preparation). The Kentucky Department of Education (KDE) has taken a thoughtful and persistent approach to systemic change in partnership with its educators and communities. Kentucky has documented the initial changes in the education system and the benefits to system alignment and cohesion for improved outcomes. This paper summarizes a few of the key actions of Kentucky's educators here, as well as some of the results of their efforts so far. In 2014, with the support of the State Implementation and Scaling up of Evidence Based Practices (SISEP) Center, Kentucky began to develop its implementation infrastructure and capacity, with goals for student learning in the area of math, as they developed the knowledge and skills to use the Active Implementation Frameworks.

The National Implementation Research Network (NIRN) and the National Center for School Turnaround (2018) published the "Four Domains for Rapid School Improvement: An Implementation Framework" as a companion to the Center for School Turnaround's (2017) recent publication of "The Four Domains for Rapid School Improvement: A Systems Framework" (ED584107). The aim of this companion paper is to describe "how" to use the improvement domains in practice.

OSEP's Results Driven Accountability is designed to assure educators have the supports necessary to address the needs of all children, including children with disabilities. OSEP-funded Centers are encouraged to support state education systems as they take charge in making systemic changes that lead to improved outcomes for students. Data are now available that demonstrate capacity for change can be purposefully developed in complex state education systems by using active implementation science as a change method. The data are based on a standard measure that has been used to systematically assess capacity development within and across states.

School districts (local education agencies) in the United States are charged with many responsibilities, by law and by custom. The most important is responsibility for assuring schools and teachers have the support they need to provide consistent, high quality, and effective education to promote student learning for all students every year. Measures of student outcomes are now commonplace in education. Those measures, while important, have not led to improved student outcomes (Manna, 2008; National Center for Education Statistics, 2013). Recognizing the key role of teachers, what is missing is a measure of district supports for schools and teachers. The District Capacity Assessment was developed to fill this gap. The purpose of this brief is to provide an overview of the District Capacity Assessment and provide resources for additional information and learning.

The purpose of this piece is to provide the research and rationales behind Practice Profiles. To achieve outcomes and develop effective implementation supports, innovations need to be "teachable, learnable, doable, and assessable." Practice Profile methodology facilitates the development of innovations and their necessary infrastructure. Specific training on NIRN's Practice Profile methodology can be found on the Active Implementation Hub.