Search

Extant coverage of multi-tiered systems of support (MTSS) in the research literature has tended to focus on student outcomes as opposed to the process of adult implementation. This conceptual artic...

The purpose of this study was to examine how an early literacy intervention, Kindergarten Peer-Assisted Learning Strategies (K-PALS; Mathes et al. Kindergarten peer-assisted literacy strategies (K-PALS): Sopris West, 2001), can be intensified to extend the positive outcomes to students who do not initially demonstrate an adequate response to intervention. K-PALS was intensified by incorporating a self-monitoring package that included self-graphing. An alternating treatment design was used to compare the effectiveness of K-PALS with self-graphing alone and K-PALS with a self-monitoring package to baseline/K-PALS for the purpose of increasing student performance on the early literacy measure of phonemic awareness. The results of the study indicate that while the self-monitoring package was the more effective intervention for two of the three student participants, the self-graphing alone intervention also resulted in positive outcomes for both of these participants. There was no change in the phonemic awareness performance for the third participant throughout the alternating treatment phase. Implications for intensifying academic interventions within the context of a Response to Intervention framework are discussed.

Self-management has been shown to be effective for increasing the on-task behaviors and decreasing the problem behaviors of students. However, little is known regarding the individual and combined ...

School-Wide Positive Behavioral Interventions and Supports (SWPBIS) is a broad set of evidence-based strategies designed to create school environments that promote and support appropriate b...

Innovations in information technology have generated an array of options for the professional learning for educators. Face-to-face, remote, and hybrid formats for professional learning each have their advantages and disadvantages for advancing educators’ knowledge and skills in evidence-based practices to promote student achievement. The purpose of this study was to better understand teacher and educational leaders’ preferences for professional learning formats in relation to the intended learning objective. The results indicate that educators preferences for professional learning format varied by the intended learning objective of the session. Remote professional learning was preferred relative to face-to-face and hybrid formats for a session in which the primary objective was to share information uni-directionally from the presenter to the audience (53% versus 25% and 22% preferred hybrid). The respondents were equally split in their preference for either a face-to-face or remote format when the learning objective was to understand new concepts and learn how to apply the concepts to their setting. The results have important implications for designing professional learning.

Although mobile phones and tablets can be effective tools for searching and retrieving information efficiently, accessing interactive learning platforms, and capturing audio/video, the inappropriate use of mobile devices can create distractions in the classroom that are detrimental to learning. This column presents a classroom management strategy for reducing the inappropriate use of mobile devices and increasing academic engagement in middle and high school classrooms, including students with emotional and behavioral disorders.

As technology becomes more accessible and acceptable in the preschool setting, teachers need effective strategies of incorporating it to address challenging behaviors. A nonconcurrent delayed multiple baseline design in combination with an alternating treatment design was utilized to investigate the effects of using iPad tablets to display video self-modeling and activity photos for three preschoolers during circle time. During baseline, all three children demonstrated low levels of engagement and high levels of off-task behavior compared to peers. The intervention phase consisted of alternating between showing the child the self-video and photos prior to circle time. A child preference phase was conducted whereby each child self-selected the video or photo prior to circle time. For all three children both videos and photos led to increased engagement and decreased off-task behaviors. During the child preference phase, all three children selected the video most frequently. Social validity data demonstrated teacher and child preference for the video self-modeling condition.

The demand for results-driven accountability is pervasive in all aspects of education today. The shift in emphasis from compliance to accountability for closing achievement gaps and improving student outcomes creates challenges and opportunities for school-based professionals. School-based professionals have a significant role to play in supporting school improvement efforts and improving outcomes for students in their role as program evaluators. Despite the need for advanced knowledge and skills in program evaluation, resources dedicated to developing the competencies of school-based professionals in evaluation approaches, methods, and tools are scarce. This book addresses this need by serving as a practical resource for school-based professionals seeking to use program evaluation to enhance data-based decision making and accountability as informed by implementation research and within the context of a multi-tier system of supports framework. Throughout this book, school-based professionals are positioned as change agents guiding their school or school districts in the adoption, implementation, and evaluation of evidence-based interventions and systems of support to meet students’ academic and behavioral needs.

The objective of this chapter is to describe the evaluation methods, techniques, and tools involved in measuring implementation fidelity. Implementation fidelity is defined and the distinction between process evaluation and formative evaluation is described. The body of knowledge known as implementation research is introduced, and the National Implementation Research Network’s Implementation Drivers Framework and implementation stages are presented. The use of manuals, protocols, checklists, and practice profiles to support high-fidelity practitioner behaviors are illustrated. Research-validated tools for assessing implementation fidelity within a multi-tiered system of support initiative are highlighted. This chapter concludes with a discussion regarding the use of established criterion for judging implementation fidelity and the critical importance of measuring implementation fidelity linked to intended student outcomes.

This chapter consists of three case studies that illustrate how the evaluation approaches, methods, techniques, and tools presented in Chapters 1 to 5 can be translated into practice. The first case study describes an evaluation of the Dyslexia Pilot Project, a statewide multi-tier system of supports (MTSS) initiative targeting early literacy. In this evaluation, special attention was paid to the evaluating the cost-effectiveness of serving students in kindergarten to grade 2 proactively. The second case study features the use of single-case designs and corresponding summary statistics to evaluate the collective impact of more than 500 academic and behavioral interventions provided within an MTSS framework as part of the annual statewide evaluation of the Ohio Internship Program in School Psychology. The third case study focuses on efforts to evaluate the fidelity of implementation for teacher teams’ use of a five-step process for data-based decision making and instructional planning.

The objective of Chapter 4 is to provide practical guidelines for developing evaluation plans. Practical tips for writing measurable goals and objectives are provided first following by guidance for developing a logic model based on the program’s theory of change. Guidance is provided for writing evaluation questions and linking the questions to evaluation methods. Useful templates for designing and communicating evaluation plans and roles, responsibilities, and milestones are presented in this chapter. This chapter on developing an evaluation plan incorporates content and concepts from the previous chapters on program evaluation approaches (Chapter 1), evaluating program implementation (Chapter 2), and evaluating teacher professional learning opportunities (Chapter 3).

The topic of data visualization and the effective communication of evaluation findings has received considerable attention in recent years. The objective of Chapter 5 is to present practical strategies for communicating findings through data visualization and report writing that maximize the impact of the evaluation. The chapter highlights the use of web-based data systems and data dashboards for communicating implementation and outcome data relative to a multi-tiered system of support initiative. Practical guidelines for developing communication plans, maintaining a data-driven focus to communications, and linking current results to proposed actions are provided. Special attention is paid to the use of targeted, tailored communication as informed by diffusion theory.

Professional learning in the form of training and coaching/consultation is central to supporting evidence-based practices in schools. The objective of this chapter is to review what is known about effective strategies to support adult learning and to compare frameworks for evaluating professional learning for educators. Five critical levels have been identified for evaluating educator professional learning. These are (a) participants’ reactions/perceptions of satisfaction, (b) participants’ learning, (c) organization support and change, (d) participants’ use of new knowledge and skills, and (e) student learning outcomes. Other topics discussed include evaluation approaches, methods, and tools for assessing the implementation and impact of teacher professional learning opportunities in a multi-tiered system of support initiative.

The objective of this chapter is to provide the school-based professional a concise introduction to program evaluation. Program evaluation is defined and the distinguishing characteristics of program evaluation and research are described. An overview of the most prominent evaluation approaches relevant to a school-based context is provided. The distinction between formative and summative evaluation is presented. Throughout this chapter, evaluation approaches, purposes, and foci are presented as they relate to the evaluation of a multi-tier system of supports (MTSS) framework. Finally, the distinction between an internal and external evaluator is outlined in terms of the relative advantages and disadvantages of each. The role of the school-based professional as an internal or external evaluator is illustrated given the current context in education emphasizing results-driven accountability.

Universal screening is a critical foundation for school-based prevention and early intervention. This study examined the technical adequacy of the SPF-7, an instrument designed to assist school personnel in the early identification of students at-risk for emotional and behavioral concerns. The SPF-7 differs from other measures used in identifying emotionally and behaviorally at-risk students, as the SPF-7 focuses on protective factors rather than risk factors. The results of this study suggest that the SPF-7 has adequate test-retest reliability and criterion validity, but limited inter-rater agreement. The SPF-7 is a potentially promising tool for identifying students in need of support within a multi-tiered system of increasingly intensive interventions.

Universal screening systems in schools are an essential component of a multi-tiered system of supports (MTSS) framework is a comprehensive service delivery model designed to meet the academic and behavioral needs of students in schools. Universal screening data are used to drive decisions in the interest of reducing risk and promoting positive outcomes for students. This commentary reviews the considerations for selecting screening approaches for assessing social, emotional and behavioral risk within a proactive MTSS framework and advocate for the use of universal screening systems that involve the development of local norms, including local norms for the presence of protective factors. An increasing body of research supports the value of protective factors (e.g. connected with teachers and school, empathy, engagement, pro-social peer groups) as a means of addressing students’ needs proactively through a competency-based lens.

The research literature on evidence-based academic interventions focuses predominantly on measures of intervention effectiveness. Intervention efficiency measures, in contrast, include the dimension of time required to achieve a level of effectiveness. The purpose of this study was to review the research literature for single-case design studies that included measures of both effectiveness and efficiency for academic (i.e., reading, math, writing, and spelling) interventions in schools. The study reviewed single-case design research on academic interventions published in six dominant peer-reviewed journals in school psychology between 2003-2013: School Psychology Review, School Psychology Quarterly, Psychology in the Schools, Journal of School Psychology, Journal of Behavioral Education, and the Journal of Evidence-Based Practices for Schools. The results of this study suggest that intervention efficiency measures are largely absent from the academic intervention research. The implications of this study are that both effectiveness and efficiency measures need to be considered as they each provide a unique contribution to determining the impact of an academic intervention. Key words: intervention efficiency, single-case designs.

The current study used a single-case design to examine two performance feedback conditions varying in intensity on the frequency of naturalistic math instruction in preschool classrooms during non-instructional times (transition, lunch, free play). Three Head Start teachers received professional development that combined information on four math-oriented milieu teaching strategies (incidental teaching, time delay, mand model, and model), milieu teaching scripts, and goal setting. Teachers then received a feedback package of relatively higher intensity associated with a performance criterion followed by lower-intensity, continued support feedback package. Teachers increased the use of math-oriented milieu teaching strategies and positive attention across intervention conditions. In addition, two of the three teachers evidenced generalization of milieu teaching strategies across contexts and content areas. Results are discussed in terms of conceptualizations of intervention intensity and implications for practice, limitations, and future directions.

The call for school psychologists to demonstrate accountability in the evaluation of services at the individual, group, and system levels comes at a time when school districts nationally are pursuing personnel evaluation models that link teachers’ instructional practices to student achievement. School psychologists have an opportunity to take a leadership role in determining how the impact of their service delivery will be evaluated. The purpose of this article is to: (a) describe the legislative context and historical trends for professional accountability for school psychologists; (b) provide four key principles to consider in designing an accountability system for school psychologists; (c) outline the advantages, disadvantages, and recommended guidelines for using case studies (i.e., single-case designs) and rubric-based approaches for evaluating school psychological services; (d) illustrate how case studies are used to demonstrate accountability in a state-wide school psychology internship program; and (e) discuss the implications for the graduate preparation of school psychologists.

Disruptions can prevent the classroom from being an effective learning environment. The Good Behavior Game (GBG) is a group contingency that has been proven to effectively prevent disruptions and increase engagement. However, the traditional methods of the GBG include teacher scanning for negative student behaviors, and may not align with Positive Behavioral Interventions and Supports (PBIS), which is becoming widely adopted in many schools. Extending the findings of Wright and McCurdy, the current study compared the effects of the GBG and a positive version of the GBG, called the Caught Being Good Game (CBGG), on student behavior, including engagement and off-task behaviors, as well as teachers’ use of positive and negative statements. Results showed both the GBG and the CBGG improved student behavior, with data not clearly indicating one was superior. Neither intervention led to an increase in positive teacher statements. Implications for teachers and suggestions for further research are discussed.

This article describes a multiagency initiative to evaluate academic tutoring services by focusing on the processes that contribute to effective program implementation. Community-based tutoring service providers serving students in the Cincinnati Public Schools (OH) partnered to initiate a “Seal of Approval” process for promoting evidence-based practices among tutoring providers, assigning merit to effective programs, and eliminating or remediating ineffective practices. The consultant-driven process evaluation was designed to be fair and equitable among an array of tutoring service providers. This case example has implications for consultants, tutoring program directors, and school district administrators seeking to establish an accountability system for tutoring service providers.

Internships play a pivotal role in the future of school psychology, and internship training is influenced by disciplinary trends including (a) an increased focus on accountability and demonstrating the positive impact of trainees' practices on student outcomes and (b) growing emphasis on scientifically based and empirically based practice. These influences also include greater sophistication in decision making as well as different ways of outcome-focused decision making. System-wide approaches such as Positive Behavior Support (PBS) and response to intervention (RTI) require high training demands to develop intervention-related decision and accountability skills. Programmatically, accountability systems to meet the training needs of intervention services require scaling up. This article describes a multiyear developmental project bringing diverse universities together to institutionalize a statewide system of accountability. We share data from 3 years (N of interns = 278) and suggest recommendations for outcome-oriented evaluation for other universities, agencies, school psychological services delivery units, or states. © 2009 Wiley Periodicals, Inc.

Although current research indicates that psychotherapeutic change both affects and is affected by spiritual concerns, relatively little is known about the degree to which spirituality is used as an intervention in counseling and how it is perceived by clients and mental health professionals. The purpose of this study was to examine the perceptions of clients and professionals regarding the use of spirituality in counseling. The results suggest that more professionals may be using spirituality in counseling than has previously been reported.

The Critical Incident Stress Management (CISM) model for crisis intervention was developed for use with emergency service personnel. Research regarding the use of the CISM model has been conducted among civilians and high-risk occupation groups with mixed results. The purpose of this study is to examine the social validity of the CISM model for school-based crisis intervention as perceived by school psychologists and school social workers trained in the model. The results suggest that the CISM Model had a positive effect on service delivery, but the training itself did not adequately address the developmental appropriateness or the cross-cultural appropriateness for school-based application. © 2007 Wiley Periodicals, Inc. Psychol Schs 44: 765–777, 2007.

The Critical Incident Stress Management (CISM) Model for crisis intervention was devloped for use with emergency service personnel. Research regarding the use of the CISM Model has been conducted among civilians and high-risk occupation groups with mixed results. The purpose of this study is to examine the effectiveness of the CISM Model for school-based crisis intervention as perceived by teachers and staff. This study compares baseline and CISM Model data over a five-year period. The results suggest that the CISM Model had a positive effect on teacher and staff perceptions of service delivery components (i.e., informing students, meeting with staff members), but no effect on perceptions of the impact of CISM on student outcomes.

School-wide , selective , and individual positive behavioral supports are critical elements of a comprehensive , tiered model of service delivery. Positive peer reporting (PPR) involves brief highly structured sessions during which peers are provided the opportunity to praise the behavior of target students. PPR procedures were modified in this study to provide all children in a classroom the opportunity to provide or receive peer praise. The effects of class-wide PPR on teacher-recorded critical events were evaluated using a multiple baseline design across two classrooms. Results indicated marginal improvement across the two settings , as well as supplemental measures of behavior outside of the classroom and class-wide sociometric nominations. Implications for future research and emerging models of school-wide PBS are discussed. Class-wide positive behavior supports have tremendous appeal over interventions targeting individual students. Class-wide positive behavior supports target many more students in need of support (cost-effective), exclude no students from the benefits of the intervention (equity), and capitalize on the superior efficacy of prevention and early intervention efforts. More importantly, perhaps, class-wide positive behavior supports address a primary limitation of interventions that occur outside of the student's natural instructional environment, which is the failure to demonstrate generalization of effects to the student's system of peers and teachers (DuPaul & Eckert, 1994).

Despite the national proliferation of technical assistance as a driver for school reform and as a model for embedded and sustained professional development, very little is known about the organizational structures and processes needed to support technical assistance. The purpose of this paper is to describe a structured needs assessment process whereby three organizational supports were identified by technical assistance providers. The context for the provision of technical assistance was a state-wide multi-tiered system of support initiative that integrated Positive Behavioral Interventions and Supports and Response to Intervention. The results of this study are informed conceptually by the Instructional Hierarchy Model and the need to match organizational structures and processes to address the identified needs.

Catholic elementary schools must continue to invest in the professional development of math and science teachers in order to prepare students for the challenging work that lies ahead of them. The purpose of the study was to examine the degree to which the Initiative for Catholic Schools (ICS), a 2-year professional development program for science and math teachers, demonstrated positive outcomes within the context of Catholic elementary education across the five levels of impact for a professional development program: participants’ reactions, participants’ learning, organization support and change, participants’ use of new knowledge and skills, and student learning outcomes. The results provide evidence of positive outcomes in the participants’ reactions, participants’ learning, organization support and change, and participants’ use of new knowledge and skills. The impact on student learning outcomes was less consistent and varied by grade level.