Search

In this four-investigation Monte Carlo simulation study, we examined the properties of nonparametric randomization and permutation statistical tests applied to single-case ABAB...AB and alternating treatment designs based on either systematically alternating or randomly determined phase assignments. Contrary to previous admonitions, when appropriately conceptualized nonparametric tests were conducted the systematic designs maintained Type I error rates at or less than their nominal values whenever either no or positive autocorrelation was present in the series. Statistical power for all designs varied as a function of both the number of phases and the number of observations per phase. It was concluded that to increase methodological rigor and statistical precision in the face of autocorrelation, (a) whenever possible, researchers should adopt a randomization scheme in assigning the orders in which the A and B phases are administered and (b) in situations where randomized phase assignment is not feasible, researchers should adopt a systematic phase assignment scheme consisting of individually alternating A and B observations, with a cost of sacrificing some degree of statistical power. Implications for single-case intervention research in school psychology are discussed. (Contains 2 tables and 15 figures.)

The purposes of this paper are to (a) propose an operational standard for defining a "practice," (b) encourage development of professional standards for visual and statistical analysis of single-case research, and (c) propose a standard for using single-case research results to identify practices that are "evidence-based." These topics are not new for behavioral researchers, but they become critical for the larger conversations now occurring among researchers, policy makers, and educators working to implement effective practices in psychology and education.

The purpose of this article is to provide some perspectives on Lilienfeld, Ammirati, and David's (2012) paper on distinguishing science from pseudoscience in school psychology. In many respects their work represents an intervention for "grandiose bragging," a problem that has occasionally occurred when various non-evidence-based or discredited interventions receive sensationalized positive endorsement for adoption in school psychology practice. In this paper, the implications of the Lilienfeld et al. work are discussed within the context of the scientist-practitioner gap, scientific thinking and evaluation of scientific thinking, and negative results research. The authors have advanced our thinking on evidence-based practices in school psychology and education. (Contains 2 tables.)

The American Psychological Association Task Force on Evidence- Based Practice for Children and Adolescents (2008) recommended a systems approach to enhancing care in order to improve outcomes for children and adolescents with mental health needs and redress persistent systemic problems with the structure of services. Recommendations for enhancing an ecological approach to the adoption and implementation of evidence-based practices are offered through increased attention to practice-based research frameworks for adoption and dissemination. Five criteria are discussed for providing acceptable evidence, including (a) systematic evidence searching and adoption of evidence-based prevention and intervention practices, (b) implementation and adherence to intervention integrity, (c) invoking standards for drawing inferences from interventions, (d) using quality assessments to measure outcomes, and (e) adopting formal data analysis procedures to assess intervention outcomes. Each criterion is illustrated with an example. Future research and policy agendas are outlined. (Contains 3 tables and 3 figures.)

Three studies were designed to measure various indicators of the reliability and validity of the Preschool Behavior Screening System (PBSS) as a screening instrument for emotional and behavioral disorder risk status of children of preschool age. Study 1 and Study 2 served to establish evidence for content validity. Study 3 investigated one type of reliability evidence (i.e., internal consistency) and four types of validity evidence (i.e., internal structure validity, validity evidence based on external relations, concurrent validity, and cross-informant agreement).Teachers and parents of children (N = 113) enrolled in preschool and childcare programs completed Phase 1 and Phase 2 of the PBSS and the Behavior Assessment System for Children (BASC-2). Cross-informant agreement between parent and teacher ratings on the PBSS was generally moderate, and internal consistency was high. PBSS Phase 1 and PBSS Phase 2 were correlated in the expected directions, and sensitivity and negative predictive value were high. Results suggest that the PBSS may yield psychometrically sound scores for helping states meet child-find requirements. Directions for future development of the PBSS are provided.

The purpose of the present study was to investigate the statistical properties of two extensions of the Levin-Wampold (1999) single-case simultaneous start-point model's comparative effectiveness randomization test. The two extensions were (a) adapting the test to situations where there are more than two different intervention conditions and (b) examining the test's performance in classroom-based intervention situations, where the number of time periods (and associated outcome observations) is much smaller than in the contexts for which the test was originally developed. Various Monte Carlo sampling situations were investigated, including from one to five participant blocks per condition and differing numbers of time periods, potential intervention start points, degrees of within-phase autocorrelation, and effect sizes. For all situations, it was found that the Type I error probability of the randomization test was maintained at an acceptable level. With a few notable exceptions, respectable power was observed only in situations where the numbers of observations and potential intervention start points were relatively large, effect sizes were large, and the degree of within-phase autocorrelation was relatively low. It was concluded that the comparative effectiveness randomization test, with its desirable internal validity and statistical-conclusion validity features, is a versatile analytic tool that can be incorporated into a variety of single-case school psychology intervention research situations. (Contains 6 tables and 4 figures.)

The evidence-based practice movement has focused on identifying, disseminating, and promoting the adoption of evidence-based interventions. Despite advances in this movement, numerous barriers, such as the lack of treatment integrity assessment methods, remain as challenges in implementation. Accurate teacher self-report could be an efficient treatment integrity assessment method. A multiple baseline across participants design was used to evaluate the effect of the (a) Treatment Integrity Planning Protocol on treatment integrity, (b) accuracy of teachers' self-report when collected daily or weekly, and (c) Good Behavior Game on students' behavior. Results indicate teachers demonstrated a relatively high level of treatment integrity across phases, teachers accurately reported their treatment integrity during daily reporting, and there was no relationship between treatment integrity and behavioral outcomes. (Contains 2 footnotes, 1 figure and 5 tables.)

Accurate identification of young children at risk for mental health problems is a key step in establishing early childhood preventive intervention programs. Without psychometrically valid identification procedures, children in need of early intervention may not be identified and may not receive appropriate care. This article provides a review of universal screening measures to help teachers, school psychologists, and other mental health professionals make informed decisions about selecting tools to assist in identifying preschool-age children at risk for mental health problems. A review of the literature on broadband measures designed to screen the social and emotional functioning of children age 3 to 5 years old yielded three published measures that met predetermined inclusionary criteria. An additional unpublished measure was also included. Selected measures were reviewed according to the "Standards for Educational and Psychological Testing". Special attention was given to predictive validity indices, including sensitivity, specificity, positive predictive value, and negative predictive value. The results of the review provided information on the measures that were most psychometrically sound and cost-effective for screening young children for mental health problems. Recommendations are made for routine screening, so young children can be referred for prevention programming before their problems worsen and negatively affect their development. (Contains 4 tables.)

In recent years, single-case designs have increasingly been used to establish an empirical basis for evidence-based interventions and techniques in a variety of disciplines, including psychology and education. Although traditional single-case designs have typically not met the criteria for a randomized controlled trial relative to conventional multiple-participant experimental designs, there are procedures that can be adopted to create a randomized experiment in this class of experimental design. Our two major purposes in writing this article were (a) to review the various types of single-case design that have been and can be used in psychological and educational intervention research and (b) to incorporate randomized experimental schemes into these designs, thereby improving them so that investigators can draw more valid conclusions from their research. For each traditional single-case design type reviewed, we provide illustrations of how various forms of randomization can be introduced into the basic design structure. We conclude by recommending that traditional single-case intervention designs be transformed into more scientifically credible randomized single-case intervention designs whenever the research conditions under consideration permit. (Contains 8 footnotes, 18 tables, and 4 figures.)

The purpose of this article is to present Convergent Evidence Scaling (CES) as an emergent method for combining data from multiple assessment indicators. The CES method combines single-case assessment data by converging data gathered across multiple persons, settings, or measures, thereby providing an overall criterion-referenced outcome on which to base decisions. Examples are described using CES for monitoring treatment integrity and for treatment evaluation to demonstrate the use of the method within consultation problem-solving and response-to-intervention models. The article concludes with a discussion of the strengths and limitations of the CES method. (Contains 3 tables.)

In this article, we present a balanced view of the prescriptive authority debate in relation to five major issues and within the context of the practice of psychology in the schools: (a) appropriate training, (b) quality of care, (c) access to services, (d) continuity of care, and (e) professional identity. We also provide an analysis of the quality and relevance of each issue within the school practice context. Despite the increasing momentum of the prescriptive authority initiative at the state level, it seems unlikely that prescription privileges will be extended to psychologists working in school-based practice settings. Nevertheless, informed school psychologists have the opportunity to become involved in this issue at both the national and state levels. These efforts may take the form of supporting or opposing the initiative or of lobbying for extension of prescriptive authority into some school settings. At the individual level, informed school psychologists will have the opportunity to develop strong working relationships with prescribing psychologists in their communities and to exercise appropriate professional judgments when referring students or families for services in the community.

We evaluated a multi-family support group intervention program in elementary schools. Kindergarten through third-grade children at eight urban schools in a Midwestern university community were universally invited to participate in the Families and Schools Together (FAST) program, and made up half of the study participants; the other half were K-3 children identified by teachers as having behavioral problems and being at risk for referral to special education services. Children were initially paired on the basis of five relevant matching variables, including teacher assessment of behavioral problems, and then randomly assigned to either ongoing school services (control) or the FAST program. Parents and teachers completed pre-, post-, and 1-year follow-up assessments. Data were available and analyzed for 67 pairs. Immediate follow-up parent reports showed that FAST students declined less on a family adaptability measure relative to control group students. This effect was still present at the 1-year follow-up assessment. In addition, FAST parents reported statistically significant reductions in children's externalizing (aggressive) behaviors, as compared to the reports of control group parents. School district data showed descriptively fewer special-education referrals for FAST children (one case) as compared with control group children (four cases). Results are discussed in relation to future research on universal prevention programs. (Contains 4 tables and 2 figures.)

The Treatment Integrity Planning Protocol (TIPP) provides a structured process for collaboratively creating a treatment integrity assessment within a consultation framework. The authors evaluated the effect of the TIPP on the implementation of an intervention designed to improve the consistency of students' mathematics performance. Treatment integrity was assessed via permanent products and teacher self-report assessments resulting from TIPP completion. Results showed that (a) 3 teachers' treatment integrity declined within the first week of implementation and increased subsequent to completion of the TIPP, (b) teachers accurately reported their level of treatment integrity, and (c) the relationship between treatment integrity and student outcomes was unclear. (Contains 2 tables, 1 figure and 1 footnote.)

The present study examined the effect accommodations have on test results of students with and without disabilities and documented experts' judgments about the appropriateness of testing accommodations. Test score data were collected from 218 fourth-grade students with and without disabilities on mathematics and science performance tasks and from eight testing experts who evaluated the fairness and validity of a sample of testing accommodations used with these students. Results indicated that, for most students with disabilities and some students without disabilities, packages of testing accommodations had a moderate to large effect on performance task scores. Expert reviewers rated most accommodations for a student with disabilities as being both valid and fair, and they gave accommodations listed on a student's individualized education program (IEP) significantly higher validity and fairness ratings than accommodations that were not listed on the student's IEP. Interpretations of these data are provided and implications for practice and future research are discussed. (Contains 5 tables and 1 figure.)

Treatment integrity (also referred to as "treatment fidelity," "intervention integrity," and "procedural reliability") is an important methodological concerning both research and practice because treatment integrity data are essential to making valid conclusions regarding treatment outcomes. Despite its relationship to validity, treatment integrity has been largely overlooked in education research and related fields. Influences over the past 5 years have increased attention to this construct, yet many questions remain in regard to how it should be addressed. The purpose of this article is twofold. first, we provide a brief overview of the current state of (a) conceptual models of treatment integrity; (b) research on assessment, promotion, and relationship to student outcomes; and (c) treatment integrity related to response to intervention models of service delivery. Second, we suggest future directions for research, practice, and policy related to treatment integrity as we move toward a better scientific understanding of this construct. The article concludes with a description of the purposes of this special series. (Contains 2 figures.)

The Concerns-Based Adoption Model (CBAM) is proposed as a heuristic for school-based consultants interested in identifying impact points for facilitating implementation of research-based practices and programs in classrooms and schools. The article begins with a discussion of the change facilitator's role in promoting changes in educational practices and a description of six essential strategies consultants can use to facilitate change. Background on CBAM's development and its applications in school environments also are provided. In addition, CBAM's three diagnostic frameworks (i.e., Stages of Concern, Levels of Use, and Innovation Configurations) are outlined and proposals for using CBAM assessment data to support educators' use of research-based practices are discussed. Limitations of the model and implications for further research are presented. (Contains 5 figures and 1 footnote.)

The evidence-based practice (EBP) movement has the potential to significantly advance the quality of psychological and educational services provided by psychologists working in schools. Training psychologists in EBP has challenged the profession and caused faculty in graduate programs to reevaluate and retool professional training curricula and instructional practices. Four domains of challenges in graduate training are identified: (a) integrating the EBP knowledge base into the curriculum, (b) expanding models of research training, (c) expanding training in prevention science, and (d) expanding training in problem-solving consultation and school contextual issues. For each of these, the author discusses the range and scope of the challenge and possible solutions for advancing graduate training in psychology relevant to school practice.

The transportability of an evidence-based teacher professional development program, the Incredible Years Classroom Management Program, was evaluated. This study compared the impact of two training methods: self-administered videotape modeling (VM) and self-administered videotape modeling plus consultation (VMC) on teachers' use of classroom management strategies, reductions in disruptive behaviors, acceptability, and contextual barriers of sustaining EBIs in practice settings. Four pairs of teachers (N = 8) were randomly assigned to VM or VMC conditions in a multiple probe design. Students (1-2 per classroom; N = 13) displaying high levels of externalizing problems also participated. Statistically significant between-groups differences in teacher confidence ratings, use of positive instructional practices, and acceptability favored the VMC condition. Positive trends favoring VMC students, including clinically significant increases in social competence, were also found. Although students in both groups exhibited reductions in disruptive behaviors, significant between-groups differences related to disruptive behavior were not observed. Findings related to contextual variables (e.g., time, cost, institutional support, adaptability of the program) have implications for future adoption of this program.

We provide an overview of the role professional development plays in multitiered prevention and intervention models. Specifically, professional development is discussed within the context of establishing sustainable improvement in schools as professionals implement multitiered models of prevention and intervention services, programs, and practices related to response to intervention. We discuss the system capacity for sustainable improvement and system-wide procedures that can be implemented to support multitiered prevention models. We also discuss various models, levels of professional development, and the relationship between professional development implementation and sustainability of school improvement. We conclude with a discussion of the challenges schools are likely to face in establishing response to intervention. We also note what preservice and in-service professional development activities can offer to help overcome some of these challenges and barriers to system-wide change in quality prevention and intervention programs. (Contains 4 tables.)

In the current paper we discuss the treatment of childhood anxiety disorders using a problem-solving consultation framework. The role of consultation as a service delivery model in a school setting is elaborated on, as well as the contribution that consultation has in the movement towards evidence-based practices in school psychology. Additionally, a description of the role of consultation specifically in the treatment of childhood anxiety is provided. The role of parents and teachers in treatment is further elucidated, and the separate influence each may have on traditional treatment outcomes is presented. Finally, we discuss the benefits of using a conjoint behavioral consultation model (CBC) of treatment, whereby a mental health professional consults with a child's parents and teachers, who collaborate to assess, monitor, and treat the child's problems. We conclude with a discussion of the benefits of this collaborative approach to treatment of childhood anxiety and illustrate the application of CBC with a case example of selective mutism.

This study examined student, parent, and teacher perceptions of the use of testing accommodations and the relationship between student perceptions of testing accommodations and their disability status and grade level. Students with and without disabilities completed math and reading achievement tests with and without accommodations. Students, parents, and teachers completed a questionnaire to share their views on the use of testing accommodations. Significant differences were found in the proportions of students with and without disabilities who reported that the accommodated test condition was easier, more comfortable, and a better indicator of their knowledge. Most parents and teachers perceived testing accommodations to be fair and valid for students with disabilities. Consequential aspects of testing are an important part of validity evidence for large-scale assessment systems. The perceived positive consequences of testing accommodations provide further evidence of their continued use and their role in facilitating valid test scores. (Contains 1 table.)

Major national, state, and local systems reform efforts are under way to close the gap between science and service through implementation of a range of evidence-based practices (EBPs). A set of rhetorical distinctions and ensuing debates about the relevance of science versus service and efficacy versus effectiveness threatens to widen rather than close the gap. However, methodological and conceptual approaches, drawing upon strong theory-driven models, offer considerable promise for an integrated science on implementation effectiveness. This paper outlines some of the major scientific and policy challenges in strengthening the link between science and service in school psychology within the context of the papers in this series.

School psychology is facing a major shortage of faculty in graduate training and education programs. To deal with this shortage and the issues that surround it, we propose a conceptual framework that incorporates a number of impact points that graduate training programs can use to educate and sustain individuals in academic careers. The impact points include: selection of students, program-related training variables, post-program transition variables, and sustainability of academic careers. Each of these impact points is discussed within the context of the role that current faculty and practitioners in the profession can play in graduate education and training of academic scholars. Among the variety of potential solutions to the shortage, we introduce the concept of the "virtual university" to promote future education and sustainability of faculty within our graduate training programs. A case scenario from graduate students at the University of Wisconsin-Madison is presented as a context for the impact points raised in the article. We argue that those of us in the profession think systemically and lead the way into a new era of collaborative work across our graduate programs and among our colleagues in clinical, counseling, and related areas of applied and professional psychology graduate training.

In this study, 18 teachers and 32 of their students who had reading difficulties were randomly assigned to one of two assessment conditions. In the functional assessment condition, consultants identified functional relationships among environmental events and targeted reading behaviors to develop an intervention plan. In the empiric condition, consultants selected an intervention plan based on interventions matched to various reading problems (i.e., omitting individualized problem analysis). Consultants facilitated interventions consistent with the two conditions through a problem-solving consultation model and collected multiple outcome measures. Both the functional assessment and empiric conditions led to improved outcomes in targeted reading behaviors, with most improvements falling within the moderate to large range of single-participant effect sizes. No significant differences were found between the functional assessment and empiric conditions on the outcome variables of intervention effects, curriculum-based assessment, consumer satisfaction, or material costs, although empiric assessment took less time than functional assessment. Implications regarding the generalizability of these results and directions for further study regarding the treatment utility of assessment for consultation are discussed.

Assessments of consultants, clients, and consumer satisfaction were used to examine the effects of a competency-based consultation training program conducted over 4 years. Using a multiple-baseline framework to assess training effects on consultants and single-case study designs to evaluate changes in client behavior, a number of significant results were found. As predicted, consultants ( N = 24) significantly increased their consultation skills and knowledge, but with no change in attitudes toward behavioral intervention techniques. Supervised consultation with preschool teachers and parents resulted in a range of behavior change in clients ( N = 39), with an overall effect size of 0.51. Consumers reported a high level of satisfaction with training and consultation services. Results of a long-term follow-up with consultants indicated positive views and use of consultation. The discussion focuses on these results, and implications for consultation training and future research directions.

We present an overview of issues related to evidence-based practice and the role that the school psychology profession can play in developing and disseminating evidence based interventions (EBIs). Historical problems relating to and the recurring debate about the integration of research into practice are presented as a context for the current challenges faced by those engaged in the EBI movement in psychology and education. Potential solutions to the problems posed by the adoption of EBIs in practice are presented within the context of the directions to be taken by the Task Force on Evidence-Based Interventions in School Psychology (Task Force). Five assumptions are presented that can guide the Task Force in addressing the integration of EBIs in practice. These assumptions are followed the Task Force for the promotion of EBIs in practice. The action plans are conceptualized as a shared responsibility of school psychology researchers, trainers, and practitioners. Future directions and implications for policy among groups with a common agenda for promoting EBIs are also presented.

We present an overview of issues related to evidence-based practice and the role that the school psychology profession can play in developing and disseminating evidence-based interventions (EBIs). Historical problems relating to and the recurring debate about the integration of research into practice are presented as a context for the current challenges faced by those engaged in the EBI movement in psychology and education. Potential solutions to the problems posed by the adoption of EBIs in practice are presented within the context of the directions to be taken by the Task Force on Evidence-Based Interventions in School Psychology (Task Force). Five assumptions are presented that can guide the Task Force in addressing the integration of EBIs in practice. These assumptions are followed by five recommendations that can be translated into action plans to be adopted by the Task Force for the promotion of EBIs in practice. The action plans are conceptualized as a shared responsibility of school psychology researchers, trainers, and practitioners. Future directions and implications for policy among groups with a common agenda for promoting EBIs are also presented.

The article "Implementation, Sustainability, and Scaling Up of Social-Emotional and Academic Innovations in Public Schools" by Elias, Zins, Graczyk, and Weissberg (2003) is a thought-provoking contribution, and one that begs for more application. Some of the points the authors raise have been articulated in the school and clinical evidence-based intervention (EBI) literature and, although these issues may deserve repeating, it is time to move to an action agenda to advance research and practice in this area. In fact, the Task Force on Evidence-Based Interventions in School Psychology (2003) has a committee specifically focused on the Research-to-Practice agenda and the Division 53 of American Psychological Association (APA) has a Task Force devoted to this topic, both of which are addressing scaling up and adoption of innovation issues in school and community settings. Within this context, the authors provide a good look at the big picture and some ideas that can guide implementation of social-emotional interventions in schools. In this commentary, the authors aim to reinforce and extend some of the key points made by Elias and his colleagues. The authors' thoughts are organized into two main sections: (1) reflections and reactions, and (2) needed research and research methods. (Contains 1 table.)

Research compared effectiveness of two different approaches of conjoint consultation using a manual versus a videotape series as the main components of training parents and teachers to treat children's behavioral difficulties. Parent and teacher reports characterized students as meeting their overall behavior goals; and reported high rates of treatment acceptability and satisfaction with manual and videotape programs. (Contains 62 references and 12 tables.) (ADT)

Provides a rationale for the name change of the new Standing Section of "School Psychology Quarterly" from "Empirically Supported Interventions" to "Evidence-Based Interventions" (EBI). Furnishes updates on the intent of publishing articles in the EBI section and highlights the types of articles the journal hopes to feature in the future. (GCP)