What Works Clearinghouse™ Procedures Handbook, Version 4.1

The What Works Clearinghouse (WWC) is an initiative of the U.S. Department of Education's Institute of Education Sciences (IES), which was established under the Education Sciences Reform Act of 2002. It is an important part of IES's strategy to use rigorous and relevant research, evaluation, and statistics to improve the nation's education system. The WWC examines research about interventions that focus on improving educationally relevant outcomes, including those for students and educators. The WWC systematic review process is the basis of many of its products, enabling the WWC to use consistent, objective, and transparent standards and procedures in its reviews, while also ensuring comprehensive coverage of the relevant literature. The WWC systematic review process consists of five steps: (1) Developing the review protocol; (2) Identifying relevant literature; (3) Screening studies; (4) Reviewing studies; and (5) Reporting on findings. A separate WWC Standards Handbook describes step 4, including the standards used by the WWC to review studies (ED602036). This new "Procedures Handbook" updates the previous Version 4.0 to Version 4.1. The following substantive updates were made: (1) The removal of the "substantively important" designation; (2) The addition of standard error calculations for all effect sizes; (3) The addition of single-case design (SCD) procedures for synthesizing SCD study findings using design-comparable effect sizes; (4) The addition of procedures to estimate effects from regression discontinuity designs (RDDs); (5) Clarification of decision rules determining the use of difference-in-difference effect sizes; (6) The synthesis of studies within intervention reports using a fixed-effects model; (7) The modification of the intervention report effectiveness rating; and (8) Levels of evidence in practice guides. The remainder of this handbook is organized as follows: Chapter II describes the steps that the WWC uses to develop a review protocol. Chapter III describes how the WWC identifies the relevant literature. Chapter IV describes the screening process to determine whether a study is eligible for review, and chapter V describes the procedures used to review eligible studies. Chapter VI describes how the WWC summarizes evidence of effectiveness. Organizational procedures used by the WWC to ensure an independent, systematic, and objective review are described in the appendices. As the WWC uses and applies the procedures in this "Procedures Handbook," reviewers and other WWC staff may occasionally need additional guidance. If necessary, the WWC will produce guidance documents to provide clarification and interpretation of WWC procedures. [For version 4.0 of this handbook, see ED577035.] [For "Supplement to the What Works Clearinghouse Procedures Handbook, Version 4.1," see ED608943.]