Search

This technical assistance paper provides policy and guidance to individuals with test administration responsibilities in adult education programs. The Florida assessment policies and guidelines presented in this paper are appropriate for state and federal reporting. Therefore, guidance and procedures regarding the selection and use of appropriate student assessment are also included. Additionally, the following important information for adult education programs is reviewed: (1) The definition of key terms and acronyms; (2) Selection of appropriate assessments by student and program type; (3) Appropriate student placement into program and instructional level; (4) Verification of student learning gains, Educational Functioning Level, and/or program completion; (5) Accommodations for students with disabilities and other special needs; (6) Assessment procedures for Distance Education; and and (7) Training for all staff who administer the standardized assessments. [For the "2018-2019 Florida Adult Education Assessment Technical Assistance Paper," see ED600582.]

The Partnership for Assessment of Readiness for College and Careers (PARCC) is a state-led consortium designed to create next-generation assessments that, compared to traditional K-12 assessments, more accurately measure student progress toward college and career readiness. The PARCC assessments are aligned to the Common Core State Standards (CCSS) and were administered operationally for the first time in the 2014-2015 academic year. PARCC comprises assessments in both English Language Arts/Literacy (ELA/L) and Mathematics in grades 3 to 8 and high school. The information provided in this technical report is intended for use by those who evaluate tests, interpret scores, or use test results in making educational decisions. It is assumed that the reader has technical knowledge of test construction and measurement procedures, as stated in "Standards for Educational and Psychological Testing" (American Educational Research Association [AERA], American Psychological Association [APA], & National Council on Measurement in Education [NCME], 2014). The purpose of this technical report is to describe the first operational administration of the PARCC assessments and includes the following topics: (1) Background and purpose of the assessments; (2) Test development of items and forms; (3) Test administration, security, and scoring; (4) Test taker characteristics; (5) Classical item analyses and differential item functioning; (6) Reliability and validity of scores; (7) Item response theory (IRT) calibration and scaling; (8) Performance level setting; (9) Development of the score reporting scales and student performance; (10) student growth measures; and (11) Quality control procedures.

The "Principles & Standards for Quality Charter School Authorizing," first established in 2004, reflect the lessons learned by experienced authorizers. This foundational resource for National Association of Charter School Authorizers (NACSA) has been used to guide authorizing practices across the country and is referenced in state statutes. It focuses on the ends that authorizers should aim to attain in creating and upholding high expectations for the schools they charter while recognizing there are many means of getting there. The Principles articulate a set of beliefs about quality charter school authorizing. The Standards identify core authorizer responsibilities and describe how the principles are upheld within each responsibility. Consult this resource often as a guide to understanding and fulfilling the critical responsibilities of charter school authorizers. [For the previous edition, see ED580966.]

Disproportionate rates of suspensions and expulsions among racial/ethnic groups are a local, state, and national concern. Maryland has made significant progress in decreasing the use of exclusionary discipline, but there is substantial work left to do to move toward equity. Maryland has cut the use of exclusionary discipline practices by half in the last decade, from 8.7 to 4.3 percent since the 2005-2006 school year. However, when Maryland discipline data is disaggregated by student group, it becomes apparent that the experience of individual student groups varies widely. According to 2015-2016 data, 8.1 percent of African-American students and 9.9 percent of students with disabilities in Maryland received an out-of-school suspension or expulsion, compared to just 2.3 percent of white students and 3.6 percent of students without disabilities. It was review of data such as this that sparked reform in the area of equitable discipline and caused the Maryland State Board of Education (MSBE) to direct the Maryland State Department of Education (MSDE) to develop a method to analyze local school system discipline data. Data indicators can help school climate or leadership teams identify racial/ethnic disproportionality in school discipline practices. The purpose of this guide is to help educators understand the MSDE's method for analyzing out-of-school suspension and expulsion data to determine whether school discipline practices have a disproportionate impact on students of color and students with disabilities. This guide outlines a process for examining disproportionality within schools, as well as root cause analysis to help systems and schools identify areas of need on which to focus to reduce and eliminate disproportionality in schools. The guide is a part of a statewide effort to reform school discipline and ensures that policies and practices related to student conduct foster appropriate behavior create a positive school climate, and safe environments. In addition, this guide is a framework for Maryland schools and systems to use in analyzing discipline-related policies and practices that may lead to disproportionality.

The Partnership for Assessment of Readiness for College and Careers (PARCC) is a state-led consortium designed to create next-generation assessments that, compared to traditional K-12 assessments, more accurately measure student progress toward college and career readiness. The PARCC assessments are aligned to the Common Core State Standards (CCSS) and were administered operationally for the first time in the 2014-2015 academic year. PARCC comprises assessments in both English Language Arts/Literacy (ELA/L) and Mathematics in grades 3 to 8 and high school. The information provided in this technical report is intended for use by those who evaluate tests, interpret scores, or use test results in making educational decisions. It is assumed that the reader has technical knowledge of test construction and measurement procedures, as stated in "Standards for Educational and Psychological Testing" (American Educational Research Association [AERA], American Psychological Association [APA], & National Council on Measurement in Education [NCME], 2014). The purpose of this technical report is to describe the second operational administration of the PARCC assessments in the 2015-2016 academic year and includes the following topics: (1) Background and purpose of the assessments; (2) Test development of items and forms; (3) Test administration, security, and scoring; (4) Test taker characteristics; (5) Classical item analyses and differential item functioning; (6) Reliability and validity of scores; (7) Item response theory (IRT) calibration and scaling; (8) Performance level setting; (9) Development of the score reporting scales and student performance; and (10) Quality control procedures.

The Partnership for Assessment of Readiness for College and Careers (PARCC) is a state-led consortium designed to create next-generation assessments that, compared to traditional K-12 assessments, more accurately measure student progress toward college and career readiness. The PARCC assessments are aligned to the Common Core State Standards (CCSS) and were administered operationally for the first time in the 2014-2015 academic year. PARCC comprises assessments in both English Language Arts/Literacy (ELA/L) and Mathematics in grades 3 to 8 and high school. The information provided in this technical report is intended for use by those who evaluate tests, interpret scores, or use test results in making educational decisions. It is assumed that the reader has technical knowledge of test construction and measurement procedures, as stated in "Standards for Educational and Psychological Testing" (American Educational Research Association [AERA], American Psychological Association [APA], & National Council on Measurement in Education [NCME], 2014). The purpose of this technical report is to describe the first operational administration of the PARCC assessments and includes the following topics: (1) Background and purpose of the assessments; (2) Test development of items and forms; (3) Test administration, security, and scoring; (4) Test taker characteristics; (5) Classical item analyses and differential item functioning; (6) Reliability and validity of scores; (7) Item response theory (IRT) calibration and scaling; (8) Performance level setting; (9) Development of the score reporting scales and student performance; and (10) Quality control procedures.

Rhode Island educators believe that implementing a fair, accurate, and meaningful educator evaluation and support system will help improve teaching and learning. The primary purpose of the Rhode Island Model Teacher Evaluation and Support System (Rhode Island Model) is to help all teachers improve. Through the Model, the goal is to help create a culture where all teachers have a clear understanding of what defines excellence in their work; are provided with prioritized, specific, and actionable feedback about their performance; and receive support to continuously improve their effectiveness, regardless of the number of years they have been teaching. The purpose of this Guidebook is to describe the process and basic requirements for evaluating and supporting teachers with the Rhode Island Model. For aspects of the Rhode Island Model that have room for flexibility and school/district-level discretion, the different options have been clearly separated and labeled with a "Flexibility Factor." For the purposes of this Model, "teacher" means any public school employee working under a teacher certification whose primary responsibilities include instructional planning, managing a classroom environment, and student instruction. To help educators better understand "how" to best implement various aspects of the Rhode Island Model, additional resources are available on the Rhode Island Department of Education (RIDE) website, including online training modules, sample Student Learning/Outcome Objectives, and a suite of calibration protocols designed to help school and district leaders facilitate ongoing calibration exercises. Four appendices are included: (1) Student Learning Objective Scoring Lookup Tables; (2) Online Resources; (3) Teacher Professional Practice Rubric; and (4) Teacher Professional Responsibilities Rubric.

This guide is designed to help researchers make sure that their research reports include enough information about study measures so that readers can assess the quality of the study's methods and results. The guide also provides examples of write-ups about measures and suggests resources for learning more about these topics. The guide assumes that researchers have: (1) clearly articulated their research questions; (2) completed a rigorous review of the leading measures for assessing each necessary component of the theory of change and the relevant domains and constructs; and (3) selected measures that are aligned with the intervention's theory of change (also referred to as a logic model), and that address the study's research questions. These measures may include contextual factors, inputs to implementation, expected intervention activities, out- puts, and both short-term and long-term outcomes (Lugo-Gil et al. 2011; W.K. Kellogg 2004). Also provided are five checklists to help researchers provide complete information describing: (1) their study's measures; (2) data collection training and quality; (3) the study's reference population, study sample, and measurement timing; (4) evidence of the reliability and construct validity of the measures; and (5) missing data and descriptive statistics. The brief includes an example of parts of a report's methods and results section illustrating how the checklists can be used to check the completeness of reporting. A bibliography is also included. The appendix contains "Measures Reporting Checklist for Researchers".

The purpose of this Guidebook is to describe the process and basic requirements for the student learning measures that are used as part of the building administrator evaluation and support process. For aspects of the process that have room for flexibility and school/district-level discretion, the different options have been clearly separated and labeled with a "Flexibility Factor." For the purposes of this Guidebook "building administrator" means any public school employee working under a Building Level Administrator certification. To help educators better understand how to best implement various aspects of student learning process, additional resources are available on the Rhode Island Department of Education (RIDE) website, including online training modules, sample Student Learning/Outcome Objectives, and a suite of calibration protocols designed to help school and district leaders facilitate ongoing calibration exercises. Two appendices are included: (1) Student Learning Objective Scoring Lookup Tables; and (2) Online Resources.

The purpose of this Guidebook is to describe the process and basic requirements for the student learning measures that are used as part of the support professional evaluation and support process. For aspects of the process that have room for flexibility and school/district-level discretion, the different options have been clearly separated and labeled with a "Flexibility Factor." For the purposes of this Guidebook, "support professional" refers to the following roles: library media specialists, school nurse teachers, reading specialists, school counselors, school psychologists, speech language pathologists, and school social workers. To help educators better understand how to best implement various aspects of student learning process, additional resources are available on the Rhode Island Department of Education (RIDE) website, including online training modules, sample Student Learning/Outcome Objectives, and a suite of calibration protocols designed to help school and district leaders facilitate ongoing calibration exercises. Two appendices are included: (1) Student Learning Objective Scoring Lookup Tables; and (2) Online Resources.

The purpose of this Guidebook is to describe the process and basic requirements for the student learning measures that are used as part of the teacher evaluation and support process. For aspects of the process that have room for flexibility and school/district-level discretion, the different options have been clearly separated and labeled with a "Flexibility Factor." For the purposes of the Guidebook, "teacher" means any public school employee working under a teacher certification whose primary responsibilities include instructional planning, managing a classroom environment, and student instruction. To help educators better understand "how" to best implement various aspects of student learning process, additional resources are available on the Rhode Island Department of Education (RIDE) website, including online training modules, sample Student Learning/Outcome Objectives, and a suite of calibration protocols designed to help school and district leaders facilitate ongoing calibration exercises. Two appendices are included: (1) Student Learning/Outcome Objective Scoring Lookup Tables; and (2) Online Resources.

In accordance with the ENQA (European Association for Quality Assurance in Higher Education) membership criteria laid down in the Statutes of ENQA, member agencies are required to undergo external reviews against the membership criteria, and thereby the Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG) as adopted by the European Ministers in charge of higher education in Bergen in 2005, at least once every five years. These present Guidelines apply to ENQA coordinated reviews to be conducted periodically, for a first application for Full/Associate membership of ENQA and for inclusion in the European Quality Assurance Register for Higher Education (EQAR). According to the Statutes of ENQA, applications for Full/Associate Membership are considered only if accompanied by an independent external review report on the agency's conformity with the membership criteria. The membership criteria of ENQA consist of Part III of the ESG and some additional requirements and guidelines. However, due to the specific structure of ESG, agency reviews are not assessing Part III of the ESG only. Indeed, according to the first standard of Part III, agencies are required to demonstrate compliance with ESG Part II. Likewise, the first standard of Part II requires agencies to demonstrate that they take into account the effectiveness of internal quality assurance mechanisms (Part I). The review against the membership criteria can also include other aspects of the agency's work or organisation (see section 2). In any case, the same principles and processes outlined in these Guidelines are applied. It is recommended that the second and subsequent rounds of reviews aim at striving for improvement. In addition to judging the compliance with the ENQA membership criteria/ESG, the reviews should follow an enhancement-led approach. Agencies are invited to reflect on how they could further improve their compliance with all membership criteria/ESG and to identify areas for development in their self-evaluation report. It is also possible to accommodate a more thorough consideration of some of the criteria if the agency wishes. The specific areas of development will be considered in the next external review of the agency. Appended are: (1) Introductory section and structure of the final review report; (2) Financial Guidelines for ENQA coordinated reviews; (3) Statutes of ENQA; (4) Standards and Guidelines for Quality Assurance in the European Higher Education Area (Parts I, II and III); (5) Scrutiny for reconfirmation of or application for Full/Associate Membership; and (6) Mapping grid for panel members. A glossary is included. (Contains 2 footnotes.)

The Race to the Top Reform Support Network (RSN) designed this toolkit to help States and school districts implement student learning objectives (SLOs) with the highest degree of quality. The toolkit contains templates, guidance documents, sample SLOs and other tools, which States and school districts can select or adapt for their own purposes as they implement a system of quality control.

The Teacher Education Accreditation Council's (TEAC's) "Guide to Accreditation" is primarily for the faculty, staff, and administrators of TEAC member programs. It is designed for use in preparing for both initial and continuing accreditation. Program personnel should understand and accept all the components of the TEAC accreditation process before entering into it. Everyone in the program who is responsible for some or all parts of the program's accreditation (including annual reports) is encouraged to read and use this guide. The goal in writing the guide was to make each step of the process clear and to make the accreditation process itself transparent. For example, throughout, this guide explains the rationale behind the process as well as each step. Also included are details about formative evaluation, the audit, and the accreditation decision. Because this guide spells out in detail what is expected from the members and how all the steps in the process fit together, TEAC auditors and members of the Accreditation Panel and Accreditation Committee should also find the volume useful. Appended are: (1) List of Tables, Figures, and Formats; and (2) TEAC Glossary. An index is included. Individual chapters contain footnotes and references. (Contains 8 tables and 1 figure.) [For "Guide to Accreditation, 2011-2012," see ED516752.]

This study is the first randomized study of the impacts of the vocabulary instruction program Kindergarten PAVEd for Success (K-PAVE) (Hamilton and Schwanenflugel 2011) on low-income students in kindergarten and grade 1. The study has two components. The first component is a test of the impacts of one year of the K-PAVE vocabulary instruction in kindergarten, before students have encountered formal reading instruction (the kindergarten study). The second component is a test of whether the effects of the vocabulary instruction in kindergarten are sustained beyond the intervention period, in grade 1 (the grade 1 follow-up study). This report presents the findings from the grade 1 follow-up study, which followed students who were part of the kindergarten sample into grade 1. During grade 1, no students received K-PAVE; the study was designed to determine whether the impacts found at the end of kindergarten were sustained in grade 1. The report also presents supplemental findings on impacts at the end of kindergarten and discusses outcomes that were not addressed in the kindergarten report. Appended are: (1) K-PAVE Materials Provided to Teachers; (2) Sample Weekly Unit from K-PAVE Program; (3) List of K-PAVE Target Words; (4) Mississippi Counties with Study Schools; (5) Exploratory Analyses of Kindergarten Impacts on Components of Classroom Vocabulary and Comprehension Support, Student Lexical Diversity, and Teacher Lexical Diversity; (6) Statistical Power Analysis; (7) Random Assignment; (8) Recruitment and Random Selection of the Student Sample; (9) Comparison of Students Missing and Not Missing Baseline Assessment; (10) Model Specifications; (11) Sensitivity Analyses; (12) Imputation of Missing Data; (13) Data Collection Procedures; (14) Data Quality Assurance Procedures; (15) School, Teacher, and Student Covariates; (16) Unadjusted Sample Means and Standard Deviations for Outcome Measures; (17) Classroom Observation Measures Used to Create Vocabulary and Comprehension Support Outcome Variables; and (18) Teacher Survey. (Contains 6 figures, 53 tables, 1 map and 69 footnotes.) [For related report, "The Effectiveness of a Program to Accelerate Vocabulary Development in Kindergarten (VOCAB): Kindergarten Final Evaluation Report. NCEE 2010-4014," see ED512900.]

The Teacher Follow-up Survey (TFS) is sponsored by the National Center for Education Statistics (NCES) on behalf of the U.S. Department of Education and is conducted by the U.S. Census Bureau. TFS is a follow-up survey of selected elementary and secondary school teachers who participated in the Schools and Staffing Survey (SASS). SASS is the largest, most extensive survey of K-12 school districts, schools, teachers, and administrators in the United States today. It provides extensive data on the characteristics and qualifications of teachers and principals, teacher hiring practices, professional development, class size, and other conditions in schools across the nation. TFS focuses on a sample of teachers who participated in SASS, including both teachers who left and teachers who remained in the K-12 teaching profession. This report covers all phases of the Teacher Follow-up Survey (TFS), from survey planning through data file availability. The TFS determines how many teachers remained at the same school, moved to another school, or left the profession in the year following the Schools and Staffing Survey (SASS) administration. This report contains chapters on preparation for the 2008-09 TFS, frame creation and sample selection procedures, data collection, response rates, data processing, imputation procedures, weighting and variance estimation, a review of the quality of TFS data, structure of TFS data files and information on merging data files, and user notes and cautions. Appended are: (1) Key Terms for TFS; (2) Questionnaire Availability; (3) First Cognitive Testing of TFS Items: Summary of Findings and Recommendations; (4) Second Cognitive Testing of TFS Items: Summary of Findings and Recommendations; (5) Teacher Status Form (Form TFS-1); (6) Results of the Unit Nonresponse Bias Analysis; (7) Quality Assurance for Keying and Mailout Operations; (8) Changes Made to Variables During the Consistency and Logic Edits, by Data File; (9) Imputation Changes to Variables, by Data File; (10) Weighting Adjustment Cells; (11) Evaluation of an Alternative Nonresponse Adjustment Method; (12) Frame and Created Variables; and (13) Crosswalk Among Items in the 2000-01, 2004-05, and 2008-09 TFS and With the 2007-08 SASS. (Contains 40 tables and 14 exhibits.)

"No student left out" should be the motto of the entire student movement. It can be understood in many different ways. No student should indeed be left out of higher education, both in terms of access and progress towards completion in higher education but also in terms of representation in governance structures at all levels. Leaving no student out is also a challenge for student unions in the sense of openness, democracy and representativeness, three pillars which the student movement ESU promotes. A strong student union is a union that can be inclusive for all students in a democratic way and that can represent and defend the rights of all students. With this handbook the authors hope to provide readers with an overview of how a student union can successfully represent students in an inclusive way and how student participation in governance on all levels has grown throughout history. (Contains 9 footnotes.)

Key stakeholders who implement Supported Education may find themselves asking two questions: (1) Has Supported Education been implemented as planned?; and (2) Has Supported Education resulted in the expected outcomes? Asking these two questions and using the answers to help improve Supported Education are critical for ensuring the success of one's program. "Evaluating Your Program" shows quality assurance team members how to evaluate the effectiveness of Supported Education. It includes the following: (1) A Readiness Assessment; (2) The Supported Education Fidelity Scale; and (3) Outcome measures that are specific to Supported Education. Readers will also find instructions for conducting assessments and tips on how to use the data to improve one's program. Appended are: (1) Cover Sheet--Supported Education Fidelity Scale; (2) Supported Education Fidelity Scale and Scoresheet; (3) Supported Education Fidelity Scale Protocol; (4) Outcomes Report Form; (5) Instructions for the Outcomes Report Form; and (6) Assessor Training and Work Performance Checklist. (Contains 2 figures and 2 tables.)

The Teacher Education Accreditation Council (TEAC) "Guide to Accreditation" includes a full description of TEAC's principles and standards, the accreditation process and audit, and detailed instruction on writing the "Brief." This revision includes expanded information on (1) preparing an "Inquiry Brief Proposal" and the audit of the "Inquiry Brief Proposal"; (2) Annual Report expectations (including information on responding to stipulations); (3) responsibilities of the program, TEAC, and the auditors in preparation for and during the audit; and (4) updates regarding new USDE (United States Department of Education) regulations (for programs offered through distance education and for submitting annual headcount data). Appended are: (1) List of Tables, Figures, and Formats; and (2) TEAC Glossary. An index and TEAC's accreditation framework are also included.

This handbook is primarily for the Teacher Education Accreditation Council (TEAC) auditor. It is intended to help in preparing for audits of "Inquiry Briefs" and "Inquiry Brief Proposals" and to contribute to the writing of the audit report. This handbook contains a full description of the audit process, the responsibilities of the program and TEAC staff, and auditors' responsibilities. It also provides guidelines for making audit decisions and developing the audit opinion. In addition, it includes an array of audit tools: documents and templates that will be used in each audit. This handbook begins with ten principles that describe the qualities of a TEAC auditor and can serve to guide auditors in their role and responsibilities. Following that introduction, the guide has four parts. Part One describes the TEAC audit in detail. It outlines the particular responsibilities of the program, TEAC staff, and the auditors and explains each phase of the audit. Part Two provides information and guidance on making decisions. Part Three contains the audit tools auditors will use before, during, and after the audit including the TEAC Accreditation Framework for both teacher education and educational leadership programs, templates for the Summary of the Case and the parity and capacity tables, and information about writing findings to audit tasks. Part Four covers practical matters such as schedule and costs and addresses the most frequent questions auditors have about their work.

This Toolkit forms part of the project entitled "Time for a New Paradigm in Education: Student-Centred Learning" (T4SCL), jointly led by the European Students' Union (ESU) and Education International (EI). This is an EU-funded project under the Lifelong Learning Programme (LLP) administered by the Education, Audiovisual and Culture Executive Agency (EACEA). T4SCL, as a pioneer initiative, aims to assist policy makers in designing sound student-centred learning (SCL) strategies and approaches and to increase the capacity of student and staff representative organisations to be active partners in spreading a culture of SCL in higher education institutions across Europe. This toolkit represents a key outcome of the project. It follows extensive research and a series of events at both European and national levels, combined with trainings on the subject-matter. This toolkit is designed for "effective, practical implementation of SCL by teaching staff, students and institutional leaders." It aims to show how "practical and achievable SCL" is, as well as being highly beneficial for learners. (Contains 5 figures and a list of 24 sources for further reading.)

Available resources and indicators of high quality inclusive practices are presented in this compilation. Assembling many different resources in one place allows for easy comparison of potential indicators of quality. Excerpts and adaptations of the resources are intended to provide some familiarity with the content of each resource and encourage further examination via links to more complete information. National and state-developed resources contained within this document have been designed for a variety of audiences, and may be useful for: (1) families, practitioners and program administrators; (2) technical assistance (TA) systems personnel and pre-service instructors; (3) researchers; and (4) state administrators. There are five main sections within this compilation: (1) Quality Inclusion Practice Considerations; (2) Classroom Observation Rating Tools; (3) Individual Child Focused Considerations; (4) Collaborative Inclusion Practices; and (5) Quality Rating and Improvement Systems (QRIS).

The Rural and Low-Income School (RLIS) program is part of the Rural Education Achievement Program (REAP) that was authorized under Title VI, Part B of the "Elementary and Secondary Act of 1965" ("ESEA"), as amended by the "No Child Left Behind Act of 2001" ("NCLB"). The RLIS program provides additional funds to help rural districts serving low-income students make adequate yearly progress (AYP) as described in Section 1111(b)(2) of the "ESEA". RLIS funds may be used to support a variety of activities, including teacher recruitment and retention; teacher professional development; support for educational technology; parental involvement activities; activities authorized under the Safe and Drug-Free Schools Program; and activities authorized under Title I, Part A, and Title III of the "ESEA". RLIS funds are distributed to state education agencies, which then distribute money to the school districts that meet the following criteria: (a) the district is not eligible for a grant from the Small, Rural School Achievement (SRSA) program, which serves rural school districts that have fewer than 600 students or that serve extremely sparsely populated areas; (b) 20 percent or more of the children ages 5 through 17 served by the district are from families with incomes below the poverty line; and (c) all of the schools included in the district must have a National Center for Education Statistics (NCES) locale code of 6 (small town), 7 (rural), or 8 (rural near an urban area). The RLIS program distributed almost $86 million to 41 states in the 2009-10 school year. In turn, the states distributed RLIS funds to 1,497 districts. Award amounts in 2009-10 averaged approximately $57,000 per district and $29 per pupil in RLIS-funded districts. This report includes findings from interview and survey data obtained from state and district RLIS coordinators who were selected for the interview or survey samples based on their state or district's receipt of RLIS funds for the 2007-08 school year. It describes state and district implementation of the RLIS program, priorities for RLIS funds, and RLIS district characteristics. It also includes results from an analysis of extant data at the state and district levels on student achievement outcomes from state assessments used for "NCLB" accountability. Appendices include: (1) Supporting Data for Chapter 3; (2) Student Achievement Technical Appendix; (3) State Coordinator Interview Guide; (4) District Coordinator Interview Guide; (5) RLIS State Coordinator Survey; and (6) RLIS District Coordinator Survey. (Contains 76 exhibits and 13 footnotes.) [For "Evaluation of the Implementation of the Rural and Low-Income School (RLIS) Program: Interim Report," see ED517416.]

The authority to operate a charter school is granted through a limited-term, renewable contract. Contract renewal is not automatic--it must be earned through strong academic results and operational effectiveness. Good schools should be cultivated and poor ones shut down. This is one of the definitive functions of a charter school authorizer. This policy guide discusses the following: (1) Background on Charter School Renewal; (2) Key Considerations for Policymakers Committed to Supporting Quality Charter School Renewal; (3) Recommendations and Best Practices for State Policy on Charter School Renewal; and (4) Resources and Further Analysis. (Contains 5 endnotes and 11 resources.)

The 2003 National Assessment of Adult Literacy (NAAL) assessed the English literacy skills of a nationally representative sample of more than 19,000 U.S. adults (age 16 and older) residing in households and correctional institutions. NAAL is the first national assessment of adult literacy since the 1992 National Adult Literacy Survey (NALS). The NAAL and NALS produced direct estimates of Prose, Document, and Quantitative literacy, each reported on a 0 to 500 scale and on four performance levels: Below Basic, Basic, Intermediate, and Proficient based on this scale. This report provides information on the 2003 NAAL development and implementation, including development of survey instruments; field test and main assessment sample design; assessment items and scoring rubrics; data collection process and quality of the data; weighting, scaling, treatment of missing data, and data analysis. In addition, the NAAL technical report includes detail information on the development of NAAL innovative additions--Fluency Addition to NAAL (FAN) and Adult Literacy Supplement Assessment (ALSA). Other sections of the report contain information about health literacy cognitive items, data collection and analysis in the US correctional institutions and a user's manual on how to use statistical programs for analyzing NAAL data. The NAAL Technical Report and Data File User's Manual is a key resource for secondary data analysts and for those who want to understand how NAAL was developed and implemented. Chapters include: (1) The 2003 National Assessment of Adult Literacy: An Overview (Mark Kutner and Elizabeth Moore); (2) Development of the Survey Instruments (Elizabeth Greenberg); (3) Field Test (Michelle Amsbary, Martha Berlin, Thomas Krenzke, Leyla Mohadjer, Lyn Clark, and Jacqueline Hogan); (4) Field-Test Scoring and Analysis of Field-Test Data (Elizabeth Greenberg); (5) Field Test of the Fluency Addition to NAAL (Michelle Amsbary); (6) Field Test of Adult Literacy Supplemental Assessment (Michelle Amsbary and Barbara Forsyth); (7) Sample Design (Leyla Mohadjer and Thomas Krenzke); (8) Main Household Study Data Collection and Quality Control (Michelle Amsbary, Martha Berlin, Thomas Krenzke, and Leyla Mohadjer); (9) Correctional Institution Study Data Collection and Quality Control (Michelle Amsbary, Terri Annis, and Martha Berlin); (10) Reducing the Risk of Data Disclosure (Thomas Krenzke, Sylvia Dohrmann, and Laura Alvarez-Rojas); (11) Response Rates and Nonresponse Bias Analysis (Thomas Krenzke and Leyla Mohadjer); (12) Weighting and Variance Estimation (Thomas Krenzke and Leyla Mohadjer); (13) Scoring (Justin Baer, Jared Bernstein, and Michelle Amsbary); (14) Item Analysis, Scaling, and Estimates of Subpopulation Proficiencies (Stephane Baldi and Justin Baer); (15) The Literacy of Adults without Cognitive Data (Ying Jin and Stephane Baldi); (16) Variable Construction and File Development (Ying Jin); and (17) The NAAL Health Literacy Component (Ying Jin and Eric Dunleavy). Appended are: (1) Derived Variables; (2) Instructions On Using AM Software to Analyze the 2003 NAAL Data; (3) Electronic Codebook For Windows User's Manual; (4) NAAL Item Descriptions and Parameters; (5) P-Values Of NAAL 2003 Items; (6) NAAL Household Background Questionnaire; and (7) NAAL Prison Background Questionnaire. (Contains 135 tables, 19 figures, and 1 exhibit.)

The fundamental concepts of Achieving the Dream--using evidence to develop and evaluate strategies for improving student learning and success--are also important to successful efforts to meet accreditation requirements. Following the Achieving the Dream approach can help community colleges organize and document improvement efforts in ways that are a natural fit with the expectations of the regional accrediting agencies. During the past decade, the six regional accrediting agencies have changed their requirements and now encourage colleges to adopt processes and practices that demonstrate effectiveness in teaching and learning and improvements in institutional quality. Those processes and practices involve planning, evaluation, and outcomes assessment with the goals of enhancing student learning, strengthening programs, and improving services. This guide is designed to show how colleges can use the methodology of Achieving the Dream to provide evidence of compliance with key accreditation standards and to drive their institution's quality improvement agenda. A list of resources is included.

The "Fiscal Year 2008 Illinois Community College System's Performance Report" contains summary information about state and common institutional outcome indicators. External comparative benchmarks are cited whenever possible using the latest available data--national reporting typically trails what is available at the state level. Additionally, the state report contains reviews of selected mission-specific indicators. For the single narrative focus area--Affordability--the state report includes Highlights of Accomplishments and Plans and Selected Challenges. Appended are: (1) Mission-Specific Goals Selected by College; and (2)Common Measure Goal Statements.

In 2006, the Canadian Council on Learning produced the first national overview of post-secondary education in Canada. The report, "Canadian Post-secondary Education: A Positive Record--An Uncertain Future," identified eight goals and objectives derived from the strategic plans for post-secondary education (PSE) that had been developed by provinces and territories--and presented a series of indicators for each of the eight goals. It cautioned that serious challenges exist and must be resolved with urgency to keep Canada at the forefront educationally and economically. The report underlined that, in order for Canada to advance as a country, it is essential to: (1) articulate a set of explicit, well-defined goals and objectives for the PSE sector at the national level; (2) develop a clear set of indicators and measures to enable continuous assessment of performance and progress toward the defined goals and objectives; and (3) establish mechanisms at the national level that accomplish the tasks in the preceding two points. "Strategies for Success," the second annual report from the Canadian Council on Learning on the post-secondary education sector in Canada, builds on the priorities identified last year. Like the 2006 report, Part I of "Strategies for Success" examines the sector from a countrywide perspective, drawing on domestic and international statistics and indicators for the eight identified goals and objectives for PSE. While some modest gains have been made, Canadians can take little comfort from this year's findings. This country has fundamental data gaps. For example, Canada: (1) does not have the information required to assess PSE capacity versus labour-market needs; (2) has no useful picture of the country's private providers of PSE (who they are, what they do, their capacity, their enrolment figures, what happens to their graduates); (3) has very little information since 1999 about its community colleges regarding faculty, enrolment or capacity; and (4) can provide only a limited picture of part-time faculty at its universities. To remedy the incomplete picture of the PSE landscape, Part II of "Strategies for Success" proposes a comprehensive, pan-Canadian data strategy to provide the information needed to strengthen the country's PSE sector. Part III of "Strategies for Success" outlines how some of Canada's major trading partners are monitoring and reporting on the state of PSE in their countries. Any complacency about Canada's positive record in PSE is misplaced, given that many countries have developed: (1) national strategies for collecting robust data; (2) national goals for PSE; and (3) benchmarks and targets with which to measure progress toward those goals. That Canada's global competitors have developed such systematic approaches to optimize the benefits of PSE only heightens the urgency for practical steps to be taken within Canadian borders. It is in this context that Part IV of "Strategies for Success" proposes, as the section title suggests, working "Toward a Pan-Canadian Framework for PSE." It examines how a more cohesive and systematic approach could assist in addressing specific challenges, including: quality assurance and accreditation; student mobility and credit transfer; and prior learning assessment and recognition (PLAR). A bibliography is included. Individual parts contain endnotes. (Contains 62 tables and 75 tables.) [For related report, "Canadian Post-Secondary Education: A Positive Record--An Uncertain Future. Report on Learning in Canada 2006," see ED525257.]

This guide offers advice and guidance to key skills managers, coordinators and other practitioners who are responsible for ensuring that the standards of portfolio assessment in their center are consistent across the center and with national standards. It describes and offers advice and guidance on the stages of assessment and internal moderation of key skills portfolios and on liaising with an awarding body about external moderation. The guide should be read in association with "The Key Skills Qualifications Standards and Guidance: Communication, Application of Number and Information and Communication Technology: Levels 1-4 2004," published by QCA Publications. Following an introduction, "About Assessment", Portfolio Assessment, Internal Moderation, and External Moderation, the following chapters are contained in this guide: (1) Internal Moderation Sampling Plan; (2) Checklist for Quality Assuring Assignments; and (3) Linking Moderation to the Critical Success Factors. The following are appended: (1) Internal Moderation Sampling Plan; (2) Checklist for Quality Assuring Assignments; (3) Linking Moderation to the Critical Success Factors; (4) Glossary; and (5) Useful addresses. (Contains 5 tables and 4 figures.) [This publication was produced by the Learning and Skills Network, London, UK.]

Most policymakers, charter school operators, and others immersed in the charter school movement since it began in the early 1990s have focused their attention primarily on charter schools, not on the public bodies that license these schools to operate. As the charter school movement has grown, there has been increasing recognition that effective charter school authorizing is critical to the success of the charter school sector. Charter school authorizers are entities charged by law to approve new schools, monitor their compliance with applicable laws and regulations, and evaluate their performance to make decisions about charter renewal and closure. The role of charter authorizers has become particularly important in the context of increasing accountability under the No Child Left Behind Act of 2001 (NCLB). Intended primarily for policymakers and charter school authorizers and potential authorizers, this guide describes various ways that authorizers and policymakers can achieve quality authorizing. It provides detailed information designed to help policymakers at the state and national levels and to help current and potential authorizers replicate these successful models and practices. In order to provide specific illustrations of abstract concepts involved in successful charter authorizing, this guide highlights the practices of eight charter authorizers that have fostered the development of high-quality charter schools. Part 1 of this guide describes the common practices of the eight charter authorizers profiled here. This guide describes how effective authorizers: build a strong organization; develop a strong talent pool; select for quality; support new school operators; provide meaningful and transparent oversight; and hold schools accountable for meeting performance goals. Part II describes the kinds of policy factors that can either support or hinder quality charter authorizing practices. Part III offers profiles of each of the eight authorizers that are highlighted throughout Parts I and II. The profiles cover the history of each of these authorizers and provide more detail on the contexts in which they operate. Appended is the research methodology and a list of resources of organizations, Web sites, essays, and research studies that address elements of successful authorizing and oversight of charter schools. (Contains 16 figures and 2 tables.) [This document was also prepared by Public Impact.]

Family day care providers are responsible for creating a high-quality program where children have opportunities to grow, learn and thrive. Part of providing high-quality child care includes complying with the family day care regulations from the New York State Office of Children and Family Services (OCFS). This Handbook will help day care providers: (1) understand how the regulations promote the health, safety and development of children in their care; (2) use the regulations as the foundation of their programs; and (3) gain resources that they can use to support the children and families with whom they work. This Handbook is designed to be used along with the New York State OCFS family day care regulations. Includes an appendix on health and hygiene. [This Handbook was produced by the New York State Office of Children and Family Services.]

Spurred by the "No Child Left Behind Act" of 2001, virtually every educational reform program now includes an accountability component that requires sound data collection and reporting. Improving data quality has thus emerged as a high priority for educators and policymakers across the country. The list of programs for which data quality is relevant is extensive, and the scope of the issues involved is vast. As its title--"Improving Data Quality for Title I Standards, Assessments, and Accountability Reporting"--suggests, this set of guidelines addresses only one discrete slice of the universe of data quality issues: those associated with the annual Report Card required of all States, local educational agencies (LEAs) and schools receiving Title I, Part A funds under Sec. 1111(h) of "No Child Left Behind" ("NCLB"). Recommendations from the U.S. Department of Education's Office of the Inspector General (OIG) and the Government Accountability Office (GAO) provided the impetus for these guidelines. In September 2004, GAO identified numerous data quality problems in the States related to NCLB accountability and reiterated OIG's call for guidelines. This document, focused on State, LEA, and school Title I Report Cards, tailors its guidance specifically to address the OIG/GAO recommendations. Summary Checklists are appended. [This paper was produced by DTI Associates.]

As a result of increasing mobility of students and knowledge, cross border education, especially in higher education, is receiving general attention in the Asia and Pacific region. The toolkit serves as a reference tool to assist local policymakers in the formulation of a regulatory framework for cross-border education that is growing rapidly in the Asia and Pacific region. The toolkit is developed to support the UNESCO/OECD project on Guidelines on Quality Provision in Cross-Border Higher Education in 2004. It looks at different requirements and regulatory needs from the perspectives of both receiver and provider. For either of them, the toolkit provides details on the functions of such a regulatory framework, types that are being implemented in selected Asia and Pacific countries, and steps for implementation and enforcement. The Guidelines provide recommendations to six stakeholders groups: governments, academic staff, student bodies, quality assurance and accreditation bodies, academic recognition bodies, and professional bodies; and it refers to case studies from China, Hong Kong, Malaysia, New Zealand, UK and USA as either receiver or provider country to examine the regulatory framework and institution. A draft of the toolkit was presented to the Asia Pacific Quality Network (APQN) annual general meeting in 2006 and the current version incorporates comments and suggestions made at that meeting. The toolkit highlights important issues and considerations, different models of regulatory frameworks, the practical steps in setting up a framework and possible pitfalls drawn from the experience of some existing systems. Also included are: a glossary; and a list of 31 related resources and web sites. [This document was published by the UNESCO Asia and Pacific Regional Bureau for Education and is part of The Asia-Pacific Programme of Educational Innovation for Development (APEID Unit). It represents a collaborative project of the United Nations Educational, Scientific and Cultural Organization (UNESCO) and the Asia-Pacific Quality Network (APQN).]

DC has a larger percentage of students in charter schools than any state. To help oversee DC charter schools, Congress established two authorizers--the Board of Education (BOE), which has an Office of Charter Schools responsible for oversight, and the independent Public Charter School Board (PCSB). Congress required the GAO to conduct a study of the authorizers. This report, which completes GAO's May 2005 study--examines: (1) authorizers' resources; (2) oversight practices; and (3) actions taken once charter schools close. GAO examined BOE and PCSB monitoring reports, revenue and expenditure documents, and closure procedures. GAO recommends that the BOE Office of Charter Schools implement a risk-based oversight system to target monitoring to new and at risk charter schools. Appended are: (1) Objectives, Scope, and Methodology; (2) Comments from the DC Board of Education; (3) Comments from the PCSB; and (4) GAO Contact and Staff Acknowledgments. Contains 4 tables, 4 figures, and a list of abbreviations.

GAO prepared this report under the Comptroller General's authority as part of an effort to assist policy makers in determining how federal disability programs could more effectively meet the needs of individuals with disabilities and addressed it to each committee of jurisdiction. In this report, GAO assesses: (1) the extent to which state Vocational Rehabilitation (VR) agencies assist individuals in achieving employment; and (2) performance measures and monitoring practices Education uses to manage this decentralized program and achieve legislative goals. Of the more than 650,000 individuals exiting the state VR programs in FY '03, one-third (217,557) obtained a new job or maintained their existing job for at least 90 days after receiving customized services. Education's data showed that the remaining two-thirds exited the VR program without employment most often because the individual refused services or failed to cooperate with the VR counselor. Among recommendations to improve Education's performance measures and its monitoring of state VR agencies, are: (1) Reevaluate Education's performance measures values and ensure that sufficient data are collected to measure the performance of this program; and (2) Take steps to continue improving the timeliness of the performance data Education collects and analyzes. Appended materials include: Scope and Methodology; State VR Agency Exit and Employment Rates, FY '03; State VR Agency Population Proportions by Primary Impairment, FY '03; Social Security Beneficiaries' Employment Rates by State VR Agency, FY '03; State VR Agency Percentages of Total Agency Service Budget Spent on Each Service Category, FY '03; State VR Agency Total Administrative Costs as a Percentage of Total Expenditures, FY '03; State VR Agency Average Total Expenditures in FY '03 Per Person Exiting with Employment in FY '03; Comments from the Department of Education; and GAO Contacts and Staff Acknowledgments.

This resource toolkit for working education service providers contains four sections. Section 1, "Roles Responsibilities, and Relationships," contains: (1) "Purchasing Services from an Educational Management Organization," excerpted from "The Charter School Administrative and Governance Guide" (Massachusetts Dept. of Education); (2) ESP Agreement Process (Central Michigan University); (3) Educational Service Provider Policies (Central Michigan University); (4) Legal Opinion Template (Central Michigan University); (5) Disclosure of Financial Interest Form (Massachusetts Dept. of Education); and (6) Annual Conflict of Interest Disclosure Form (Central Michigan University). Section 2, "The Charter Application Stage," contains: (1) Selected Questions, "Charter Schools Application Kit," 5th ed. (State University of New York); (2) Existing Design Applicant Evaluation (Mayor's Office of Indianapolis); (3) Excerpt, "Annotated Guide to the California State Board of Education Model Charter School Application"; (4) Public School Academy Board Application (Central Michigan University); and (5) Questions for Prospective Charter School Board Members, "Charter Schools Application Kit," 5th ed. (State University of New York). Section 3, "Reviewing School Management Contracts," contains: (1) Educational Service Provide Review Sheet (Central Michigan University); (2) Excerpt, "The Charter School Administration and Governance Guide" (Massachusetts Dept. of Education); and (3) Management Contract Review Checklist (Massachusetts Dept. of Education). Section 4, "Additional Resources and Sample Documents," contains: (1) Comprehensive Management Services Contract Requirements (Chicago Board of Education); (2) Charter Contract Provision Concerning Public Information (Ferris State University); (3) ESP Quality and Performance Assessment (School Lane Charter School); and (4) Sample School Management Contract (Chicago). Appended are: (1) Maintenance, Repairs and Improvements; (2) Certain Insurance Requirements; and (3) Dispute Resolution. Individual sections contain footnotes.

This document was prepared to serve as an aid in the planning, design, implementation, and ongoing evaluation of Alternative Education for Disruptive Youth Programs in Pennsylvania. School Codes and Public Laws are cited for guidance on mandatory issues. The suggestions and implementation strategies provided in this document are not requirements for Pennsylvania's Alternative Education for Disruptive Youth Programs, but may be used as a tool for new program development and to assist program staff in creating new programs or modifying existing programs that are flexible and effective in helping students overcome their difficulties and address their conduct, so that they can learn, graduate, and become productive members of society. Contents include: (1) Background of Alternative Education in Pennsylvania; (2) How to Use This Document; (3) Self-Assessment Guide; (4) Additional Assistance; (5) Program Design Overview; (6) Needs Assessment/Identification of Target Population; (7) Legal Considerations; (8) Approved Private Providers; (9) Budget Design; (10) Site Selection; (11) Transportation; (12) Food Services; (13) Health Services; (14) School Climate and Environment; (15) Parental Involvement; (16) Community Partnerships; (17) Student Eligibility and Entrance Criteria; (18) Student Resources and Services; (19) Curriculum Development; (20) Staffing; (21) Instructional Practices; (22) Student Assessment: (23) Exit Criteria and Transition; (24) Annual End-of-Year Report; (25) Self-Assessment and Monitoring; and (26) Record keeping Requirements.

Visually impaired people have the same information needs as sighted people. Just as sighted people might read a newspaper, listen to a CD or download electronic information from the Internet, visually impaired people also want access to relevant information in their chosen accessible format. Developing an efficient library service for print-disabled people is extremely important, because there are significantly fewer books available commercially in accessible formats compared to what is published in print for the general public. The need to build collections in alternative formats and make them available for readers who are unable to browse shelves makes it necessary to develop special services. The purpose of these guidelines is to provide libraries, governments, and other stakeholders with a framework for developing library services for people who are print disabled. Where appropriate the guidelines are supported by examples from many countries around the world. For purposes of these guidelines, print disabled will be used to describe those who are unable to read print due to blindness, low vision, a learning disability, or a physical disability. Following the preface, the 10 chapters contained herein are: (1) Introduction; (2) History; (3) The Framework for Service; (4) Customer Care: Meeting User Needs; (5) Cooperation and Networking; (6) Collection Development; (7) Access to Service and Collections; (8) Production of Alternative Formats; (9) Management and Marketing; and (10) Quality Assurance. Appended are: (1) IFLA Declaration of Fundamental Right to Access and Express Information; and (2) Guidelines for Library Service to Braille Users. This volume also includes a glossary.

This document describes the accomplishments of the Office of Inspector General (OIG), U.S. Department of Education from October 1, 2003 through March 31, 2004. During this period, 71 audit and inspection reports were issued and 94 investigations were closed. Congress was provided with an analysis of issues for consideration in its reauthorization of the Higher Education Act and conducted several briefings on proposals for congressional staff. The March 2003 audit of Educational Credit Management Corporation identified the need for the Department to recall $103 million of excess funds. At the time of the final audit, the Department concurred with recommendation. In February 2004, the Department recalled the funds, resulting in the return of $103 million to the U.S. Treasury. The campaign continues to raise public awareness about the threat of identity theft among students and other participants in the Department's student financial assistance programs. OIG special agents gave identity theft presentations at major Federal Student Aid national conferences and at regional conferences for financial aid administrators Articles were submitted about identity theft efforts that were published in a number of publications, including the "Federal Bureau of Investigation Bulletin" and "Fraud Busters," publications of the FBI and the Federal Trade Commission, respectively Identity theft prevention tips were provided to many campus security websites and in many publications and websites for student financial assistance participants. Special agents also continued to pursue cases in which identity thieves attempted to rob students of their financial aid. In response to the importance of maintaining the security and integrity of the Department's information systems, a new unit that combines audit, investigative, and computer evidence recovery resources was established. This unit will work to identify risks to the Department's systems, to analyze computer data to determine fraud indicators, and to investigate and prevent computer hacking. Independent oversight and recommendations are essential to the Department's mission and goals and helping ensure that education dollars reach the intended recipients.

This investigation of the comparability of writing assessment prompts was conducted in two phases. In an exploratory Phase I, 47 writing prompts administered in the computer-based Test of English as a Foreign Language[TM] (TOEFL[R] CBT) from July through December 1998 were examined. Logistic regression procedures were used to estimate prompt difficulty and gender effects. A panel of experts reviewed selected prompts, and a taxonomy of prompt characteristics was developed and related to prompt difficulty and gender differences. In Phase II, 87 prompts administered from July 1998 through March 2000 were analyzed. All of the prompts used in Phase I, together with 40 new prompts, were analyzed using the larger Phase II database. Recommendations are made for statistical quality control procedures to identify less comparable prompts. Appended are: (1) Logistic Regression Model for Polytomous Items: The Proportional Odds-Ratio Model; (2) Number of Essays, Means, and Standard Deviations of English Language Ability and Essay Scores for Male and Female Examinee Groups; (3) Mean Expected Essay Scores, Residuals, and Standardized Mean Group Differences; (4) Uniform and Nonuniform Effect Sizes; (5) Prompt Difficulty Indices; (6) Taxonomy of TOEFL CBT Writing Prompts (Phase I); and (7) Scoring Rubrics for TOEFL CBT Writing Prompts. (Contains 13 tables, 6 figures, and 1 note.)

The identification and recruitment (ID&R) process is an important element in the delivery of migrant services. Federal regulations currently require each state to find eligible migrant children residing within its borders. This chapter discusses essential factors that affect the ID&R of migrant children by educators at the state, regional, and local levels. Development of a realistic plan for finding and determining the eligibility of a state's or region's migrant population is critical. One key to a good recruitment plan is understanding the area's agricultural processes and employment patterns. The state's departments of agriculture and labor and cooperative extension programs can provide such information. Three major aspects of developing a recruitment plan are: (1) planning/logistics (deciding where to recruit, following protocol and regulations, targeting priority students, networking, and having knowledgeable recruiters); (2) initial staff training and plan implementation (the Certificate of Eligibility, liberal versus conservative interpretation of eligibility guidelines, assessment of child and family needs, characteristics of good recruiters, cultural and language implications, and community-based recruiting); and (3) monitoring and evaluating success (recruiter training and supervision, record keeping, friendly audits, and annual checks of plan elements). (SV)