Search

There is a growing interest in ensuring that students are prepared for college and a career when they finish high school. Research identified several indicators that are associated with college preparedness including attendance, grade point average (GPA), and test scores. However, it is difficult to use these indicators to determine the preparedness level of the nation's students in general. This study used results from National Assessment of Educational Progress (NAEP) mathematics assessment, the "gold standard" assessing what students know and can do in grade 12 to establish college preparedness benchmarks. This study linked ninth-grade student background data and school-reported high school and postsecondary transcript data from the National High School Longitudinal Study of 2009 (HSLS:09) to student item responses on the 2013 National Assessment of Educational Progress (NAEP) mathematics assessment to examine the relationship between college preparedness and NAEP mathematics achievement. NAEP scale scores imputed for all HSLS:09 participants via marginal maximum likelihood regression analyses were used to predict college preparedness. This study extended the earlier investigations of college preparedness based on NAEP grade 12 mathematics data from the prediction of college enrollment alone to the prediction of additional postsecondary outcomes including remedial course-taking and first-year college GPA based on postsecondary transcript data. Defining college preparedness as enrolling in a 2- or 4-year college without remedial course-taking and acquiring a first-year college GPA of 2.7 or higher as did National Assessment Governing Board, this study showed that the probability of college preparedness increased from 26 percent for students performing at the NAEP Basic level to 58 percent at the NAEP Proficient level and 87 percent at the NAEP Advanced level. The probabilities at preparedness increased when the definition of college preparedness omitted remedial course-taking. Although similar patterns were observed for various student groups including race/ethnicity, gender, and socioeconomic status, there were some differences in results across groups. Since the report shows the probability of college preparedness (under various definitions) at each NAEP achievement level, it is thereby also providing important validity information for these NAEP achievement levels.

This paper summarises exploratory analysis undertaken to evaluate the effectiveness of using machine learning approaches to calculate projected completion rates for vocational education and training (VET) programs, and compares this with the current approach used at the National Centre for Vocational Education Research (NCVER) -- Markov chains methodology. While the Markov chains methodology currently used by NCVER has demonstrated that it is reliable, with predictions aligning well with the actual rates of completion for historical estimates, it has not been reviewed for some time and it does have some limitations. The evaluation of machine learning techniques for predicting VET program completion rates was undertaken to overcome some of these limitations and with a view to improving our current predictions. This report includes: (1) an overview of the methodologies: Markov chains and two machine learning algorithms that were applied to predict completion rates for VET programs (XGBoost and CatBoost); (2) a comparison of the accuracy of the predictions generated by both methodologies; and (3) an evaluation of the relative strengths and limitations of both methodologies.

This publication examines current challenges with the articulation of learning and argues that a new means of articulating learning is necessary. Specifically, drawbacks and inefficiencies developed to accommodate the credit hour measure of student learning are examined. As an alternative, a new competency-based measure of learning is proposed. Benefits of such a system would include: (1) Efficiencies and clarity when transferring between postsecondary institutions; (2) Improved effectiveness of costly prior-learning assessment processes; (3) Simplification of capturing and sharing learning that occurred while serving in the military; (4) Enhancing the effectiveness of competency-based hiring processes, particularly in the contemporary workforce that is global and increasingly values contingent employment; (5) Improved articulation of work-based learning, including apprenticeships and internships; (6) Ability to adapt to existing technology and information systems; (7) Applicability toward digital and analog competency development and management systems; and (8) Adaptability to new, innovative, and global learning models. Further, a new taxonomy of learning, based on an index of competencies, would naturally aid competency-based hiring practices. These opinions are informed by existing literature as well as feedback from thought leaders representing various sectors

This report reviews the accomplishments of higher education accreditation relative to its symbiotic relationship with assessment, acknowledges serious criticisms and proposed reforms, and indicates how accreditation might reform itself so as to disarm calls for radical change, improve its performance, strengthen the institutions and programs it serves, and enhance public understanding of and appreciation for higher education.

Technology has revolutionized corporate learning and leadership development. The number of organizations that use learning management systems is higher than ever before, and thanks to massive open online courses (MOOCs), small private online courses (SPOCS), microlearning, nanolearning, and other new media learning platforms, digital learning and training is at an all-time high. Many learning and leadership development professionals, however, still consider expensive, face-to-face, instructor-led training preferable to digital learning. This paper will explore six strategies from the Center for Creative Leadership (CCL®) that talent leaders can leverage to make digital learning initiatives more effective within their organization. When properly designed, delivered, and evaluated, digital learning initiatives can change a company's corporate culture, improve employee engagement, and increase retention. On the flip side, if done poorly, digital learning initiatives can end up as meaningless corporate activities. The six strategies explored in this paper are addressed under the following headings: (1) Less is More; (2) Support from the Top; (3) Learner-Centered Design; (4) Leaders as Teachers; (5) Learning Partnerships; and (6) What Gets Measured Gets Done.

The Institute of Education Sciences (IES), the research arm of the U.S. Department of Education, is the nation's leading source for rigorous, independent education research, evaluation, and statistics. IES's National Center for Education Research (NCER) supports rigorous research that addresses the nation's most pressing education needs from early childhood to adult education. IES's National Center for Special Education Research (NCSER) supports a comprehensive program of education research designed to expand knowledge and understanding of infants, toddlers, and youth with and at risk for disabilities to improve their developmental, education, transition, and postsecondary outcomes. This paper makes the case that IES Development and Innovation research can support the development of usable, feasible, and affordable approaches (practices, programs, or policies) to help schools meet the mental health needs of their students and staff. The goal of this research is to ensure that school-based interventions are contextually appropriate, implemented with high fidelity, and more likely to produce equitable outcomes than current practice. [This paper was published in the "School Mental Health".]

This discussion paper represents one of the National Association for College Admission Counseling's (NACAC's) first post-Testing Commission steps in advancing the knowledge base and dialogue about test preparation. It describes various types of test preparation programs and summarizes the existing academic research on the effects of test preparation on standardized test scores. The paper also presents newly published data collected by the author in cooperation with NACAC and its members about how colleges are currently using test scores in the process of making admission decisions. The purposes of this report are to (1) describe and summarize formal and informal methods of admission test preparation; (2) synthesize and summarize existing academic research on the effects of admission test preparation; (3) arrive at conclusions about the effectiveness of test preparation for admission testing; and (4) suggest future research needs in this area. The report concludes with recommendations to admission officers and high school counselors for implementing policies and training that can account for the effects of test preparation in the college admission process. Appended are summaries of SAT Coaching Studies. (Contains 6 tables, 6 figures, 12 footnotes, and bibliographies on coaching studies and coaching reviews.)

There is wide demand for engineering graduates to be capable of working well in teams. The engineering accreditation body (ABET, www.abet.org) has responded to this need by requiring engineering programs to demonstrate that their graduates have "an ability to function on multidisciplinary teams" (Haag, Froyd, Coleman, & Caso, n.d.), and many engineering instructors have integrated the use of student teams into their courses. The characteristics of effective student teams have been widely studied, and there is ample research on what makes student teams succeed. Johnson, Johnson, and Smith (2007), for example, define five traits of effective student teams, and they note that each one is critical for success. The five traits are: (1) positive interdependence; (2) individual accountability; (3) face-to-face interaction; (4) interpersonal and small-group skills; and (5) assess its performance. The purpose of this Occasional Paper is to provide instructors with a framework for ensuring that student teams possess these five traits and are set up for success. A sample peer evaluation form is appended.

Standards-based education reform has a more than 20-year history. A standards-based vision was enacted in federal law under the Clinton administration with the 1994 reauthorization of the Elementary and Secondary Education Act (ESEA) and carried forward under the Bush administration with the No Child Left Behind Act (NCLB) of 2001. In a recent survey of policy makers, standards were acknowledged as the central framework guiding state education policy. Yet, despite this apparent unanimity about the intuitively appealing idea of standards, there is great confusion about its "operational" meaning: exactly what should the standards be, how should they be set and by whom, and how should they be applied to ensure rigorous and high-quality education for American students are the central questions that challenge policy makers and educators. For example, "content" standards (subject-matter descriptions of what students should know and be able to do) are often confused with "performance" standards (which are more like passing scores on a test), and very different theories of action are used to explain how standards-based reforms are expected to work. Ambitious rhetoric has called for systemic reform and profound changes in curriculum and assessments to enable higher levels of learning. In reality, however, implementation of standards has frequently resulted in a much more familiar policy of test-based accountability, whereby test items often become crude proxies for the standards. This disconnect between rhetoric and reality is one of the reasons for the failure of prior reforms. Standards-based education is still the core idea guiding education policy and education reform. But the foregoing issues need to be addressed if the promises of standards-based education are to be kept. As yet, neither state content standards nor state tests reflect the ambitions of standards-based reform rhetoric, and the link between high expectations for all students and capacity building has been almost forgotten. The intentions of standards-based education--to focus greater attention on student learning, to ensure the participation and success of all students, and to provide guidance for educational improvement--are in the best interest of the country. This paper offers the following recommendations: (1) The federal government should encourage the redesign and clear connection of content and performance standards--and the curricula, teacher training, and high-quality assessments to go with them--with the goal of developing clearly articulated statements of the expected progression of learning. Efforts to develop these components may involve partnerships among states, universities, groups of teachers, scholars, and the private sector; (2) The federal government should support research on accountability system indicators to reflect both the status and growth of students. Performance standards should set ambitious but realistic targets for teaching and learning, and they should communicate to the public, parents, educators, and students themselves what is to be learned. Assessment results should be reported in ways that recognize progress all along the achievement continuum; (3) The federal government should support the redesign and ongoing evaluation of accountability systems to ensure that they contribute to school improvement. Less than satisfactory school performance should trigger closer investigation of school operations before remedies or sanctions are applied, and stellar performances should also be verified. Different investigative approaches, including audit assessments, data-driven analyses, or expert constituted inspectorates, should be considered; and (4) The federal government should support an intensive program of research and development to create the next generation of performance assessments explicitly linked to well-designed content standards and curricula. (Contains 77 notes.)

In March 2003, the National Center for Education Statistics asked the Human Resources Research Organization to participate in a special study of 2002 reading assessment results for Delaware. This working paper summarizes the findings of that study. (AMT)

"Can You Hear Me? The Right of Young Children to Participate in Decisions Affecting Them" emphasises that participation enhances children's self-esteem and confidence, promotes their overall capacities, produces better outcomes, strengthens understanding of and commitment to democratic processes and protects children more effectively. Participation provides the opportunity for developing a sense of autonomy, independence, heightened social competence and resilience. The benefits are therefore significant, and adults with both direct and indirect responsibility for children need to acquire a greater humility in recognising that they have a great deal to learn from children. But the case for listening to young children goes beyond the beneficial outcomes. It is also a matter of social injustice and human rights. All people, however young, are entitled to be participants in their own lives, to influence what happens to them, to be involved in creating their own environments, to exercise choices and to have their views respected and valued. (Contains 4 footnotes.)

This paper reviews the literature on academic quality programs and examines issues faced at one institution in which two initiatives emphasizing student outcomes and faculty performance as indicators of quality are underway. In its review of the literature the paper focuses on concepts such as total quality management, peer or student evaluation of faculty, organizational change, and human and technological factors; and it defines the characteristics of quality academic programs as including a culture of shared purpose, an active learning environment, high expectations for students and faculty, continuous feedback for both students and faculty, adequate resources, and good administrative support. At the St. John's University (New York) Centers of Excellence, resource allocation is integrated with assessment of student outcomes and faculty performance. The university uses four categories of measurement to select academic departments as centers of excellence: input (ability to attract high-quality students, excellent facilities, favorable faculty/student ratio); process (programs demonstrating long-term interest and need, and outstanding curriculum); value-added features (evidence of student satisfaction); and output (productive faculty, graduate placement, and evidence of student satisfaction after graduation). Key to the success of the program is careful attention to the balance between the technical (data analysis), organizational (shared values), and personal frames. (Contains 30 references.) (CH)

This paper addresses issues in assessment of college students, outcome studies, and attrition research in the context of trends toward requiring greater accountability from institutions of higher education. First the paper considers how assessment, student outcomes studies, and attrition research terminology might be more clearly defined. It suggests that these three areas of inquiry, although distinctive, are highly interrelated phenomena. The paper then attempts to operationalize definitions and outline a recursive research model appropriate for research in these areas. Finally, it briefly describes how one small liberal arts university is currently thinking through the process of implementing a longitudinal assessment program including student outcomes studies and student attrition research through a recursive research model and operationalized definitions. Stressed is the need for institutions to plan their research to be consistent with the educational goals of the institution. (Contains 17 references.) (LEE)

This paper examines the many facets of administrative information management on the college or university campus. It is argued that, depending on the situation, an effective information manager can adopt the outlook of an architect/designer, data administrator, editor, analyst, reporter, planner, broker, collaborator, interpreter, or marketer. These roles are discussed in the context of an information management model that begins with the campus office or department that supplies the raw data and moves from data base development and administration to the production of analyses and subsequent presentation of these analyses to the customer (the president, deans, committees, governing board, or legislature) who ultimately uses the information in implementing a policy or strategic decision. This model is discussed in relation to Sheehan's (1977) Three-Hat Theory (which focuses on the roles of decision maker, intermediary, and technician), and extends his theory by positing numerous personae who are integral to effective institutional research. One of these many faces--the evaluator--transcends Sheehan's three hats and suggests a further linkage with the Plan-Do-Check-Act sequence as detailed by McLaughlin and Snyder (1993). (Contains 18 references.) (MDM)

This paper provides the current rationale and historical background for educational standards and discusses various meanings and interpretations attached to this term. It then provides a comparative analysis of these sets of publications that are seen as providing national standards for the American Association for the Advancement of Science (AAAS) and the National Research Council (NRC). Next, the role of assessment in setting standards is discussed, in particular the science frameworks used by the National Assessment of Educational Progress (NAEP) and the Third International Mathematics and Science Study (TIMSS), the work of the New Standards Project (NSP), and the expectations built into rigorous university entrance exams as represented by advance placement examinations. The paper also addresses the current status of state science curriculum frameworks, including commonalities and variations among them. It concludes with a brief discussion of standards and framework documents, nationally and state developed, as policy levers for reforming science education in elementary and secondary school. (Author/KHR)

This paper identifies some issues that may prevent the implementation of evaluation results in government agencies. A major reason for the lack of implementation in such agencies often is that evaluations are expected to be used both for information purposes and as accountability mechanisms. These two purposes do not always work together. When this is the case, the focus of the evaluation, the way it is conducted, and the manner in which the results are disseminated may not be conductive to program personnel implementing the results. There are other reasons why evaluations may not be implemented in government agencies, and this paper suggests some strategies for dealing with these problems, including the presentation of the results in varied settings and formats and, where appropriate, the establishment of steering committees or project groups to ensure that the results are implemented appropriately. (Author)

During school year 1998-99, military leaders and parents expressed discontent about the quality of education in Department of Defense (DoD) dependent schools in Europe. Concerned that these education issues were having an adverse impact on the quality of life of military members and families in Europe, and were creating morale and potential retention problems, the Under Secretary of Defense (Personnel and Readiness) requested that the Institute for Defense Analyses conduct an independent review of DoD schools. Study goals were to provide an objective assessment of the quality of education in Department of Defense Education Activity (DoDEA) schools, examine command and parental issues and concerns about educational quality and expectations for students' educational experience, examine issues and concerns identified by DoDEA personnel, and make recommendations about how to respond to any differences between parental perceptions and expectations, and actual performance of DoDEA schools. After the summary, seven chapters address: (1) background; (2) study objectives and approach; (3) student achievement; (4) curriculum; (5) teachers and administration; (6) school environment; and (7) Department of Defense Education Activity (DoDEA) management and relationships. Appendices include a condensed history of DoDEA, study objectives and approach, student achievement, curriculum, teachers and administrators, school environment, DoDEA management and relationships, recommendations, glossary, and references. (Contains 186 references.) (RT)

This volume compiles, and presents in integrated form, the Institute for Defense Analyses' (IDA) quantitative analysis of educational quality provided by the Department of Defense's dependent schools. It covers the quantitative aspects of volume 1 in greater detail and presents some analyses deemed too technical for that volume. The first task in conducting the study involved reviewing relevant literature to determine the most important factors on which data should be obtained and analysis performed. The second task was to become familiar with the data. The third task was to integrate the data and conduct the analysis. After the introduction (chapter 1), the following topics are covered in their own chapters: student achievement, educational resources, college attendance, teacher quality, measuring the Department of Defense Educational Activities (DoDEA) contribution to student achievement, and conclusions and recommendations. Recommendations include continuing to focus on and deliver good student achievement as evidenced in this study, concentrating on maintaining DoDEA's high level of intellectual capability of its teachers when recruiting and managing them, and introducing a system to measure the contribution of DoDEA to student achievement. The report concludes with 30 tables. (RT)

This report summarizes the outcomes of the "Colleges Going Green" project that sought to develop a widely applicable core of environmental learning outcomes (curriculum objectives) and illustrative learning assignments and to review activity in colleges and provide guidance on introducing environmental policy. This report also presents a unit of generic environmental learning outcomes specified at Level 2 and Level 3 (in the Further Education Development Agency (FEDA) and national qualifications frameworks) and illustrates how these can be implemented through learning assignments in various vocational areas together with assessment criteria enabling student achievement to be assessed. The units can be implemented free standing or the outcomes can be wholly or partly integrated into a vocational or other program. (JRH)

Many of the first-wave reform initiatives of the 1980s have been successfully implemented on a widespread basis and are having an important influence on the schooling process. The reform movement was expected to fail because of strong financial, political, and organizational arguments. Reform initiatives were said to be insufficiently funded, lacking a comprehensive approach, employing inappropriate policy mechanisms and tools, and giving responses that were too indirect to be effective. Professional intransigence, deep-rooted norms in organizational culture, and the institutional, bureaucratic nature of loosely linked systems in schools also contributed to the movement's expected failure. Speculations on reasons for the reform movement's success are: (1) efforts built improvements on existing organizational structures; (2) changes rendered schools more receptive to reform than in the past; (3) arguments on organizational propositions were incongruent with the current school systems; and (4) shifts in values had redefined the context in which reform was enacted. (66 references) (EJS)

A matrix model for the collection, management, and utilization of data was developed at Eastern Iowa Community College District, (Davenport, Iowa) for evaluating institutional effectiveness. The model was examined in relation to various assessment instruments and the evaluation of functional areas and programs. Surveys of six different target groups were conducted for each program under review; these groups were: program administrators; part-time faculty; currently enrolled students; graduates of the program during the last three years; program advisory council members; and employers of program graduates. The process of constructing the matrix required a thorough examination of each survey instrument, the identification of items common to the various surveys, and the determination of uniformity in measurement of those various functional areas or components. This resulted in the design of two matrices: the Program Evaluation matrix and the Student Survey matrix. The matrix model facilitated integration of data from a variety of survey instruments into a composite report on any given service or functional area. An additional outcome of the matrix model was that it provided a rationale for the revision of survey instruments. Contains four references. (GLR)