Search

The classroom assessment process can have encouraging results when it begins with "early assessment" that addresses student learning, as well as the social and emotional needs of student(s) in the classroom. This paper presents a SWOT (Strengths, Weaknesses, Opportunities, and Threats) analysis for classroom assessment. It will review literature and other research documents citing contributing factors to inadequate classroom assessment and suggest solutions to this pervasive problem. The SWOT analysis is a commonly employed framework in the business world for analyzing the factors that influence a company's competitive position in the marketplace with an eye to the future (Rizzo, Kim, 2005). However, the SWOT paradigm can also be usefully applied outside of the business domain and is here used to analyze the academic activity of the classroom. It should become obvious in this paper how SWOT can be usefully applied to assess and guide classroom organization and outcomes.

Over the past few decades, educators in general, and language teachers in specific, were more inclined towards using testing techniques that resembled real-life language performance. Unlike traditional paper-and-pencil language tests that required test-takers to attempt tests that were based on artificial and contrived language content, performance tests are authentic so that the test-taker is asked to perform language tasks that he or she will need to perform in real-life interactions. A very valuable type of performance test is called portfolio assessment in which a record of students' performance across a wide range of language tasks over a logical period of time is kept so that a profile of performance can be obtained for the evaluation of achievement. This paper will define performance assessment, trace its origins and development, explain how performance tests can be constructed, and describes the nature and advantages of portfolios.

It is the school board alone that determines the quality of education provided within the school district. Without periodically examining its own performance, the board's capability to create a quality educational environment can be compromised. Evaluation is a control mechanism that allows the school board to make judgments about performance based on a set of predetermined and understood objectives. The function of school board self-evaluation is to improve the performance of others within the school district by improving the performance of the board itself. Eight reasons for self-evaluation of school boards are described and cited, based on the work of Kowalski (1981). They are to identify and clarify the board's purpose, to identify strengths and weaknesses, to assess successes and failures, to inform the public, to promote the concept of accountability, to avoid the abuse of power, to enhance the understanding of the purpose of evaluation, and to provide the framework for goalsetting. Further details are concerned with who evaluates school boards, the benefits and pitfalls of self-evaluation, appropriate criteria, and effective procedures. Two sets of guidelines are listed for school board self-evaluation. A discussion on individual board member and meeting evaluations concludes this paper. (RR)

Washback or backwash, also known as measurement-driven instruction, is a common term in applied linguistics referring to the influence of testing on teaching and learning, which is a prevailing phenomena in education. It is a truism that "what is assessed becomes what is valued, which becomes what is taught." This paper aims to share the discussion of this education phenomenon from different perspectives both in the area of general education and in language education. It discusses the historical origins of washback; the definition and scope of washback; and the function and mechanism of washback, and efforts, both recent and not, to mitigate its negative effects. It is concluded that the ultimate reason for the persistence and widespread nature of this problem is the existence of high-stakes testing. Few educators would dispute the claim that these sorts of high-stakes tests markedly influence the nature of instructional programs. Whether they are concerned about their own self-esteem or their students' well-being, teachers clearly want students to perform well on such tests. Accordingly, teachers tend to focus a significant portion of their instructional activities on the knowledge and skills assessed by such tests. (KFT)

Evaluation is important to public education and training programs if the United States is to be competitive in a global economy. Six major factors have had an impact on evaluation efforts: decreased financial resources, increased public dissatisfaction, changed management approaches, enhanced evaluation methods and procedures, redesigned organizations, and increased privatization efforts. Today, three strategies are being used for evaluating educational programs, and they are likely to continue to be used into the next century. These strategies are outcome assessment, performance assessment and measurement, and value-added assessment. Ten recommendations are offered to improve evaluation efforts: (1) build a commitment to conducting high-quality evaluations; (2) develop individuals' evaluation capacity; (3) foster a clearer understanding of mission, customers and their needs, and desired results; (4) find a closer link between performance measures and the outcomes that they asses; (5) focus programs on a limited number of outcomes for which they will be accountable; (6) programs should not avoid having outcomes that are difficult to measure; (7) strengthen the systems and processes for generating and using financial and program information in conducting program evaluation; (8) use greater innovation and creativity in developing evaluation and accountability techniques; (9) develop more efficient and cost-effective evaluation and assessment techniques; and (10) keep politics out of the process. (Contains 24 references.) (KC)

Evaluation affects the people who participate and the situation studied, and there is no guarantee that the effects will be democratic, constructive, or equitable. A case study illustrates that even the most scrupulous attention to ethical principles and procedures for the release of information cannot always protect the interests of participants in the study, or the integrity of the study itself. At issue was a male school principal instructing a female teacher to change the record of what she had said in evaluating a language curriculum project. The objectives of the paper are the following: (1) to demonstrate the way in which local reactions to an evaluation directly threatened its validity; (2) to demonstrate that not all participants in a qualitative evaluation study understand or defer to the commitments of democratic evaluation; (3) to indicate that the metapolitical analyses of evaluation approaches do little to inform the nature of local site effects; and (4) to identify some new theses about the practice of "democratic" evaluation. (MLF)

Indicators of school-level achievement, such as the percentage of students who are proficient in a particular content area, are subject to random year-to-year variation in much the same way that the results of an opinion poll will vary from one random sample to another. This random variation, which is more pronounced for a small school, should be taken into account by education officials when evaluating school progress in a policy climate of high stakes. To do otherwise is to unnecessarily risk the false identification of a failing school. In this monograph, I describe the application of confidence intervals to the evaluation of "adequate yearly progress" for No Child Left Behind (NCLB). Throughout, I demonstrate the particular relevance of confidence intervals for small schools. Upon completion, readers will understand why 27 states included confidence intervals in their NCLB accountability plans (and perhaps wonder why the remaining states did not). (Contains 13 footnotes, 5 figures and 1 table.)

This keynote address argues that policy makers should consider a project approach to implementation of research results regarding foreign language learning; they should pay attention to what is being revealed by a growing number of evaluation and impact studies concerning effective innovation and project effects. Second, language teaching research needs to develop better definitions of linguistic and non-linguistic outcomes if research is to address the complexity of the task of foreign language learning. There appears to be a pervasive lay belief in the essential simplicity of the task of learning a foreign language, encouraged by the mistaken notion that foreign language learning and first language acquisition are comparable. Such beliefs may partially explain the commercial, if not educational, success of teaching methods and materials that make claims of effectiveness in remarkably short periods of time. Careful research has failed to bear out such claims or to support such beliefs. Even if effective approaches were known, it is naive to believe that research results can be translated into action without consideration of the conditions under which innovations take hold and succeed. The language teaching profession has failed to define adequately what it means by success in language learning; there is little agreement on expected or desired standards of achievement. There is also little attention to the principles that are accepted in the language testing profession for the construction of measures of achievement and proficiency. Language teaching research will need to develop much better definitions of the linguistic and non-linguistic outcomes that are expected of the various programs and projects if research is both to address the complexity of the task of foreign language learning as well as to contribute to our understanding of the conditions that can lead to successful language learning. (Contains 58 references.) (Author/NAV)

With the growing demand for accountability by legislators, outcomes assessment has become a much higher institutional priority for community colleges. According to a recent survey, two-thirds of the states have implemented formal assessment initiatives, compared to a very few states only 1 year earlier. Institutions, however, must be careful that assessment is not used only to ensure accountability, but also to enhance institutional quality. Effective and meaningful assessment evolves within a collaborative framework in which both legitimate legislative needs and the integrity of institutional autonomy are respected equally. Follow-up studies and job placement surveys are often adequate for institutional needs but do not sufficiently fulfill state requirements. Some colleges have therefore implemented comprehensive assessment programs which attempt to evaluate quality at all levels, over time, through systematic reviews of academic programs and services and longitudinal studies of educational trends. One model identifies six areas of concern faced by all colleges: access and equity; employment preparation and placement; college/university transfer; economic development; college/community partnerships; and cultural and cross-cultural development. In 1988, a national study of outcomes measures investigated the use of academic progress and employment outcomes, student learning outcomes, and student satisfaction outcomes among a sample of 675 institutions of the American Association of Community and Junior Colleges. Over 90% of the respondents expected outcomes measures to maintain their current priority or increase in priority over the next 3 to 5 years. The results of outcomes assessment should help colleges identify where present efforts and priorities lie and where they should be placed. (JMC)

In the United States today, policymakers and citizens believe that the quality of their schools must be improved. Reform policies and strategies are the focus of the first section of this report. The four areas identified are the perception and interpretation of the problems of quality, the methods of preparing reform or ad hoc measures, the main features of reform or ad hoc measures, and the strategies of implementing adopted policies. Evaluation and monitoring of policies and reform comprise the second section. In contrast to other countries, the limited role of U.S. federal government in monitoring and evaluation is examined. Seven areas are identified: quality evaluation policies and their methods, evaluation of the quality of schools, evaluation of educational quality, evaluation of teacher quality, evaluation of curriculum quality, evaluation of the quality of other educational services, and changes in the monitoring and supervisory mechanisms. The final section reports on the first results of evaluation and monitoring. (RR)

This discussion paper is intended to inform the further development of the National Research and Evaluation Strategy (NRES) by strengthening its focus on evaluation in Australian vocational education and training (VET). Chapter 1 discusses the implications for the role of evaluation that changes in VET have caused in the past decade. Chapter 2 focuses on these dimensions of evaluation: for what purpose, by whom, and using which approach. Chapter 3 outlines these focus areas of current evaluation activities, from the largest scale to the smallest: evaluation of national performance; evaluation of policies and national initiatives; evaluation of practice; consumer evaluation of training provision; and critique of policies and programs. Chapter 4 describes these three outcomes to assist the use of evaluative methods to support decision making that could be achieved by extending the NRES: (1) a useful level of evaluative activities; (2) a capacity to evaluate; and (3) an informed and robust critique. Two appendixes are "The Performance of the VET System: Further Details" and "Some Possible Topics for Evaluation of National Initiatives." (YLB)

This monograph examines selected issues in the design and conduct of program and project evaluation in developing countries, with a focus on the evaluation of international technical assistance programs. Chapter 1 introduces the evolving role of efficiency criteria in the evaluation of educational systems. Chapter 2 provides detail on the nature of internal efficiency, its operationalization, and the special efficiency issues that exist in the developing world. The major constraints on educational efficiency enhancement efforts in developing nations are also discussed. The third chapter discusses the nature of the evaluation process (as distinct from research), stressing the meaning of evaluation in terms of context and timing and the crucial role of the evaluator. Chapter 4 examines the critical role of criteria, standards, and indicators in designing and conducting evaluation studies. The special problems of developing and applying evaluation criteria within the international technical assistance system are also discussed. Chapter 5 discusses basic procedural steps common to all evaluations, and chapter 6 focuses on the evaluation issues that are most problematic in enhancing efficiency in educational assistance programs in developing nations. The final chapter offers a summary and recommendations. (Contains 101 references.) (LMI)

Critiques school reform evaluation and common misconceptions about it. The article warns against using evaluation to sort students in ways that hinder them from gaining quality education or force teachers into implementing school changes that are unresponsive to students' learning problems. Study findings involving school improvement efforts are reviewed and recommendations for increasing reform evaluation's importance are provided. (GR)

That theoretical debates and developments in organizational analysis can be a source of inspiration for evaluation utilization research is explored through consideration of the societal function of evaluation and its historical development. A discussion of policy evaluation in Danish political administration illustrates evaluation theory and practice. (SLD)

Examines forces that shape the design of assessment systems for teachers and some inherent contradictions with regard to curriculum and assessment. Proposes a system of assessment that is continuous, dynamic, and principled and does not remove from teachers the responsibility of defining what is important to student learning. (SD)

A subordinate evaluation instrument was used to identify factors that employees of Dayton-Montgomery County (OH) Public Library value in their supervisors. Results suggest that subordinates place considerable value on supervisors as role models and expect supervisors to treat them with respect and to encourage their opinions. Implications of these results are discussed for library administrators. (Author/JKP)

Reflects on one person's efforts to develop a collaborative action research methodology for evaluating professional development. Focuses on the development of an inside methodology for teachers. Sets out a range of issues and identifies some tentative principles of procedure. (CFR)

Outlines a procedure which training professionals and business managers can use to assess the effectiveness of their corporate evaluation studies. Meta-evaluation is explained; and phases in the training evaluation process, including plan, develop, obtain, analyze, and report, are described based on experiences at Motorola. (Contains two references.) (LRW)

A former editor of this journal, 1969-72, selects an article from his tenure that stands as an outstanding example of evaluation research. The article selected describes the comprehensive evaluation of the Individually Prescribed Instruction Project in mathematics in three Alberta elementary schools. The project was evaluated via classroom observation; student achievement; and student, teacher, and parent attitudes. (SV)

In spite of many inherent problems, impact evaluation should remain an integral part of program evaluation, both because impact evaluation brings its budget justification with it, and because its focus on causal hypotheses is an essential part of evaluation. Methodology based on the author's body of work is reviewed. (SLD)

Conjectures about the future use of controlled randomized experiments for planning and evaluating social programs suggest that they will frequently involve institutions and organizations rather than individuals and that their applications and the publicity surrounding them will increase as they are used in new and controversial ways. (SLD)

Provides a comprehensive perspective on assessment in community colleges, indicating that there is great variance among institutions in the extent and sophistication of their assessment practices. Suggests that there are three categories of outcomes: student performance at community colleges, transfer institutions, and occupations; student learning levels; and student development. (MAB)

Discussion of the evaluation of new educational technologies and their effectiveness in public education focuses on the use of distance learning. Topics addressed include evaluation audiences; the role of evaluation; the students and teachers using the technology; educational administrators; identified needs and instructional effectiveness; evaluation methodology; costs; and determining cost effectiveness. (LRW)

Maintains that early childhood teacher evaluation practices urgently need reform and that effective practices should (1) draw on multiple data sources, including peers, parents, children, administrators, and teachers themselves; (2) view teacher growth as an ongoing process; (3) examine individualized goals and methods; and (4) involve teachers as decision makers. (MDM)

Analyzes Hughes and Nelson's study of placement testing at Riverside Community College and offers an alternative method of evaluating placement systems using logistic regression. Reanalyzes data from an ASSET placement test validity study using logistic regression. (DMM)

This report considers a range of issues involved in doing case-study research with emphasis on the potential for and the problems associated with this approach in distance education. The report begins with an introduction to the case-study research methodology, noting that it is not widely recognized as an approach in distance education. Substantive general issues are identified as ethical issues, problems of validity and reliability, generalizability, and the extent to which case-study research can contribute to the building of theory. Two case studies in distance education are then presented to illustrate different aspects of this type of research, both in the nature of the findings and the theoretical issues involved. The first example looks at the process of developing distance education courses with particular attention to the experience of the Classroom Processes course at Deakin University (Victoria, Australia). The second example is an interview-based evaluation study of the introduction of home computers into an Open University (United Kingdom) second-level course, The Fundamentals of Computing. The more detailed report for the second study covers theoretical perspective; methodology; the types of students (i.e., professionals using computers, students wanting a degree for a job change, women returning to work, students with a general interest in computers, and computer experts); study patterns and coping with the course; and issues of openness and access. The two studies are then compared, and a discussion of the development of theory in distance education concludes the report. (8 references) (DB)

Comments on the educational quality indicators projects undertaken by Alberta school districts, noting their dependence on process-product research; inclusion of social and affective aspects of education; use of provincial achievement tests as indicators; and emphases on accountability, teacher collaboration, and professional development. (SV)

The idea of making college students more comfortable with their campus environment through involvement in extracurricular activities, thus helping them remain in school, has never been argued by those in the college union field. Nevertheless, efforts and theories in this endeavor have been rarely supported by research and evaluation. Reliable and valid research data can help higher education institutions properly invest in housing, student centers, and recreational facilities and programs for students. The Schine Student Center (Syracuse University, New York) administration and directors of units housed in the student center developed a comprehensive plan for the evaluation of their center including the physical environment, programs offered, and the impact of the Center on student quality of life. With the use of interviews, surveys, focus groups, and record analysis, the Center administrators learned two major lessons: (1) that this type of research is time-consuming and expensive requiring a major commitment, and (2) that most of the college student's intellectual development occurs outside of the classroom, and if a proper space is provided, adequate programs developed, and adequate public notice given about scheduled events, the learning environment becomes stronger, and thus, the university becomes stronger. Contains 4 references. (GLR)

Describes the findings of the National Assessment of Educational Progress's "Reading Report Card" issued in September 1993. Critiques the report for methodological reasons. Argues that the report is virtually worthless regarding the reading skills of American students and should receive a failing mark. (HB)