Search

This Digest summarizes trends in grading practices and introduces issues related to standards-based reporting. A large majority of schools use a traditional grading system, and many teachers have a great deal of autonomy in making decisions about grades. Given the latitude that teachers have in developing grading policies and differences in curriculum across schools, districts, and states, it is not surprising that there may be disconnects between students' grades and student achievement as measured by test performance. One possible way to find out if an "A" at one school represents the same level of achievement as an A at another school is to compare student performance on end-of-course examinations. Fifteen states now use end-of-course examinations. Most states have embraced standards-based education, and it might be possible to make grades reflect the progress students have made toward the standards. Report cards that combine traditional grades and information about progress toward standards are also an option. K. Marzano describes such an approach and also recommends that teachers reorganize their grade books around standards by allocating columns to standards rather than to assignments and tests. Some current classroom grading practices are thought to be detrimental to the goals of standards-based education. It is essential that clear information be provides about how far along each child is in mastering specific knowledge and skills that all are expected to learn. (Contains 11 references.) (SLD)

This Digest is intended to help teacher candidates systematically gauge their progress toward the teaching profession by developing a portfolio. Portfolios are one way to assess teacher candidates, and they are a major requirement for experienced teachers seeking board certification from the National Board for Professional Teaching Standards. The contents of a portfolio depend on what it is intended to demonstrate to whom. Written work and products might come from many different sources. What keeps the portfolio from being merely a scrapbook is the reflection the teacher candidate undertakes about what each artifact demonstrates about his or her educational philosophy. The Digest also discusses the four stages of portfolio development: (1) collection of materials and artifacts; (2) reflection on the collected materials; (3) reduction to select the artifacts that demonstrate mastery; and (4) display of the selected artifacts to demonstrate mastery and permit review. Portfolio development is not a scavenger hunt that results in the creation of a scrapbook, but a responsive and purposeful activity that engages reflective capacities of preservice teachers. (SLD)

This Digest draws from the current literature and the author's experience to identify suggestions for developing performance assessments and their accompanying scoring rubrics. This Digest, part 1, addresses writing goals and objectives and developing performance assessments. Before a performance assessment or scoring rubric is written or selected, the teacher should clearly identify the purpose of the activity. The statement of goals should provide a clear focus for instruction and assessment, and both goals and objectives should reflect knowledge and information that is worthwhile for students to learn. The relationship between a goal and the objectives that describe that goal should be apparent, and all important aspects of the goal should be reflected through the objectives. Objectives should describe measurable student outcomes, and then should be used with goals to guide the selection of an appropriate assessment activity. Recommendations are also given for developing performance assessments: (1) the selected performance should reflect a valued activity; (2) the completion of performance assessments should provide a valuable learning experience; (3) the statement of goals and activities should be clearly aligned with the measurable outcomes of the performance activity; (4) the task should not examine extraneous or unintended variables; and (5) performance assessments should be fair and free from bias. (Contains 12 references.) (SLD)

This Digest addresses developing scoring rubrics, administering performance assessments, and scoring, interpreting, and using results. In developing scoring rubrics, the criteria within the scoring rubric should be clearly aligned with the requirements of the task and the state goals and objectives. They should also be expressed in terms of observable behaviors or product characteristics. Scoring rubrics should be written in specific and clear language, with points that make sense, and a clear separation between score levels. It is also important that the statement of the criteria be fair and free from bias. Recommendations are also offered for administering performance assessments. Explanations of tasks should be clear and consider, and appropriate tools should be available to support the completion of the assessment activity. Scoring rubrics should be explained to students ahead of time. In scoring, interpreting, and using results, guidance is offered for using rates and anchor papers. The connection between the score or grade and the scoring rubric should be immediately apparent. Recommendations in this Digest can guide a teacher from planning, gathering, interpreting, and using, the four phases of classroom assessment. (Contains 10 references.) (SLD)

This Digest provides educators with an overview of some important facets of cognitive science research and suggests implications for classroom assessment. Education researchers study the thinking of experts in various subject areas to understand the concepts and procedures that are the most important to teach and to help determine how to move students along the continuum towards real-world subject mastery. Researchers consider the common pathways toward acquiring knowledge in the classroom. More research has been done about domain structure in some disciplines than in others, but much work is being done on how learners store and access knowledge. Assessment designers can use findings from cognitive science. Cognitive science calls for test developers to work from a deep knowledge of the central concepts and principles of a given subject area and to identify tasks that let students demonstrate their understanding and skills, as opposed to rote memorization. Overall, cognitive science asks test developers to let students show what they know and can do. At the classroom assessment level, cognitive science findings encourage teachers to teach learners how and when to apply approaches and procedures and to teach metacognitive skills and problem solving. Teachers need to be aware that acquiring important knowledge and skills at an in-depth level takes a significant amount of time, practice, and feedback. (Contains 20 references.) (SLD)

This Digest, intended as an instructional aid for beginning research students and a refresher for researchers in the field, identifies key factors that play a critical role in determining the credibility that should be given to a specific research study. The needs for empirical research, randomization and control, and significance testing are discussed, and seven review criteria are proposed. These criteria are: (1) randomization; (2) control; (3) reliability; (4) validity; (5) implementation of treatment variable; (6) analysis issues; and (7) interpretation issues. (SLD)

This Digest outlines an appropriate way to handle score normalization in a fair and equitable manner. Using raw scores to calculate final grades may not entirely capture a student's true performance within a class. As variation in performance evaluation increases, so does the impact on the student's final ranking. Ideally, the distribution of individual student performance for all examinations should be equal, and fortunately the methodology for placing diverse assignments on an equitable scale is straightforward. Appropriate normalization requires nothing more than adjusting the examinations' means to be equal as well as their variances. A template for the normalization process is included, and an example derived from real data from a college biology course is given. (SLD)

This Digest provides a review and applications of the concepts of normal distribution, standard deviation, effect size, and translation of effect size into percentile gain, the foundations for the understanding of meta analytic results. Meta analysis is a statistical technique that enables the results of a number of studies to be combined to determine the average effect of a given technique. Comparisons can then be made about the relative effectiveness of various techniques for increasing student achievement. Meta analysis involves identifying, collecting, reviewing, coding, and interpreting scientific research studies. Study outcomes are translated to a common metric to allow results to be compared. Following the explanation of concepts important to meta analysis, the Digest provides an example and lists some resources for further information and tips for locating examples of meta analysis. (Contains 10 references.) (SLD)

This Digest addresses the benefits of formative assessment and provides examples and resources to support its implementation. Formative assessment is the diagnostic use of assessment to provide feedback to teachers and students over the course of instruction. Since the goal of formative assessment is to gain an understanding of what students know and dont know in order to make responsive changes in teaching and learning, techniques such as teacher observation and classroom discussion have an important place alongside analysis of tests and homework. P. Black and D. William (1998) encourage teachers to use questioning and classroom discussion to increase their students knowledge and improve understanding. They suggest several strategies to involve all students. Teachers can assess students understanding in a number of ways that include interviews and brief, in-class writing assignments designed to determine understanding. Tests and homework can be used formatively, as can portfolios. Because formative assessment is tightly linked with instructional practices, teachers must consider how their classroom activities, assignments, and tests support learning aims and allow students to communicate what they know, and they must use this information to improve teaching and learning. Some useful resources for teachers interested in formative assessment are listed. (Contains 15 references.) (SLD)

The goal of this Digest is to explore some of the issues involved in data transformation, with particular focus in the use of data transformation for the normalization of variables. The Digest is intended to serve as an aid to researchers who do not have extensive mathematical backgrounds or who had not had extensive exposure to this issue. It focuses on three of the most common data transformations used to improve normality: (1) square root; (2) logarithmic; and (3) inverse transformations. It is recommended that researchers always examine and understand data prior to performing analyses, and that they then know the requirements of the data analysis technique to be used. Data transformation should be used with care, and never unless there is a clear reason, and the researcher must ensure that the variable is anchored at a place where the transformation will have the optimal effect. (SLD)

This Digest presents a discussion of the assumptions of multiple regression that is tailored to the practicing researcher. The focus is on the assumptions of multiple regression that are not robust to violation, and that researchers can deal with if violated. Assumptions of normality, linearity, reliability of measurement, and homoscedasticity are considered. Checking these assumptions carries significant benefits for the researcher, and making sure an analysis meets the associated assumptions helps avoid Type I and II errors. Attending to such issues as attenuation due to low reliability, curvilinearity, and nonnormality often boosts effect sizes, usually a desirable outcome. There are many nonparametric statistical techniques available to researchers when the assumptions of a parametric statistical technique are not met. These are often somewhat lower in power than parametric techniques, but they provide valuable alternatives for researchers. (SLD)

Current practices for assessment of language in bilingual students frequently involve the use of tests translated from English to the target language or tests designed for, and normed on, monolingual people. This Digest explains why these common approaches are not well suited for a bilingual population and provides guidance regarding more suitable approaches. There are many problems with test translation. Tests translated from one language to another do not retain their psychometric properties, and issues of equivalence of both vocabulary and grammar surface. Bilingual school children usually are circumstantial bilinguals, and their vocabularies in the different languages may reflect the different circumstances in which the vocabulary is used. Language tests can be improved if test developers: (1) ensure that concepts and linguistic features are appropriately represented for each language; (2) use conceptual scoring systems to eliminate underestimation of ability; (3) select an appropriate mix of item types to gain the maximal amount of information about language ability in each language; and (4) consider the frequency of occurrence of the words. (Contains 11 references.) (SLD)

This Digest summarizes the results of a survey administered to the American Educational Research Associations Survey Research Special Interest Group regarding the use of electronic surveys and discusses their responses in the context of the existing literature base. Topics include the conditions under which use of e-mail or Web-based surveys would be most appropriate, sampling issues, weaknesses of the approach, and guidelines for other researcher who plan to use e-mail or the Internet for survey research projects. Overall, respondents (n=63) reported frequent use of, and a high level of confidence in, electronic mail and the Internet. The only area in which they expressed concern was creating and maintaining a Web page. Participants generally responded favorably to statements regarding the use of e-mail and Web-based surveys. Respondents were most positive about the reduction of cost. Less favorable responses focused on concerns about the survey respondents' knowledge of, and experience with, technology. A review of the literature supports the position that Web-based electronic surveys must use principles of sound survey design, and researchers must focus on the adaptability of principles of survey design to electronic survey formats. (Contains 18 references.) (SLD)

This Digest introduces ethical considerations related to acquiring and analyzing online data and provides resources to support sound practice. Because online conversation is relatively new and unfamiliar, and takes place at a distance, participants may not realize or may not remember that their conversations could be made public. A researcher affiliated with a university or similar institution will have an institutional review board to help with ethical issues. Those who are not associated with an organization with the equivalent of an institutional review board should follow ethical principles as laid out in several resources that are readily available. The least problematic conversations are those that take place entirely in the public domain in situations in which people know that they are publishing to a public area with unrestricted viewing. If a domain has established any degree of privacy, communications should be considered privileged. If a researcher sets up a site to collect data, he or she is ethically bound to let potential participants know that their contributions will be data in a research project. Researchers engaged in online conversations should begin with established techniques and then adapt these to the online environment. (SLD)

Web-based surveying is very attractive for many reasons, including reducing the time and cost of conducting a survey and avoiding the often error prone and tedious task of data entry. At this time, Web-based surveys should still be used with caution. The biggest concern at present is coverage bias or bias resulting from sampled people either not having access to the Internet or choosing not to use it. Both coverage bias and the question of familiarity with Internet tools should become less of an issue in the future. Although research on Internet-based surveying is limited, findings are beginning to appear in the literature. Several studies have suggested that response rates for Internet surveys are lower than the response rates for mail surveys. At present, most Internet surveys are done using HTML forms with potential respondents often contacted through e-mail cover letters. A set of software tools, Web Survey Mailer System, that provides many of the complex Web survey administration functions included in many Web surveying packages has been developed. This set and other tools will make Web-based surveys easier to conduct, but at present researchers should use the technique with caution. (Contains 11 references.) (SLD)