Search

One Midwestern state has chosen a model of state assessment in which local school districts are responsible for developing the strategies to measure and report their students' performance on state-adopted content standards. When students are not measured on common instruments, district accountability becomes an added challenge. This paper presents a strategy for evaluating locally developed assessments as part of the state assessment system that can be used to inform the need for state accountability. An application of the strategy is described in which a 16-member District Assessment Evaluation Team was recruited to evaluate district assessment portfolios for the state. An analysis of inter-rater agreement on the common districts evaluated by all 19 raters (16 team members and 3 anchor raters) was performed. A goal of 80% agreement or higher was established, but there was some variance in the levels of agreement for each criterion and the overall ratings. Results indicate there was reasonable consistency among raters. Benefits of the proposed strategy include an emphasis on formative rather than summative feedback and on improving assessment strategies at the local level. Locally developed assessments that are aligned to the state's content standards and integrated into the district's curriculum are likely to produce data that is meaningful to the state, yet can inform instruction in the classroom. appended are tables of data used in the study. (Author/SLD)

A study was conducted to explore empirically the effect of different definitions of the target examinee on the judgment of panelists setting a passing score. Two cut score studies were done in a school district for the same test within a 6-month period, and different definitions of the target candidate were provided for each study. In October 1998, 15 teachers, all of whom were middle school or high school mathematics teachers, participated in the first standard setting exercise. The target student was called the "Barely Master Student," a student whose skills are sufficient to justify graduation, "but just barely." The methods and procedures for the April study were essentially the same, but the target examinee was designed to be more skilled. The 10 panelists who participated in both studies typically set higher standards when the target student definition represented a more skilled student. The findings suggest that under the same conditions, cut scores can be very reliably set across time and panels, but when elements of the cut score study change, the cut score may change accordingly. (SLD)

While earlier, more blatant forms of sex discrimination may have declined, some researchers have suggested the existence of more subtle forms of bias, based less on gender than on gender-related attributes. The investigation of bias related to either gender or gender-related attributes requires a scale to address both the gender-relatedness of characteristics and how positively or negatively the stereotypical characteristics are viewed. Twenty masculine, 20 feminine, and 20 neutral gender-related characteristics with positive connotations related to employability were selected for the development of the Positive Sex Role Inventory (PSRI). The 60 attributes were placed in an instrument and presented to 284 undergraduates who rated the attributes for their desirablity for a counseling psychologist. Analysis of this instrument showed that the scale had a three-factor structure and satisfactory subscale internal consistency reliability. Five masculine and feminine item pairs were then identified from the subscales so that the mean difference between the rating of the two items was not statistically significant. This study yielded two experimentally useful products: the PSRI can be used in experimental research involving positive sex-role attributes and the set of five item pairs developed to be of equal market value provides a potentially useful methodology for sex bias research. (NRB)