Search

We developed Behaviorally Anchored Rating Scales (BARS) for measuring teaching practice, and compared them to the well-established Framework for Teaching (FfT; Danielson, 2013). BARS provide behavioral anchors delineating levels of performance via a set of behaviors. Our BARS focused on two dimensions of teaching, leading a classroom discussion and making content and practices explicit. We examined how a) rater agreement for BARS compares to rater agreement using the FfT, and b) how BARS and the FfT compare regarding perceived ease of use, perceived accuracy, and perceived advantages and disadvantages. Nineteen raters, who are users of the FfT and were trained to use BARS, independently evaluated video-taped teacher lessons using both methods. Rater agreement was higher for the FfT, which may, in part, be a factor of the raters' greater experience with that rating system. Nonetheless, raters reported that there are many aspects of BARS that are desirable.

A licensure assessment's purpose is to measure the relevant knowledge and skills required for safe and effective professional practice. Given the important role of licensure assessments, great care must be paid to the issue of validity: Is the assessment measuring what it claims to measure? In particular, evidence of a licensure assessment's content-related validity must be collected and evaluated prior to incorporating the assessment into a licensure process. The School Leaders Licensure Assessment was developed to be part of state licensure processes for entry-level school principals. To evaluate the use of the assessment for this purpose, a multistate panel of professionals examined the assessment and rendered judgments concerning its appropriateness. The results of this content-related validity study support the use of the School Leaders Licensure Assessment by affirming the relevance and importance of the content being assessed.

The Test of Written English (TWE), administered with certain designated TOEFL® examinations, consists of a single essay prompt to which examinees have 30 minutes to respond. It was introduced in 1986 to provide TOEFL score users with a direct measure of examinees' writing ability. Preliminary studies had indicated that the two different kinds of prompts: prose compare, contrast, and take a position, and describe or interpret a chart or graph, elicited comparable writing performance. However, questions were subsequently raised with respect to continued comparability of different TWE® prompts administered under operational conditions. The present study was designed to elicit essays for prompts that differed in both subject matter (Topic) and in the level of explicitness with which the essay task was presented (Topic Type). Eight different prompts were spiraled worldwide at the October 1989 TOEFL administration, with each prompt eliciting approximately 10,000 essays. The results of the analyses indicated that there were small differences among the prompts. The most notable differences were obtained among the scores for topics using the explicit comparison. Across all the prompts, the chart-graph with the explicit comparison statement produced the highest mean scores. Because it was the first study of its kind to focus on the comparability of prompts in a major testing program, the authors had difficulty making definitive statements regarding the meaningfulness of the obtained differences. While many of the differences in means observed in this study were so small as to be of no practical significance, differences observed across prompts in the numbers of examinees at each score level were not. Such differences may warrant further consideration by the TOEFL program.

The purpose of this study was to explore the use of two alternative item response theory estimation models in the scaling and equating of TOEFL – a modified one-parameter model (M1PL) and a modified two-parameter model (M2PL) – and to compare item scaling and test equating results based on these two alternative models with results based on the three-parameter model (3PL) that is currently being used to scale and equate the TOEFL. The study employed a design in which a typical TOEFL equating was simulated using artificial data. The simulated equatings were compared in terms of correlations between estimated and generating parameters, model-data fit, and concordance of simulated score conversions with conversions based on the generating parameters. The results of the study clearly indicated that the 3PL model performed better than the M1PL and M2PL models on the basis of each of the evaluation criteria. There was also evidence that the M2PL model performed better than the M1PL model, particularly in terms of model-data fit and in the weighted root mean square difference statistics used to evaluate the simulated score conversions. The results of the study also indicated that discrepancies between score conversions based on the M1PL and M2PL model and those based on the 3PL model tended to occur at the lower and upper ends of the score scales. Finally, the results of the study for the 3PL model indicated that while correlations between item parameter estimates and generating parameters tended to be affected by sample size, neither the quality of model-data fit nor the quality of simulated equatings appeared to be sensitive to the different sample sizes used in the study.

The present investigation compared two statistics attitude scales, the Statistics Attitude Survey (SAS) by Roberts and Bilderback (1980) and the Attitudes Toward Statistics (ATS) by Wise (1985). Wise had claimed that the SAS was partially invalid because many of the items appeared to be achievement oriented rather than attitudinal in nature. The SAS and ATS were combined in one scale and administered to 280 introductory statistics students at the beginning of the course. Course grades were obtained at the end of the course. Analyses showed high reliabilities for both SAS and ATS but a correlation of nearly .9 between the two scales. In addition, none of the correlations between the two scales with several criteria, including course grades, were significantly different. It was concluded that the ATS was essentially an alternate form of the previously developed SAS.