Search

The Guidance Services and Research and Development Departments of the College Board coordinated an investigation of the results of a national high school survey. This survey, administered in 2000, follows a similar, large-scale survey of high schools in 1993 by the College Board. The current research report is designed to provide the reader with detailed and updated information about high schools in the United States. It provides basic facts, figures, and trends about common features among the participating institutions. The report also presents evidence in support of observable trends among the educational and demographic factors of concern as of 2000 and 1993. Two appendices are included: (1) 2000 Advanced Placement Examination® (AP®) Participation by Total School Enrollment and by Institution Type; and (2) Methodology and Survey Instrument.

The 1999 Advanced Placement[R] (AP[R] Psychology Examination contains items drawn from 13 factors related to the study of psychology. This factor structure had not been explored previously. This study focuses on evaluating the fit of confirmatory factor analysis (CFA) models to examination items. Since examination items were dichotomous and polytomous, the CFA models were fit to polychoric correlation matrices using weighted least squares with the inverted matrix of asymptotic variance/covariance estimates serving as the weight matrix (W-1). A rationale for using this method is provided. The correlations among items, as well as their asymptotic variances and covariances, were estimated with PRELIS 2.3, and the CFA was performed with LISREL 8.3 (K. Joreskog and D. Sorbom, 1999). Results indicate that the proposed CFA models fit the data well, which suggests that the theoretical factor structure of the examination is plausible. The paper discusses limitations and next steps. (Contains 1 table, 13 figures, and 25 references.) (Author/SLD)

This study used confirmatory factor analytic methods to investigate whether the subscales of the Advanced Placement International English Language examination (APIEL[R]) measuring Writing, Speaking, Listening, and Reading were invariant across 2 groups, 197 Chinese students and 434 German students. Since categorical responses were observed on APIEL[R] indicators, models were fit to the polychoric correlation matrix and the matrix of asymptotic variances and covariances. PRELIS 2 was used to estimate the correlation matrix and the asymptotic variances and covariances of the examined indicators; LISREL 8 was used to perform confirmatory factor analysis (K. Joreskog and D. Sorbom, 1996). Results indicate that while the factors comprising the APIEL[R] are valid across groups, examinees did not interpret the content of indicators equivalently across groups. The paper discusses limitations of this research and next steps to take. (Contains 2 figures, 4 tables, and 19 references.) (Author/SLD)

Setting standards on educational tests is extremely challenging. The psychometric literature is replete with methods and guidelines for setting standards on educational tests; however, little attention has been paid to the process of setting standards on computerized adaptive tests (CATs). This lack of attention is unfortunate because CATs are becoming more widely used, and setting standards on these tests is typically more difficult than setting standards on nonadaptive (linear) tests. This paper discusses some of the issues to be addressed when setting standards on CATs, presents the results of a standard setting study conducted on a computerized adaptive placement test, and discusses the implications of the findings for future research and practice in this area. Thirteen mathematics experts participated in the standard-setting study using ACCUPLACER (College Board) scores. The results of the study suggest that standards can be set on CATs using subsets of items from a CAT item pool, and that methods designed to gather test-centered standard setting data more quickly than traditional methods show promise for setting standards on CATs. (Contains 2 figures, 5 tables, and 14 references.) (Author/SLD)

Because different types of computerized tests exist and continue to emerge, the term "computer-based testing" does not encompass all of the various models that may exist. As a result, test delivery model (TDM) is used to describe the variety of methods that exist in delivering tests to examinees. The criterion that is used to distinguish between the various TDMs is the extent to which the test is adaptive to an examinee's performance during the test session. On one side of the continuum, there are linear tests that do not change in light of the examinee's performance. On the other side of this continuum, there are tests in which each item presented is dependent on the examinee's performance (adaptive). Types of TDMs and the architecture of a computer-based testing system are described.

The purpose of this paper is threefold. First, the description of a research plan for a large-scale computer adaptive testing program migrating from a network to Internet version is presented. The general areas needed in such a research plan include: (1) comparability studies; (2) psychometric analyses; (3) development projects; (4) validity studies; and (5) audits. Second, some data showing the comparability between the network and Internet version of one test (i.e., Elementary Algebra) are presented. Simulations used actual test questions with previously calibrated parameters at known ability groups. The comparability results examine comparability in ability estimates, scale scores, performance levels, course placements, and content specifications. The third aspect of this paper is to suggest the use of the "Association of Test Publisher's Provisional Guidelines" in the development of a research plan. Limitations in the simulation data require additional research. Next steps to replicate the comparability studies are presented. Appendixes contain a discussion of characteristics of the Elementary Algebra test, summaries of item pool parameters, a summary of performance level statements, and an outline of system functionality specifications. (Contains 8 figures and 19 references.) (Author/SLD)

The purpose of this study was twofold: to develop teachers' assessment skills and to enable teachers to apply this knowledge to the creation of assessment tasks for the language arts curriculum thereby linking curriculum to assessment. Using a newly developed language arts curriculum, 79 urban Connecticut teachers were asked to develop the assessment component after they participated in a 6-session (12 contact hours) assessment training inservice. It was found that the participants were satisfied with this inservice program. They also reported that information and skills in assessment increased, and suggested that their approaches to teaching were influenced. Suggestions to improve this inservice included more time to develop the assessments and more hands-on work. Using this type of inservice in conjunction with curriculum guide development seemed to provide a means of integrating curriculum and assessment and developing the assessment skills of teachers were developed. Investigations into the impact on student learning are suggested. (Contains 3 tables and 11 references.) (Author/SLD)

The most popular methods for setting passing scores and other standards on educational tests rely heavily on subjective judgment. This paper presents and evaluates a new procedure for setting and evaluating standards on tests based on cluster analysis of test data. The clustering procedure was applied to a statewide mathematics proficiency test administered to 818 seventh-grade students in a small urban/suburban school district. Content area subscores were derived from the test specifications to serve as clustering variables. Subsequent course grades in mathematics were used to validate the cluster solutions and the stability of the solutions were evaluated using two random samples. The three-cluster (K-means) solution provided relatively homogeneous groupings of students that were consistent across the two samples and were congruent with school mathematics grades. Standards for "intervention,""proficient," and "excellent" levels of student performance were derived from these results. These standards were similar to those established by the local school district. The results suggest that cluster analytic techniques may be useful for helping set standards on educational tests, as well as for evaluating standards set by other methods. Suggestions for future research are provided. (Contains 2 figures, 7 tables, and 23 references.) (Author/SLD)

The selection of supervision models from the point of view of practitioners has become overwhelming, especially within the contextual forces of a school or school system. The purpose of this study was to use multidimensional scaling (MDS) to describe four theoretical models of teacher supervision (clinical, artistic, technical, and reflective) empirically on common dimensions. Seven graduate students in a class on supervision and the professor completed a questionnaire asking them to: (1) rate the similarity between pairs of models of supervision; and (2) rate the importance of various adjectives in describing these models. The information from this questionnaire was used to perform an MDS scaling analysis. Two dimensions were found to distinguish among the four models of supervision. An MDS approach may facilitate the development of empirically based descriptions of these various models. However, additional research would be necessary. (Contains 2 tables, 1 figure, and 15 references.) (Author/SLD)