Search

This paper presents an alternative means of assessing faculty teaching that can be used by administrators. This method has been derived from in-depth faculty and student interviews, and its criterion of quality has been validated with empirical data through computer sensitivity analysis. The assessment process has been successfully tested in clinical teaching trials. Interviews with faculty have identified that their implicit expectations for professional development can be described in terms of technical skills, professional competence, and professional attitudes. This Three Ability Framework (3AF) is explained to the students by the lecturer, who is then rated by the students on each of these abilities. The lecturer rates himself or herself on the same abilities, and the alignment between the self-rating and the students' ratings is used as a measure of the effectiveness of teaching. The indicator of effective teaching is that the students and the instructor are working toward the same changes. This method promotes a positive teaching and learning culture in direct and indirect ways as it encourages teaching that promotes students' critical and evaluative thinking and high standards in technical skills and professional values. (Contains 33 references.) (SLD)

This paper presents the results of research into a technique for the measurement of effective teaching that avoids the problems inherent in student evaluations of teacher effectiveness. The method takes as little as 5 minutes to administer and so can be used in-course by the teacher to track his or her teaching effectiveness. It results in a single number that can be used at the end of a course as an administrative decision point measure. The initial research with 3 classes of teacher training students (n=12, n=12, and n=23) was done in an Australian university and is now being replicated in other universities. The measure operationally defines the three basic abilities of the Three Ability Framework (3AF): technical skills, professional competence, and professional attitudes. The 3AF method assesses the degree of alignment between the changes the students expect in these three abilities and the changes toward which the teacher is working. Correlations between the scores of academic attainment, degree of teacher/student alignment, and course satisfaction indicated that when the students and their teacher were working towards the same proportion and amount of these three abilities, the students had higher academic attainment and greater course satisfaction. The four steps of the application of the method are outlined. (Contains 1 table and 28 references.) (SLD)

The mathematics achievement and mathematics attitudes of eighth-grade female students attending independent coeducational middle schools were compared with those of students in independent all-girls middle schools. Participants were 467 8th-grade girls from 10 all girls middle schools, 208 girls from 10 independent coeducational middle schools, and 123 11th and 12th grade girls from 2 independent high schools. An attitude questionnaire was completed by all subjects, and achievement test data were obtained. Results from high school students were used to support attitude measures. The only statistically significant difference in mathematics achievement was found for one (higher) level of the Comprehensive Testing Program. Overall, large differences were not found for mathematics achievement quantitative ability or attitude for mathematics for eighth-grade girls in single-sex or coeducational schools. Implications for mathematics education for females are discussed. (Contains 6 tables and 62 references.) (SLD)

The convention for selecting starting points (that is, initial items) on a computerized adaptive test (CAT) is to choose as starting points items of medium difficulty for all examinees. Selecting a starting point based on prior information about an individual's ability was first suggested many years ago, but has been believed unimportant provided that the CAT is reasonably long. However, starting with a medium difficulty item for all examinees has two potential disadvantages: unnecessary uses of the first one or two items and overuse or overexposure of the items around the medium difficulty. This study analyzed simulated CAT results and suggests significant benefits from administering the first CAT item at a difficulty level suitable to each examinee. Such an adjustment can reduce the use of items around the medium difficulty in the item pool, providing extra help in controlling the exposure rate of the items beyond what standard exposure control methods can achieve. The effect of selecting examinee-appropriate starting points can vary depending on the quality of the information used about examinees' ability levels and the test termination rules applied. (Contains 3 tables, 26 figures, and 8 references.) (Author)

This paper explores the issue of why some students in the Caribbean do not do as well as expected in assessed groupwork. The study was based on peer assessment in 2 university courses taken by 57 students who worked in 8 groups (group sizes 4 to 10). Both males and females participated, their ages ranging from 20 to 46 years. The study was designed to focus on fundamental learning problems rather than on problems that might be associated with learning the content of a particular course. This was done by separating the assessment of the final quality of each group's work from the assessment of individual contributions to the work and by duplicating the study in two courses. Findings indicate that a fundamental problem associated with less able students is their lack of discrimination about what the work entails. These findings are robust in that the correlations show that the effect is not just apparent for low achieving students, but that the effect decreases as the ability of the student increases. Findings were the same across groups of different sizes and across content areas. Suggestions are made about how students might reduce this problem. (SLD)

A two-stage design provides a way of roughly adapting item difficulty to test-taker ability. All test takers take a parallel stage-one test, and based on their scores, they are routed to tests of different difficulty levels in the second stage. This design provides some of the benefits of standard computer adaptive testing (CAT), such as increased precision of ability estimates over a paper-and-pencil design. In addition, the item selection and scoring algorithms in a two-stage design may be easier for test takers and test-result users to understandan important feature for gaining public acceptance of new test designs. This simulation study incorporated testlets (or collections of items) into the two-stage design and compared the precision of the ability estimates derived from this design with those derived from a standard CAT design and from a paper-and-pencil test design. The results indicate that if the testlets are carefully assembled, a two-stage testlet design can produce more precise ability estimates in the middle ability range than those obtained from a paper-and-pencil design with twice as many items. Results of this study provide a baseline against which future research that incorporates content constraints can be compared. (Contains 6 figures, 5 tables, and 12 references.) (Author/SLD)

This paper describes a new, but tried and tested, paradigm in teacher evaluation: the Three Ability Framework (3AF). Previous research has identified three fundamental abilities that faculty implicitly expect to develop in their courses: technical skills, professional competence, and professional attitudes. The 3AF monitors the alignment of the student's and teacher's expectations of these developments. The process respects the professional freedom of lecturers to be responsible for how they believe their subjects should be taught. Because the process is so efficient, it can be used two or three times during courses to monitor the quality of teaching. This evaluation method also links student assessment to the professional awareness of the teacher, resulting in a single number that is used as an administrative decision point. The 3AF is applied in a five-step process that begins with training provided the lecturer by the institution. Alignment between student ratings and the lecturer's own rating is calculated. It has been found in previous research that the alignment of percentage ratings correlates with academic attainment as measured by examination and coursework and that the alignment of proportions of the three abilities correlates with enjoyment of the course. (Contains 14 references.) (SLD)

This paper presents an alternative means of assessing faculty teaching that can be used by administrators. This method has been derived from in-depth faculty and student interviews, and its criterion of quality has been validated with empirical data through computer sensitivity analysis. The assessment process has been successfully tested in clinical teaching trials. Interviews with faculty have identified that their implicit expectations for professional development can be described in terms of technical skills, professional competence, and professional attitudes. In practice, lecturers explain these three skill areas to their students and describe how students will be taught and assessed. Students are asked to rate the lecturer on these three areas, and then the lecturer rates himself or herself. The indicator of effective teaching, that the students and the lecturer are working toward the same changes, is measured by the change expected from the students subtracted from the change expected by the lecturer. A "zero" is the score that indicates perfect alignment. This method promotes a positive teaching and learning culture by encouraging teaching and learning that promotes students' critical and evaluative thinking. (Contains 31 references.) (SLD)

In conventional practice, most educators and educational researchers score cognitive tests using a dichotomous right-wrong scoring system. Although simple and straightforward, this method does not take into consideration other factors, such as partial knowledge or guessing tendencies and abilities. This paper discusses alternative scoring models: (1) credit for omissions; (2) disproportionate correction for wrong versus omitted items (correcting for guessing); (3) scoring only for items that a given examinee is expected to get right based on one-parameter item response theory (Lawson, 1991); and (4) scoring using various partial credit models, including misinformation. The literature regarding the utility of each algorithm, including validity and reliability, is also summarized briefly. Psychologists should be familiar with alternative scoring strategies, since such strategies can be useful in the design, administration, or analysis of results from measures of cognitive abilities, especially in high stakes testing. Findings from this exploration indicate that correction for guessing formulas do not show significant benefits over conventional scoring (no correction), and while results on partial credit scoring algorithms are inconclusive, the observed slight increases in reliability and validity do not justify the additional complexity, time, and cost involved in developing, administering, scoring, and interpreting test results. (Contains 1 table and 20 references.) (Author/SLD)

Mathematics achievement and the relationship of low socioeconomic status (SES) students with teachers have been a major concern for educators. Although there are social and economic explanations of failure of low SES students, ultimate success and failure may be significantly affected by differing personal beliefs about learning among students and teachers. C. S. Dweck's implicit beliefs about intelligence and mathematics were examined using a bi-directional methodology with R. D. Lang's Interpersonal Perception Method (IPM) in algebra 1. Incremental/entity beliefs, gender of students, gender of teachers, and low achievers and high achievers were subjected to an analysis of variance. Eighty low SES students and 43 male and 37 female teachers responded to the incremental beliefs that reflected abilities as changing and increasing with effort and to entity beliefs that reflected abilities as unchanging or fixed. Individual incremental female students confirmed the prediction that incremental believers would have higher final grades than the female students endorsing entity beliefs. Students matched with incremental-belief male teachers confirmed the dyadic prediction that incremental intelligence beliefs would lead to higher achievement than for a dyad endorsing entity beliefs. Dweck's personal theory integrated with the IPM examined several complex issues about real students' and teachers' experiences. Examining these experiences revealed some adaptive and maladaptive patterns that lead to math achievement for the low SES female student. Issues of methodology for examining student teacher relationships, power of projections over accuracy of perceptions (particularly for female students' expectations from their teachers), and the necessity of collecting data from both students and teachers for a clearer understanding of the issues affecting low SES students' math achievement are discussed. (Contains 1 table, 10 figures, and 44 references.) (Author/SLD)

Students taking the paper-based Scholastic Assessment Test (SAT) mathematics test are permitted to bring and use their own hand-held calculators, and this policy was continued for the computer-adaptive tests (CAT) designed for use in talent search programs. An on-screen calculator may also be used with the CAT. The bring-your-own option has raised some fairness concerns (because all students cannot afford the most sophisticated calculators) as well as security concerns (because questions could be entered into a calculator's memory and taken from the testing session), but forcing all students to use an unfamiliar on-screen calculator raises different fairness issues. This study of the computerized SAT compared the performance of 360 students tested under the current policy (bring-your-own or on-screen) with the performance of 373 students who had only an on-screen calculator available. These students had already taken a national administration of the SAT and were invited to participate. Across ethnic, gender, and ability groups, students who had to use the on-screen calculator performed as well as students who were permitted to use their own calculators. Nevertheless, students expressed a strong preference for using their own calculators. (Contains eight tables and one figure.) (Author/SLD)

A common procedure for obtaining multiple readings (ratings) for a constructed response item, especially in high-stakes tests, is to have two readers read the papers independently, with a third reading if the results differ by more than one point. This necessitates a scoring rule that specifies how the ratings will be aggregated into a single item score. Two plausible scoring rules involve averaging the readings and rounding either to the nearest half point or the nearest integer, but it is not known which results in a greater precision of measurement. This study investigated the precision and accuracy of ability estimates obtained under the two scoring rules for mixed format tests calibrated under an item response theory model. Eleventh-grade reading, mathematics, and science test results and a fifth-grade mathematics test result were analyzed, with more than 1,200 students available for each form. There was little substantive difference in score information or the standard errors of ability estimates due to the type of rounding (integer versus half point), above the floors of three of the four tests, but in the fourth (11th grade reading) there was less error in the integer-rounded ability estimates at the lower portion of the scale. Integer-rounded estimates generally produce slightly larger predicted percent of maximum (test) scores, though not throughout the entire ability range of all the four tests studied. The expected larger positive differences or rounding bias for number correct estimates were observed. Within-subject differences between scale score estimates derived using integer versus half-point scores were generally small for both pattern and number correct ability estimates. The lack of substantive improvement in measurement precision that could be attributed to half-point rounding, coupled with the documented instance of increased error induced by that type of rounding in a portion of the ability range of students taking one test, would seem to argue for rounding average ratings to the nearest integer. Rounding up gives the preponderance of students the benefit of the doubt concerning the acceptability of their responses. (Contains two tables, four figures, and eight references.) (SLD)

This aim of this study was to analyze the content of students' verbal interactions within a problem-based learning context in biology. This was achieved through the qualitative analysis of the verbal protocols of three groups of two clases of ninth-grade female students (average/high ability, high/high ability, and average/average ability). The three groups were audio and video taped as they worked collaboratively in the problem-based learning environment to solve a problem case created by another group of students. Data were transcribed and analyzed using the idea unit approach. Overall, these groups were successful at negotiating roles and materials to be used to help them solve the problem case. Members in all three groups alternated in terms of researching and obtaining information, reading materials, and note-taking. Few disagreements were found, and when they did occur, they were often quickly resolved. When the students were of equal academic ability, the division of power was shared, but in the average/high ability group, the higher ability student tended to dominate the conversation and assume a more authoritative role. The implications of these findings, particularly of the inequality in participation among students of differing academic abilities, for instruction are discussed. (Contains 2 tables and 12 references.) (SLD)

Latent trait models introduced the concept of the latent trait, or ability, as distinct from the test score. There is a recent tendency to treat the test score as through it were a substitute for ability, largely because the test score is a convenient way to place individuals in order. F. Samejima (1969) has shown that, in general, the amount of test information decreases if the test score is the estimate of ability. This paper introduces a new family of models that has a high level of substantive validity and inner consistency in ordering individuals. This family is called the logistic positive exponent family (LPEF). The development of these models is traced, and how to define the item characteristic curve in the LPEF family is demonstrated. One of the most important characteristics of the LPEF family is that a point-symmetric (logistic) model is treated as one of the infinitely many models of the family. LPEF provides more appropriate models for human behavior those based on error distributions. (Contains 2 tables, 5 figures, and 12 references.) (SLD)

Three aspects of the usual approach to assessing local item dependency, Yen's "Q" (H. Huynh, H. Michaels, and S. Ferrara, 1995), deserve further investigation. Pearson correlation coefficients do not distribute normally when the coefficients are large, and thus cannot quantify the dependency well. In the second place, the accuracy of item response theory person ability estimates is always relative to the standard error of the estimation, and in the third place, Yen's approach does not have a criterion to determine the significance level of the index of dependency between a pair of items (Q3). This paper addresses these concerns by proposing Fisher's "Z" as the index for item dependency. It compares the statistical properties of Fisher's Z with Yen's Q and weights the observed residuals by the standard error of person ability. Items for the study were from the second administration of part of a national medical licensing examination in 1995 taken by 361 first-time test takers. Fisher's Z accurately estimated the zero mean dependency for stand-alone items and successfully differentiated stand-alone items from clustered items based on the degree of dependency. This suggests that Fisher's Z is a valid, reliable, and sensitive index for quantifying item dependency. Two appendixes provide a sample of stand-alone items and a sample of item cluster. (Contains four figures and three references.) (SLD)

This paper explains the multifacet technology for analyzing performance examinations and the fair average method of setting criterion standards. The multidimensional nature of performance examinations requires that multiple and often different facets elements of a candidate's examination form be accounted for in the analysis. After this is accomplished, all candidate ability estimates are located on the same scale. The fair average standard setting method, while substantially different from more traditional methods, provides a criterion standard in a score metric that is easy for most people to understand. Yet, it accounts for the influence of the particular facets elements in each test form, and can be used to establish a pass point that applies appropriately to all candidates regardless of the raters or tasks or problems on each candidate's examination. Performance examinations are extremely complex because of the number of possible examination forms that may occur during an administration. The technology provided by computer programs, as exemplified by the FACETS program, is essential to sorting out the impact of each facet element on candidate scores and setting a criterion standard that accounts for the impact of all examination facets. (Contains two figures, three tables, and nine references.) (Author/SLD)

This paper illustrates a method to study rater severity across exam administrations. A multi-facet Rasch model defined the ratings as being dominated by four facets: examinee ability, rater severity, project difficulty, and task difficulty. Ten years of data from administrations of a histotechnology performance assessment were pooled and analyzed to establish a scale. Next, the 17 individual administrations were anchored to that scale and reanalyzed. The severity of the nine raters who graded most often were listed and plotted by administration. These plots show the consistency of each rater's level of severity. Results show that: (1) raters have an individual level of severity; (2) raters can usually maintain that level; and (3) some raters maintain their level of severity more consistently than others. The implications for equating performance assessments prospectively are discussed. (Contains 2 tables, 10 figures, and 5 references.) (Author/SLD)

This paper presents a detailed description of maximum parameter estimation for item response models using the general EM algorithm. In this paper the models are specified using a univariate discrete latent ability variable. When the latent ability variable is discrete the distribution of the observed item responses is a finite mixture, and the EM algorithm for finite mixtures can be used. Maximum likelihood estimates of the item parameters and of the discrete probabilities of the latent ability distribution are given using the EM algorithm for finite mixtures. Results are presented in general for both dichotomous and polytomous item response models. The relation between the EM estimates and the Bock Aitken marginal maximum likelihood estimates is discussed. Estimates for the item parameters will depend on the specific form of the item response functions, and will usually require iterative numerical procedures. The EM algorithm is the same as the Bock-Aitken algorithm (R. D. Bock and M. Aitken, 1981) for marginal maximum likelihood estimation of the item parameters. (Contains 28 references.) (Author/SLD)

Dichotomous item response theory (IRT) models can be viewed as families of stochastically ordered distributions of responses to test items. This paper explores several properties of such distributions. The focus is on the conditions under which stochastic order in families of conditional distributions is transferred to their inverse distributions, from two families of related distributions to a third family, or from multivariate conditional distributions to a marginal distribution. The main results are formulated as two theorems that apply immediately to dichotomous IRT models. One theorem holds for unidimensional models with fixed item parameters. The other theorem holds for models with multiple abilities or with random item parameters as used, for example, in adaptive testing. (Contains 2 tables and 36 references.) (Author/SLD)

This study demonstrates the utility of a HYBRID psychometric model, which incorporates both item response theoretic and latent class models, for detecting test speededness. The model isolates where in a sequence of test items examinee response patterns shift from one providing reasonable estimates of ability to those best characterized by a random response pattern. The study applied the HYBRID model to three distinct data sets: (1) simulated data representing the performance of 3,000 examinees on a 70-item test; (2) data from a statewide field test (n=5,997) of a 40-item reading comprehension test with a fixed time limit; and (3) data from a study of urban university students (n=752) who took a similar 40-item reading comprehension test under varying time limits. The HYBRID model successfully identified "switch points" in examinees' item response patterns in all three data sets. This paper discusses applications of the model used to detect speededness and to provide adjusted estimates of item parameters and examinee abilities. (Contains 4 tables, 4 figures, and 38 references.) (Author)

Although computerized adaptive tests acquire their efficiency by successively selecting items that provide optimal measurement at each examinee's estimated level of ability, operational testing programs will typically consider additional factors in item selection. In practice, items are generally selected with regard to at least three, often conflicting, goals: (1) to maximize test efficiency by measuring examinees as quickly and accurately as possible; (2) to protect the security of the item pool by controlling the rates at which popular items can be administered; and (3) to assure that the test measures the same composite of multiple traits for each examinee by balancing the rates at which items with different content properties are administered. This paper focuses on the goals of maximizing test efficiency and controlling item exposure rates, avoiding a discussion of content balance. Problems in existing algorithms for accomplishing these goals are outlined and illustrated, and some alternative algorithms that offer at least a partial solution are presented. Posterior weighted information is suggested as a new item selection method, and its usefulness is demonstrated through a simulation. Conditional exposure control is suggested to control exposure rate, and a similar simulation is presented to demonstrate its usefulness. (Contains one table, seven figures, and seven references.) (SLD)

The Appalachia Educational Laboratory qualitative study of the implementation of five aspects of the Kentucky Education Reform Act in four rural school districts is now in its fourth year. This paper focuses on findings concerning the first year of the primary program, which was put into widespread operation in 1992-93. Nongraded primary programs are neither a new idea, nor unique to Kentucky, but the Kentucky program is a large-scale attempt to implement such a program statewide. Results from eight rural schools in the four target districts indicate that the most successfully implemented of the previously identified critical attributes of such a program are (1) developmentally appropriate instructional practices; (2) multi-age and multi-ability classrooms; (3) authentic assessment; (4) qualitative reporting methods; (5) professional teamwork; and (6) parent involvement. Least successfully implemented was the seventh identified attribute, continuous progress. A number of difficulties with the program, primarily in the areas of teacher education and teachers' time constraints and work loads, are identified. Most teachers have made an effort in good faith to implement the program and can do so with adequate support. (Contains 12 references.) (SLD)

The issue of test-item selection in support of decision making in adaptive testing is considered. The number of items needed to make a decision is compared for two approaches: selecting items from an item pool that are most informative at the decision point or selecting items that are most informative at the examinee's ability level. The first approach is explored using a method based on sequential Bayes estimation and the second is based on the sequential probability ratio test (SPRT). Simulation studies used item parameters from an actual mathematics item pool developed for COMPASS, the American College Testing Program's computerized adaptive-testing package for college placement. Results could not be compared directly because matching the decision error rates for the two procedures is beyond the scope of the paper. Results do provide clear evidence that selecting items to maximize information at the decision points results in shorter average test lengths than selecting items to maximize information either at true ability (which is impossible) or at the most recent ability estimate. It is expected that these results will generalize to many adaptive-testing applications. (Contains 14 references and 2 figures.) (SLD)

This paper evaluated multidimensional linking procedures with which multidimensional test data from two separate calibrations were put on a common scale. Data were simulated with known ability distributions varying on two factors which made linking necessary: mean vector differences and variance-covariance (v-c) matrix differences. After the calibrations of multidimensional item parameters, blocks of means from item parameter estimates were used to equate the two groups. The linking was effective for mean vector differences. The linking for v-c matrix differences was less effective, but encouraging. Suggestions for future research are provided. Four tables are attached. (Contains 6 references.) (Author)

It is a common practice in item response theory (IRT) to treat estimates of item parameters, say "B" circumflex, as if they were the known, true quantities, "B." However, ignoring the uncertainty associated with item parameters can lead to biases and over-confidence in subsequent inferences such as ability estimation, especially when item-calibration samples are small. This paper demonstrates how to incorporate uncertainty about "B" with Lewis's "expected response functions" (ERFs), pointwise expected values of item response conditional on examinee proficiency averaged over posterior distributions of item parameters. This paper presents ERFs, outlines procedures for computing them and using them in practical work, and gives an illustration with data from the National Assessment of Educational Progress. Advantages of approximating ERFs response curves with members of familiar parametric families of IRT curves are noted. Two appendices present work on pseudolikelihood estimations and program documentation for two computer programs: EXPRESFN and PLOTIRF. (Contains 3 figures, 3 tables, and 18 references.) (Author/SLD)

How different evaluation formats affected test scores for different tracks of high school biology students was studied in an action research case study. For purposes of the study, "A" tracked students were generally college-bound; "B" tracked students were those who had no desire to attend college, especially those in vocational education, or who were below average students whose advisors thought they would not be able to pass a "college-bound" course. Forty-eight A-tracked students and 45 B-tracked students were the experimental groups. Twenty-nine A-tracked students and 42 B-tracked students served as comparisons. Students in the experimental group took the same test control group students did, but, in addition, they were given an additional three essay questions. Their attitudes about the essay test evaluations were noted. Results suggest that adding essay questions to objective tests supplied by textbook publishers is recommended for A-tracked students, who performed well and liked the essay format. However, B-tracked students found the essays difficult and did not do well. The disadvantages of the essay format outweighed the advantages for the students in the B track. Action research was a good approach to investigating the effects of different evaluation formats for students of different levels of ability. (Contains 26 references.) (SLD)

Essay questions and the effects of their use on student scores are examined by a teacher through collaborative action research. Collaboration with the teacher's graduate school faculty resulted in refinements to the research question and development of a practical study methodology. During the two years of the study, the teacher taught four science classes of college-bound high school students of average and above average ability and four classes of science for vocational students of average and below average academic ability. In any one term, he would use essay questions in two classes and not use them in the other two. A total of 77 college-bound students and 97 vocational students completed the study. In general, essay questions, when used, accounted for about 25% of the overall test score. Time of day the class was held had no effect on student achievement, and, as expected, the classes that took the same textbook-supplied objective tests had similar average scores and attitudes about testing. When essay tests were used, the college-bound students' test scores went up almost 10% overall, with increased interest in studying for tests and increased motivation for learning the material. In the vocational group, the overall average test scores decreased by over 10%, with 5 students decreasing their scores for every one that increased his or her score. Students expressed dislike for the essay questions and tended to leave them unanswered. Additional research is necessary regarding the use of essay questions with below-average students. (Contains 51 references.) (SLD)

This study assessed whether providing examinees with a choice between computerized adaptive testing (CAT) and self-adaptive testing (SAT) affects test performance in comparison with being assigned a CAT or SAT, and evaluated variables influencing examinee choice of either test form. The relative influences of test type and test choice on examinee anxiety were also examined. Subjects were 244 undergraduate and 133 graduate students from a large midwestern university. Students were randomly assigned to SAT, CAT, and choice conditions for an algebra test. Test-related anxiety was assessed with a paper-and-pencil measure in pretests and posttests. It was found that, for students with high mathematics anxiety, providing a choice between CAT and SAT led to significantly higher mean proficiency estimates, lending support to the hypothesis that examinees can cope with a stressful situation more effectively if they feel that they have some control over the source of the stress. Expected differences in estimated proficiency and posttest state anxiety between CAT and SAT conditions were not found, but a strong relationship was seen between examinee test type choice and mathematics anxiety level. Higher anxiety examinees have a greater preference for the control provided by SAT. Six tables and two graphs summarize findings. (SLD)

The effects of previous mathematics, statistics, and computer science coursework; attitudes toward statistics and computers; and mathematics ability on statistics achievement were studied for 289 college students over 4 semesters. A secondary purpose of the study was to determine the effect of the computer laboratory component of an inferential statistics class on students' end of course attitudes. Instruments were administered to determine attitudes toward statistics and computers, anxiety about statistics, algebra and mathematics skills, and biographical data. The proportion of variance in statistics achievement accounted for by the following parameters was not statistically significant: (1) attitudes toward computers; (2) attitudes toward statistics; (3) mathematics background; (4) computer science background; (5) mathematics ability; (6) statistical anxiety; and (7) Graduate Record Examination scores (available for only 83 students). Coefficient alpha reliabilities are reported for each of the scales used to measure attitudes and abilities. No statistically significant differences were found between students taught with a computer laboratory and those taught without the computer component for attitudes toward statistics, but those taught by computer exhibited more positive attitudes toward the computer and less statistical anxiety at the end of the course. Five tables present study findings. Contains 18 references. (SLD)

The relative contributions of four individual difference variables to ability to solve analogically related problems in four separate problem sequences and for spontaneous and assisted transfer was examined. Subject were 116 university undergraduates enrolled in an introductory psychology course who volunteered to participate for course credit and 65 high school seniors who volunteered to participate for extra credit from their social studies teacher. Specifically examined was whether combinations of individual difference variables served as significant predictors of problem-solving success in four different source-target problem sequences. For spontaneous transfer, the results indicate that under certain circumstances different combinations of individual difference variables contribute to the ability to predict whether a subject would provide a particular solution to a target problem. For assisted transfer, individual differences do not account for the ability to predict whether a subject would provide a particular solution to a target problem. Results are discussed in terms of why the individual difference variables enhance prediction for spontaneous transfer only under certain circumstances. In addition, suggestions for pursuing this line of inquiry are provided. Four tables present study data. Appendix A contains a logic problem from the study, and Appendix B contains the source and target analogues. (SLD)

This paper reports a method for assessing the adequacy of existing item banks for computer adaptive testing. The method takes into account content specifications, test length, and stopping rules, and can be used to determine if an existing item bank is adequate to administer a computer adaptive test efficiently across differing levels of examinee ability. An example is presented that shows that the adequacy of the bank can depend on the stopping rule implemented. The example is from an item bank with 183 items from 4 content subtests, with 50% of items from subtest 1, 24% from subtest 2, 20% from subtest 3, and 6% from subtest 4. The use of information functions for both subtests and the total test gives a picture of the adequacy of the item bank across content areas. The procedure can be modified for use with other item response theory models as long as the item parameters are known. Ten figures illustrate the discussion. (Contains 13 references.) (Author/SLD)

In 1990 a routine study confirming the accuracy of the Rasch model in predicting item difficulties led to an in-depth analysis of the impact of guessing on the Rasch model when used with multiple-choice items. This paper reviews the highlights of that research. Seventeen linked vocabulary tests of 110 items were each administered to groups of 400 to 600 elementary school students. The obtained data were then analyzed using the BIGSTEPS computer program (B. Wright and M. Linacre, 1992) in order to obtain item difficulties and person ability estimates for the 1,304 items and 7,711 students. The actual performance of each person-item interaction was compared to predicted performance at each interaction point in order to construct an empirical item characteristic curve. The empirical curve was compared to the theoretical Rasch based curve and the discrepancies noted. When restrictions were placed on the analysis, the differences between the empirically derived curve and the theoretical curve were decreased. (Contains two figures, two tables, and five references.) (Author/SLD)