Search

This study assessed the strengths, weaknesses, and degree of implementation of the Manitoba (Canada) Physical Education curriculum in grades 4, 8, and 11. This report presents the conclusions and recommendations resulting from a review and interpretation of the findings by the Physical Education Educators on the grade level Technical Advisory Committees. At each grade level, the assessment consisted of a student performance test, a student written test, and a teacher survey. The six major learning goals at each grade level were consistent even though the specific objectives varied from one grade level to another; the fitness portion of the performance test was consistent across the three grade levels as well. In general, results were positive and indicated that the curriculum was well covered, with males showing higher fitness than females, except in flexibility. At Grades 4 and 8, a sizeable proportion of students did not receive the recommended minimum of time for physical education classes; at all three grade levels, there was a de-emphasis in Rhythm and Dance, Aquatics, and Gymnastics. In reporting to parents, evaluation concentrated on behavior, participation, and effort; skills, fitness, and knowledge of concepts received little attention. The level of teacher qualification correlated with teaching assignment; the least qualified teachers were found at Grade 11. Appendices include the detailed procedures of the Physical Education Assessment and a listing of the members of the Technical Advisory Committees. (NAV)

This study assessed the strengths, weaknesses, and degree of implementation of the Manitoba (Canada) Physical Education curriculum in grades 4, 8, and 11 in French Immersion Schools. This report presents the conclusions and recommendations resulting from a review and interpretation of the findings by the Physical Education Educators on the grade level Technical Advisory Committees. At each grade level, the assessment consisted of a student performance test of fitness and motor skills, a student written test of knowledge and attitudes, and a teacher survey; there was no test of motor skills at Grade 11. The six major learning goals at each grade level were consistent even though the specific objectives varied from one grade level to another; goals were physical fitness, basic movement skills, self-expression, independence, safety, and social interaction. Apart from certain weaknesses in Grade 11, all students demonstrated a good grasp of the subject matter covered in the curriculum with positive attitudes found toward physical activity and Physical Education classes, yet many students were found to be at risk for health related fitness. Student evaluation centered mostly on behavior with observation the most common assessment method; little student self-assessment was found. It appeared that the curriculum lacked precision in form as well as content and that most physical education teachers used the curriculum guide for reference only; professional development needs were ubiquitous. Lack of knowledge about safety measures was found in Grades 8 and 11, and unsportsmanlike behavior was found in Grades 4 and 8. A listing of the members of the Technical Advisory Committees is appended. (NAV)

Gifted underachievers (n=108) were compared to gifted high achievers (n=96). All children had Wechsler Intelligence Scale for Children - Revised (WISC-R) IQ scores of 130 or greater, but underachievers were performing at or below the 50th percentile in at least one major area of achievement, whereas high achievers were at the 96th percentile or greater in three areas of achievement: language, math, and reading. Results of analysis of variance of achievement level X WISC-R subtests revealed significant differences in scores on four verbal subtests: Information, Similarities, Vocabulary, and Comprehension. High achievers had significantly higher verbal, but not performance, IQ scores than underachievers. However, comparison of the verbal IQ-performance IQ discrepancy distributions for the two groups revealed no significant differences, negating the idea that a large verbal/performance IQ discrepancy can be used as an indicator of risk for low achievement in gifted children. Analysis of gender, ethnicity, and risk revealed a greater concentration of non-Caucasian males with at least two risk factors in the underachieving group. Present findings are consistent with earlier findings concerning the importance and discriminating power of the Information subtest in distinguishing high versus underachievers. The findings indicate that gifted underachievers are not as motivated or interested in acquiring traditional factual information as high achievers. (Contains 242 references.) (DB)

This document contains a project report and procedures manual from a project in which a group of junior colleges and postsecondary technical schools in Texas developed, pilot tested, and disseminated performance criterion-referenced occupational examinations for technical occupations. The report describes how teachers and industry personnel collaborated in developing the model and four competency-based examinations for the following subjects: drafting, electronics and instrumentation, food production, and microcomputer applications, as well as a model procedure manual for developing and conducting the examinations. The model procedure manual includes the following: (1) an introduction that explains the rationale for competency-based instruction and highlights pertinent sections of Perkins legislation that apply to technical education and competency-based testing; (2) a discussion of occupational competency testing; (3) directions for developing occupational competency examinations; and (4) eight appendixes--definitions; an example of a survey to validate tasks or competencies; samples of occupational examinations in computer science, drafting, food production, and electronics and instrumentation; project abstracts for seven U.S. Department of Education and Department of Labor grants; and a list of nine selected references. (KC)

A national study examined the system of performance measures and standards that had been approved in each of the 50 states, the District of Columbia, Puerto Rico, Guam, and the Virgin Islands in response to the 1990 Perkins Act. Descriptive survey and content analysis research methods were used to develop summary sheets categorizing the types of performance measures approved by each state/territory and the methods adopted to assess performance. The numbers of performance measures approved for secondary and postsecondary vocational education averaged 10 and 8, respectively. Most states (30) had developed a single set of measures to assess both basic and advanced academic skills at the secondary level. Other areas in which performance measures had been approved included the following: competency attainment, work skill attainment, program completion, high school graduation, placement, percentage served, and gender mix. In nearly every state, systems of core standards and measures of performance for secondary and postsecondary education had been developed and implemented in line with the requirements of the 1990 Perkins Act; however, the standards, performance measures, and types of assessment techniques approved by each state varied greatly from state to state. (Contains 13 tables, 26 references.) (MN)

A general evaluation design was developed to examine the effectiveness of a computer-based, multimedia simulation test on California smog check mechanics. The simulation test operated on an Apple Macintosh IIci, with a single touchscreen color monitor controlling a videodisc player; it had three parts: introduction-tutorial-help, data, and test. It was compared to a hands-on, actual vehicle test. The two tests were as equivalent as possible. The hands-on test was essentially a printout of the nine test items and specific questions from the simulation test. The evaluation gathered two kinds of data: test performance and subjective responses to the test experience. Currently certified smog check mechanics were recruited and paid to participate in the study. This sample was randomized between the two testing methods; 38 were tested in each group. A Rasch analysis was chosen for a reliability study of the test. The high item separation reliability indicated that, in general, persons responded consistently to the test according to their performance capabilities. The comparison of the performance between the simulation test group and the hands-on test group indicated the difference was not statistically significant. Data showed that the probability was very high that the differences between the group means were the result of chance. (Appendixes which comprise at least half of the document include the following: background and simulation test documents, mechanics survey, sample selection documents, raw data score sheets, and test proctors' observations/recommendations.) (YLB)

The use of the Wechsler Intelligence Scale for Children, Third Edition (WISC-III) and its interpretation in educational use are discussed. To measure intelligence, Wechsler believed one must measure the various aptitudes that contribute to the total behavior of the individual. The WISC-III has six verbal subtests and seven performance subtests. The verbal intelligence quotient (VIQ) is obtained by adding the scaled scores of all the verbal subtests except Digit Span, and the performance IQ (PIQ) is derived from five of the performance subtests. The PIQ is not as good a predictor of school achievement as is the VIQ. Indexes derived from the WISC-III subtests include: (1) verbal comprehension; (2) perceptual organization; (3) freedom from distractibility; and (4) processing speed. Subtest scatter and possible causes for significantly high and low subtest scores are explored and include reasons for differences between VIQ and PIQ. Factors that have been identified based on the WISC editions are listed, and ways in which subtest scores may suggest possible brain damage and emotional disturbance are reviewed. A worksheet is described that can enable the evaluator to examine a number of WISC-III factors for making placement and curriculum recommendations. (Contains 44 references.) (SLD)

Industrial and organizational psychologists for the Department of Defense have been working for the past 10 years to develop high fidelity measures of job performance for use in validating job selection procedures and standards. Information on developing and scoring performance exercises in the Job Performance Measurement (JPM) Project is presented, and lessons that might be useful in education are extracted. In many ways, the task of the industrial psychologist is easier than that of the educator because of broader agreement about how the task should be performed and close alignment between training and expected performance. Tasks identified by each Armed Service were analyzed, and scoring rules were developed. The following lessons seem especially pertinent to educational assessment: (1) careful specification of the domains assessed is essential for evaluating the adequacy of any sample selected; (2) scoring elements that assess adherence to processes that are taught will have better diagnostic value (and possibly greater validity) than will those that just reflect the quality of output; (3) scoring procedures must be anchored to observable criteria; and (4) generalizability theory provides a useful framework for evaluating alternative scoring rubrics. One table lists the JPM occupational specialties, and two figures illustrate the discussion. An attachment summarizes the lessons to be learned. (SLD)

The instructional validity of computer administered tests was studied with a focus on whether differences in test scores and item behavior are a function of instructional mode (computer versus non-computer). In the first of 3 studies, performance test scores for approximately 400 high school students in 1990-91 for tasks accomplished with the computer were correlated with objective paper-and-pencil test scores for comprehension of the same tasks. In the second study, a comparison was made between test scores of 77 high school students in 1991-92 taking an objective test via computer and scores on the same test in paper-and-pencil format. In the third study, the item characteristics of tests given in a computer format were compared to the item characteristics of the same tests presented in a paper-and-pencil format. Data from the first two studies were used. Findings from the three studies indicate that performance tests for computing are not equivalent to objective paper-and-pencil tests even when the content is the same. Test scores and item statistics for tests in a paper-and-pencil format and computer format do not differ for content taught in a combined computer and lecture mode. Whether students not taught by computer are hampered by computer-based tests is a logical extension of this research. Seven tables present study findings. (SLD)

Findings of a study that investigated the relationship between the socioeconomic characteristics of Ohio school districts and district test performance are presented in this paper. Methodology involved ANOVA analyses of data derived from the 611 Ohio school districts that participated in the state's 1989-90 testing program. The independent variable was district wealth and the dependent variable was district performance based on students' scores on the state's reading, language, and mathematics tests. Wealth was operationally defined as the average family income per tax return in the first analysis, and as the percentage of district families receiving aid to dependent children (ADC) in the second analysis. Findings show that district performance rankings were largely a measure of family wealth, which did not reflect school effectiveness or instructional quality. School districts should not be ranked according to test scores that are highly influenced by wealth, but on the basis of three categories of educational quality: access to knowledge; district expectations; and the teaching environment. Two tables are included. (LMI)

The microcomputer has increased interest in performance testing, which samples what a person can do rather than what he or she knows. Conventional psychometric theory is based on knowledge tests, but in performance testing the unit of analysis is a trial, and it is unreasonable to assume that mean performance and interim correlations are independent of order of administration. For example, performance typically improves with practice. Both reliability and temporal stability frequently encounter optima as a performance test is lengthened. Scoring all trials administered may not yield the best predictive validity; rather, scoring a subset of trials frequently yields higher predictive validities. Subset analysis serves the same ends in performance-test theory as does item analysis in conventional psychometrics. Serial averaging and its applications (reliability and stability optima, optimal scoring for predictive validity, and subset analysis) are explained and illustrated. Results from the Project-A computer-administered tests, served as the database. Ten performance tests were used with two samples of college undergraduates (102 in each) and samples of Army enlisted people ranging from 8,892 to 9,269. Twenty-three references are included, and 12 tables and 11 graphs provide illustrative data. (SLD)

Drawing tests are often used to provide confirmatory and supporting evidence of intelligence or intellectual functioning. The relationship between scores on these types of instruments and academic achievement was studied among graduate students seeking degrees in education. The sample consisted of 125 graduate students in education (93 women and 32 men). Subjects had to draw a lady walking in the rain, a standardized measure that is not difficult to score. "Lady Walking in the Rain" protocols were scored by the researcher and correlated with achievement measures including grades and scores on achievement tests. Results indicate that scores on human figure drawing tests are not consistent with other measures of achievement in these adult students. There was considerable variation in scores on the various measures. Using such a "language free" measure of intellectual functioning requires additional study prior to acceptance for adult subjects. One table presents descriptive statistics and a correlation matrix. An appendix contains the scoring system for the drawing task. (SLD)

A demographic and educational profile was constructed of learners enrolled in California employment training programs. Summary data were derived from 47,318 Employability Competency System (ECS) Appraisal answer sheets received by the Comprehensive Adult Student Assessment System (CASAS) office from 1987 through 1990. ECS provides reading, math, and communication assessment linked to competencies and competency-based curriculum materials appropriate for adult basic education, pre-employment/work maturity, and English-as-a-Second-Language programs. ECS Appraisal forms the foundation of the system. ECS reading and math appraisal provides an initial assessment of a learner's basic functional literacy skill levels in a training and employment context. (Demographic and test performance information is summarized. The demographic profile includes self-reported information on the participants' gender, age, ethnic background, and native language. The educational profile presents attainment information relating to the highest grade level completed and the highest degree earned by the respondents. Demographic and educational profile data are cross-tabulated with test score performance in reading and math. Two appendices are included: (1) components of ECS; and (2) participating counties and service delivery areas.) (NLA)

Construction of a work sample test, the Investigator Planning Exercise (IPE), for the job of detective in the Chicago (Illinois) Police Department is described. Simulated crime scenarios, a mock crime scene, and five checklists of necessary skills (i.e., ability to summarize and communicate facts, identify inconsistencies, and determine the next action) were prepared. To screen the maximum number of candidates, the IPE was designed to be administered to between 400 and 600 applicants in a single day by 50 boards of 3 raters each. Safeguards included using raters who did not know each other, assigning as many minority and female raters as possible to the boards, allowing the applicant to reject up to two boards for cause, and requiring raters to explain extremely divergent ratings on the six scales making up the evaluation. In all, 189 police sergeants received rater training, which began with a 5-hour classroom session examining a similar selection process. The process for youth officers was used. Rater trainees viewed videotapes of mock review boards and practiced rating applicants. Although raters were not given the actual checklists, they did receive explanations of restrictions on the rating process. Additional training was given in a 2-hour hands-on session in which the actual check lists were reviewed. Trainees were also offered optional training in the test process in 15 2-hour sessions introducing test-taking strategy. On the test day, 619 applicants were tested. The psychometric results are reviewed briefly. Four tables of study data and an eight-item list of references are included. (SLD)

Homeless individuals are eligible for adult education services, such as literacy training and basic skills remediation, under the Stewart B. McKinney Homeless Assistance Act. In California, 7 agencies in 1989 and 13 in 1990 were selected as service providers to serve 100 homeless adults each per project year. Each funded agency assessed individuals' basic skills using vocational assessment instruments and the Employability Competency System (ECS) Appraisal developed by the Comprehensive Adult Student Assessment System (CASAS). Demographic characteristics--gender, ethnicity, age, education, and native language--were compiled for 542 participants in 1989 and 1,686 in 1990. CASAS test scores for reading and math at four levels of functional literacy were determined. In 1989, a Goal Attainment Survey received 840 responses. Participants reported the main reasons for enrolling: 29% wanted to obtain a better job; 44% reported education as a primary goal; and 26% reported improvement of personal/communication problems as a primary goal. (Tables and charts present demographic and test score data. Test results reported on the CASAS scale are divided into four functional levels: (1) below 200; (2) 200 to 214; (3) 215 to 224; and (4) above 225. Goal attainment by reason for enrollment, the goal attainment survey, and the ECS Appraisal form are included in the 1989 report.) (NLA)

In the domain of performance assessment, the errors-and-omissions (EOT) test falls between a work-sample test and a simulation test. The examinee works with a sample of material from the work environment. The correct answers and the exact criteria for acceptable performance are known in advance. For this study, EOTs were used for assessing entry level teachers in the areas of commercial art and furniture repair and upholstery. Thirty examinees were drawn from the pool of existing experienced teachers in Florida. A committee of six to eight experts in each field identified the competencies and skills needed to be an effective teacher. The committee members modified hypothetical examples of EOT tests, which were reviewed by additional experts in the fields, and used the examples in pilot testing with one teacher from the pool for the furniture repair test and three for the commercial art test. There were three raters for each test. The level of rater agreement was good. Future research will require larger samples to test the methodology proposed. Four appendices provide fictitious examples of commercial artist examinations. (SLD)

One of the goals in the Commission's Performance Assessment Framework for higher education is that all high school students in California receive an education that gives them the opportunity to enter a college or university. There are many possible measures that could be used to gauge preparation for higher education. These include SAT scores, number of Advanced Placement (AP) courses taken, standardized test scores, and completion rates of the a-g college preparatory curriculum. It is not always clear from the data which of these measures is most closely related to progress in higher education. Commission staff are developing an analytic method that looks at the relationship between the different measures available and progress in higher education. This analysis will help staff in selecting a measure that is the best gauge of movement toward the goal of giving students from all backgrounds the opportunity for higher education. This fact sheet reports progress on a study exploring academic proficiency scores of public high school students, as these measures are linked directly to the knowledge and skills that students need to acquire to be successful at the university level. Staff found that a measure based on scores in high school proficiency tests correlates slightly better with university student persistence than does the school's Academic Performance Index (API). A distinct advantage of the Commission's measure is that it can be calculated by gender and by ethnic/racial group within schools. Thus, it can be used as an indicator of how well schools are doing in giving all students an education of a quality that prepares them for higher education. API is available only for a school as a whole, so it is not a good indicator of differences in the quality of preparation within a school. (Contains 3 displays.)