Search

An evaluation was made of an educational television series called "Infinity Factory." The series was designed to teach elementary school children various math skills useful in everyday life. This volume consists of the appendixes to the evaluation report. It includes abstracts of the analysis of the first eight shows, pretest/posttest instruments, various rating scales and questionnaires, and the raw data that were used in the evaluation. Program guides for the eight shows that were studied are also included. (JY)

This document reports efforts to address some of the problems of competency-based assessment by developing and utilizing unique measures of competence. Seven conceptual guidelines are identified for test development and utilization: (1) new competencies must be identified and operationally defined; (2) new competencies should have general significance to a wide variety of career and life outcomes; (3) new definitions of competencies and measures developed for their assessment should be easy for faculty and students to understand; (4) competencies should be empirically linked to external realities; (5) the discovery of new ways of measuring competencies should acknowledge levels of performance required for entry into roles outside of academic settings; (6) standards for awarding credentials should acknowledge levels of performance required for entry into roles outside of academic settings; and (7) new attempts to define and assess learning outcomes should not be guided solely by attempts to make them functionally equivalent substitutes for traditionally assessed school achievement. A number of measures were developed along these guidelines; they are organized according to three outcome domains--cognitive, effective, and social. Descriptions of these measures are presented, and their applicability for competency based education and use in postsecondary institutions is discussed. Finally, the critical concept of the meaning of measurement is discussed along with some additional problems to be faced by educators, researchers, and funding agencies if the problems of assessment in higher education are to be adequately redressed. (MM)

The efficacy of small group instructional activities dealing with transitive inference, conservation, and class inclusion of length was investigated in a sample of kindergarten (N=44) and first grade (N=45) children (mean ages of 5 years, 7 months and 6 years, 7 months respectively). The two instructional sessions (10 to 15 minutes each) employed a game-like atmosphere, peer-groups/teacher interaction, and positive feedback within a small group context. Assessment of instructional effects included pretesting, immediate posttesting, and delayed posttesting (three months subsequent to training) on the above concept domains in addition to a quasi-standardized series of tasks based upon the concrete operations period "groupements." Preliminary analyses indicated a lack of order of presentation effects, sex differences, tester and teacher biases, and pretest distinctions among the experimental conditions (six treatment conditions, three control conditions). The normative order of item difficulty and the associated differential instructional susceptibility indicated that concepts of transitive inference precede conservation which are in turn mastered prior to class inclusion understanding. Apparent task ceiling effects precluded transitivity instructional effects. Significant specific instructional effects were shown only for the conservation training conditions. Intraconcept transfer to the counterpart weight concept cases was not shown. In relative contrast, some evidence for interconcept generalization was demonstrated for the conservation and transitivity concept domains and to the logical "groupement" performances. Little evidence was found for significant treatment/grade-level interactions. Implications of the present findings for the status of cognitive stages of development are discussed. (MS)

This report assesses the progress of the six summative Home Start projects as evaluation families completed their first twelve months of enrollment. Home Start, a federally-funded 3-year (1972-1975) home-based demonstration program for low-income families with 3- to 5-year-old children was designed to enhance a mother's skills in dealing with her own children and to provide comprehensive social-emotional, health and nutritional services. Aspects examined in this report include: overall national statistics on families, staff, and costs; organization and staffing of local projects; kind and quantity of services delivered; operational problems previously identified; mother and child outcomes; descriptive program costs; program cost-effectiveness; and cost-relevant program characteristics. The data examined in this report were gathered in fall 1974, through site visits, outcome measures, and quarterly information reports. Findings are presented in three different parts of this volume: program analysis results, summative evaluation results and cost-effectiveness results. (Author/MS)

These summative evaluation results are part of the National Home Start Evaluation interim report. Home Start, a federally-funded 3-year (1972-1975) home-based demonstration program for low-income families with 3-to 5-year-old children was designed to enhance a mother's skills in dealing with her own children and to provide comprehensive social-emotional, health and nutritional services. The data presented in this report were collected from 434 families in six of the 16 Home Start sites operating in the spring of 1974. The families belonged to one of three groups, the Home Start group (192 families), the delayed-entry control group (130 families), or the Head Start group (112 families). Included are reviews of psychometric analysis such as Preschool Inventory, Denver Developmental Screening Test, Child 8-block Task, Child Food Intake Questionnaire, Schaefer Behavior Inventory, Pupil Observation Checklist, High/Scope Home Environment Scale, Mother Behavior Observation Scale, Parent Interview and, 8-Block Sort Task. A number of tables and charts are included in this report. (Author/MS)

This interim evaluation report focuses on process (formative) and outcome (summative) data collected in spring 1975 on the National Home Start Program. Home Start, a federally-funded 3-year (1972-1975) home-based demonstration program for low-income families with 3- to 5-year-old children was designed to enhance a mother's skills in dealing with her own children and to provide comprehensive social-emotional, health and nutritional services. Chapter II presents information on the characteristics of the national program and the individual Home Start projects and describes program costs and services delivered. Chapter III analyzes several issues related to program operations--program duration, support extended after family graduation, national office support, home visitor supervision, nutrition services and projects' future plans. Chapter IV presents data bearing on several questions relating to the home visit process, program costs and program effects. (Author/MS)

The focus of this paper is on innovative evaluation, evaluation methodologies, and the development of evaluative instruments. Innovative evaluation is discussed as related to comprehensive evaluation, general evaluation with special emphasis, problems in comprehensive and general evaluations, and five components of an assessment model (accessibility, flexibility, personalization, synthesis, and efficacy of resources). Four models that may be applied in the evaluation of innovative and nontraditional programs are discussed; these models are: Competency-Based Evaluation, Self-Evaluation, Systems Analytic Evaluation, and Fortune/Hutchinson Methodology of Educational Evaluation. The development of evaluative instruments is presented as to guidelines, and examples of instruments are given. An appendix provides a copy of a Faculty Acceptance of Innovation questionnaire, an Occupational Status of Former Occupational Students questionnaire, and two handscoring forms. A list of references is included. (DB)

This report reviews the progress made by the New York State Education Department and local school districts in developing and implementing a comprehensive student evaluation program for elementary and secondary education. With the assistance of local school district personnel and technical consultants, the department concentrated its efforts on identifying the most pressing information needs and the characteristics of instruments adequately supportive of the decisionmaking tasks of today and tomorrow. The "interim" program first described in the report makes the best and the most systematic uses of existing methods and information to evaluate current categorical and general educational programs, and gives the best answers currently possible to pressing, performance-related questions being asked by the educational community. A "projected" program, described in the final part of the report, provides an examination of two state-of-the-art problems that must be overcome if the performance-related questions asked by educators, the State government, or the public at large are to be answered. (Page 140 may reproduce poorly.) (Author/JF)

A study was made: (1) to determine whether children allowed to choose the difficulty levels of their arithmetic problems in a Computer Assisted Instruction (CAI) task would show greater engagement in learning than children who were not given a choice, (2) to discover possible patterns in the choices made, and (3) to determine the relationship of locus of control (LOC) attributions to engagement in the task. The subjects were 4th and 5th grade students, mostly Mexican-Americans, in a low-income school. Pairs of subjects were matched on age, sex, and initial achievement level in math. Subjects in both conditions used a modified form of the Math Drill and Practice program by Suppes, et al, 1974. Task specific attention or inattention and locus of control were measured. Among the major finds were: (1) that the choice group was significantly higher in engagement, (2) for both groups, engagement decreased significantly over a 15 day period; and (3) distinctive choice patterns did occur. The findings also showed that children will choose problems that result in poor academic performance; if performance in choice situations is to be improved, training methods that use information about children's unique patterns of choices should be designed. (Author/WCM)

This report concerns a comparison of the course and teacher evaluation form used by New College of Hofstra with another form recently developed for use in other units of Hofstra University. The comparative analysis is based on data derived from administration of the Hofstra University Course and Teacher Evaluation Questionnaire and the New College Evaluation Questionnaire to 368 New College respondents at the conclusion of the spring semester 1974. The purpose of this study was to compare results on the two forms to determine: (1) if similar information was gained from the two forms, (2) if one or the other of the forms alone provided information appropriate to the needs of New College, and (3) if in the future it would be necessary to administer both forms. Overall, it might be concluded that the two forms do get at basically the same information, since significant correlations do exist between many items and many items do have "face" or content similarity. Yet, as previously indicated, much higher correlations might have been expected among certain items that are in fact identical. The possibility of some systematic bias in responding, as suggested by the review of individual respondee sheets, and by the relatively low intercorrelations between forms with relatively high interacorrelations between items on the same form, limits any extensive interpretation of comparability of forms. Reproduced from best available copy. (Author/PG)

This report presents comparative data collected from 122 North Texas State University elementary education students enrolled in the Dallas Teacher Center Professional Semester (DTCPS) program during the 1972-73 and 1973-74 academic years. Three instruments--One Word Sentence Completion, Self Report Inventory, and Directed Imagination Test--were used to gather data on the following topics for factor analysis: (a) self-perception ability, (b) optimism-hope, (c) empathy-view of others, (d) general adjustment and mental health, (e) attitude toward teaching, (f) view of children, and (g) confidence regarding classroom discipline. Results indicate the following: (a) in all of the attitudinal dimensions sampled, the student teachers experienced positive growth during the DTCPS program; (b) inner-city and middle-class schools were suitable settings for this significant positive attitudinal growth to take place; and (c) the total impact of the DTCPS program would appear to have been the influence producing these results. Also, a high proportion of the participants who completed the DTCPS program saw themselves and others favorably, tended to be well-adjusted adults, had hope and optimism for the future, and were committed to the teaching profession. Raw scores averages for each instrument are included in the tables. (PD)

Reported is the development and field testing of the Science Classroom Observation Form (SCOF), an instrument focusing on the interactive characteristics among students, the environment, and teacher in elementary and junior high school science classrooms. This instrument was developed to be used in research investigations of openness and inquiry orientation. The data from field tests resulted in SCOF IV which consists of 39 statements or questions about characteristics of science classroom interactions, each rated on a 5-point scale following classroom observation. Items are grouped into "environment,""teacher," and "student" for observer convenience. SCOF has been used in 43 junior high school science classrooms in the Province of Saskatchewan. This report includes descriptive statistics of those data as well as a discussion of the results of factor analysis of the SCOF instrument. Validity and reliability are also discussed. A copy of the instrument is included. (PEB)

Reported is the development of an experimental instrument for use in monitoring opinions of preservice and inservice science teachers concerning science teaching. Also reported is a study in which this instrument was used. The instrument, The Science Teaching Opinion Survey, consists of 20 statements to which respondents react via a five point scale ranging from "strongly agree" to "strongly disagree." (A copy of the instrument is included in this report.) Use of the instrument with preservice teachers enrolled in a science methods course and with inservice teachers attending NSF institute programs appears to indicate that it is sensitive, reliable, and easy to score. It requires a minimum of time to administer and lends itself readily to statistical analysis. (Author/PEB)

The Institutional Goals Inventory (I.G.I.), an integral part of the Institutional Goals-Setting Model developed at Brevard Community College during the fall of 1973, was field-tested during the period December 15, 1973 through February 1, 1974. The Inventory consists of 90 statements of possible institutional goals, to which the respondent gives two replies--(1) How important is the goal at this institution at this time? and (2) How important should the goal be at this institution? The results of the field testing are recorded in this report. (Author/SGM)

This project identified the specific goals sought by a group of off-campus study program directors having much experience who developed an appropriate instrument to assess whether or not the student participants perceived these goals as having been actualized within themselves. The research instrument developed for the assessment was the Individual Opinion Inventory (IOI). Results emerging from this phase of the project indicated: (1) that students of different academic majors responded differently to the instrument, with students in the humanities scoring category items significantly more favorably than students in the sciences; (2) associations of statistical significance were found between the 13 experimental categories of the instrument and (a) the student's present interest in the area of off-campus study, (b) the student's present amount of reading about that area, and (c) the student's present amount of correspondence with individuals living in the off-campus study location. (MJM)

The Education Voucher Demonstration is a large-scale educational and social intervention that began in the Alum Rock Union Elementary School District, San Jose, California, in September 1972. First funded by the Office of Economic Opportunity, it is now sponsored by the National Institute of Education. This technical appendix is the third of four Rand reports that describe and analyze the workings of an education voucher demonstration during its first year (academic year 1972-73). The appendixes in this volume contain additional technical material related to the contents of the main volume of the series and are numbered to correspond with the section numbers of that report. (Author)

This report of the Task Force on Measurement and Evaluation in the Study of English (appointed by the National Council of Teachers of English) analyzes the present state of the art of testing and recommends the use of common sense in selecting and using tests and in interpreting the information derived from testing. The report views standardized tests in current use as inadequate and advises makers of tests to be more explicit in stating the limitations of their tests, to do more to describe the populations on which norms are based, and to be more effective in informing users about proper and improper uses of tests and test data. Among the alternatives which the Task Force suggests as superior to testing are portfolios of student work, interviews, peer evaluation, and observations of classroom performance. (JM)

The Career Maturity Inventory (CMI) has been used by some school districts to study the career development of high school students, as well as to evaluate the effectiveness of the career education project. Two questions which were raised about the selection of the CMI evolved into the objectives of this research. Those questions were: (1) Do counselor ratings of students' career knowledge, career planning, and attitude toward work correlate with those concepts as measured by the CMI?; and (2) Are the students of the CMI measuring independent factors of career maturity or are they so highly intercorrelated that they measure only one or two factors? One hundred eleven tenth-grade students were given the CMI, and four counselors were asked to rate each student on three factors--concepts of knowing about jobs, looking ahead and attitudes toward work. Results of the study showed that the CMI subtests were highly interrelated, and that, while the counselor's ratings of students' career maturity correlated significantly, the relationship between what the counselors measured and what the CMI measured were not highly related at all. (Author/PC)

This study investigates the underlying constructs of two instruments which purport to measure the locus of control of reinforcement variable. Instruments used were the IAR and the Career Development Responsibility Scale (CDR). Principal components analyses were computed separately for each instrument from the intercorrelation matrix of the items of the respective instruments. Data were obtained from a cross-sectional student group of ninth graders. The study provides evidence that instruments designed to measure locus of control in domains other than academic achievement serve to increase the number of dimensions. Intercorrelations of the factor score derived from the IAR and CDR are sufficiently low as to question whether or not they are, in fact, measuring the same construct. Developers of locus of control instruments should be aware that the balance of situations and sources of control items will affect the scores obtained from the instrument, and thus the relationship with other similar instruments. (Author/PC)

This study was a replication of an earlier study by Mazyck (1971) which was designed to analyze characteristics of paraprofessional child care workers, as determined by ratings given on a scale of paraprofessional worker characteristics. The original study is reviewed and the development of the Mazyck Rating Scale for Paraprofessionals, is discussed. The Scale was composed of two categories of characteristics: Personal-Social, and Educational-Biographical-Working Relationships. In the present study, the category Reaction to Stress was added, and the original experiment was replicated with a different and smaller group of child workers and specialists similar to those used in the first study. Findings show that from both studies there are 20 scale items that each of the groups agree on as being significant characteristics of paraprofessionals. The next step is to devise ways of measuring the identified characteristics and to develop instruments useful for persons interested in selection of child care workers. Extensive statistical data are included, as well as a copy of the Rating Scale as used in the second study. (ED)