Search

This paper discusses how an experimental design can be applied to a large-scale evaluation. The purpose of the study described is to assess the success of the National Science Foundation (NSF) in achieving its goal for five comprehensive projects. This paper is divided into four main sections: (1) design; (2) sampling procedures; (3) data gathering techniques; and (4) conclusion. The majority of the paper is given to a description of the sampling and data gathering techniques used. Throughout, an emphasis is placed on procedures used and decisions made together with the reasoning behind the decisions made. Actions taken which reduced the experimental design power and advantages of using the design are discussed. It is concluded that the experimental design deserves to be considered in the evaluator's tools to be used in improving both the decision-making process and the resulting decisions. (Author/HM)

One of the objectives of Project TACT is to determine the potential of a gamut of educational media. The working papers in this set have a basis in pictorial information produced through computer graphics. These papers are intended to serve as a basis for sharpening questions, delineating the context within which the answers might be significant, and determining whether or not interesting experiments are feasible and rewarding. (Author/MK)

Selected aspects of the problem of obtaining unbiased program or product evaluation are discussed. An evaluator who is a member of the project staff will have difficulty producing an evaluation which is credible and valid. Project monitors will also have a problem since they are often required to assume the conflicting roles of external evaluator and project advocate. Therefore, no unit should rely entirely on a given subunit for evaluative feedback about that same subunit. Evaluative feedback systems require renewal or replacement to prevent deterioration of their independence. Evaluators should arrange for replication of their own work by independent evaluators. Four further approaches for reducing bias in evaluation include: (1) standardizing the qualitative aspects of evaluation procedures by using a checklist; (2) upgrading evaluator training procedures; (3) reducing sources of bias external to the evaluator; and (4) comparing the project, programs, or products with alternatives. (SDM)

Conventional approaches to program evaluation have followed the experimental and psychometric traditions dominant in educational research. Their aim (unfulfilled) of achieving fully objective methods had led to studies that are artificial and restricted in scope. Illuminative evaluation is introduced as a total re-appraisal of the rationale and techniques of program evaluation. It follows a contrasting 'anthropological' research paradigm. Illuminative evaluation recognizes that an innovative instructional program cannot be separated from its learning milieu--a network of cultural, social, institutional, and psychological forces. The evaluator concentrates on processes within the classroom rather than on outcomes (goals, objectives) derived from a specification of the instructional system. Data is collected from observation, interview, questionnaires, and documentary information. Illuminative evaluation thus concentrates on the information gathering rather than on the decision making component of evaluation. The paper concludes with a discussion of the problems and potentialities of the new approach: its range of applicability; objectivity; validity; obligations of the research worker; how explorations of the learning milieu can meet the need for theoretical advance; and how the illuminative approach can clarify decision-making. (Author/CP)

An individual's socialization to occupational roles is reviewed in terms of the concepts and of the link between childhood socialization, social class, and occupation. The educational system and peer group influences are briefly reviewed. Occupational socialization is a general progression in which specific job training occurs only near the end of the process. Although several studies suggest the importance of seemingly indirect influences of cultural values, economic values, and the economic system, the relative neglect of occupational socialization in research suggests that it is considered in narrow terms. However, researchers have found numerous links between socialization practices and social class and between social class and occupation. Parental values that are passed on to children are the values that parents need to function in the workplace. Kohn (1969) suggested that occupation reflects all three of the major dimensions of stratification: economic (privilege); honorific (status); and political (power). Other research indicates that occupational socialization occurs as a result of the emphasis on discipline, subordinancy, external reward systems, and a bureaucratic rationale in the educational system. Further research should focus on historical experiences and links between socialization practices and models of social change. (KC)

Comprehension of the quantifiers "some" and "all" was studied with 202 children, three to nine years old. Thirty-two quantifier sentences dealing with descriptions of circles and squares were presented to the children. Wooden objects were presented to some children to see if results were affected by the choice of abstract objects, but no differences were found. The relationship of "over-interpreting" quantifiers to acquisition was analyzed. "Over-interpretation" is to understand "all" as occupying both possible sentence positions: as a sentence quantifier and as a quantifier in an adjective phrase. Interpretations of "all" as an adverb and as an adjective are analyzed, and deep structure positions and over-interpretation are discussed. "Double-some" sentences, the extensive"some," the intensive "some," reversals, and quantifier order are also discussed. (SW)

The purpose of this investigation was to classify a sample of high school biology students into concrete and formal operational levels using three separate instruments: (1) a battery of Piagetian tasks (the pendulum, bending rods, and the balance beam); (2) a written biology examination consisting of questions requiring concrete and formal operations; and (3) a subject-free examination also consisting of questions requiring concrete and formal reasoning. Using the obtained data, the authors' purpose was to determine if the logical operations under question were closely tied to specific content. Sixty-eight high school students (32 males and 36 females) enrolled in a biology course at Delphi High School in Delphi, Indiana served as subjects. The subjects did not perform more formally on the subject matter free Longeot examination and on the biology examination than on the Piagetian tasks. It was concluded that Piagetian tasks are relatively content free and can serve as realistic indicators of concrete and formal thinking abilities. (Author/DS)

Students ask "why?" questions in the survey stage of using TICCIT lessons either because (1) they have holistic learning strategies, (2) they are searching for familiar bases upon which to attach new material, or (3) they desire to find out what relevance the lesson has for their future lives. In the design phase of TICCIT, an effort was made to answer the "why" question as it relates to the survey aspects of learning by introducing material that showed why a particular lesson or unit was important. However, the teaching and proctoring staff of a TICCIT class have the responsibility to deal with the additional "why?" questions which can only be answered by perceptive human beings, and to prompt students in the use of improved learning techniques. (CMV)

The TICCIT command language is a synthesis between work in instructional psychology and cybernetics, and broadens the degrees of control available to students well beyond the capabilities of existing CAI systems. Student control commands manipulate important instructional variables and information is provided to guide the student's choice of presentation form, and inter-display relationships (easy-hard) as well as requests for instructional help. TICCIT is designed to provide students with optimal and transferable learning strategies, increased motivation, and attitudes of individual responsibility. (Author/CMV)

This document presents the design of an evaluation study for the Program of Comprehensive Grants for Teacher Education which is funded by the National Science Foundation (NSF). The main concerns of the evaluation were the need to provide context information to individual project directors and to develop a summative strategy for the national program. A total of 17 instruments were selected to collect a variety of cognitive and affective data for both science and mathematics projects from students, teachers and administrators. Approximately one-half of these instruments were developed for this evaluation while the remainder were chosen from existing instruments. A total of 354 schools from five experimental regions and five control regions participated in this study. The data gathered by this national design will be processed and analyzed during the next several months and presented to Comprehensive Project Directors and to the NSF. (HM)

This research study investigated student perception of the social learning environment in biology, chemistry and physics courses. A stratified random sample of secondary schools from three regions was selected. The principal of each sampled school randomly selected a biology, chemistry or physics teacher who, in turn, randomly selected one of his classes to complete two instruments: Learning Environment Inventory (LEI); and the Test of Achievement in Science (TAS). The statistical procedures used were multivariate and univariate analyses of variance, and discriminant function analyses. Achievement level (high, middle, low) and science course (biology, chemistry, physics) were the two independent factors. The combined mean score for each science course on ten scales of LEI were the dependent variables. Results showed that there were significant differences between the students' perception of their environment in the three science courses. (HM)

This study focused on the evaluation of the delivery system of five different projects supported by the National Science Foundation. Seven general factors proposed by Havelock were used to identify the difference among various delivery systems. A series of on-site visits, interviews, observations, and questionnaires were used to describe the major characteristics of the delivery system of each project. A panel of nine judges used the collected information to rate each delivery system against Havelock's factors. A two-way analysis of variance with repeated measures was used to analyze the data. Results showed that differences existed among projects in the extent to which they met the factors taken as a group. (HM)

This research study sought to determine the extent of the relationship between teacher characteristics and student outcomes and to provide an indication of the order of importance among the teacher characteristics. A stratified random sample was selected of 236 secondary science teachers including 84 biology teachers, 111 chemistry teachers, and 41 physics teachers from 14 states. The selected science teachers completed four instruments. Each teacher randomly selected one of his/her classes to complete four instruments. Stepwise regression and canonical correlation were used to analyze the data. Results showed that the selected teacher characteristics accounted for one-fifth to one-third of the total variance in student scores. It was also indicated that the self-improvement variable seemed to be the most consistent predictor for student outcomes. (HM)

This study provides information on science teachers' perception of themselves and their working conditions. A stratified random sample of junior and senior high school science teachers from three regions (12 states) of the United States was selected to respond to a questionnaire and an attitude measure. The questionnaire contained 20 items dealing with teacher satisfaction on teaching skills and 25 items dealing with teacher satisfaction on school conditions. Results showed that both junior and senior high school teachers rated their knowledge and abilities as satisfactory to good on all the 20 ability items. However, five of these items showed a significant difference between the ratings of the junior and senior high school teachers. Results showed also that the junior high teachers rated their school conditions as significantly poorer than did the senior high school teachers on eight of the 25 items. It was concluded that these science teachers are generally satisfied with their own abilities, but believe their school conditions need improvement. (HM)

In the states of Bihar and Uttar Pradesh in northern India the speech styles of men and women differ markedly in a number of ways. One mode of communication, tuneful weeping, is exclusive to women. This behavior is distinguished from spontaneous crying caused by pain, and is used only in certain prescribed social situations, and not necessarily in sorrow. It is accompanied by a well-organized set of statements on certain topics and themes, "wept statements," that are used only with weeping. Vocal embellishments to wept statements enrich their emotional charge, and the degree of affection in the situation can affect the length of the weeping. It is often done with a weeping partner, and the usual topics are memories of the past lived with the weeping partner, the fear of an unknown future in an unknown place, apologies for not being as dutiful or respectful as possible, and appeals for forgiveness and requests not to be forgotten. Male response to female weeping is in a normal conversational tone. Typical weeping situations include a woman's leaving her home for her husband's, visits from kinsmen, reunions of women long separated, a period after initial mourning in which the women sit around the dead body and weep, or in a quarrel when grievances are being aired. Under the given conditions, women are obliged to weep or be considered antisocial. However, many modern young women are refusing to weep and this mode of communication may become obsolete. (MSE)

African Wolof society is divided into a number of ranked status groups or castes, the largest of which is the high-ranking noble caste. Wolof conceive of two styles of speaking, the restrained or noble-like and the elaborated or "griot"-like, and the two styles are connected by the presence or absence of "kerse," honor and self-control. The linguistic phenomena relevant to social rank are found in all aspects of speech performance: paralinguistic phenomena (such as intonation), phonology, syntax, and lexicon. Intonational features provide the most striking differences in style. Phonological differences seem to be related to differences in speech tempo. The griot style uses syntactic devices that lengthen the total utterance but add little or nothing to its meaning, while the noble style uses shorter, blander forms and intentional errors as a sign of high rank. The variants can also signal high and low rank within a caste. The use of intentional error can create confusion between a high-ranking individual and a linguistically incompetent foreigner. The linguistic system as a whole may have shifted in the direction of the high-status variant, and the reduction in surface elaboration is consistent with high-rank restraint and nonfluency, possibly arising from the general population's desire to appear more high-ranking. Thus the cultural valuation of speech can influence both the form and the direction of language change. (MSE)

An assessment of educational game playing in the classroom is presented. The assessment is based mainly on observations and reports of the game Empire (Educational Services Incorporated). In separate sections of the report, the author considers arrangements for game playing, the motivational appeal of educational games, and how and what children learn from games. The expenditure of time and money and the utility of educational games are also considered. Six reasons for the motivational appeal are: the classroom routine is interrupted; children are encouraged to talk to fellow students; children like the manipulative aspects of games; children like to win; game playing is a familiar and non-threatening activity; and not much reading is required for participation. In discussing intrinsic and extrinsic learning from games, the author points to problems of misleading the child through falsifications and oversimplifications of the reality the game is intended to portray. Conceptual and factual learning is also examined. In addressing these problems, the report stresses the necessity of game design and teacher follow-up. The author concludes that games are most useful at the junior high level and are least successful with bright students. (KC)

Written in 1966, the author examines the evolution of teaching in relation to the development of the elementary social studies course, Man: A Course of Study. The act of teaching is traced from play practice of primates to teaching-in-context of primitive societies to the more complex abstract teaching in separate schools of contemporary society. Five specifications about how a society must proceed in order to equip its young are noted: it must convert what is to be known into a form capable of being mastered by a beginner; the learner must be saved from needless learning; a society must place emphasis on how one derives a course of action from what one has learned; all societies must maintain interest among the young during the learning process; and a society must assure that its necessary skills and procedures remain intact from one generation to the next. The author then examines the relationship among teaching in separate schools, how a society equips its young, and the content of Man: A Course of Study. The recurring questions of what is human about human beings, how did they get to be that way, and how can they be made more so, form the structure of the course which examines the humanizing forces of tool making, language, social organization, man's prolonged childhood, and the urge to explain. Problems in constructing the course are noted in terms of the psychology of subject matter, stimulating thought in the school setting, the personalization of knowledge, and the encouragement of self-conscious reflectiveness. (KC)

Written in 1965, the author describes the initial stages in the development of Man and Politics, a three year social studies curriculum for students ages 12-14. The author notes that the reason for choosing the political theme is that the school is the most influential agent of political socialization in the child's life. The report outlines the concepts of intellect and education upon which the curriculum is based, the goals, rationale, central theme, and framework of the curriculum, and describes the three units. The curriculum is based on discovery learning and the goals are described as providing children with experience in generalization, valuing, and causality. Following a discussion of the framework of the course, the three units are outlined. Inventing the Western World (Grade 7) encompasses the time span from the 5th century B.C. to 1600 A.D. and presents a variety of case studies related to power and political culture. From Subject to Citizen (Grade 8) draws its material from 17th and 18th century British and American experience. The Civic Culture (Grade 9) examines the nature, problems, prospects, and evolution of American political culture. Materials are designed to: provide children with the kind of data that scholars themselves work with, present data that demand deductive analysis, give children an opportunity to discover structure in the relationship of ideas, and increase the power of the student to deal with the modern condition. (Author/KC)

Written in 1965, the report provides the rationale for including the study of the mythology and art of cultures alien to Western civilization in the social studies curriculum: Man: A Course of Study. The context of the course, a basic theory of myth, principles of art, and examples of Bushmen and Eskimo myths are presented. The author suggests two reasons for the study of the Eskimo, Pygmy, Bushmen, and a group of Australian Aborigines: the societies are relatively simple and their structure is easier to understand than the complex Western societies; and the study of hunting societies offers a student contrast with his own culture. Included in the section on mythology is a definition, a discussion of myth as communication and as document, and of structural analysis, lyrics, ceremonials, folklore of hunters, riddles and proverbs, rites of passage, and the functions and interconnectedness of symbols. In the section on art, the author outlines differences between myth and art, the ecology of art, art and cognition, art in instruction, and the functions of art. Tales included in the final section are the Bushmen origin of marriage, of the sun, and of death, and the Eskimo tale, Earth Gives Men Their First Children and a poem, My Breath. The author concludes that the aim of the unit is to provide an understanding and respect of other cultures. (KC)

Written in 1965, the author describes the initial stages in the development of the elementary level curriculum, Man: A Course of Study. The structure and form of the course and three pedagogical techniques are discussed. The course is organized around the humanizing forces of tool-making, language, social organization, the management of man's prolonged childhood, and man's urge to explain. Plans for the section on language include a consideration about what communication is, the design features of a language, arbitrariness, acquisition, and the role of language in shaping human characteristics. The tool-making section is designed from a philosophical approach; the object is to explore how tools affected man's evolution. The unit on social organization focuses on the nature of structure in a society, roles filled by people, and the world view of a society. The childrearing unit centers around three themes: the extent to which and the manner in which the long human childhood leads to dominance of sentiment in human life, the human tendency toward mastery of skill for its own sake, and the shaping of man by the patterning of childhood. The fifth unit concerns itself with man's drive to explicate and represent his world through symbolic systems. The three pedagogical techniques emphasized to achieve the goals of these units are contrasting, using games, and stimulating self-consciousness about assumptions. The author states his plan to create far more units than could possibly fit into a school year in order to provide the teacher with flexibility in planning. (KC)

Since ethnic studies in schools is an integral part of the larger social system in this country, its current status, problems and strengths must be examined within the total social context. Ruling groups determine the formulation and dissemination of knowledge (particularly negative social science research about ethnic groups and the poor). This knowledge is designed to support the status quo, and to legitimize the position of those in power. Ethnic studies programs must be replanned and novel teaching strategies formulated in order for students to effectively challenge and invalidate the assumptions about ethnic groups on which white-dominated social science research is often based. A primary goal of ethnic studies must be to help minority students to psychologically liberate themselves from myths and stereotypes and to help them to develop a commitment to social change. Unfortunately, reform of current programs is needed, as dissension among different ethnic groups, lack of funds, and inadequately qualified instructors have caused them to be self defeating. Despite the difficulties that minority leaders encounter, they should demand this reform and take steps to see that ethnic studies programs survive and gain integrity. (Author/KR)

A mainline CAI system is an individualized instructional system of major scope which uses the computer as a tool for instructional management and information transmission. Such systems are developed according to a design science approach and used as a less costly alternative to conventional instruction. In contrast, adjunctive systems have a much smaller scope and are used in support of regular classroom or laboratory instruction. TICCIT is classified as a mainline CAI system because (1) it was not designed by a single faculty author for a single course, (2) it is not used on existing computer systems, (3) it does not use existing languages, and (4) it does not employ normative grading. (CMV)

The formative evaluation process described was designed to be used by Brigham Young University personnel in 1974 to evaluate and improve TICCIT programs and materials for community college credit courses. Involving the use of editorial judgment and data on student use, this process includes five levels of evaluation and revision: (0) materials are reviewed for subject matter accuracy, instructional psychology, and message design; (1) materials pass through several cycles of formal debug, using skilled and critical students to look at every display with different types of mental sets characteristic of students and catch problems in answer processing; (2) lessons and units are tested on 20 or more students enrolled in convenient institutions to identify those lessons, segments, and displays which produce confusion or difficulty; (3) courses, lessons, and units are tested on 20 or more community college students to discover any remaining areas causing difficulties; and (4) courses, units, and lessons are tested on several hundred community college students. Modifications indicated by the results in each step are part of the on-going process. The most advantageous role during this period for Educational Testing Service (ETS), which would perform a separate summative evaluation of the final revised version, was seen as facilitator of interchange between the developers and the users and as a facilitator of the formative evaluation process. (Author/CMV)