Search

Selected papers from the Conference for the Evaluation of Instructional Materials treat the area of evaluation by describing Richard Dershimer's three-part evaluative schema, the Educational Products Information Exchange approach to evaluating instructional materials, the evaluation procedures in Montgomery county (Maryland), the Consumers Union Model, and a proposed management model for evaluation data. References accompany the first two papers, and commentary remarks regarding the middle three papers are provided. (LE)

This paper suggests that the variety of decision making proposed by Professor Lortie will not afford the luxury of evaluative systems of the kind he describes. Professor Gordon feels that, had Professor Lortie pursued a line of functional analysis of many outcomes, he would have arrived at an entirely new analysis of the justification for complexity of evaluation. (Author)

Professor Stake feels that Professor Glaser's paper does not give adequate attention to how descriptions and judgements of the instructional process can be used to facilitate educational decisions. He also asserts that behavioral specification, as it is usually defined, is often impractical and unnecessary, and he challenges the primacy of goal specification as a first logical step in instruction or evaluation. Objectives must be reported, but attention must be given to how they change over time and the priority given by different groups to different objectives. (Author)

THIS BOOK COMPRISES THE MAJOR PAPERS DELIVERED AT A CONFERENCE ON MARCH 20-22, 1967, SPONSORED BY THE POLICY AND DEVELOPMENT COUNCIL, AN ADVISORY UNIT OF THE DEPARTMENT OF EDUCATION IN ONTARIO. THE CONFERENCE WAS ATTENDED BY REPRESENTATIVE PERSONS FROM DEPARTMENTS OF GOVERNMENT, UNIVERSITIES, AND MAJOR SCHOOL SYSTEMS ACROSS CANADA. THE CONFERENCE HAD TWO OBJECTIVES--(1) TO GATHER PLANNERS AND STUDENTS OF PLANNING TO DISCUSS THE STATE OF THE ART, ITS TECHNICAL PROBLEMS, AND ITS FUTURE, AND (2) TO EXPLAIN TO CANADIAN EDUCATORS WHAT IS MEANT BY "PLANNING." FOUR OF THE PAPERS DEALT WITH NATIONAL OR INTERNATIONAL EXPERIENCE AND EMPHASIZED ONE PARTICULAR ASPECT OF PLANNING EXPERIENCE--THE ORGANIZATIONAL PROBLEMS, THE COMPLICATIONS OF A FEDERAL SYSTEM, OR THE DATA PROBLEMS. FOUR PAPERS WERE TECHNICAL, DEALING WITH ECONOMIC ASPECTS OF PLANNING AND WITH SOCIAL GOALS. SCHEMES FOR TRAINING PLANNERS WERE THE SUBJECT OF TWO PAPERS. (HW)

The management field of Organization Development approaches institutions from the social, technical, and economic systems with major emphasis on group process. Organization Development contains many possibilities for libraries and their staff to meet the challenge of our greatly changing environment. This paper outlines the methods by which an Organization Development Team could be created and operate at Wayne State University Libraries. (Author)

A paper presenting the objectives of the National Association of Black Adult Educators is given. The association was formed to take action to seek and implement pragmatic solutions to the problem of the exclusion of blacks and other minorities from the decision making process in American society. Specific objectives include: (1) To provide a permanent organization for all Adult Educators in the United States of America, (2) To provide meaningful involvement of blacks in the activities of the major professional organizations, (3) To encourage the employment of blacks as directors and supervisors of Adult Education programs at national, state, and local levels, (4) To encourage the employment of blacks as deans, directors, and professors at the colleges and universities, (5) To design and implement quality ethnic oriented programs with special emphasis placed on reducing racial tension, promoting legal justice, and promoting cultural and economic awareness, (6) To design relevant training programs for teachers, administrators, paraprofessional and peer-teachers involved in projects dealing with minority group education, (7) To develop racial/ethnic curriculum materials, (8) To promote agreement with Federal Agencies to perform contract monitoring, evaluation, and research services on projects relating to minority education, and (9) To promote fellowship among members. (Author/CK)

The Public Law enacted in 1971 requires that fundamental subject matter areas taught in Wisconsin's public schools be assessed. It is proposed in this paper that assessment of the State's social studies programs needs to focus its attention on the school environment, which educators can and should control, change, and improve, rather than assessing student performance or on breaking social studies into component parts. In essence, evaluation procedures need to focus on the cultural environment; i.e. what the schools are doing to provide quality educational experiences and opportunities permitting examining, questioning, and testing evidence and beliefs, which influence the thought and actions of students and which contribute to the social studies goal of increasing human understanding. When assessment focuses on academic achievement of students it only measures the circumstances of birth and social situation. Three questions describing the existing pattern of social studies experience are: 1) What opportunities are provided by teachers to involve students in the action of problematic inquiry? 2) What is the role of student and teacher in social study? and 3) Do the methods and materials of social studies instruction maintain the integrity of the social disciplines? (SJM)

Students are clients of the educational system, but traditionally have had little voice in decisionmaking about the services they receive. Students are now demanding greater involvement, and administrators have tried a number of programs designed for student participation, generally with success. Some examples of successful programs are (1) student centers that provide a focus on students and serve to improve relations with students, (2) inclusion of students on deliberative and decisionmaking bodies to encourage greater communication and understanding between staff and students, and (3) involvement of students in producing innovative educational programs and services that better meet the needs and interests of students. (RA)

This paper examines (1) attempts at educational reform in Venezuela and Colombia by means of decentralization, and (2) the impact of decentralization on two contrasting educational systems. In Venezuela, the national school system was highly centralized with even routine decisions being made in Caracas, which resulted in a rigid, unresponsive relationship between local men and government officials. In Colombia, State and local units were relatively autonomous, and government funds were divided among these units so that control over allocations of educational money and enforcement of standard teacher qualification requirements were in the hands of the local units. In both countries, decentralization programs were designed that set up a series of regional offices to strike a better balance between local and national authorities. (RA)

The usual approach to improving discipline in the schools is the repressive approach -- more rules, stricter enforcement, more efficient pupil surveillance, suspensions of privileges, or additional rules imposed by the board of education. School administrators, faced with increased public concern and lacking well-defined alternatives to such repressive measures, tend to accede to public pressure, thereby making matters worse. This paper is designed to serve the educator who wants to take positive rather than negative steps towards improving school discipline. The projects suggested here, taken together, should add up to a comprehensive effort to modify the learning environment. Three major pieces of this environment are considered here -- the belief and value system of the school, the school's communications system, and the manner in which decisions are made within the institution. Significant changes in these important components of the institution should do much to remedy the causes of discipline problems. (Author)

The circulation systems of three divisional units of Wayne State University libraries (Education, Humanities, Social-Science) needed centralization because their separate operations had created many inconsistences and unnecessary inconveniences. This paper outlines the socio-technical decisions made in the organizational change. It describes the problems of the existing organization and shows how the reorganization process involved physical changes, record re-arrangement, staff reassignments and training, procedural changes, and reorganization of divisional functions. The obstacles and other planning deficiencies involved in the process are also indicated. Procedural suggestions are made for future organizational change. (CH)

Simulation is the process of synthetically manipulating the variables in a model of a system for the purpose of understanding, experimenting with, and predicting the behavior of that system. Many different models are now being developed by university administrators to aid them in making decisions. Simulation models have been developed for predicting instructional programs, academic program structures and space utilization. Many of these models must wait for implementation until adequate information is collected which will allow meaningful values to be placed on the variables. At the present stage of development of the art, the more experimentation carried on in different kinds of institutions with different kinds of approaches, the greater the progress will be. As individuals develop, test, and publish their efforts, the trading and borrowing of ideas and methods should begin to shape a comprehensive and diversified technology. (TC)

This paper discusses "the classroom as a social system" as an idea which has implications for teacher training. Reference is made throughout to the author's own recent research (ED 003 449 and ED 017 965) and his teaching of undergraduate and graduate educational psychology. Section 1, "The Conceptual Framework," includes (1) definition of the classroom as a social system, (2) description of a teacher decisionmaking model based on a prediction system and a value system, and (3) analysis of the importance of "initial teacher roles" with models depicting the initiation of the activity structure and development of classroom control. Section 2, "The Preservice Training Problem," (1) analyzes the special problems of the preservice trainee and (2) discusses each of several program components which attack such problems through focus on the interdependencies of the social system position: microethnography of the classroom, educational psychology laboratory, and a "two-by-two apprenticeship" (a semester practicum in which trainees spend two weeks teaching each of nine grade levels). A brief section on inservice teacher training describes a course (including specific writing exercises) in which teachers analyze their own classrooms as social systems. The concluding section outlines a process model of the "Phases and Emphases in Developing a Professional Teacher." A 34-item bibliography is included. (JS)

A management information system (MIS) is a systems approach to data processing. It provides relevant, timely, and accurage data to decisionmakers, as well as information storage, retrieval, and transfer functions through the use of computers. Educational administrators have been slow to implement computerized data processing either because they fail to understand information systems or because systems planning has been neglected. Population growth and the changing role of education necessitate implementation of computerized data handling. Data processing personnel are in the best position to give educators guidance in data processing and should be given greater roles in decisionmaking. Only with a good information management system can current administrative and educational confusion be ended. (RA)

Planning, programing, budgeting is an approach to rationalizing budgeting and strengthening analysis as a policy change and planning tool. Organizational development applies behavioral science research to organization to enhance the commitment and involvement of personnel for smoother and more effective system function. It is argued that implementation of PPB can be facilitated by proper attention to the management philosophies of OD; and that by the concerted, coordinated use of these approaches, planning and system growth will undergo considerable improvement. The special philosophies of OD are discussed in planning contexts, and the federal government's implementation of PPB is examined for problems it encountered. A paradigm is subsequently developed that specifies more explicitly the proposed PPB-OD interaction. The special contribution that OD can make to program planning and budgeting processes is developed in detail. (Author)

Research is needed to determine whether American Indian students really benefit from counseling as perceived by non-Indians, the nature of current methods of formal and informal counseling of Indians, effective forms of counseling for Indians, the relationship between Indians' self-esteem and employment, factors affecting vocational decision-making, the impact of current occupational and career materials, family and tribal interests, and occupational difficulties. In terms of methodology, researchers must be aware that Indian students are unique, that English is a second language for most Indian students, that special measurement instruments having validity for Indians must be developed and used, and that results from subpopulations of Indians must not be generalized to the total Indian situation. (LS)

The evidence in the literature is that both students' efforts and administrators' responses have fallen far short of what it takes to change a system substantially. The categories of literature covered in this bibliography include student and educator perceptions of conditions in schools that generate conflict and impetus for change; empirical studies of students' powerlessness and surveys of the actual and desired extent of student participation in school decisions; case studies of individual schools in which students' actions brought about changes; and prescriptions and strategies for involving students in the shaping of education. (Author/JF)

The analytical part of the Coleman report has such serious methodological shortcomings that it offers little guidance for policy decisions. The specification of the theoretical model is inadequate, and without the theoretical framework to provide order, and a rationale for the large number of variables, there is no way to interpret the statistical results. Even in those instances where the Coleman Report makes clear the justification for the use of a variable in the regression model, the criterion used to assess the statistical performance of the variable is inappropriate. In order to suggest a more meaningful approach to the problem of measuring determinants of educational achievement for policy purposes, the role of a variable in affecting objectives must--and can--be interpretable in the context of a proper model; further, when such a model is in the form of a regression equation, an appropriately scaled regression coefficient is the most useful statistic to measure the importance of the variable for the purposes of policy action. (JM)

The objective of evaluation in education is distinct from the objectives of measurement and assessment or testing. Accordingly, a synthesis of the three procedures into a single theory is not viable. Evaluation is a method of gathering empirical evidence for decision making by answering questions about the worth of educational materials, activities, and programs. Testing, with its emphasis on measuring and predicting human characteristics, supports evaluative efforts but the relationship is one of assistance and not identify. A related document is EA 002 819. (JK)

Evaluation specialists need to decide when to use quantitative techniques for evaluation and when to use more qualitative techniques. Different situations require different strategies for evaluation. Some factors which can be used to differentiate situations include: (1) Current state of knowledge; (2) cost of the evaluation process in terms of time, manpower, or general economic resources; (3) controllability of independent variables; and (4) complexity of the stimulus. All possible combinations of the above factors generate almost 24 different types of evaluation techniques. These techniques, presented in tabular form, comprise a multiple model theory of evaluation. Related documents are EA 002 410 and EA 002 551. (MLF)

Cost-effectiveness represents an attempt to apply one methodology to decision-making in education. Three recent trends in the behavioral and social sciences are apparent: Rationalization in the decision process, institutionalization of research in education, and the beginning of a general theory of organizational behavior. The problem posed is how to combine the decision process of cost-effectiveness with the behavioral processes of institutional decision-making. A general theory of instructional evaluation is needed to unify evaluations. Questions are raised about the specific problem area of concern, the relevant variables, and the environmental influences of the school. The openness of the school system is affected by its interactions with other systems, by the criteria used in evaluation, and by feedback. Cost-effectiveness is only a part of the systems approach in the evaluation of instructional programs. Related documents are EA 002 407 and EA 002 534. (MLF)

Evaluation of the ratio of cost to benefit of instruction must play an important part in the formation of a public policy on education. However, it is doubtful if evaluation is sufficiently developed to play such a role, because evaluation is based on student responses to test items which contain an indefinite bias and cannot be accepted as reliable data. A more reliable theory of test writing must be developed before evaluation can perform an important role in the making of public policy. Related documents are EA 002 475 and EA 002 534. (MLF)

Nine group exercises for developing organizational skills and personal and interpersonal insight in inner city teachers are presented. The exercises, which consist of rationale and instructions for the user, are devoted to developing an understanding of verbal communication; investigating the roles of a helper, of a white counselor and black teacher in a conflict, and of a superintendent of schools and consultant in a race crisis; motivating interracial discussions; developing group skills in revising and reacting to policy documents; and exploring strategies for creative conflict. The latter also proposes how decisionmaking should be structured in an organization. An additional exercise (not necessarily for a group) explains the advantages of a time bar diagram (Critical Path Method network) over a conventional bar chart in long range planning. (LP)

In this model for program planning and evaluation, the State Director of Vocational Education acts as the chief program manager or decision-maker. In addition, a program evaluator manages an information system which provides the decision-maker with a means of assessing the efficacy of the course he has chosen. Elements of the model include individual attributes, needs of society, vocational education goals and objectives, operational procedures, material and human resources, and actual outcomes or products of the program. Evaluation may be in terms of an appraisal of the process, or it may be directed toward an appraisal of the outcomes. The decision-maker is responsible for administrative functions, such as allocating resources and ordering objectives into a hierarchy, while the program evaluator is responsible for the management system. This model, which may be used at any level, illustrates the information that must be provided by the program administrator to the decision-maker if appropriate alternatives are to be selected, objectives attained, and goals realized. Evaluation is a key element in progress and goal realization. (SB)

This dissertation begins with a description of some methods employed in making public-sector resource-allocation decisions, with conclusions on the appropriateness of each method for evaluating the ongoing programs of a school system. The second section has been rewritten and published as "A Comprehensive Theory of Cost-Effectiveness" (EA 002 920). This section develops the theoretical substantiation for cost-effectiveness analysis which can serve as the groundwork for evaluation of ongoing school systems and which may be used for planning as well as evaluation, taking the decision maker's preferences into account. A case study is then considered which generates the data systems required by the model. The model provides outputs, whose utility for the decision maker is discussed. The sensitivity of model outputs to various changes in the decision maker's value system is also explored. The final section provides specific recommendations to decision makers as a result of the case study and presents a more general set of cautions and recommendations. Suggestions for future research are presented, and a 61-entry bibliography is attached. (Author/DE)

This monograph provides a theoretical substantiation for benefit-cost analysis and cost-effectiveness analysis in school system planning. Nine examples of decision-making situations are presented. A simple case of a single objective to be attained with one plan being selected from a set of proposed plans with no resource constraints evolves into a more complex and more realistic case with multiple objectives and several activities or programs to be chosen from a host of possibilities under resource constraints. Cases include not only a priori decision-making schemes but evaluation schemes as well. A summary of the nine cases offers specific recommendations about methods that can be used for generating and processing the kind of "data types" required for educational decision making. A 36-entry bibliography is appended. This document is a rewritten chapter of a dissertation entitled, "A Cost-Effectiveness Evaluation Approach to Improving Resource Allocations for School Systems" (EA 002 937). (DE)

As part of a project designed to construct a mathematical model of the operation of an academic library, a research team determined the usefulness of a quantitative approach to making decisions about the number of copies of each item that a university library should provide. Using the reserve collection of the University of Lancaster Library as a basis, a mathematical relationship was established between the number of requests, the length of the loan period, the number of copies, and a standard of service (defined as "immediate availability"). It was found that the Poisson distribution could justifiably be used to relate the average request rate to the number of copies and the degree of availability likely to be achieved. A table was produced which can be used in a predictive manner insofar as the request rate itself can be predicted. A method of estimating the average request rate from incomplete data was also devised. It was concluded that a mathematical relationship between the factors concerned can be established. The chief difficulty lies in predicting the level of demand for library services and, more generally, in understanding the factors which determine it. (JB)