30,750 results
Search Results
152. The Many Functions of Evaluation in Education
- Author
-
Phillips, D. C.
- Abstract
This paper focuses upon the many functions that are served by evaluations, and by the process of conducting them. Many forms or types of evaluation have evolved to serve these different functions; and a brief account is given of some of the most common of these forms and the issues or controversies that they have engendered. The discussion opens, after a brief historical note, by describing the differing views of Lee Cronbach and Michael Scriven about whether an evaluator should aim to educate stakeholders about the program so that they can make informed decisions about it, or whether evaluators should go further and make a value judgement about it. The discussion then moves on to the importance of not overlooking the unintended effects of a program that is under study; and after presenting a list of functions that evaluations can have, the remainder of the discussion deals with the "pros" and "cons" of, and the differences between, formative and summative evaluations.
- Published
- 2018
153. Toward an Intercultural Approach to Evaluation: A Perspective from the National Institute for Educational Evaluation in Mexico (INEE)
- Author
-
Schmelkes, Silvia
- Abstract
The National Institute for the Evaluation of Education (INEE) in Mexico has begun to meet the challenges in evaluating indigenous children and teachers and the educational programs and policies targeted to them. Several evaluation projects are described in this paper. One is the "Previous, Free and Informed Consultation of Indigenous People," which focuses on quality of education they receive. A second is the design of a protocol for reducing cultural and linguistic bias in standardized tests, which requires oversampling of indigenous students and the involvement of anthropologists, linguists and indigenous teachers in item development. A third is an indigenous language evaluation for candidates for entry into the teaching profession, which they must pass before they can work in indigenous schools. A fourth is the development of a qualitative instrument for evaluating teacher performance. The instrument asks evaluated teachers to contextualize their planning; scorers decide whether the plan is adapted to the cultural context and the characteristics of the children. The projects described are only a starting point. In the near future, several dilemmas, such as the apparent trade-off between contextualization and quality, have to be faced and solved.
- Published
- 2018
154. Student Perceptions of the Effectiveness of Formative Assessment in an Online Learning Environment
- Author
-
Ogange, Betty Obura, Agak, John O., Okelo, Kevin Odhiambo, and Kiprotich, Peter
- Abstract
Assessment is an integral part of the teaching-learning process in both conventional and distance education contexts. Literature suggests that with the increase in the use of Information and Communications Technology in the delivery of learning, a number of institutions are resorting to formative assessment practices that are mediated by technology to not only provide flexible and more efficient means of assessment but also attain improved learning outcomes. This paper investigated student perceptions of the effectiveness of different types of formative assessment used in online learning environments. A 31-item questionnaire was used to gather data on student perceptions. On the level of difficulty, students generally perceived the various types of formative assessment as having no significant differences. Results further indicated that students received more prompt feedback from peer assessment and computer-marked assessment, compared to teacher-marked assessment. The findings of this study will support practitioners in eLearning to use formative assessment and feedback mechanisms more effectively to influence student engagement as well as learning outcomes.
- Published
- 2018
155. Critical Issues in Designing and Implementing Temporal Analytics
- Author
-
Chen, Bodong, Knight, Simon, and Wise, Alyssa Friend
- Abstract
The importance of temporality in learning has been long established, but it is only recently that serious attention has begun to be paid to the precise identification, measurement, and analysis of the temporal features of learning. From 2009 to 2016, a series of temporality workshops explored temporal concepts and data types, analysis methods for exploiting temporal data, techniques for visualizing temporal information, and practical considerations for the use of temporal analyses in particular contexts of learning. Following from these efforts, this two-part Special Section serves to consolidate research working to progress conceptual, technical and practical tools for temporal analyses of learning data. In addition, in this second and final editorial we aim to make four contributions to the ongoing dialouge around temporal learning analytics to help us move towards a clearer mapping of the research space. First, the editorial presents an overview of the five papers in Part 2 of the Special Section on Temporal Analyses, highlighting the dimensions of data types, learning constructs, analysis approaches, and potential impact. Second, it draws on the fluid relationship between 'analyzed time' and 'experienced time' to highlight the need for caution and criticality in the purposes temporal analyses are mobilized to serve. Third, it offers a guide for future work in this area by outlining important questions that all temporal analyses should intentionally address. Finally, it proposes next steps learning analytics researchers and practitioners can take collectively to advance work on the use of temporal analyses to support learning.
- Published
- 2018
156. Measuring Research Impact in Australia
- Author
-
Gunn, Andrew and Mintrom, Michael
- Abstract
The implementation of the national Research Engagement and Impact Assessment in Australia provides a timely opportunity to review attempts to improve the non-academic impact of academic research. The impact agenda represents a new phase in academic research evaluation and funding, characterised by a heightened need to demonstrate a return on public investments in research. New imperatives seek the reorientation of some academic research towards more directly driving national innovation, meeting the needs of business, and contributing to improved social and economic outcomes. This paper reviews the policy journey of research impact in Australia from the proposed, but never implemented, Research Quality Framework (RQF) to the National Innovation and Science Agenda (NISA). Our analysis of policy developments from the Howard to the Turnbull Governments highlights the controversial nature of research impact assessment and the political and methodological challenges that have accompanied its implementation.
- Published
- 2018
157. Factors Influencing the Assessment Perceptions of Training Teachers
- Author
-
Crossman, Joanna
- Abstract
The paper describes a study revealing a number of factors that influence how Bachelor of Education students perceive assessment. These factors include personal histories, student teacher relationships, opportunities for personalisation and deep learning, notions of relevance, and anxiety issues. "Personal histories", as a term used to describe students' previous experiences, provided participants with an experiential reference point for thinking and talking about aspects of the assessment process. However, the learning and assessment context together with individual motivations, rather than personal histories, seemed to determine the learning approach adopted. The paper is likely to be of interest to those involved in designing assessments for training teachers.
- Published
- 2004
158. Thin layer chromatography coupled to paper spray ionization mass spectrometry for cocaine and its adulterants analysis.
- Author
-
De Carvalho, Thays C., Tosato, Flavia, Souza, Lindamara M., Santos, Heloa, Merlo, Bianca B., Ortiz, Rafael S., Rodrigues, Rayza R.T., Filgueiras, Paulo R., França, Hildegardo S., Augusti, Rodinei, Romão, Wanderson, and Vaz, Boniek G.
- Subjects
- *
ELECTROSPRAY ionization mass spectrometry , *COCAINE abuse , *THIN layer chromatography , *CRIME laboratories , *BENZOCAINE , *AMIDES , *CAFFEINE , *COCAINE , *COMPARATIVE studies , *DRUG adulteration , *LIDOCAINE , *MASS spectrometry , *RESEARCH methodology , *MEDICAL cooperation , *NARCOTICS , *RESEARCH , *EVALUATION research - Abstract
Thin layer chromatography (TLC) is a simple and inexpensive type of chromatography that is extensively used in forensic laboratories for drugs of abuse analysis. In this work, TLC is optimized to analyze cocaine and its adulterants (caffeine, benzocaine, lidocaine and phenacetin) in which the sensitivity (visual determination of LOD from 0.5 to 14mgmL(-1)) and the selectivity (from the study of three different eluents: CHCl3:CH3OH:HCOOHglacial (75:20:5v%), (C2H5)2O:CHCl3 (50:50v%) and CH3OH:NH4OH (100:1.5v%)) were evaluated. Aiming to improve these figures of merit, the TLC spots were identified and quantified (linearity with R(2)>0.98) by the paper spray ionization mass spectrometry (PS-MS), reaching now lower LOD values (>1.0μgmL(-1)). The method developed in this work open up perspective of enhancing the reliability of traditional and routine TLC analysis employed in the criminal expertise units. Higher sensitivity, selectivity and rapidity can be provided in forensic reports, besides the possibility of quantitative analysis. Due to the great simplicity, the PS(+)-MS technique can also be coupled directly to other separation techniques such as the paper chromatography and can still be used in analyses of LSD blotter, documents and synthetic drugs. [ABSTRACT FROM AUTHOR]
- Published
- 2016
- Full Text
- View/download PDF
159. Gallup Goes to School: The Importance of Confidence Intervals for Evaluation 'Adequate Yearly Progress' in Small Schools. Policy Brief
- Author
-
Rural School and Community Trust, Washington, DC. and Coladarci, Theodore
- Abstract
Indicators of school-level achievement, such as the percentage of students who are proficient in a particular content area, are subject to random year-to-year variation in much the same way that the results of an opinion poll will vary from one random sample to another. This random variation, which is more pronounced for a small school, should be taken into account by education officials when evaluating school progress in a policy climate of high stakes. To do otherwise is to unnecessarily risk the false identification of a failing school. In this monograph, I describe the application of confidence intervals to the evaluation of "adequate yearly progress" for No Child Left Behind (NCLB). Throughout, I demonstrate the particular relevance of confidence intervals for small schools. Upon completion, readers will understand why 27 states included confidence intervals in their NCLB accountability plans (and perhaps wonder why the remaining states did not). (Contains 13 footnotes, 5 figures and 1 table.)
- Published
- 2003
160. From Usable to Useful Assessment Knowledge: A Design Problem. CSE Report.
- Author
-
National Center for Research on Evaluation, Standards, and Student Testing, Los Angeles, CA., California Univ., Los Angeles. Center for the Study of Evaluation., and Baker, Eva L.
- Abstract
Policymakers are fond of saying that we have enough research knowledge; however, one problem is that nobody is applying the knowledge we already have. This paper offers a discussion of the kinds of knowledge needed to improve students learning, what to do about this in the assessment and testing arena, and where assessment falls short. Types of knowledge are discussed, including research knowledge, which must be both usable and useful. Distinctions are made between usable and useful knowledge, and a case is made for how we might ultimately design our systems and our own actions to help us act with greater intelligence. (Author)
- Published
- 2003
161. Uptake and impact of priority setting exercises in nutrition research publications.
- Author
-
Hawwash, Dana, Pinxten, Wim, Raneri, Jessica E., Kolsteren, Patrick, and Lachat, Carl
- Subjects
EXPERIMENTAL design ,RESEARCH ,CROSS-sectional method ,RESEARCH methodology ,MEDICAL cooperation ,EVALUATION research ,COMPARATIVE studies ,EXERCISE ,HEALTH planning - Abstract
Objective: To assess how priority setting exercises for nutrition research are considered in publication.Design: Cross-sectional design.Settings: First, a citation analysis of priority setting exercises found in nutrition research until 2019 was conducted. The reasons for citation were extracted from the text of citing papers and the reasons were defined as: (i) acting on the research questions identified as priorities, (ii) acknowledging the priority setting exercise, (iii) using the same method, or (iv) previous knowledge to support evidence. Second, a survey with authors of the priority setting exercises was done to understand priority setters' perspectives on the impact and satisfaction of their work.Participants: Twenty-one priority setting exercise papers were included. In all, 434 citing papers were found, of which 338 were considered in the citation analysis. A sample of 17 authors representing 13 priority setting exercise papers completed the impact and satisfaction survey.Results: Half of the priority setting exercise papers were published by 2013. After excluding self-citations (n = 60), the priority setting papers had on average 18 citations. Priority setting exercises had a median of 1 (IQR = 0-1) citing manuscript that acted on the recommendations produced from priority setting exercises. Authors of the priority setting exercises expressed a desire for increased uptake of the results of the priority setting exercises by funding agencies. Key barriers for uptake were identified as challenges in involving stakeholders and the general public for participation in the priority setting exercise.Conclusions: Priority settings exercises are important efforts to guide nutrition research toward effective allocation of resources. However, there seems to be a limited consideration of these priority setting exercises in research papers. [ABSTRACT FROM AUTHOR]- Published
- 2021
- Full Text
- View/download PDF
162. Evaluating Online Instruction: Adapting a Training Model to E-Learning in Higher Education.
- Author
-
Hallett, Karen and Essex, Christopher
- Abstract
This paper presents a model for the evaluation of postsecondary online distance education courses and programs. To better address the unique nature and audience for these courses and programs, and the related institutional needs for assessing their success or failure, the focus is on a model from corporate training that provides a comprehensive, multi-level evaluation of a distance course or program. This model is based on Kirkpatrick's (1975) levels of evaluation. A general description is provided of how this model can be implemented via online means to provide the assessment information required by postsecondary institutions. (Contains 12 references.) (Author)
- Published
- 2002
163. Some Principles of Assessment and Evaluation in Teacher Education
- Author
-
Sedere, Upali M.
- Abstract
This paper offers a broader framework for assessment and evaluation of teachers in developing countries. Teacher is seen not only as some one who imparts knowledge and skills but in the developing world, teacher has a wider role to play. (Contains 2 tables, 3 figures, and 13 footnotes.)
- Published
- 2001
164. Assessment and Evaluation Modeling. Symposium 38. [AHRD Conference, 2001].
- Abstract
This symposium on assessment and evaluation modeling consists of three presentations. "Training Assessment Among Kenyan Smallholder Entrepreneurs" (George G. Shibanda, Jemymah Ingado, Bernard Nassiuma) reports a study that assessed the extent to which the need for knowledge, information, and skills among small scale farmers can promote effective management of the off-farm enterprises. It recommends business counseling as an integral part of extension education. "Business-Focused Evaluation: A New Collaborative Model" (Valerie Stead) presents a case study of a business-focused evaluation model developed as part of an organizational development intervention for a global semi-conductor company. It assesses the model's value against three principles concerning collaboration, joint ownership, and integration drawn from client and provider needs and current thinking in evaluation research. "Challenges in Measuring and Evaluating Diversity Efforts in Human Resource Development" (T. Marshall Egan) examines challenges faced by practitioners and scholars interested in evaluation and measurement of diversity efforts in organizational contexts. It considers an organizational systems model illustrating the diversity-related interchanges between an organization and the macro environment and examines challenges to measurement and evaluation due to the latency of the diversity, performance, and performance improvement constructs. All three papers include substantial bibliographies. (YLB)
- Published
- 2001
165. Reclaiming the Word 'Standards': Professional Standards for Teachers of English Language and Literacy in Australia (the STELLA Project).
- Author
-
Doecke, Brenton, Gill, Margaret, and McClenaghan, Douglas
- Abstract
In 1998, the Australian Research Council provided funding for a three year research project to develop professional standards and assessments for the English teaching profession. The project team is a consortium of researchers from three major universities together with the two national English teaching associations and representatives from state government education departments. In this paper, two of the researchers, Margaret Gill and Brenton Doecke from Monash University, and a teacher, Doug McClenaghan, reflect on the collaborative nature of the project and the central role the profession itself is playing in developing and validating nationally recognized sets of standards which will define what both the beginning and accomplished English teacher should believe, know, and be able to do. (Contains 13 notes.) (Author/EF)
- Published
- 2000
166. Field evaluation of the CATT/Trypanosoma brucei gambiense on blood-impregnated filter papers for diagnosis of human African trypanosomiasis in southern Sudan.
- Author
-
Chappuis, F., Pittet, A., Bovier, P. A., Adams, K., Godineau, V., Hwang, S. Y., Magnus, E., Büscher, P., and Büscher, P
- Subjects
AFRICAN trypanosomiasis ,TRYPANOSOMA brucei ,MEDICAL screening ,AGGLUTINATION tests ,DISEASE relapse ,ANIMAL experimentation ,BLOOD collection ,COMPARATIVE studies ,DIAGNOSTIC reagents & test kits ,FILTERS & filtration ,IMMUNOGLOBULINS ,INDUSTRIES ,RESEARCH methodology ,MEDICAL cooperation ,PROTOZOA ,RESEARCH ,RESEARCH evaluation ,TRYPANOSOMIASIS ,EVALUATION research - Abstract
Most Human African Trypanosomiasis (HAT) control programmes in areas endemic for Trypanosoma brucei gambiense rely on a strategy of active mass screening with the Card Agglutination Test for Trypanosomiasis (CATT)/T. b. gambiense. We evaluated the performance, stability and reproducibility of the CATT/T. b. gambiense on blood-impregnated filter papers (CATT-FP) in Kajo-Keji County, South-Sudan, where some areas are inaccessible to mobile teams. The CATT-FP was performed with a group of 100 people with a positive CATT on whole blood including 17 confirmed HAT patients and the results were compared with the CATT on plasma (CATT-P). The CATT-FP was repeated on impregnated filter papers stored at ambient and refrigerated temperature for 1, 3, 7 and 14 days. Another 82 patients with HAT, including 78 with a positive parasitology, were tested with the CATT-FP and duplicate filter paper samples were sent to a reference laboratory to assess reproducibility. The CATT-FP was positive in 90 of 99 patients with HAT (sensitivity: 91%). It was less sensitive than the CATT-P (mean dilution difference: -2.5). There was no significant loss of sensitivity after storage for up to 14 days both at ambient and cool temperature. Reproducibility of the CATT-FP was found to be excellent (kappa: 0.84). The CATT-FP can therefore be recommended as a screening test for HAT in areas where the use of CATT-P is not possible. Further studies on larger population samples in different endemic foci are still needed before the CATT-FP can be recommended for universal use. [ABSTRACT FROM AUTHOR]
- Published
- 2002
- Full Text
- View/download PDF
167. The evaluation and validation of Phadebas® paper as a presumptive screening tool for saliva on forensic exhibits.
- Author
-
Wornes, Danielle J., Speers, Samuel J., and Murakami, Julie A.
- Subjects
- *
SALIVA analysis , *BODY fluid analysis , *ALPHA-amylase , *FORENSIC sciences , *BLOOD , *COLLECTION & preservation of biological specimens , *AMYLASES , *BLOOD testing , *COMPARATIVE studies , *FECES , *RESEARCH methodology , *MEDICAL cooperation , *FORENSIC medicine , *MUCUS , *NASAL mucosa , *PERSPIRATION , *PHYSICS , *RESEARCH , *SALIVA , *SEMEN , *TEARS (Body fluid) , *TEMPERATURE , *URINE , *EVALUATION research , *EQUIPMENT & supplies - Abstract
The Phadebas® Forensic Press Test is routinely used for the detection of saliva. However, assessment of the use of Phadebas® paper for this purpose has not been studied extensively. The suitability of Phadebas® paper as a presumptive screening tool for saliva on forensic exhibits, was investigated by analysing the following: (1) sensitivity, (2) specificity, (3) effects of temperature on sensitivity and specificity, (4) detection of saliva in mixed body fluid samples, and (5) influence of substrate porosity. The results of this study demonstrated that Phadebas® paper is more sensitive to α-amylase activity and less specific for saliva than previously reported. The use of an examination temperature of 37°C had no effect on sensitivity, but increased the incidence of cross-reactivity with other forensically relevant body fluid stains. Blood, urine and vaginal secretions can inhibit the detection of α-amylase activity with Phadebas® paper in mixed stains of saliva and body fluid. Substrate porosity is a weak predictor for the time taken for a saliva stain to achieve a strong positive result on Phadebas® paper. Overall, this study demonstrated that the Phadebas® Forensic Press Test has limitations as a presumptive test for the accurate identification of saliva. [ABSTRACT FROM AUTHOR]
- Published
- 2018
- Full Text
- View/download PDF
168. Study of Flare Assessment in Systemic Lupus Erythematosus Based on Paper Patients.
- Author
-
Isenberg, D., Sturgess, J., Allen, E., Aranow, C., Askanase, A., Sang-Cheol, B., Bernatsky, S., Bruce, I., Buyon, J., Cervera, R., Clarke, A., Dooley, Mary Anne, Fortin, P., Ginzler, E., Gladman, D., Hanly, J., Inanc, M., Jacobsen, S., Kamen, D., and Khamashta, M.
- Subjects
SYSTEMIC lupus erythematosus diagnosis ,CLINICAL competence ,COMPARATIVE studies ,CONSENSUS (Social sciences) ,DECISION making ,RESEARCH methodology ,MEDICAL cooperation ,MEDICAL records ,RESEARCH ,RESEARCH evaluation ,EVALUATION research ,PREDICTIVE tests ,RESEARCH bias ,SEVERITY of illness index ,DISEASE progression - Abstract
Objective: To determine the level of agreement of disease flare severity (distinguishing severe, moderate, and mild flare and persistent disease activity) in a large paper-patient exercise involving 988 individual cases of systemic lupus erythematosus.Methods: A total of 988 individual lupus case histories were assessed by 3 individual physicians. Complete agreement about the degree of flare (or persistent disease activity) was obtained in 451 cases (46%), and these provided the reference standard for the second part of the study. This component used 3 flare activity instruments (the British Isles Lupus Assessment Group [BILAG] 2004, Safety of Estrogens in Lupus Erythematosus National Assessment [SELENA] flare index [SFI] and the revised SELENA flare index [rSFI]). The 451 patient case histories were distributed to 18 pairs of physicians, carefully randomized in a manner designed to ensure a fair case mix and equal distribution of flare according to severity.Results: The 3-physician assessment of flare matched the level of flare using the 3 indices, with 67% for BILAG 2004, 72% for SFI, and 70% for rSFI. The corresponding weighted kappa coefficients for each instrument were 0.82, 0.59, and 0.74, respectively. We undertook a detailed analysis of the discrepant cases and several factors emerged, including a tendency to score moderate flares as severe and persistent activity as flare, especially when the SFI and rSFI instruments were used. Overscoring was also driven by scoring treatment change as flare, even if there were no new or worsening clinical features.Conclusion: Given the complexity of assessing lupus flare, we were encouraged by the overall results reported. However, the problem of capturing lupus flare accurately is not completely solved. [ABSTRACT FROM AUTHOR]- Published
- 2018
- Full Text
- View/download PDF
169. State Funding of Head Start.
- Author
-
Idaho State Legislature, Boise. Office of Performance Evaluation.
- Abstract
This background paper details Head Start, a federally funded program serving preschool age children from low-income families, and focuses on the program's effectiveness and the adequacy of historic federal funding levels. The paper provides an overview of the Head Start Program, describes federal requirements for local programs, and describes Head Start services in Idaho. The paper also details federal Head Start appropriations, enrollment, and costs per child from fiscal year 1995 to 1999, both nationally and for Idaho. In addition, the paper lists the states providing no state funding for Head Start, those providing direct state funding for Head Start, and those with state-funded pre-kindergarten programs other than Head Start. The paper notes although it is difficult to determine the overall impact and effectiveness of Head Start nationwide because of the variation in local programs, recent research concerning local programs has generally found them to be effective in preparing children for school. However, the General Accounting Office has concluded that the existing body of studies was not sufficient to develop a national estimate of the impact of Head Start and recommended that the Department of Health and Human Services conduct a nationwide assessment of the impact of Head Start programs. The Department of Health and Human Services maintains that existing research has already provided clear evidence of the positive impacts of Head Start services. The paper concludes with a list of proposed evaluation questions for Idaho Head Start programs. (KB)
- Published
- 1999
170. Monitoring Education Quality in Schools: Framework and Technology.
- Author
-
Cheng, Yin Cheong
- Abstract
This paper provides a framework for the discussion of policy and development of educational quality. Based on the choice of indicators, the purpose of assessment, and the type of evaluation, the paper proposes a whole-matrix approach to addressing the critical issues in assessing and monitoring educational quality. These critical issues include maintaining a balance between the basic functions of monitoring and between school self-evaluation and external evaluation, choosing an appropriate combination of quality indicators, determining who decides which indicators and standards to examine, and managing the differences between school-level and system-level evaluation. The whole-matrix framework, which uses multiple indicators to monitor multiple levels, was used to map profiles of educational quality in Hong Kong schools. A survey of 62 primary schools was conducted in 1991 and 1993. The more comprehensive 1993 survey was administered to 56 principals, 1,407 teachers, 219 classrooms, and 7,102 students. The paper describes how the framework served the purposes of both internal development and external accountability. The information gained from such an evaluation can be then used for educational improvement. Two approaches to enhancing educational quality include the external-control approach and school-based management. Four figures are included. (LMI)
- Published
- 1995
171. Standardization, Diversity, and Teacher Evaluation of Writing.
- Author
-
Pounds, Buzz R.
- Abstract
A study examined whether or not the composition faculty at the University of Northern Iowa agreed on criteria for evaluating student writing and to what extent they would give similar scores even if their criteria did not agree. Using 15 sample student papers written for the Writing Competency Examination (WCE), the scores given by current writing faculty using a 4-point holistic scale in addition to one 3x5 index card's worth of comments were analyzed. Results indicated that scores reflected a strong tendency toward the middle range. Statistical analysis on the subgroups revealed that of several factors the most significant was the relationship of scores to whether or not the rater made more content or form comments, followed by percentage of positive comments and teaching classification. Findings suggest that since significant differences were not found among the subgroups, standards are not sacrificed in spite of the current practice which reflects a more diverse and possibly less experienced group of teachers. The study suggests that: (1) the current system produces scores which tend toward the average with few highs and fewer lows; (2) no standardization of criteria exists among teachers of the College Reading and Writing (CRW) course; and (3) the university is served by promoting academic freedom for all teachers of first-year composition. (Contains 7 tables of data and 28 references; various sample forms--a recruitment letter for the study, a demographic information questionnaire, and two additional tables of data--are appended.) (CR)
- Published
- 1998
172. Evaluating a College Success Course.
- Author
-
Rasor, Richard A.
- Abstract
This paper addresses the methodological issues encountered when attempting to evaluate a college course's effectiveness by determining whether students who enroll in it perform at higher levels than students who did not enroll. The paper identifies resource people who may be helpful in the evaluation, and offers guidelines for the research question, which defines the purpose of the project, and the null or research hypothesis. The necessity of operational definitions is also emphasized, with examples and definitions of key words. Suggested factors to consider in planning the research project include cost, ethics, time, choice of students for the study, availability of students, commitment, faculty, the experimenter effect, student retention during the study, outcome measures and when to examine them, treatment rigor, and necessary sample size. The four research models provided for evaluation of the college success course are: (1) the experiment with randomly assigned partially matched students; (2) the experiment with randomly assigned unmatched students; (3) the post-facto design; and (4) the time series design. Selection of a particular model depends on the collective purpose of the evaluation. The paper concludes with a suggested outline for the ensuing research report.(YKH)
- Published
- 1998
173. Evaluating the Net Impact of School-to-Work: Proceedings of a Roundtable.
- Author
-
Department of Education, Washington, DC., National School-to-Work Opportunities Office, Washington, DC., and Department of Labor, Washington, DC.
- Abstract
This volume brings together the reports and discussion connected with a roundtable convened by the Departments of Education and Labor and the National School-to-Work Office to discuss issues surrounding the conduct of a net impact evaluation of school-to-work. After an introduction that summarizes the approach and intentions of the School-to-Work Opportunities Act and describes evaluation activities that have been commissioned separately from a net impact effort, six commissioned papers are presented. The papers are as follows: "Evaluating Early Program Experiences in the School-to-Work Opportunities Act: Policy and Design Issues" (Gary Burtless); "Net Impact of School-to-Work: Exploring Alternatives" (Charles Dayton); "Net Impact Evaluation of School-to-Work: Desirable but Feasible?" (Robert Glover, Christopher T. King); "Evaluation of School-to-Work Transitional Programs" (James J. Heckman); "Evaluating the School-to-Work Opportunities Act of 1994" (Robert A. Moffitt); and "Net Impact Evaluation of School-to-Work: Contending Expectations" (Hillard Pouncy, Robinson Hollister). Following the papers is a summary of the roundtable discussion and an epilogue that synthesizes both the papers and the discussion. Contains a list of participants, 190 references, and some papers includes tables/figures. (MO)
- Published
- 1997
174. Assessment, Technology and Democratic Education in the Age of Data
- Author
-
Perrotta, Carlo
- Abstract
This paper contends that powerful techniques to manipulate data, enabled by technological and economic developments, can be easily co-opted to serve the restrictive frameworks of hyper-controlling, managerial accountability that characterise current cultures of summative assessment in education. In response to these challenges, research is urgently needed to increase our understanding of the impact that assessments have on individuals and society. The paper concludes that social research ought to contribute to the identification of responses--educational, technological and political--that can minimise inequalities and potential abuses through the encouragement of data literacy across society. (Contains 2 notes.)
- Published
- 2013
- Full Text
- View/download PDF
175. Heuristics for Improving the Interpretation of Research Reports.
- Author
-
Forsyth, G. Alfr
- Abstract
Many students view statistics as their worst college course. Four heuristics that can improve students' proficiency in statistics and in interpreting reports of research are presented in this paper. The heuristics guide students' judgments about significance, generalizability, cause-and-effect, and strength of independent-dependent variable relationships. Previous studies of students' abilities to interpret research indicated that few of them understood random sampling and random assignment research methodologies as determinants of generalizability and cause-and-effect conclusions. It is supposed that students may fail to interpret research correctly because of the exclusive attention given to factual knowledge and statistical procedures rather than to interpretation abilities. It is recommended that a large set of vignettes (research-report summaries) can provide the critical methods needed for assessing student interpretations. One way to improve students' interpretation-of-research abilities is to develop a taxonomy that guides students in their judgments. A set of taxonomies is provided that can further aid students in answering questions about vignettes. A sample interpretation-of-research vignette is presented in which seven questions are posed to students when interpreting any research report. The students' answers to the questions can identify specific interpretation problems. (RJM)
- Published
- 1996
176. Toward Guidelines for the Research & Development of Interactive Multimedia: The Arizona State University TMMUIV Project.
- Author
-
Bitter, Gary G. and Pryor, Brandt W.
- Abstract
This paper chronicles the Teaching Mathematics Using Interactive Videodisk (TMMUIV) pilot project. The project emerged in response to the need to train pre-service teachers in, among other things, the use of mathematics manipulatives. The project now can offer a video database and pilot instructional program that combines full-motion video, digitized audio, text, and graphics. Two instructional modes, classroom presentation and stand alone tutorial, help the teacher develop a repertoire of techniques. The report catalogs all the system's components and provides a brief history of the genesis of the project and subsequent improvements. Formative evaluation was conducted at approrpiate times throughout the course of the project and included informal review by project staff, review by outside experts and the advisory board, and field trails conducted over a 2-year period in mathematics methods classes. Summative research was conducted when the system was close to completion. Four elementary mathematics methods classes were used to test the utility of the TMMUIV classroom presentation system. Two classes received only conventional instruction and two received the same instruction enhanced by TMMUIV. Overall, the group receiving the pilot TMMUIV enhancement reported being better prepared to teach and more highly motivated to teach. Also, they reported gaining more knowledge for teaching than the group receiving conventional instruction alone. The TMMUIV group also scored higher on each of three objective performance measures. These findings suggest the potential impact on teacher training for pre-service and in-service teacher utilization of interactive multimedia. (BEW)
- Published
- 1996
177. A Review of the Literature on the Validity of the Child Behavior Checklist.
- Author
-
Latkovich, Sharon A.
- Abstract
The Child Behavior Checklist (CBCL) is used to assess the behavioral problems and social competencies of children. Its broad usage in both a practitioner and research context has led to misapplications as compared to the CBCL's original intended purpose. Practical applications vary from the more traditional mental health centers and medical contexts to schools and forensic applications. Furthermore, an exhaustive number of citations of the CBCL can be found in the research literature. These 1990's uses of the CBCL most probably do not coincide with the original intent or purpose of the CBCL. Additionally, the composition of the normative sample presents particular difficulties for establishing validity across different groups. Thus given the broad usage of the CBCL combined with potential validity biases, it is the purpose of this study to review the literature regarding the validity of CBCL. This review examines the original purpose of the test, the normative sample, and test development from the purview of content, concurrent, predictive, and construct validity. A call is made for additional research so as to facilitate practitioners' and researchers' understanding of the nature of the CBCL in regards to a given use on a specific population. (Contains 43 references.) (Author/TS)
- Published
- 1996
178. The Value of Research and Evaluation in Language Education.
- Author
-
Council for Cultural Cooperation, Strasbourg (France). and Alderson, J. Charles
- Abstract
This keynote address argues that policy makers should consider a project approach to implementation of research results regarding foreign language learning; they should pay attention to what is being revealed by a growing number of evaluation and impact studies concerning effective innovation and project effects. Second, language teaching research needs to develop better definitions of linguistic and non-linguistic outcomes if research is to address the complexity of the task of foreign language learning. There appears to be a pervasive lay belief in the essential simplicity of the task of learning a foreign language, encouraged by the mistaken notion that foreign language learning and first language acquisition are comparable. Such beliefs may partially explain the commercial, if not educational, success of teaching methods and materials that make claims of effectiveness in remarkably short periods of time. Careful research has failed to bear out such claims or to support such beliefs. Even if effective approaches were known, it is naive to believe that research results can be translated into action without consideration of the conditions under which innovations take hold and succeed. The language teaching profession has failed to define adequately what it means by success in language learning; there is little agreement on expected or desired standards of achievement. There is also little attention to the principles that are accepted in the language testing profession for the construction of measures of achievement and proficiency. Language teaching research will need to develop much better definitions of the linguistic and non-linguistic outcomes that are expected of the various programs and projects if research is both to address the complexity of the task of foreign language learning as well as to contribute to our understanding of the conditions that can lead to successful language learning. (Contains 58 references.) (Author/NAV)
- Published
- 1996
179. The relevance of user evaluation for e-health-application in the homecare setting (paper)
- Author
-
De Rouck, Sofie, Leys, Marcus, Jordanova, M., Lievens, F., and Medical Sociology
- Subjects
user needs assessment ,evaluation research ,telehomecare applications ,e-Health - Abstract
BACKGROUND: Within the broad field of e-health specific attention is being paid to telehomecare applications. In this domain more attention has to be paid on the intertwining of technological design and the contexts of use. In order to optimise an effective and efficient implementation of telehomecare technology the user perspective needs particular attention. Lessons can especially be learned from the social construction of technology literature focussing on the technology as an artefact and the social context in which the technology is embedded and from evaluation research. OBECTIVE: We will demonstrate the importance of User Needs assessment, knowledge and insight in the particularities of the healthcare sector as a critical part of the design and implementation of telehealth applications. METHODS: Experiences in two pilot telehomecare projects in Belgium, demonstrate the added value of user evaluation research. The first project designed and introduced a telemonitoring application (blood pressure and glycaemia) in the homecare sector. The second project is the development of an eHomecare platform supporting the care process of patients in the primary healthcare sector. Using a constructive technology assessment (CTA) approach the follow-up, development and introduction of the telehomecare applications in the primary healthcare field is being done. The perspectives of the different stakeholders, with their own needs, expectations and values, are studied, evaluated and integrated in an evaluation framework, and taken into account when designing and implementing the technology. RESULTS: The institutional field of healthcare plays an important role in the use and acceptance of telehealth applications. In healthcare, consumer behaviour is to a large degree triggered by the package of reimbursed care (e.g. resistance of future use of telemonitoring applications by diabetes patients because of non-repayment). Specific requirements of homecare applications are identified when doing user needs research (e.g. guaranteeing mobility, usability aspects, particular needs for particular pathologies). Furthermore, user research clearly demonstrates stakeholders values and interests. The existing tension between the medical and paramedical domain can induce resistance from certain users (e.g. general practitioners who refuse interference by homecare nurses in their medical field). CONCLUSION: A thorough understanding of the social and healthcare culture and structure is essential for the acceptance, acceptability and feasibility of an application. The healthcare field is characterised by specific financial features, power relations, organisational aspects and needs of different types of users, which partially determines the successes and failures of e-health-systems. These insights and considerations need to be integrated developing and implementing future telehomecare applications.
- Published
- 2006
180. Developing theory- and evidence-based counseling for a health promotion intervention: A discussion paper.
- Author
-
Tiitinen, Sanni, Ilomäki, Sakari, Laitinen, Jaana, Korkiakangas, Eveliina Elisabet, Hannonen, Heli, and Ruusuvuori, Johanna
- Subjects
- *
HEALTH counseling , *HEALTH promotion , *COUNSELING , *BEHAVIOR , *RESEARCH , *META-analysis , *RESEARCH methodology , *BEHAVIOR therapy , *MEDICAL cooperation , *EVALUATION research , *COMPARATIVE studies - Abstract
Although the use of theories and evidence is often stressed in the development of health promotion interventions, this does not guarantee the success of an intervention. Thus, we need to reflect on intervention development processes that use different types of theories and evidence. In this paper, we provide a reflective discussion on how we identified evidence-based behavior change techniques and counseling themes for a health promotion intervention. In addition, we discuss the challenges that we encountered and what we learned during the process: a) a lack of previous research and meta-analyses, b) inconsistencies in evidence, c) integrating evidence and theories that have different starting points, and d) collaborating with researchers who represent different evidence and theories. During the process, we benefitted from having the clear goal of conducting evidence- and theory-based work. We solved the challenges by, for example, utilizing different types of evidence and being reflective about the reasons behind any inconsistencies in the evidence. In retrospect, we would have benefitted from closer collaboration between the teams that worked separately with different evidence. These kinds of reflective descriptions of development processes and the challenges encountered during them may help other researchers and professionals avoid encountering the same challenges. [ABSTRACT FROM AUTHOR]
- Published
- 2020
- Full Text
- View/download PDF
181. Developing and Assessing Students' Abilities To Interpret Research.
- Author
-
Forsyth, G. Alfr
- Abstract
A recent conference on statistics education recommended that more emphasis be placed on the interpretation of research (IOR). Ways for developing and assessing IOR and providing a systematic framework for creating and selecting instructional materials for the independent assessment of specific IOR concepts are the focus of this paper. The recommended assessment procedure to evaluate IOR abilities consists both of vignettes (research-report summaries) and questions designed to assess the students' interpretations of those vignettes. Vignettes could be selected to systematically vary on different features, such as random sampling as opposed to using an available group of subjects. A systematic framework is introduced here, consisting of 4 features, that can be used to write sets of vignettes for each cell of a 16-cell taxonomy. The features are: (1) random assignment versus classificatory independent variable; (2) a dependent variable that is either life-experience meaningful or not; (3) results that are counter to popular beliefs or for which there is no clear expectation of outcome; and (4) the independent variable having levels that are quantitatively different versus the independent variable not having an underlying continuum. A booklet containing a sample vignette for each cell of the taxonomy is appended. (RJM)
- Published
- 1995
182. Preservice Teachers Assess the Writing Quality of Language Minority and Nonmainstream Students: An Experiment.
- Author
-
Oliver, Eileen I.
- Abstract
This study of 20 preservice teachers in a secondary English methods course explored whether materials on diversity, information about language of wider communication, and literature from parallel cultures would have a positive effect on their ability to fairly assess work by high school students from diverse populations. At the beginning of the semester, the students scored and discussed two sets of essays written by tenth- and eleventh-grade students. These essays had already been scored by experienced teachers; these scores and the authors' ethnic identities were hidden from the education students. During the rest of the semester students studied pedagogy, assessment, and issues of diversity including differences in prose development from culturally diverse writers. They also read secondary level literature by nonmainstream writers. At the end of the semester students evaluated the essays again. A comparison between student ratings in the first and second round found that: (1) all but one preservice teachers' scores were lower than those of the expert raters; (2) in all cases but one, preservice teachers' first scores either remained constant or were raised substantially by the end of the semester; and (3) essays which received large jumps in scores were written by students whose ethnicities were very apparent in their work. The paper includes scoring guide, table of the experts' and preservice teachers' scores, essay assignment, and a student essay, "The 'Addquin' Streets of the 'Barrio'." (JB)
- Published
- 1995
183. Describing the Enacted Curriculum: Development and Dissemination of Opportunity To Learn Indicators in Science Education.
- Author
-
Smithson, John L.
- Abstract
Recent calls for high standards and hard content for all students have brought a growing interest in Opportunity-To-Learn (OTL) measures. OTL indicators provide valuable information for anyone interested in knowing what instruction is like, whether it is equitably distributed, and whether it is improving over time. This paper describes an effort to develop OTL indicators for science education through a collaborative project of 14 states. The Council of Chief State School Officers is working with the State Collaborative on Assessment and Student Standards (SCASS) Science Project to develop innovative approaches to assessment in elementary and secondary science education. A project component of high interest and potential application for local educators is information on students' opportunity to learn the science, content, knowledge, and skills that are assessed. The SCASS member states anticipate using methods of collecting and reporting on opportunity to learn the science content in conjunction with science assessments. The paper discusses the approach to OTL data used in SCASS and some of the issues in designing OTL data collection instruments. The questionnaire is appended. (LZ)
- Published
- 1995
184. Teacher Perceptions toward the Interpretation of Results from the New Norm-Referenced Portion of the Mississippi Assessment System. A Pilot Study.
- Author
-
Howe, Mary E.
- Abstract
Standardized testing, usually in the form of a multiple choice test, has dominated educational reform throughout Mississippi for the past 2 decades. Because of the minimal impact that standardized testing has traditionally had on curriculum decisions and classroom instruction, a paradigm shift in assessment format was adopted in the State from a narrow, criterion-referenced minimum competence to a broader, performance-based model. This pilot study examined teachers' perceptions of this new Mississippi Assessment System (MAS) and its extent as well as its impact on teacher judgment, modifications to classroom instruction, and non-traditional as well as traditional assessment formats. A 2-part questionnaire and teacher interviews were developed to collect both demographic and teachers' perceptions data. Participants were K-12 teachers (N=220) located in 2 small school districts in southern Mississippi. Data were analyzed using canonical correlation procedures. Results suggested that 10 demographic contributors influenced teachers' judgments, instructional change, and assessment/program evaluation. Quantitative and qualitative findings indicated that overall teacher perceptions, based on professional judgment, supported the changes in the MAS that combined traditional and non-traditional testing formats and that assessment results could help them select appropriate teaching strategies, curriculum, and educational programs. Participants also expressed a need for change in their classroom instruction to better prepare children for the new, non-traditional formats. (Contains 12 references.) (NAV)
- Published
- 1995
185. Evaluating Technology in Schools: Implications of a Research Study.
- Author
-
Kell, Diane
- Abstract
This report provides an overview of the issues related to a recently completed study of the use of computers in primary classrooms in six school districts and some special program classes, i.e., K-2 English-as-a-Second-Language and bilingual classes at one study site and Chapter 1 pullout classes for grades K-5 at another site. The sites used were demonstration sites for a new program of computer-based language arts instruction known as the Apple Software Series: Early Language (ALS-EL). Discussion of the development of the research context, purposes, and design includes consideration of issues related to: (1) assessing the implementation of educational innovations (an understanding of the process of change was emphasized and the change experience for individual teachers was examined); (2) assessing student performance in reading and writing (the Sulzby Writing and Rereading Task was adapted, and the Test of Early Reading Ability by Pro-Ed was chosen); and (3) assessing the effects of technology in classrooms (special attention was paid to the amount of funding allowed for the study, the amount of time necessary for the study, and the development of instruments to measure the implementation of an innovation). Strengths and weaknesses of the study are discussed, and implications for future research on computers in education are noted. It is suggested that a series of site-specific studies be pursued in which site-specific evaluation strategies be developed. It is concluded that operational variations and levels of implementation quality are important reminders of the need to allow time for major changes to take place before judgement about the innovations is passed. (34 references) (DB)
- Published
- 1990
186. Reliability and Validity of the Children's Loneliness Questionnaire.
- Author
-
Nyre, Joseph E.
- Abstract
Little research has focused on children's loneliness or on the relationships between children's loneliness and depression. Children are typically labeled as lonely by the external behaviors they may or may not exhibit, rather than by their perceptions of their social relationships. This study examined the Children's Loneliness Questionnaire (CLQ) for stability over time and construct-related evidence of validity. Children (n=279) in grades three through six, from two rural midwestern schools, completed both the CLQ and the Reynolds Child Depression Scale (RCDS) two times, 6 weeks apart. The 24-item CLQ consists of 16 primary items which focus on children's feelings of loneliness. The RCDS's 30 items describe each symptom of depression. Results indicate that the CLQ and RCDS are reliable sociometric measures for children over the age of 8. Large group administration procedures may have hindered younger children's responses on the CLQ. Concurrent-related evidence of validity of both instruments was reasonably high for children in grades four through six. For children in grades four through six, the test-retest reliability coefficients for the RCDS over a 6 to 7 week period was found to be lower than in previously reported research using shorter test-retest periods. (RJM)
- Published
- 1994
187. Authentic Assessment of Self-Concept through Portfolios: Building a Model with Public Schools.
- Author
-
Ryan, Charles W.
- Abstract
Teachers commonly use only two types of assessment--written examinations that test basic skills and direct observation of student learning. Both assessment procedures have been the subject of intense criticism. This study was conducted to investigate the viability of the portfolio model for authentic assessment of student growth, to analyze self-concept related teaching activities, and to develop the reflective abilities of students in grades K-6. Data were gathered from three selected public schools and teachers who implemented portfolios as part of a drug education effort. All subject areas were included in the portfolios along with accompanying documentation in the areas of self-concept growth and drug education curriculum activities. Analysis of 40 portfolios revealed that the majority of students' materials included self-concept activities and some drug education information, and that student worksheets showed completion of both drug education and self-concept learning activities. However, findings provided little evidence of student-based reflection on what the portfolio entries meant to them. Student portfolio evaluations are summarized in tabular form. (LL)
- Published
- 1994
188. Research on the IBM Grant Program's Impact on Teacher Preparation.
- Author
-
Bitter, Gary G. and Pryor, Brandt W.
- Abstract
Technology is not often a factor in classroom activities because teachers are untrained in its use. In 1990, International Business Machines (IBM) contributed over 30 million dollars' worth of workstations, networking hardware and software, IBM courseware, cash, and training to prompt 144 selected teacher preparation programs to integrate technology into their curricula. This paper describes a study evaluating the impact of these contributions, based on data and comments received from the sites in the fall of 1992 (response rate 92%). The study discovered that 63% of teacher preparation faculty were trained in the use of IBM equipment, 44% of them because of the grant program, and that the numbers of pre-service and in-service teachers so trained increased in three successive academic years. (Contains 6 figures and 15 references.) (Author/BEW)
- Published
- 1994
189. Using National Surveys To Improve the Efficiency and Effectiveness of Broad-Based Program Evaluations.
- Author
-
Clark, Sheldon B. and Boser, Judith A.
- Abstract
The suggestions offered in this paper are based on the experiences of two scientific researchers of how evaluations undertaken in a competitive arena, in which true experimental designs are not viable, can be designed in such a way that meaningful comparative data can be examined. Case studies of the Science and Engineering Research Semester and the Laboratory Graduate Research Participation programs at the Oak Ridge Institute for Science and Education illustrate how one can make use of existing comparison groups when the establishment of control groups for each cohort in an educational study is not economically feasible. Data from national surveys are used for comparative purposes. These surveys include: (1) the Survey of Earned Doctorates of the National Science Foundation (NSF); (2) the Survey of Doctorate Recipients, another NSF survey; (3) the NSF New Entrants Survey; (4) the Survey of Experienced Scientists and Engineers, also an NSF component survey; and (5) the National Survey of College Graduates. Items from these surveys that are clearly relevant are selected for comparisons. In conclusion it is remarked that this approach is not a panacea, and requires thorough understanding and careful evaluation of the national survey, but it can be a useful and cost-effective alternative to traditional control-group designs. Ten figures illustrate survey use. (Contains nine references.) (SLD)
- Published
- 1993
190. Defining the Future: A Program Evaluation.
- Author
-
Lavoritano, Joseph E. and Segal, Patricia B.
- Abstract
The objectives of this program evaluation were to gather information and normative data about the students referred for psychoeducational evaluation; to determine the diagnostic profile of students referred for special education; and to use the information gathered to determine future program direction and possible program modifications. Students (N=55) in non-public schools referred for psychoeducational evaluation because of learning problems were evaluated. Students were evaluated using the Wechsler Intelligence Scale for Children III and the Wide Range Achievement Test-Revised (WRAT-R). After the evaluation these students were assigned to one of three dispositions which determined an intervention of increasing intensity in behalf of the student. The first level was teacher/psychologist consultation. The second level was multidisciplinary team meeting including the parent. The third level was a multidisciplinary team meeting with referral for an individualized educational planning meeting. Results indicated that there were no significant differences in psychologists' case dispositions using intelligence scores as dependent variables. As achievement scores in reading and spelling on the academic screening measure, the WRAT-R, yielded the only significant differences among case dispositions, a classroom based assessment of academic skills would be a viable alternative to the varied assessment batteries currently being administered. (ABL)
- Published
- 1993
191. Improving Sociometric Methods for Defining Friendship.
- Author
-
Hardy, Cindy
- Abstract
Using a theoretical conceptualization of friendship as a context for development, this study evaluated modifications of standard sociometric procedures to assess friendship. A culturally diverse sample of 237 children in grades 3 through 6 completed two sociometric nominations over a 1-month period and rated the quality of their best friendship and their social competence. Sociometric nominations involved children's degree of friendship with their classmates; nominations were coded for mutuality and for stability of mutuality over the 1-month period. From the results of the study, it is concluded that: (1) the rates of stability of friendship observed were comparable to those reported in studies using standard sociometric techniques; (2) the techniques in this study permitted identification of good friendships; and (3) the assessment of short-term stability of mutuality of friendship enhanced the prediction of social competence, beyond the level achieved by the assessment of mutuality of friendship alone. (MM)
- Published
- 1993
192. Evaluation of Music Educators: Toward Defining an Appropriate Instrument.
- Author
-
Brophy, Tim
- Abstract
This paper questions whether it is appropriate to evaluate music educators exclusively on general competencies, or whether effective music teaching involves certain teaching behaviors, characteristics, and attributes that are so significantly unique that they demand their own set of evaluative criteria. The paper reviews selected relevant research from 1989 to 1993 and discusses pertinent practical issues involved in the evaluation of public school music teachers, and synthesizes eight general areas for further development as potential criteria for music teacher evaluation. Both empirical and nonempirical evidences are studied, and attempts are made to reconcile the discontinuity that exists between evaluative criteria appropriate for all teachers and that which is appropriate for teachers of music. An appropriate evaluation instrument for in-service music teachers should include the following broad areas that would serve as defining attributes: (1) personal characteristics, as evidenced through student teacher, teacher faculty, and teacher administration interactions; (2) musical competence and performance skill, as evidenced when teaching a lesson; (3) effective use of nonverbal strategies, such as modeling and demonstrating; (4) effective use of verbal strategies, such as eliciting performance from students; (5) classroom management, including discipline, group participation, and the creation of a positive learning environment; (6) effective planning for concept learning and aesthetic appreciation across a wide age span; (7) an objective assessment of teaching style based on empirically supported criteria; and (8) relevant and appropriate professional development activities. Contains 10 references. (DK)
- Published
- 1993
193. Prestige and independence-controlled publication performance of researchers at 14 Hungarian research institutions between 2014 and 2018 - a data paper.
- Author
-
Tóth, János and Demeter, Márton
- Subjects
PRESTIGE ,RESEARCH evaluation ,EVALUATION research ,SCARCITY - Abstract
This data article describes a dataset showing the five-year performance of 471 researchers from 14 Hungarian research institutions, with a total of 3219 observations. Each observation represent items produced between the 1st January 2014 and the 31th December 2018 by a researcher employed in the sampled research institutions from one of six research output types. Due to a prestige and independence-controlled categorization of research output, and the scarcity of easily accessible, well-structured data curated for research performance evaluation, this dataset can play an important role in new research evaluation policies at Hungarian research institutions aiming to enhance global competitiveness by fostering scientific excellence and innovation. [ABSTRACT FROM AUTHOR]
- Published
- 2021
- Full Text
- View/download PDF
194. Course Selection and Performance of Very High Ability Students: Is There a Gender Gap?
- Author
-
Stocking, Vicki Bartosik and Goldstein, David
- Abstract
Recent attention has focused on the decreasing levels of scholastic achievement of youth in the United States, particularly in areas of mathematics and science. In particular, concern has been expressed about the involvement and achievement of girls in traditionally sex stereotyped curricula, such as mathematics and science. This paper reports on an investigation exploring the course-taking and achievement patterns of academically talented girls and boys, selected by Scholastic Aptitude Test (SAT) scores. The subjects in this study were enrolled in the Talent Identification Program's Summer Residential Program at Duke University, an intensive 3-week academic experience. The study found that girls and boys performed equally well in all types of classes. Main effects in achievement and motivation were found only for type of class. The success of the students indicated that the SAT is a valid selection instrument. In addition, the usefulness of the SAT as an identifier of very high academic ability is described. (DK)
- Published
- 1992
195. Identifying a Range of Performance Improvement Solutions--High Yield Training to Systems Redesign--through Evaluation Research.
- Author
-
Dean, Peter J.
- Abstract
This paper describes a needs assessment process that identifies a range of performance improvement strategies using the steps of the Evaluation Research Model (Geroy and Wright, 1988). A detailed description of a project conducted by the Professional Development Committee of a mid-sized community college in suburban Philadelphia to identify professional development activities for the administrators, faculty, and support personnel of the college is used to illustrate this process. All of the challenges identified by outside consultants were addressed: (1) all employees (stakeholders) were involved equally in identifying professional development needs; (2) the planning committee recognized that certain non-training actions were required to maximize the worth of training; (3) data were elicited from all 280 employees that represented both their wants and their needs; and (4) decision makers were provided with a systematic process for selecting professional development activities that will yield maximum results for cost and effort. The project illustrated the number and kind of responses generated through the nominal group technique (NGT), as well as the range of professional development options that emerge from NGT responses. It reflected the cutting edge of performance technology by virtue of its application in an academic setting, involving all employees of the organization, addressing the desire of the organization for training while simultaneously presenting alternative performance improvement solutions, and introducing performance technology as the decision making model for all HRD (human resource development) functions of the organization. The project also provided techniques, references, and examples that can be used in convincing decision makers of the value of conducting a needs assessment. (5 references) (BBM)
- Published
- 1992
196. On Ambition, Learning, and Co-Evolution: Building Evaluation as a Field
- Author
-
Sridharan, Sanjeev and De Silva, Soma
- Abstract
The papers in this dialogue help develop evaluation as a field (both in South Asia and other regions). What comes through in all of the papers is both humility of where evaluation is as a field and imagination and ambition of where evaluators could be in the near future. Reading the papers in this forum makes them enthusiastic about being evaluators. The papers are a testament to the need for ambition about the field--it is probably time to move beyond narrow debates on projects and program evaluation to larger questions of how evaluations can contribute toward development and perhaps more generally social betterment. The papers collectively focus on the importance of learning from prior capacity building experiments and adventures. The papers in this forum serve to celebrate the tremendous opportunities that exist in South Asia to translate this aspiration into "real capacity" that serves the goals of development and social betterment. The authors' view is that this task of translating the vision into practice will take focused ambition, a culture of learning, and an openness to create multiple systems of feedback in learning between the West and South Asia. They summarize and reinforce how the many points in these papers address the "evaluation gap" discussed by Carden. (Contains 6 notes.)
- Published
- 2010
- Full Text
- View/download PDF
197. Exploring Values Underlying the Assessment of Teaching Competence: An Application of Social Judgment Theory.
- Author
-
Hogge, James and Murrell, John
- Abstract
This study was conducted to examine individual differences with respect to values underlying the assessment of teaching competence in order to enhance the effectiveness and fairness of assessment. Nine classroom teachers who were preparing to be mentors of beginning teachers and five teacher educators indicated the relative importance (expressed values) they attached to the following aspects of teaching: teacher/pupil relationships; preparation and management; curriculum content; classroom interaction; assessment and records; and self-evaluation. They then rated the competence of 50 hypothetical student teachers for whom scores were available. Relative weights (implemented values) applied to the aspects of teaching were computed for each participant by employing a regression model, utilizing scores as predictors and participants' overall ratings as criteria. Results found individual differences with respect to both expressed and implemented values, and several participants exhibited discrepancies between expressed values and implemented values. The latter result may reflect a lack of self-awareness and could adversely affect communication among assessors and between assessor and assessee. (Author/LL)
- Published
- 1991
198. Utilizing Participatory Evaluation in a Public School Setting: An Assessment of Teacher Involvement in Decision Making.
- Author
-
Sick, Paul and Shapiro, Joan Poliner
- Abstract
Participatory evaluation is an assessment approach that involves the participants of a study from beginning to end. The evaluation strategy was implemented in a public school district to assess teacher involvement in decisionmaking, determine whether teachers desire more participation, and evaluate the assessment strategy itself. Data were collected through interviews with 12 teachers and 1 principal as well as through results from a survey instrument developed by the interviewees and distributed to their district's entire elementary teacher population. Results from the data analysis indicate that teachers participate in decisions involving curriculum/instruction and student personnel but do not desire more participation in the two areas. However, teachers do want more participation in decisions regarding staff personnel, supervision, budget/finance, and school facilities. Observations made while implementing the evaluation strategy indicate that the participants became actively immersed in the project, explorations of issues were facilitated due to the trusting relationship between participants, discussions of topics were easier because of the participants' broad knowledge base, and representatives of the teacher population felt that the survey findings were accurate. Recommendations for implementing the participatory evaluation strategy include making certain that all participants are knowledgeable insiders. Three tables and a figure are included. (41 references) (EJS)
- Published
- 1991
199. Evaluation Concepts and Practices in Selected Distance Education Institutions.
- Author
-
FernUniversitat, Hagen (West Germany). Zentrales Inst. fur Fernstudienforschung Arbeitsbereich., Schuemer, Rudolf, Schuemer, Rudolf, and FernUniversitat, Hagen (West Germany). Zentrales Inst. fur Fernstudienforschung Arbeitsbereich.
- Abstract
This report contains, in addition to the introduction and preface, 13 papers written by individuals working in the field of evaluation who present the concepts and practices of evaluation at their own particular distance education institutions. The introduction (Schuemer) gives a short outline of the evaluation nomenclature and an overview of the contributors to the report. The papers are: "Orientation and Conceputalization of the Academic Quality Control Centre" (Alavarado et al.); "Evaluation at the FernUniversitat" (Bartels); "System Evaluation: Some Reflections on Current Tertiary Practices in Australia and Hong Kong" (Castro); "Evaluation at a Distance" (Ganor); "Evaluation at Indira Gandhi National Open University, New Delhi, India" (Koul); "Assessment at Empire State College--Strategies and Methods Used in Evaluating Distance Education" (Lehmann, Granger); "Evaluation of Danish Distance Education at University Level" (Lorentsen); "Institutional Evaluation of the National Open University of Venezuela (U.N.A.)--A Proposal" (De Machado, Machado); "Evaluation in University Distance Education" (Moehle, Gerhardt); "Evaluation Concepts and Methods in Distance Education at Humboldt University (HUB)" (Nerlich); "Evaluation Concepts and Practices at NKI, Norway" (Rekkedal); "Evaluation in the Distance Courses Established by the Department of Education of 'La Sapienza' University (Rome)" (Vertecchi); and "Evaluation at the British Open University" (Woodley). (YLB)
- Published
- 1991
200. Misunderstanding and Misrepresentation: A Reply to Hutchison and Schagen
- Author
-
Gorard, Stephen
- Abstract
The author previously published a paper discussing how to conduct an analysis based on a cluster sample. In that paper, the author outlined several widely adopted alternative approaches, and pointed out that such approaches are anyway not needed for population figures, and not possible for non-probability samples. Thus, the author queried the prevalence and relevance of complex approaches such as multi-level modelling (MLM) for most real-life analyses. Hutchison and Schagen (2008) responded with a paper that misrepresents what the author said, in an attempt to defend their favoured technique of MLM. In this article, the author wants to establish their misunderstanding of this important topic. This paper deals first with areas in which they agree, and then areas in which Hutchison and Schagen (2008) simply restate their position, before turning to the more substantive consideration of why it is not necessary or appropriate to use MLM in most real analytical situations. (Contains 2 notes.)
- Published
- 2009
- Full Text
- View/download PDF
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.