Search

The National Assessment Summit was designed to bring together users and producers of assessments and engage them in a discussion of unmet assessment needs and untapped assessment capacities. The meeting was truly a summit in that national foreign language organizations and associations were asked to nominate representatives to bring their organization's perspective to the forum and also to take back to their organization ideas for action. Specifically, the National Assessment Summit aimed to identify a set of priorities for the next two years (2005-07). At a follow-up National Assessment Summit in 2007 in Portland, Oregon, under the sponsorship of the Center for Applied Second Language Studies (CASLS) at the University of Oregon and the Center for Advanced Research in Language Acquisition (CARLA) at the University of Minnesota, these organizations will gather to report on progress made and to consider an agenda for the following two years (2007-09). The National Assessment Summit identified seven priorities for the coming two years. Each priority is addressed in a section of this series of papers and is authored by one of the participants. These priorities are: (1) Articulation; (2) Assessment Literacy; (3) Suite of Assessments; (4) Test Database; (5) Models and Anchors; (6) Research; and (7) Advocacy. Taken as a whole, these papers describe state-of-the-art language assessment and articulate a two-year agenda for improving assessment and educational practice. This publication includes: (1) Introduction (Carl Falsgraf); (2) Achieving Articulation through Assessment (Jacque Bott Van Houten); (3) Assessment Literacy (Peggy Boyles); (4) Suite of Assessments (Ann Tollefson); (5) Foreign Language Assessment Database (Margaret E. Malone and David W. MacGregor); (6) Models and Anchors (Elvira Swender); (7) Assessments Research (Ursula Lentz); (8) Promoting Language Advocacy Through Assessment (Martha G. Abbott); and (9) Conclusion (Paul Sandrock). Individual papers contain references and footnotes.

This document contains the five papers presented at a meeting at which key issues in evaluating workplace literacy programs were discussed. In "Key Components of Workplace Liteacy Projects and Definitions of Project 'Modules,'" Judith A. Alamprese describes the context for evaluating the National Extension Program, components of workplace literacy programs, and validation of program components. Design alternatives for a national program evaluation and design options for an effective practices study are explored by Jorie W. Philippi in "Measuring Workplace Literacy Program Effectiveness: Evaluation Purposes and Methodologies." In "Advantages and Disadvantages of Longitudinal Designs for Evaluating Workplace Literacy Programs," Elisabeth Hayes examines the following: the short-term nature of literacy gains, random assignment versus other methods, availability of comparison/control groups, and practical feasibility of alternative designs and procedures. The last two papers, "Articulation and Measurement of Participant Outcomes in Workplace Literacy Programs" (Larry Mikulecky) and "Articulation and Measurement of Program Outcomes" (Anthony R. Sarmiento), focus on the relationships between curriculum/instructional design and various participant/program outcomes and assess the availability and appropriateness of data and various data collection methods. Most papers include substantial bibliographies. Appended are the work group conference agenda, biographies of the presenters, and a list of work group participants. (MN)

Evaluation use is a complex, iterative process involving diverse stakeholders and activities. Factors influencing evaluation use has long been discussed, however, little is known about current conceptualizations evaluation use including what counts as use, what encourages use, and how to measure use. Following two previous scoping reviews, we identify enablers and constraints to evaluation use through a scoping review of literature published since 2006 (n = 47). The aims of this scoping review are two-fold: (1) to characterize recent evaluation use literature to identify factors influencing evaluation use, and (2) to coalesce findings, implications, and areas for future practice and research related to use. Our review generated insights regarding publication trends, terminology, factors influencing evaluation use and informing evaluation practice.

Intentional synthesis of research findings is necessary to inform practice, particularly within the research on evaluation (RoE) literature. This study expands upon the work of Coryn et al. (2017) to synthesize the RoE studies pertaining to the domain of evaluation context. Findings from this study demonstrate that organization and program's size, stage, resources, and values/mission can all affect evaluation processes and outcomes. While ordinal dimensions often offer clues as to how the evaluation may be impacted, evaluators are advised to anticipate and explore the systemic role of values/mission throughout the other context domains and dimensions. The findings suggest the specific evaluation context descriptors in the organization and program dimension represent an opportunity for developing research informed practice based on RoE.

Since educational statistics, which are relatively easy to obtain, can only attempt to measure "quality," this paper asks how quality in higher education is assessed and how educational rankings, which are defined as benchmarks or attempts to measure, contribute to this process. The paper notes that while attempts to rank institutions of higher education date to the late nineteenth century, two seminal studies published in the 1980s and 1990s generated immense interest in the academic community and prompted much analysis. Since then there has been a proliferation of annual publications by the media, but the paper cautions that many are not published on a timely basis nor are their data comparable. Some discussion is devoted to the four major ranking methodologies--reputational rankings, citation analysis, faculty productivity, and statistical rankings--and five articles, which are considered classic studies, are noted as are efforts at international rankings. The paper then offers eight requirements for perfect educational rankings, including standardized reporting methods, peer review, multidimensional techniques, and focus on interdisciplinary programs and emerging disciplines. The paper concludes that while statistics-based rankings are based on hard facts, consumers need to be aware of the shortcomings of opinion-based rankings. (Contains 34 reference notes.)(CH)

Research practitioners have traditionally collected and analyzed data using experimental designs and inferential statistical procedures. In addition to being difficult to learn and apply, these quantitative procedures generally leave out such variables as attitudes, feelings, and emotions. This paper describes the nature and role of qualitative data collection and analysis procedures in the context of evaluation research or impact assessment. This method is often used by social service professionals to determine if a given program, process, or procedure has worked; focusing only on readily quantifiable outcomes would appear to leave out quite a few essential social and behavioral elements. The document provides a framework for discussing the two complementary aspects of impact assessment, process and outcome, and relative strengths and functions of quantitative versus qualitative procedures. Following this discussion, several commonly applied qualitative procedures are identified, the power of combining numbers and words in the form of multimethod research designs is explained, and their applications are illustrated in a variety of actual evaluation studies. (Contains 23 references.) (LL)

This research aims to make a methodological contribution to the mixed methods literature by investigating the value-added of empirical studies that are explicitly framed with the purpose of interrogating divergent findings. Case studies introduce the role philosophical paradigms can play in reconciliation process and reveal the potential to advance theoretical understanding and/or to challenge prevailing views as a principal value-added as a potential outcome of research with this purpose. The research makes an additional contribution by proposing that there is a variant of the initiation design that is theoretically framed in ways that anticipates divergence and resists reconciliation.

The Teacher Assessment Program (TAP) presents alternative modes of teacher evaluation to inform the deliberations of the National Board for Professional Teaching Standards (NBPTS). Three experimental methods of assessment--simulation exercise, portfolios, and simulation-based portfolios--were implemented by various groups of teachers. Practical application and observation helped to identify problems and effectiveness. Findings indicate that the methods are useful for teacher assessment and for establishing teachers as a community of professionals. (LMI)

This paper cites several examples of the outcomes assessment activities taking place on college campuses and their impact on the freshman educational experience. Outcomes assessment is viewed as the impetus for faculty to make responsive improvements in higher education practices. Improvements derived from assessment activities are being implemented in the areas of registration and preregistration, advising and student-faculty contact, computer usage, emphasis on basic skills, general education, staff development, student-student contact, and curriculum-related matters. Includes 14 references. (JDD)

This presentation reports the results from a three year research project examining evaluation capacity building in a complex adaptive system (CAS) and using CAS as a lens for the analysis. The NSF funded Complex Adaptive Systems as a Model for Network Evaluations (CASNET) project provides new insights on (1) the implications of complexity theory for designing evaluation systems that promote widespread and systemic use of evaluation within a network, and (2) complex system conditions that foster or impede ECB within a network. The complex adaptive system used was the Nanoscale Informal Science Education Network (NISE Net); a network that has been continuously operating for ten years and is currently comprised of close to 400 science museum and university partners.

This paper provides a framework for the discussion of policy and development of educational quality. Based on the choice of indicators, the purpose of assessment, and the type of evaluation, the paper proposes a whole-matrix approach to addressing the critical issues in assessing and monitoring educational quality. These critical issues include maintaining a balance between the basic functions of monitoring and between school self-evaluation and external evaluation, choosing an appropriate combination of quality indicators, determining who decides which indicators and standards to examine, and managing the differences between school-level and system-level evaluation. The whole-matrix framework, which uses multiple indicators to monitor multiple levels, was used to map profiles of educational quality in Hong Kong schools. A survey of 62 primary schools was conducted in 1991 and 1993. The more comprehensive 1993 survey was administered to 56 principals, 1,407 teachers, 219 classrooms, and 7,102 students. The paper describes how the framework served the purposes of both internal development and external accountability. The information gained from such an evaluation can be then used for educational improvement. Two approaches to enhancing educational quality include the external-control approach and school-based management. Four figures are included. (LMI)

The Child Behavior Checklist (CBCL) is used to assess the behavioral problems and social competencies of children. Its broad usage in both a practitioner and research context has led to misapplications as compared to the CBCL's original intended purpose. Practical applications vary from the more traditional mental health centers and medical contexts to schools and forensic applications. Furthermore, an exhaustive number of citations of the CBCL can be found in the research literature. These 1990's uses of the CBCL most probably do not coincide with the original intent or purpose of the CBCL. Additionally, the composition of the normative sample presents particular difficulties for establishing validity across different groups. Thus given the broad usage of the CBCL combined with potential validity biases, it is the purpose of this study to review the literature regarding the validity of CBCL. This review examines the original purpose of the test, the normative sample, and test development from the purview of content, concurrent, predictive, and construct validity. A call is made for additional research so as to facilitate practitioners' and researchers' understanding of the nature of the CBCL in regards to a given use on a specific population. (Contains 43 references.) (Author/TS)

Standardized testing, usually in the form of a multiple choice test, has dominated educational reform throughout Mississippi for the past 2 decades. Because of the minimal impact that standardized testing has traditionally had on curriculum decisions and classroom instruction, a paradigm shift in assessment format was adopted in the State from a narrow, criterion-referenced minimum competence to a broader, performance-based model. This pilot study examined teachers' perceptions of this new Mississippi Assessment System (MAS) and its extent as well as its impact on teacher judgment, modifications to classroom instruction, and non-traditional as well as traditional assessment formats. A 2-part questionnaire and teacher interviews were developed to collect both demographic and teachers' perceptions data. Participants were K-12 teachers (N=220) located in 2 small school districts in southern Mississippi. Data were analyzed using canonical correlation procedures. Results suggested that 10 demographic contributors influenced teachers' judgments, instructional change, and assessment/program evaluation. Quantitative and qualitative findings indicated that overall teacher perceptions, based on professional judgment, supported the changes in the MAS that combined traditional and non-traditional testing formats and that assessment results could help them select appropriate teaching strategies, curriculum, and educational programs. Participants also expressed a need for change in their classroom instruction to better prepare children for the new, non-traditional formats. (Contains 12 references.) (NAV)

This report provides an overview of the issues related to a recently completed study of the use of computers in primary classrooms in six school districts and some special program classes, i.e., K-2 English-as-a-Second-Language and bilingual classes at one study site and Chapter 1 pullout classes for grades K-5 at another site. The sites used were demonstration sites for a new program of computer-based language arts instruction known as the Apple Software Series: Early Language (ALS-EL). Discussion of the development of the research context, purposes, and design includes consideration of issues related to: (1) assessing the implementation of educational innovations (an understanding of the process of change was emphasized and the change experience for individual teachers was examined); (2) assessing student performance in reading and writing (the Sulzby Writing and Rereading Task was adapted, and the Test of Early Reading Ability by Pro-Ed was chosen); and (3) assessing the effects of technology in classrooms (special attention was paid to the amount of funding allowed for the study, the amount of time necessary for the study, and the development of instruments to measure the implementation of an innovation). Strengths and weaknesses of the study are discussed, and implications for future research on computers in education are noted. It is suggested that a series of site-specific studies be pursued in which site-specific evaluation strategies be developed. It is concluded that operational variations and levels of implementation quality are important reminders of the need to allow time for major changes to take place before judgement about the innovations is passed. (34 references) (DB)

Little research has focused on children's loneliness or on the relationships between children's loneliness and depression. Children are typically labeled as lonely by the external behaviors they may or may not exhibit, rather than by their perceptions of their social relationships. This study examined the Children's Loneliness Questionnaire (CLQ) for stability over time and construct-related evidence of validity. Children (n=279) in grades three through six, from two rural midwestern schools, completed both the CLQ and the Reynolds Child Depression Scale (RCDS) two times, 6 weeks apart. The 24-item CLQ consists of 16 primary items which focus on children's feelings of loneliness. The RCDS's 30 items describe each symptom of depression. Results indicate that the CLQ and RCDS are reliable sociometric measures for children over the age of 8. Large group administration procedures may have hindered younger children's responses on the CLQ. Concurrent-related evidence of validity of both instruments was reasonably high for children in grades four through six. For children in grades four through six, the test-retest reliability coefficients for the RCDS over a 6 to 7 week period was found to be lower than in previously reported research using shorter test-retest periods. (RJM)

Teachers commonly use only two types of assessment--written examinations that test basic skills and direct observation of student learning. Both assessment procedures have been the subject of intense criticism. This study was conducted to investigate the viability of the portfolio model for authentic assessment of student growth, to analyze self-concept related teaching activities, and to develop the reflective abilities of students in grades K-6. Data were gathered from three selected public schools and teachers who implemented portfolios as part of a drug education effort. All subject areas were included in the portfolios along with accompanying documentation in the areas of self-concept growth and drug education curriculum activities. Analysis of 40 portfolios revealed that the majority of students' materials included self-concept activities and some drug education information, and that student worksheets showed completion of both drug education and self-concept learning activities. However, findings provided little evidence of student-based reflection on what the portfolio entries meant to them. Student portfolio evaluations are summarized in tabular form. (LL)

Using a theoretical conceptualization of friendship as a context for development, this study evaluated modifications of standard sociometric procedures to assess friendship. A culturally diverse sample of 237 children in grades 3 through 6 completed two sociometric nominations over a 1-month period and rated the quality of their best friendship and their social competence. Sociometric nominations involved children's degree of friendship with their classmates; nominations were coded for mutuality and for stability of mutuality over the 1-month period. From the results of the study, it is concluded that: (1) the rates of stability of friendship observed were comparable to those reported in studies using standard sociometric techniques; (2) the techniques in this study permitted identification of good friendships; and (3) the assessment of short-term stability of mutuality of friendship enhanced the prediction of social competence, beyond the level achieved by the assessment of mutuality of friendship alone. (MM)

This study was conducted to examine individual differences with respect to values underlying the assessment of teaching competence in order to enhance the effectiveness and fairness of assessment. Nine classroom teachers who were preparing to be mentors of beginning teachers and five teacher educators indicated the relative importance (expressed values) they attached to the following aspects of teaching: teacher/pupil relationships; preparation and management; curriculum content; classroom interaction; assessment and records; and self-evaluation. They then rated the competence of 50 hypothetical student teachers for whom scores were available. Relative weights (implemented values) applied to the aspects of teaching were computed for each participant by employing a regression model, utilizing scores as predictors and participants' overall ratings as criteria. Results found individual differences with respect to both expressed and implemented values, and several participants exhibited discrepancies between expressed values and implemented values. The latter result may reflect a lack of self-awareness and could adversely affect communication among assessors and between assessor and assessee. (Author/LL)

Issues surrounding the testing and evaluation of speaking in Singapore secondary schools are examined, including the questions of what skills and knowledge a speech evaluator should possess and what model of spoken knowledge--Received Pronunciation (RP) or Singapore English--should be provided. Oral tests, developed for the English language section of the Functional Objectives of Language Learning (FOLL) project by the Institute of Education in Singapore to determine the extent to which students achieve the functional objectives of syllabuses, are also surveyed. Finally, an evaluation study of the opinions and attitudes of some Singapore teachers and pupils towards the teaching and learning of spoken English is briefly summarized. It is concluded that the evaluation of speaking in secondary schools is clearly linked with the evaluation and valuing of spoken language by all teachers and by the school system itself. Techniques for evaluating speaking skills are developing, but there is a great need for administrators to acknowledge the complexity and importance of these skills both for teachers and for pupils. (JL)

The purpose of this research was to evaluate the impact of a staff development program on the performance of principals in one of their most important tasks: the annual evaluation of teachers. The objective was to determine whether the effects of the staff development training could be seen in the quality of documentation provided by the principals in their annual review of teachers whom they found to be "unsatisfactory." The data source consisted of a random sample of personnel files of teachers who had been rated "unsatisfactory." Analyses of the results indicated that the quality of documentation improved over the years and that principals were doing more observations. Other findings reported that documentation was more detailed, improvement suggestions were richer, and instructional terminology was being used. Nearly all the records determined classroom organization to be the single most prominent weakness. Appended are 10 references, 4 figure graphs, and 5 data tables. (SI)

This lecture describes the organization and kinds of pedagogical research undertaken in the United Kingdom Open University and, through a case study of educational research in the recently introduced Arts Foundation Course (A102), to show how one type of research can contribute to course organization and development. The Institute of Educational Technology (IET), a unit within the Open University, is primarily involved in pedagogical research and is the focus of this paper. Types of educational research discussed include comparative-evaluative research, investigative-evaluative research, and action-research or theoretical-investigative research. Research methods are also mentioned with interviews and surveys emerging as primary data collection tools. The lecture concludes with a brief discussion of some issues central to the organization, conduct, and purposes of pedagogical research in distance education. (10 references) (DB)

Schools and universities in the United Kingdom are coming to accept performance appraisal as a fact of life. The first part of this paper discusses appraisal policies as "negotiated ambiguities." Ambiguous criteria concerning wording subject to misinterpretation has created reservations on the side of the National Union of Teachers. This study demonstrates how an appraisal scheme, which appears to satisfy all the criteria specified in the government's conciliation service's framework for school teachers, can operate as a ritualistic mechanism for ideological construction. The report presents a guide that outlines the process of improving performance at all levels of the hierarchy in schools. The possibility of a counterhegemonic practice in the appraisal context is also discussed. (18 references) (SI)

The efforts of the Austin (Texas) Public School District to improve the readability and usefulness of technical reports are reviewed. Coincident with, or consequent to, a 1977 letter by W. E. Jones to the editor of an Austin paper (the "Austin American Statesman") complaining about excessively lengthy (500-page) evaluation reports, the district began to separate technical reports and summary documents. Technical reports became in-house documents and final reports took on a more newsy reporting style, with more and simpler graphics, simpler language, suggestions of actions rather than direct recommendations, standardized report features, an executive summary, and design features to guide the reader to important elements. Positive features of an evaluation report should be: (1) succinct description of the study; (2) succinct description of the analysis, in an attachment rather than in the text; (3) succinct description of the results; (4) conversational language; (5) simple presentation of numbers; (6) single idea tables; (7) placing complex tables in attachments; (8) simple, single-idea graphics; (9) popular format; (10) attractive report covers; and (11) a descriptive or inspiring title. Attachments include four pages of before and after reports reflecting the new policies. (SLD)

Research and evaluation activities have gradually evolved from the selection of an improvement and evaluation model, as well as data collection using an attitudinal survey and application of case study procedures along with focus group interviews, to the preparation of summary documents and specific policy recommendations. This document presents models of evaluation, assessment, and profiling that have emerged from the activities and findings of the Career Ladders Research and Evaluation Project. The project, in its 5th year of indepth study of 15 pilot-test school districts, has produced extensive data on specific interrelated components within school systems. The three resultant assessment models and profiling procedures can be applied to any school system (or organization) that has identified the essential interrelated components for reaching functional goals. The models--"Interrelated Components of Program Support and Focus for Effecting Change and Reform in Education,""Assessment Model for Projected District Readiness Levels," and "District Readiness Profile of Strengths and Weaknessess"--provide for individual diversities of readiness levels and allow sufficient time for change within unique systems. (7 references) (KM)

This document contains three papers presented at a symposium on university instruction in human resource development (HRD) moderated by Janice Black at the 1996 conference of the Academy of Human Resource Development (AHRD). "Facilitating Transfer of Learning from the Classroom to the Workplace" (Brenda S. Gardner, Sharon J. Korth) examines a formal educational program that uses David Kolb's experiential learning cycle as a theoretical framework for teaching students about group and team development and helping them transfer their learning to workplace activities. "Team Teaching HRD as a Mentoring Tool" (Timothy G. Hatcher, Barbara E. Hinton) discusses a study in which they examined instructors' perceptions regarding team teaching in a graduate-level course in HRD. "Evaluation Research for the Development of Effective Practice in the Video-Teleconferencing Distance Education Human Resource Development Classroom" (John A. Niemi, Kevin Owens) presents data from student journals, course evaluations, and site coordinator findings regarding the effectiveness of a graduate-level class in HRD that was delivered via video-teleconference distance education. (MN)

This presentation outlines, in operational terms, the components and products of a longitudinal evaluation model for teacher education programs, and discusses an approach to program evaluation that embodies activities within three taxonomical phases referred to as formative, summative, and confirmative evaluation. When applied to competency-based certification, a longitudinal evaluation model for a teacher education program is created that can link procedures and costs to actual effects and competencies. The paper describes this approach, the products produced, and the criteria for completion for each of the three evaluation phases. Tasks outlined set the agenda for a complete teacher education program evaluation system that would track policies and students in each specialty area program during its operational life. In this way, firm estimates of a program's long-term effects, benefits, and costs can be derived. Appendices provide criteria for completion of a formative, summative, and confirmative evaluation; examples of professional teaching competencies and indicators of each competency; information for possible consideration in a longitudinal student data base; and examples of items to include in the entrance, exit, and follow-up assessment instruments. (Contains 27 references.) (LL)

Educators in North Carolina, recognizing a need to develop a more objective, performance-based teacher evaluation system, determined that evaluation of teachers should have both formative and summative outcomes and should focus on improvement of teacher skills. The Teacher Performance Appraisal System (TPAS), a companion to the North Carolina Career Development Program, requires at least three classroom observations of a teacher by either the principal or a peer observer. Both of these programs focus on the growth and development of educators, both teachers and teacher evaluators. A large-scale, state-ordered review of the performance system, as perceived by evaluators and evaluatees, led to the present study, which examines the results of the evaluation system over a 3-year period (1985-86 to 1987-88). The study examines the evaluations of teachers (up to 6,257) in 15 different school districts. In general, results of this study confirm and illuminate the improvement in performance expected of both evaluators and teachers for the first 3 years of a 4-year pilot program. Results also confirm the developmental and interactive nature of these improvements. (IAH)

A study was conducted to determine the function of a firm within the context of public relations practice. The assessment of James E. Grunig's "Indices for Models of Public Relations" (an instrument for measuring the type of public relations practiced by an organization) was of primary concern. This instrument places public relations practice into four models: (1) one-way asymmetrical primarily serves propaganda purposes; (2) one-way symmetrical provides dissemination of information; (3) two-way asymmetrical attempts to persuade; and (4) two-way symmetrical tries to develop mutual understanding. A sampling of 136 public relations practitioners from the state of Washington responded to the 16-item "Indices" questionnaire in order to describe their job. Results indicated there is no significant relation between the one-way scales and the two-way scales and an inability to clearly distinguish job functions. This information indicates that the "Indices" may not be able to distinguish among the four types of organizations. These "Indices," however, are clearly useful in distinguishing between the two types (one-way or two-way) of public relations organizations. The low explained variance and the low loadings on two items suggest that a more fully developed scale may be more useful. A larger number of items may not only increase the reliabilities, but they may also increase the differentiation among the four models. (Four tables of data are included; 10 notes and a copy of the "Indices" and a Job Function List are attached.) (MG)

The development of a reliable and valid instrument to measure levels of school administrator burnout is the objective of this research. The stages involved in the development and assessment of the Administrator Role Perception Inventory (ARPI) were item writing, field testing, participant selection, test administration, and data analysis. The instrument was mailed to the 2,113 members of the Confederation of Oregon School Administrators, with a 62% return. Multivariate and univariate analyses were used to assess internal consistency, content validity, and criterion validity in relation to self-perceptions and perceptions of peers' levels of burnout. Findings point to the need for recognition of burnout and the importance of wellness programs to enhance administrator effectiveness. Twelve tables provide statistical substantiation of the results. (10 references) (LMI)

Academic conferences can promote academic exchanges among researchers. As the core of academic conferences, they inject soul into academic conferences. Here, we put forward the evaluation of researchers' academic conference active degree from the perspective of participation. This paper mainly proposes to apply the "h‐index" and the "p‐index" to the active degree evaluation of researchers, which is a particularly simple and useful method to describe the scientific output of a researcher in academic meetings. The relevant indices are extended to the participation indices, forming the p‐index of participation. Introduce the participation rate to improve the original h‐index and form the h‐index of participation. The evaluation effect of each index is analyzed from the consistency, difference and correlation between them. The results support the conclusion that the p(p)‐index can well reflect the value of researchers' lifetime contribution, and the h(p)‐index can reflect the degree of loyalty of researchers better. [ABSTRACT FROM AUTHOR]

This document defines the standards and characteristics of an assessment center. An assessment center consists of a standardized evaluation of behavior based on multiple inputs. Multiple trained observers and techniques are used. Judgments about behavior are made, in part, from specially developed assessment simulations. These judgments are pooled by the assessors at an evaluation meeting during which all relevant assessment data are reported and discussed, and the assessors agree on the evaluation of the dimensions and any overall evaluation that is made. A "typical" assessment center is discussed, followed by an analysis of the three major elements of an assessment center: (1) skills dimension; (2) assessment exercises and activities; and (3) assessors. Finally, a brief overview of research studies on assessment centers is offered. (16 references) (SI)

The purpose of the Australian conference on mathematical assessment was to address the challenges to traditional methods of assessment that have resulted as part of the call for reform in the mathematics curriculum. The 28 papers presented were: "Who Assesses Whom and To What Purpose?" (Leone Burton; "Assessment of the Learned Structure in Elementary Mathematics and Science" (Kevin Collis); "Concerning about Mathematics Assessment in the United States" (Thomas Romberg); "Assessment: No Change without Problems" (Jan De Lange); "Measurement Models for New Forms of Assessment in Mathematics Education" (Mark Wilson); "Adult Numeracy in an Industrial Context: An Emerging Research Agenda" (Patrick Griffin); "Assessment: As We Sow We Reap" (Gilah Leder); "Developing Assessment Strategies for Mathematics Projects" (Christopher Haines); "New Procedures for Diagnostic Assessment in Arithmetic" (Joy Cummings & John Elkins); "Assessment of School-Based Work" (Sue Jennings); "The Assessment Implications of Open-Ended Tasks in Mathematics" (David Clarke & Peter Sullivan); "Assessment Post Sixteen" (Nigel Price); "Assessment of Challenging Problems and Project Work in Senior Secondary Mathematics" (Gary Davis & Andrew Waywood); "The Development of a Clinical Tool for Initial Assessment of a Student's Mathematics Learning" (Robert Hunting & Brian Doig); "Affective Considerations in Assessing Mathematics" (Beth Southwell & Mon Khamis); "Classroom Practicalities and Constructive Assessment" (David Clarke & Jenny Reed); "Mathematics Profiles (Sue Ferguson); "Comprehensive Assessment at Senior Secondary Level in Victoria" (Max Stephens); "System-Wide Assessment of Extended Mathematical Tasks" (Robert Money & Sue Proud); "Criteria and Standards Based Assessment in Applied Statistical Mathematics" (Graham Maxwell); "Early Experiences in Measurement - Future Directions in Mathematics Programs in Primary Schools (Annette Willis); "Assessing, Grading, and Reporting on Informal Writing for Learning Mathematics" (Andrew Waywood); "Spatial Mathematics: A Group Test for Primary School Students" (Kay Owens); "Patterns of Development with Probability Concepts: Assessment for Informative Purposes: (John Izard); "Assessing Students' Mode of Understanding at the Upper Primary and Secondary Level in the Mathematical Sciences" (John Pegg); "Computerized Assessment in Pretertiary Mathematics" (Kenneth Mann); "Assessment Using a Computer Assisted Learning and Testing System (CALTS)" (Ernest Yu & Rory McAuliffe); and "International Comparison of Grade 7 Students on a Multiple-Choice Mathematics Test" (Vivekanand Mohan-Ram). (CRW)

The main failure mechanisms affecting the reliability of power devices caused by thermo-mechanical stress are the degradation of the solder layer and wire bonding. The Power Cycling (PC) test used to imitate the degradation process provides degradation data for the reliability assessment of the power device. Several studies have been conducted over the years to provide a framework for the implementation of the PC test, including its control strategies. This paper presents an evaluation of the current literature on PC test control strategies to provide the current state of knowledge of the strategies. The first section will present the prominent failure due to thermo-mechanical stress, the basics of the PC test, and its control strategies. Subsequently, the evaluation of literature relating to PC test control strategies is presented in section two. [ABSTRACT FROM AUTHOR]

Individuals are evaluated on their factual knowledge in a myriad of settings such as job interviews and exams. In this paper I ask: How to optimally design such an evaluation scheme? [ABSTRACT FROM AUTHOR]

Workforce development is the most critical factor to maintain a sustainable manufacturing industry in the US. Despite the current efforts being made, job openings in the manufacturing sector exceed applicants, primarily due to a skills gap, resulting in part from the introduction of new advanced technologies and automation. Such technologies may not be immediately included in the manufacturing curriculums in higher education, especially in engineering programs with limited resources and access to capital manufacturing equipment. Virtual Reality (VR) technology offers immersive, interactive, and engaging experiences; and 360-degree media based on real-world recordings can offer a grounded and accurate representation of the world. Through collaborating with manufacturing centers in academia and/or industry, customized 360-degree media on advanced manufacturing technologies can be filmed and then displayed remotely in a virtual environment via VR headsets. This would bridge the skills gap in today's manufacturing education by facilitating open access to these advanced technologies, obviating the need for duplicate capital equipment, and enabling university curricula to keep pace with the industry. In this paper, ongoing work regarding a VR production workflow is presented by applying 360-degree filming to reproduce the scenes of real-world additive manufacturing equipment and adding interactive information to the virtual environment. In this pilot study, 360-degree videos and images of a consumer-grade 3D printer were filmed in the laboratory. Then these 360-degree media were edited in a web browser-based online platform, for creating interactive VR storytelling through multiple 360-degree scenes featuring embedded interactive hotspots. This further enabled a cohesive and interactive VR tutorial for enhancing students' learning in 3D printer operation and additive manufacturing technology. Plans for VR content production and student assessment were also reviewed and discussed. [ABSTRACT FROM AUTHOR]

The main aim of this paper is to provide basic knowledge background for further work. The basis is classification of available methods for multi-criterial evaluation. This classification will serve as a support of project (Comparison of methods of multi-criteria evaluation with user ratings of ITIL tools). In first part of this paper are described different approaches to multi-criterial evaluation and decision-making. Second part presents basics of methods of determining criteria strength and mainly their classification and division into categories. [ABSTRACT FROM AUTHOR]

The aim of this paper is to introduce and evaluate selected determinants of e- Accessibility from the point of view of university students with visual impairment. In the years 2014 and 2015 we have evaluated the level of satisfaction with e-Accessibility of universities in Czech and Slovak Republic through the questionnaire completed by 54 university students with visual impairment with various degree of severity. Simultaneously we were investigating the preferences and most frequent practices of visually impaired students when working with websites, e-learning and academic information systems and information in digital form in general. Despite the enactment and adaptation of accessibility guidelines and various supportive technologies, 43 % of the students reported that they don't have full access to the all electronic information when compared with the sighted peers. At the same time we have confirmed that the digital gap is determined not only by the severity of visual impairment, but it also depends of the selection of adopted supportive technologies. Also we have confirmed that the level of success of visually impaired students is determined also by the quality of collaboration between the members of faculty and non teaching supportive staff, as well as a functional model of education containing elements of universal electronic environment which is inclusive for all student. [ABSTRACT FROM AUTHOR]

Over the past decades, European labour market policies, like the Portuguese, offer not only a set of simultaneous active programmes, to the universe of unemployed registered individuals, but also the possibility of successive participation in different active programmes. Such dynamic participation patterns have as a consequence different average causal effects to participants. Aware of this reality, the present paper discusses how to overcome the limitations of the traditional labour market microeconometric evaluation literature, eminently static, and empirically estimate the impact of dynamic selection. In recent years it is possible to report several developments concerning the introduction of dynamic elements in the traditional evaluation econometric models. For instance, is important to report that the causal effects of dynamic sequences make use of potential results and allow the introduction of intermediate results which determine subsequent sequences. The application presented in this research study uses an administrative dataset containing extensive individual information concerning the active public intervention on the Portuguese labour market. Based on this dataset, it will be shown that it is possible to make a quantitative analysis on the impact participation of an unemployed individual on a sequence of active measures offered by the unemployment office services. The present empirical study analyses the power of the dynamic nonparametric model as a fundamental tool for an active labour market policy evaluation, and allows, even if in an exploratory way, the evaluation of the consequences of sequential participation. Indeed, based on propensity score matching microeconometric evaluation techniques - computed through a probit model - is possible to compute the causal effect of the decision to postpone participation in an active labour programme vis-à-vis the decision of consecutive labour programme participation, on the one hand, and continuous non-participation, on the other. The evaluation results of a consecutive participation decision in comparison with the alternative of non-participation or postponed participation suggest a low degree of effectiveness by the Portuguese public intervention in the labour market. Moreover, the empirical application of a dynamic propensity score methodology seems to be a quite useful tool for the estimation of dynamic causal effects when the effectiveness of a sequence of participation decisions is tested compared to a distinct sequence of decisions. [ABSTRACT FROM AUTHOR]

The abstracted article highlights the research targeted at improving students' writing with a focus on its rhetorical aspect. With this aim in view, the author has studied students' compositions classifying the most typical errors under several headings. Part of the research concerning unclear and incoherent writing was featured in the author's previous articles published in recent years. The abstracted article focuses on the problem of ease and awkwardness. The research was based on the experiment that followed three stages. At the initial stage the author observed the students' writing and classified the rhetorical drawbacks into several categories. The next stage involved instructions on the qualities that, interacting with each other, constitute the notion of a good style. The final stage comprised the students' feedback, which was suggestive of their progress in written compositions. The author singles out typical errors, illustrates them with examples from the students' compositions, and tries to find out the causes of awkwardness in students' performance. As follows from the observations, the major reasons of awkward writing are mother tongue interference and students' carelessness. The first manifests itself in wordiness and word for word translation; the latter - in the overuse of repetition, undeliberate rhyme, and unintended alliteration. The students' feedback leads to certain improvement at the end of the experiment. [ABSTRACT FROM AUTHOR]

Mankind has faced new challenges related to the development of the Internet era, and the formation of a digital society, digital (global) consciousness, a new axiosphere associated with the development of all forms of digitalization and networking. Due to the expansion of the information space to a state of unlimited, weak or impossible assessment of its reliability, there are risks of manipulating consciousness through the "soft power" of politics, art, and education. Such risks can be mitigated by education, which, during the period of widespread digitalization, must find answers to new challenges related to the education of a person in a network society. The article deals with the preparation of a new generation of teachers with new "digital" abilities, ready to overcome the challenges of our time. The empirical study addresses the issue of the range of digital teacher professions needed for life and work in the emerging Industry 4.0. The authors formulate the conditions under which the teacher's readiness to implement digital technologies in educational practice can be achieved. [ABSTRACT FROM AUTHOR]

In modern conditions, education acquires democratic forms of openness and integration with various cultural institutions. The interaction of educational institutions and cultural centers acquires the features of a systematic and structured process. However, the emerging new forms of culture, which should include virtual museums, are not used to the full extent. In this regard, the purpose of this article is to reveal the cultural and educational potential of the virtual form of the museum as a modern technology for teaching and educating younger students. The leading research methods were: theoretical analysis of scientific sources, generalization of historical experience, as well as empirical testing methods and generalization of experimental results. The article presents the historical retrospective of the development of museums as cultural, educational and educational centers in their relationship with educational institutions. The article attaches special importance to virtual museums as the most accessible and educational potential form of culture, and also substantiates the position that virtual museums can be represented as a pedagogical technology. The article presents the results of experimental work on the development of creative activity of younger students by means of a virtual museum. The obtained data confirmed the growth in the number of primary school children with high and medium levels of creative activity, which proves the high efficiency of using Museum technology in the form of a virtual Museum in the development of the creative potential of the student's personality. The theoretical and empirical materials presented in the article indicate the need to improve the training of primary school teachers in mastering Museum pedagogy, which contains the potential for the development of students, including by means of a virtual museum. [ABSTRACT FROM AUTHOR]

Procurement in Public Sector has always been controversial and heatedly discussed. Methods and approaches aiming to support decision making have been the subject of research since the emergence of the Industrial Revolution in the late 1800s. However, and despite efforts of transparency in the public sector, it is not until automatisation of procedures and the systematic use of IT that the procurement decision making has had the need for the support of precise and unambiguous decision making methods. With the emergence of e-procurement in the public sector, both response to and decision making on tenders is now heavily dependent on Multi-Criteria Decision Analysis (MCDA) methods. The research that is the base for this paper aimed at identifying a suitable MCDA method for the specific legal and functional needs of the Maldivian Public Sector. In responding to this research question, the researchers developed a three tier evaluation research approach that is reported in this paper. This approach addressed the three different aspects that determine the suitability of an MCDA method to a particular public sector context: the legal framework, the operational requirements and the characteristics of particular MCDA methods. This resulted in the need to identify the specific legal framework in the Maldives that is defined by the National Constitution, the Public Finance Act and Fiscal and Financial regulations. Furthermore, there was a need to understand the operational aspects of the tendering process, namely the requests for tender, the analysis of the tenders and the final decision making process. Finally, it was necessary to analyse how the characteristics of the different MCDA methods perform against both legal and operational constraints and requirements identified. This paper reports on the research design created to support this process, namely a combination of literature review of both academic and grey literature, a set of focus groups interviews led in the Maldives with public servants and public managers and finally a desktop comparative analysis of MCDA methods. [ABSTRACT FROM AUTHOR]

Due to predominant effects of globalization, a massive price pressure as well as a high product variety manufacturing companies are facing tremendous challenges. To address these challenges by a unique positioning in the market and to meet customers' needs, an efficient realization of development projects is necessary. One approach to increase the efficiency and effectiveness of development projects is set-based design. The principle of setbased design states that despite parallel development of multiple product design solution alternatives the efficiency and effectiveness of the development project can be increased in comparison to a point-based approach. For the operationalization of set-based design the existing uncertainties within the development project have to be identified and mapped to the product functionalities. Based on this evaluation an uncertainty reduction due to a parallel development of product design solution alternatives can be derived. [ABSTRACT FROM AUTHOR]

Personal self-determination is an inter- and intrapsychic process of an active nature, carried out through the "trying" activity of the personality. There is a need for a pedagogical mechanism that mediates the personal self-determination of the future teacher in the process of activity. As such a mechanism, we consider pedagogical support, as the main means of support, we propose to consider the individual trajectory of personality development. The research problem lies in the poor representation of the category "individual development trajectory" in psychological and pedagogical science, the undeveloped procedure for constructing the individual trajectory of the personality development of the future teacher in the educational process. The purpose of the study is to justify the construct "personal development trajectory" of future teachers; to review support procedures for personal self-determination of future teachers using individual trajectory of development of the student. Research methods are an analysis of author's approaches to the interpretation of the terms "individual educational trajectory", "individual educational route", "individual development trajectory"; substantiation of the need to introduce the construct "individual development path" in the context of the personal orientation of education; designing the procedure for supporting the personal self-determination of a future teacher using an individual development path in the logic of a system-activity approach and a general theory of activity. The conducted research allows us to draw a conclusion about the need for further conceptualization of ideas about the individual trajectory of development. The research results can be used for further objectification and operationalization of personal self-determination of future teachers. The perspective of our research will be the questions of further detailing the procedure for supporting the personal self-determination of future teachers using an individual development trajectory as an integrative means of support. [ABSTRACT FROM AUTHOR]

As pressures on healthcare systems increase, due to an ageing population, hospital admission avoidance interventions have been emphasised. These interventions can be difficult to objectively evaluate due to non-randomised roll-out, requiring observational methods with carefully selected control groups. This study aims to identify the defining characteristics of elderly patients receiving admission avoidance home visits. We conducted a record linkage study using routinely collected data to compare characteristics and outcomes of the general elderly population and a subset of high-risk patients. Intervention patients were found to have significantly different demographics and admission rates compared to the general population, having four times higher admission rates at baseline. However, they share similarities with high-risk patients, particularly in that after a period of increased admissions, both groups experienced a reduction in the following year. Identifying defining characteristics of the target intervention population can guide the careful selection of a control group for evaluation. [ABSTRACT FROM AUTHOR]

In this work, for the first time, we investigate Electro-Magnetic (EM) attacks on GPU-based AES implementation. In detail, we first sample EM traces using a delicate trigger; then, we build a heuristic leakage model and a novel leakage model to exploit the simultaneous EM leakages in parallel scenarios. After that, we evaluate the effectiveness of EM attacks on GPU-based AES implementation. Our evaluation results show that GPU-based AES implementation is vulnerable to EM attacks. This work also suggests that GPU-based AES implementation needs to be protected against EM attacks in real scenarios. [ABSTRACT FROM AUTHOR]

The purpose of the present study is to suggest a revision of the team science concept to the more inclusive extra-territorial research team (ETRT). Translational thinking is largely marked by the perception of the team as a thing-like structure at the center of the scientific activity. Collaboration accordingly involves bringing external others (e.g., scientists, community members, and clinicians) into the team through limited or dependent participation. A sociologically promising way to see the team is as an idea: a schema for assembling and managing relationships among otherwise disparate individuals with vested interests in the problem at hand. Thus, the ETRT can be seen as a process as well as an object. This analysis is derived from a qualitative analysis, informed by symbolic interactionist principles, of the implementation of translational science at a major medical center. The case study is a team response to environmental health following an off-coast oil disaster. The ETRT in question displayed the following principles of constructive relationship management: a high sense of adventure given the quick pace and timeliness of team members' response to the oil spill; regular meetings in the community to avoid the appearance of academic hegemony; open access by lay as well as institutional scientists; integration of emergency management coordinators into the group; and the languages of public health, environmental pharmacology/toxicology and coastal culture seamlessly interwoven in discussion. The ETRT model is an appropriate strategy for mobilizing and integrating the knowledge and skills needed for comprehensive science and service responses, especially during crisis. [ABSTRACT FROM AUTHOR]

The research on the evaluation of architectural aesthetics is a complex subject but with far culture significance, which should be investigated and researched from different aspects. This article attempts to set up the evaluation system for architectural aesthetics through the AHP principle. Based on the AHP theory, the evaluation system aims to set up evaluation, analysis and decision model of architectural aesthetics by quantifying architectural aesthetics elements and aesthetic performance, through the research on architectural aesthetics essence and process. [ABSTRACT FROM PUBLISHER]

Evaluation apprehension (the fear that your work or performance may be critiqued) can inhibit knowledge exchange, an essential component of effective knowledge management. The current study examined the effects of evaluation apprehension and perceived benefit of knowledge sharing in two contexts: interpersonal (e.g., face-to-face, telephone) and group (e.g., databases, repositories). Evaluation apprehension was negatively associated with knowledge sharing intentions in both contexts while perceived benefit was only positively associated with knowledge sharing in the group context. Differences in the level of evaluation apprehension and knowledge sharing intentions were found between the two contexts, with evaluation apprehension higher and knowledge sharing lower in the group context. Gender and tenure differences were also observed on levels of knowledge sharing intention, evaluation apprehension and perceived benefit of sharing. These findings highlight the need to consider the socio-psychological aspects of knowledge sharing. [ABSTRACT FROM AUTHOR]

Increased interrater reliability was found for ratings from mixed standard rating scales comprised of generic descriptions of behavior that had been tested for stability using scalogram analysis. Empirical comparisons to behaviorally-based summated scales and graphic scales revealed no significant differences on any psychometric criteria. [ABSTRACT FROM AUTHOR]