Search

Background: Mid-semester evaluations are gaining traction as a means to gather evaluation data for formative purposes. However, it is not clear if course coordinators who conduct these evaluations are adequately equipped with evaluative knowledge and skills to guide them through their evaluative processes. Objectives: This study is a meta-evaluation of course coordinators' mid-semester evaluative practices in a Higher Education institution language centre and the extent to which their evaluative practices are attuned to RUFDATA for its preparatory design framework in the evaluative process and Utilization-Focused Evaluation (UFE) for its sets of principles with regards to uses and users and how these impact on the design and use of the evaluation. Research design & subjects: Data was gathered using meta evaluative mixed methods from three groups of participants consisting of course coordinators, academics who taught on their courses and their learners. Results: The results imply that whilst most course coordinators who participated in the study conducted mid-semester evaluations and that their evaluative practices contained some elements from the frameworks, it is problematic to align their evaluative practices and processes against these tools due to the challenges that they encountered in their evaluative processes. Conclusions: As important agents of evaluations in mid semester evaluations, it should be noted that course coordinators are not trained evaluators and conducting evaluations does not constitute a major component of their professional duties and roles. This study has implications for higher education policies and practices, commissioners, and users of evaluations.

Concerns with educational quality have led to the implementation of external school evaluation (ESE), based on the premise that these processes can provide valuable information about schools and, consequently, create conditions for improvement. Improvement is based on the feedback, commonly in the form of an evaluation report, resulting from evaluations, describing the reality of each school, and providing clues and guidance for action and progress. Nonetheless, ESE still has a relatively weak impact on overall school improvement. With this in mind, this paper focuses on the potential of evaluation reports to promote improvement, aiming to answer the question: What kind of feedback on school self-evaluation (SSE) does ESE provide to schools? Focusing on the Portuguese case, the paper analyses the feedback regarding school self-evaluation provided in evaluation reports from the northern region of Portugal. The study concludes that the feedback provided in the reports is mainly descriptive and generic, referencing issues that apply to all schools rather than targeting issues specific to each school. This leads to the hypothesis that the vagueness of ESE feedback can explain the limited contribution external evaluations make towards SSE improvement in particular, and school improvement overall. The example of Portugal and self-evaluation can help bring to light where ESE processes are underperforming and require investment to achieve their goals.

The present paper investigated how inclusive vocational excellence (IVE) practices specific to vocational education and training (VET) can be described in an effective and transparent manner. A framework for the description of such practices was proposed, and then 44 descriptions of practices were collected with this framework from an international sample of providers of such practices from four countries. Expert raters have rated the quality of these descriptions, in terms of how much they respect the proposed structure, provide relevant information about the proposed practice, provide relevant information about the manner in which the practice was implemented and offers a clear and explicit description, further usable by other interested VET institutions. Statistical analyses have then revealed that the framework generates more consistent descriptions for some domains than for others. Implications for practice are discussed. [Note: The volume and issue number (24, 1[47]) shown in the header on the PDF is incorrect. The correct volume and issue number are 25, 1(49).]

Based on interviews and case studies in nonprofit organizations, this article examines the perspectives of evaluation advocates undertaking evaluation in culturally diverse contexts where interpersonal barriers may exist. Participants provided insights into how to overcome issues by developing meaningful interpersonal relationships, finding shared goals, and providing encouragement. This article presents practical strategies that anyone promoting evaluation may find helpful to develop positive interpersonal relationships and equitable and inclusive working environments.

In the last three decades, the promotion of stakeholder involvement in evaluation has been gaining relevance in the Latin American and internationally, across varied agencies, institutions, and civic organizations. The 2030 Agenda and the Global Evaluation Agenda have also recognized the centrality of participation in evaluation. This article explores stakeholder involvement in evaluation based on collaborative work with stakeholders from 15 evaluative experiences. It shows what characterizes participatory evaluation in the region today and the principles of this practice.

Factors influencing evaluation use has been a primary concern for evaluators. However, little is known about the current conceptualizations of evaluation use including what counts as use, what efforts encourage use, and how to measure use. This article identifies enablers and constraints to evaluation use based on a scoping review of literature published since 2009 (n = 47). A fulsome examination to map factors influencing evaluation use identified in extant literature informs further study and captures its evolution over time. Five factors were identified that influence evaluation use: (1) resources; (2) stakeholder characteristics; (3) evaluation characteristics; (4) social and political environment; and (5) evaluators characteristics. Also examined is a synthesis of practical and theoretical implications as well as implications for future research. Importantly, our work builds upon two previous and impactful scoping reviews to provide a contemporary assessment of the factors influencing evaluation use and inform consequential evaluator practice.

The notions of indigenous knowledge and cultural philosophies are becoming ubiquitous in many social inquiries, and evaluation is no exception. Nonetheless, the pursuit to embed relevant indigenous philosophies in contemporary evaluation has yet to succeed. In this article, we discuss indigenous relational philosophies, approaches, and practices as they relate to evaluation. Using qualitative research approaches, we interviewed 43 Indigenous development leaders and other local representatives in three local government areas in Ghana. Utilizing evidence synthesis approaches through a triangulation process, we conclude that indigenous knowledge and other cultural ethos were distinct in community-based development evaluation processes. There was an elusive intersection between indigenous and contemporary evaluation paradigms. Indigenous evaluation has principles such as community spirit, mutual trust, self-organization, relational patterns or networks, "ubuntu" ideals, consensus building, and collective action that can complement contemporary evaluation for the effective and efficient evaluation of community development programs and social policies. We identify key indigenous elements and other indigenous relational assessment patterns to aid in the design of an indigenously driven relational evaluation framework. The evaluative competencies embedded in indigenous philosophies are vast, thus, a call for future research is proposed.

Traditional teaching methods struggle to convey three-dimensional concepts effectively. While 3D virtual models and virtual reality platforms offer a promising approach to teaching anatomy, their cost and specialized equipment pose limitations, especially in disadvantaged areas. A simpler alternative is to use virtual 3D models displayed on regular screens, but they lack immersion, realism, and stereoscopic vision. To address these challenges, we developed an affordable method utilizing smartphone-based 360° photogrammetry, virtual camera recording, and stereoscopic display (anaglyph or side-by-side technique). In this study, we assessed the feasibility of this method by subjecting it to various specimen types: osteological, soft organ, neuroanatomical, regional dissection, and a dedicated 3D-printed testing phantom. The results demonstrate that the 3D models obtained feature a complete mesh with a high level of detail and a realistic texture. Mesh and texture resolutions were estimated to be approximately 1 and 0.2 mm, respectively. Additionally, stereoscopic animations were both feasible and effective in enhancing depth perception. The simplicity and affordability of this method position it as a technique of choice for creating easily photorealistic anatomical models combined with stereoscopic depth visualization.

College and career readiness is crucial to success in postsecondary education, employment, and independent living. The College and Career Readiness for Transition (CCR4T) assessment is valid and reliable and can assist in transition planning for students with disabilities. The CCR4T assessment was designed for planning with students with and without disabilities. This article aims to provide an overview of this assessment and demonstrate how secondary educators and other stakeholders can use the data gathered to make decisions and inform the Individualized Education Program (IEP) as an age-appropriate transition assessment for youth with disabilities.

Immersive virtual reality (IVR) offers significant transformative potential for science education by supporting learning experiences that deeply engage students and improve their understanding of scientific concepts. Despite considerable interest, research on the use of IVR in science education is still in its formative stage. Currently, there is a substantial gap in a tool that can help stakeholders evaluate key elements of immersive software for science education contexts. This research addresses this gap by conceptualising and applying a framework designed to assist educators, researchers, and designers in assessing essential components of an immersive science application. The framework highlights three key components: IVR technological affordances, the exploration of science within IVR, and scientific representations. These components are synthesised into the Immersive Representations Model (IRM). Employing screen capture methodology, we evaluated the application and significance of the IRM. This study pioneers a structured approach to evaluating immersive technologies in science education.

Students' evidence-based reasoning was examined across two studies. In Study 1, students were asked to evaluate newspaper excerpts including anecdotal, descriptive, correlational, and causal evidence provided in support of causal claims as well as to justify their quality ratings for two of these excerpts. In Study 2, students' justifications for quality ratings were further probed and students were asked to select the criterion that they considered to be most important to consider in evaluating information. Key findings included that while students were fairly effective at discounting anecdotal evidence relative to evidence that was descriptive, correlational, and causal, students did not seem to distinguish among these three latter evidence types, when these were provided in support of causal claims. While students cited a variety of justification criteria for their quality evaluations, they were found to rate methods-related factors as more important to consider when evaluating quantitative, rather than anecdotal, evidence.

Educational apps can help children develop their early numeracy skills, but only if they incorporate certain pedagogical features. AppLINOU is an early numeracy app that was co-designed by a multidisciplinary team to be compatible with the way numeracy is taught in France's preschools. This study of whether incorporating AppLINOU into classroom teaching has a beneficial effect on students' early numeracy skills is the first quasi-experimental study of an educational app to be conducted in France's preschools. The study included 537 preschool students (mean age 4.42) followed up to first grade (mean age 6.32). Students in the experimental group (n = 345) used AppLINOU during their lessons to practice ten numeracy activities over 25 weeks. Students in the control group (n = 192) did not use AppLINOU. Both groups of students followed the same syllabus. Multilevel growth models show that students in the experimental group had progressed more by the end of the experiment than those in the control group.

We extended current knowledge by examining the performance of several Bayesian model fit and comparison indices through a simulation study using the confirmatory factor analysis. Our goal was to determine whether commonly implemented Bayesian indices can detect specification errors. Specifically, we wanted to uncover any differences in detecting under-fitting or over-fitting a model. We examined a conventional Bayesian fit index (the posterior predictive p-value), approximate Bayesian fit indices (Bayesian RMSEA, CFI, and TLI), and model comparison indices (BIC and DIC). We varied the type and severity of model mis-specification, sample size, and priors. We focused on the ability of these indices to detect model under- or over-fitting. We provide practical advice for applied researchers regarding how to assess and compare models using these common indices implemented in the Bayesian framework.

A good assessment system is one of the preconditions for quality education. Formative assessment is comparatively an emerging idea to assess the students throughout the academic year with the intention to identify and overcome the weaknesses of the students and enhance their learning outcome. Taking these into account, this study attempted to explore the teachers' perceptions of the use of formative assessment in enhancing teaching and learning in Bangladesh higher education. A mixed-method study was employed with survey and semi-structured interview as the data collection methods. 100 participants were randomly (simple random) selected for the survey, and 6 participants were purposively selected for the interviews. For analysing the data, descriptive analysis and content analysis were used. The findings of the study revealed majority of the participants agreed that formative assessment is crucial to enhance teaching and learning in Bangladeshi colleges. Nonetheless, there are some challenges like - teachers' biasness, shortage of teachers, large class, poor infrastructure, insufficient power supply, and heavy workload of the teachers. As a whole, this study will provide a fundamental ground for future research in formative assessment in Bangladeshi colleges specifically and for comparative study with other higher education institutions globally.

This study examines the perceptions of Jordanian science teachers' conceptions of assessment, and how these perceptions are affected by the teacher's gender, length of experience, and subject variables. The teacher conception of the assessment questionnaire was completed by 488 teachers. It consisted of 27 items spread over four dimensions: school accountability, student accountability, assessment improves education, and assessment is irrelevant. Results showed a high mean for assessment being irrelevant, followed by assessment improving education, student accountability, and school accountability. The results also showed significant differences in the teachers' perception of school accountability by gender, as the female teachers' mean score was higher than that of the males. There were significant differences in the perception of assessment that improves education, again by the teacher's gender, with female teachers scoring higher. The results showed no significant differences in the perception of any of the dimensions according to the length of teaching experience. Finally, there were significant differences in perceptions of assessment in student accountability according to the subject, the mean score for physics teachers being higher than that for chemistry and biology teachers. Perceptions of assessment improving education and assessment being irrelevant, also varied according to the subject, the mean score for mathematics teachers being higher than that of the chemistry, biology, and physics teachers.

Emerging Artificial Intelligence-enhanced technology platforms in education warrant attention to exploring new learning strategies and dynamics. Keeping up with the accelerating momentum to bring classic traditional learning activities to Artificial Intelligence-supported platforms may unbalance the interest in developing the participants' higher-order thinking. This article presents case study research of an Artificial Intelligence-based technological platform to measure complex thinking traits of higher education participants in an Ideathon learning scenario. The didactical strategy was grounded in the Transition Design approach, with Sharing Economy as the challenge. An overview of the process for developing Artificial Intelligence-supported activities, the challenges and risks identified in the development, and a classification model and enhancements for future implementation in a subsequent pilot are presented. The findings set a guideline for balancing Artificial Intelligence-powered educational activities and the development of the participants' complex thinking.

The article describes a systematic review of 109 studies using the data-informed decision-making framework to understand the sources of assessment data and the main purposes for assessment use among early childhood educators. Findings revealed assessment use is driven by the purposes of measuring and monitoring children's learning and development, informing instructional next steps, communicating with caregivers, and identifying students for early intervention. Standardized assessments of multiple domains, teacher observations, student work samples, and student background information emerged as key sources that inform decision-making. We noted several factors that likely influence teachers' assessment use to consider in preparing teachers for DIDM. We describe these considerations in terms of structural-, teacher-, child development, and assessment tool-related influences. Early childhood assessment use is different from primary or secondary contexts that largely inform the DIDM literature. We conclude with implications for preparing educators' DIDM by presenting six skillsets to foster their ability to engage in critical thinking about standardized assessments, engage in intentional observation, generate high-quality data, engage caregivers as consumers and sources of data, use multiple data sources for early intervention identification, and advocate against accountability-driven DIDM.

This study analyzes evaluation influence theories to understand their unified contributions to a conceptual framework for research on evaluation influence in non-western contexts. Specifically, these theories are analyzed according to their usefulness for interpreting the consequences of the Annual Status of Education Report (ASER)--a cyclical, large-scale educational monitoring and evaluation effort in India. An ethnographic case study of ASER facilitates an empirical investigation of the field's evaluation influence theories and uncovers a new source of influence--evaluation design. The study's findings challenge a common belief in the field: that evaluation process and results are the two fountainheads of evaluation impact. Design, as a distinct third source of influence, gives rise to a novel form of evaluation consequence--design diffusion. The phenomenon of design diffusion illuminates why ASER's model for evaluation has been adopted by several other Global South nations.

Evaluations are considered of key importance for a well-functioning democracy. Against this background, it is vital to assess whether and how evaluation models approach the role of citizens. This paper is the first in presenting a review of citizen involvement in the main evaluation models which are commonly distinguished in the field. We present the results of both a document analysis and an international survey with experts who had a prominent role in developing the models. This overview has not only a theoretical relevance, but can also be helpful for evaluation practitioners or scholars looking for opportunities for citizen involvement. The paper contributes to the evaluation literature in the first place, but also aims to fine-tune available insights on the relationship between evidence informed policy making and citizens.

Formative evaluation provides stakeholders with timely feedback to support an intervention's improvement during implementation to maximize its effectiveness. We describe the qualitative methods that guided one study within a formative evaluation of a multicomponent care delivery intervention. We then describe the challenges and lessons learned that emerged from this study, organizing them by the study's four overarching challenges: (1) addressing multiple research questions, (2) working with a large interdisciplinary team, (3) triangulating qualitative results with quantitative results, and (4) studying implementation in real-world delivery settings. Overall, the evaluation generated important findings to support improvement of the intervention during implementation. We hope that sharing the lessons learned will increase the rigor and efficiency with which formative evaluations of complex care delivery interventions are conducted and the likelihood that they will improve implementation in real time. We also hope the lessons learned will enhance the satisfaction of the researchers working on these evaluations.

Recent improvements in advanced artificial intelligence (AI), including large language models (LLMs) such as ChatGPT, and the subsequent release of these models into the public domain have instantly provided the public with technical capabilities that previously required years of engineering education to acquire. The ability of these models to perform technical tasks such as writing MATLAB code and generating corresponding technical reports challenge these and other components of a traditional engineering education with obsolescence. However, these models are inherently limited, primarily in their ability to evaluate and create, which are strictly human capabilities. The traditional focus of higher engineering education, to create what we've termed Human Calculators (HCs), should adapt to develop instead what we've termed Advanced Engineer (AEs) by embracing AI and elevating the importance of their humanity, namely through the integration of culture and the arts into the engineering curriculum.

The purpose of this study is to find an authentic assessment in field study activities that can be used to shape the professionalism of geography teacher candidates. The method used in this research is Research and Development. Research instruments: documentation study guidelines; model authentic assessment; Instruments for prospective professional teachers with peer ratings. Authentic factual model assessment limited testing of students at the Jakarta State University Geography Education. The factual model was improved to be retested. The hypothetical model was tested on 120 respondents in geography education students at Jakarta State University & Indonesian Education University. Data analysis used by path analysis, with the help of AMOS software program 5. The type of authentic assessment is positively related, namely the performance appraisal of tool use (0,36) and project assessment (0,24), which has a negative relationship namely attitude assessment (-0,13), presentation performance (-0,14), and portfolio assessment (-0,40). Teacher professionalism includes four competencies. Field study can make personality and social competencies of geography teacher candidates, but it lacks in making pedagogic and professional competences.

This study aims to investigate the opinions of teachers working at public, private, and International Baccalaureate (IB) Schools on the evaluation practices, the use of assessment tools, the frequency of documenting data, and the feedback frequencies given to both students, and parents. A questionnaire has been used to collect data from 168 teachers. 84 of these teachers were from public schools, 30 from IB, and 54 from private schools. Percentages and frequencies were examined to describe the data, and the chi-square test was conducted for their distribution. There are two main conclusions derived. The first one is that state schools and private schools differ from IB schools in terms of exam-oriented preparation and evaluations when traditional methods are employed. The second one is that IB schools differ from state schools and private schools in terms of documenting the data and reporting of assessment for learning improvement and process-based evaluation. In terms of teachers' opinions, the ones from the IB schools believed there was a difference in terms of having an assessment policy, collaborative planning processes with partners, and experts, taking into consideration the individual differences, and having an inspection mechanism to monitor of the processes.

The European Research Council (ERC) receives many high-quality applications, but funds only a few. We analyze how members of ERC review panels assess applications in the first, highly competitive step of evaluations for ERC Starting and Consolidator Grants. Drawing on interviews with ERC panel members in different fields, we show that they adopt a set of "evaluation devices" that offer pragmatic and standardized ways of evaluating in a time-constrained and highly competitive setting. Through the use of evaluation devices, panel reviewers enact and generate a distinct "reviewing expertise" that encompasses subject-specific knowledge and knowledge about how to accomplish evaluation within a situated setting. We find that ERC panel reviewers employ four evaluation devices during the first step of ERC reviews: first, reviewers base judgments on applicants' prior achievements ("delegation devices"); second, they adjust their evaluations of individual applications to the quality of a given set of applications ("calibration devices"); third, they combine multiple elements to assess the feasibility of proposals ("articulation devices"); and finally, they consider the impact of the proposed research on science and society ("contribution devices"). We show that the current use of these devices generates what we have termed "evaluative pragmatism": a mode of reviewing that is shaped by and accommodated to the need to review many high-quality proposals in a short time period with possibly limited expert knowledge. In conclusion, we discuss how the prevalence of evaluative pragmatism in the first step of ERC panel reviews shapes candidate selection, particularly regarding human and epistemic diversity in European research.

Evaluation policy has been identified as an important means of shaping and influencing organizational evaluation practice, yet, to date, little empirical research has been conducted to deepen our understanding of this relationship. The purpose of this study was to illuminate evaluation policy's role in leveraging organizational capacity to do and use evaluation. We interviewed 18 published evaluation scholars and practitioners from North America and Europe about this topic. A thematic analysis of findings underscores the importance of context, policy attributes, enablers, and organizational benefits. Based on the findings, we developed an ecological conceptual framework to guide thinking about the role of evaluation policy in capacity building. We discuss these findings in terms of practical implications of understanding context, redressing the imbalance between learning and accountability purposes of evaluation, and organizational leadership, and we conclude with some implications for research.

Many institutions implement assessment teams as resources to develop faculty and staff knowledge of and confidence in programmatic assessment processes. Additional resources may include rubrics or peer review and feedback, but effectiveness of these resources is rarely evaluated. Programmatic assessment allows institutions to examine the impact of multiple resources to determine which positively impact assessment outcomes. This quantitative study examined administrative and student affairs units' perceptions of assessment resources supported by the Office of Institutional Effectiveness (OIE) at one large public, southeastern university. The findings corroborated the positive impact of rubrics and peer review and feedback, providing a basis for continued support of many of the institution's existing resources. Although institutions cannot control the utilization of resources available, they can be more certain that the resources provided are beneficial to those who seek them.

The purpose of this sequential transformative study was to elucidate the negative experiences of teachers with performance evaluations and to juxtapose the intended use of current popular teacher evaluation reform movements to the evident implementation. One may quickly assume that negative experiences with evaluation are a result of unsatisfactory teaching practices. However, this may not accurately explain the negative experiences. This study focused on the negative experience of teacher evaluation to provide a broader understanding of the impact of new evaluation policy reform on student achievement and teacher quality. With a paucity of previous research focused on the negative impacts of teacher evaluation, this study addressed the following questions: (1) "How does the Peer Assistance and Review (PAR) teacher evaluation process negatively impact teachers?" (2) "What, if any, parallel traits exist among those teachers who had negative experiences with the PAR evaluation system?" and, (3) "How does the intended use of the PAR teacher evaluation process compare to the evident use of PAR?" Data revealed dissonance among intent and evident use of the evaluation policy. A disproportionate number of African Americans, women over the age of 55, and teachers higher on the pay scale were referred to PAR. Vague policy language was suggested as the impetus for misuse, abuse, and biased implementation at the local level. This study suggests that policymakers and school district officials take heed of multiple perspectives and consider the negative impacts of teacher evaluation reform. Evaluation systems that prioritize teacher learning over accountability are integral to successfully improving student achievement.

This study investigated the impact of 'assessment for learning' on learner performance in Life Science. Simple random sampling was used to select four schools from the King Cetshwayo District of KwaZulu Natal Province, South Africa, to participate in the study. A quasi-experimental, pretest-posttest comparison group design was used, involving four schools--two forming the 'treatment condition' while the other two served as the 'comparison group'. Altogether, 160 grade eleven learners participated in the study. Two teachers were trained to use assessment for learning (AfL) as an instructional approach, while the two teachers of the comparison group used their usual instructional approaches. Data were analysed using SPSS (V23) and the statistical technique used was the 2-factor ANOVA with repeated measures. The result revealed that learners following an AfL instructional approach performed statistically higher that those following normal classroom instruction. This result is discussed, and recommendations made in respect of both classroom practice and further research. The findings of this study had implications for policy, further research as well as instructional and assessment approaches to be used in the teaching of Life Science in the South African education system.

Evaluation has not been used to its fullest potential in environmental education (EE). Pressures from external stakeholders can cause organizations to focus on reporting requirements at the expense of conducting evaluations that support programmatic improvement. Understanding practitioners' satisfaction with their evaluation processes and the drivers of this satisfaction may reveal strategies for improving evaluation processes in the field of EE. We administered an online survey to EE practitioners in the United States via email and social media. Our results indicate low satisfaction with evaluation processes overall, but greater satisfaction from organizations engaged in systematic "formal" evaluation, particularly processes focused on adaptive management and programmatic improvement, when compared to evaluations focused on satisfying external accountabilities or using "informal" evaluation processes based on nonsystematic observation and reflection. Our results also highlight current gaps in using evaluation for adaptive management and for furthering diversity, equity, and inclusion efforts in the field.

COVID-19 is disrupting assessment practices and accelerating changes. With special focus on K-12 and credentialing exams, this article describes the series of changes observed during the pandemic, the solutions assessment providers have implemented, and the long-term impact on future practices. Additionally, this article highlights the importance of the balanced assessment system, the use of assessments both for learning and of learning, and using assessments to support for social justice, equity and inclusion. These desired uses and outcomes will continue to challenge assessment and measurement experts on how they design, develop, and implement assessments moving forward.

Increasingly, attention is being turned toward the actions of students in relation to feedback. This paper presents the outcomes of a questionnaire study researching what undergraduate and postgraduate architecture students perceive as the purpose of feedback, how they use feedback on their coursework, and what makes them more likely to act on it. An overarching theme in their conception of feedback's purpose was that of a guide for improving coursework and learning. Two key dimensions that facilitated use over the longer term were feedback being generative and informing their working methods. The study revealed students engaging in a range of strategies with both formative and summative feedback to prioritise their next work; these could form the basis of facilitating wider adoption of these methods across the student body. There were similarities, but also substantial differences, between how undergraduate and postgraduate students reported perceiving and utilising feedback, which may have impact on feedback provision to enhance uptake and use. Whereas undergraduate students saw feedback's purpose in terms of directing improvement, postgraduates were more likely to see their own role in the process. Postgraduates were more likely to question feedback's validity over the longer term.

Many documents are produced in evaluation processes; documentation of student evaluation can therefore be time-consuming. Nevertheless, we have little knowledge about how these documents are used. This paper is based upon analysis of documents that direct and report on internal student evaluation practice at a Norwegian university. Interviews with academics and students are used as supplementary data. We analyse how student evaluation practice is described in university documents compared to what academics and students express. The aim is to explore how documents can contribute to the constitution of evaluation practice. The study shows that the university mainly requested information about how evaluation was carried out and followed up. The academics reported on challenges with evaluation practice, use and results. These reports were supposed to serve as background for educational quality reports at departments and faculty level. However, the information from programme reports was nearly absent from reports at higher organisational levels; they mainly described a well-established evaluation practice. There were misalignments between how student evaluation practice was described at different levels, in documents and in interviews. This may hinder improvements to evaluation practice and may also contribute to an understanding of student evaluation as an accountability tool more than a practice promoting learning.

In "The Testing Charade: Pretending to Make Schools Better", the author's new book from which this article is drawn, the failures of test-based accountability are documented and some of the most egregious misuses and outright abuses of testing are described, along with some of the most serious negative effects. Neither good intentions intentions nor the value of well-used tests justifies continuing to ignore the absurdities and failures of the current system and the real harms it is causing. Here the author presents some options for doing better.

In this column, the author revisits who we are as assessment professionals, how we are trained, what our role is, and what our responsibilities are--to ourselves as a community, to our colleagues in education, at our institution, and to those we serve-- especially from an equity lens. They also share a few strategies to address the gaps they see.

This survey study investigated how graduate school admissions committees perceive and use the "GRE"® General Test and "GRE"® Subject Tests after the launch of the "GRE"® revised General Test in August 2011. These perceptions and uses impact the validity of the tests. Prior research about the perceptions and uses of the General Test and Subject Tests was last conducted in 2002 and, prior to that, in 2000 (for writing) and 1984. Therefore, even without test revisions, perceptions and uses of the GRE tests may have changed. Overall, by graduate discipline area, by graduate degree level sought, and by future career track preparation (research vs. professional), we examined online survey responses from 163 individuals involved in graduate school admissions. We did not find major changes in the use or perceived utility of the revised General Test or Subject Tests for admissions or funding decisions. General Test scores are valued in relation to other admissions information. Sometimes, valuation of these scores leads to practices proscribed by published GRE Program guidelines, such as the use of cut scores. Scores on the Subject Tests continue to be valued less than other admissions information. Perceptions and uses of the GRE tests often vary when decision makers consider applications from international applicants and underrepresented racial or ethnic minorities. That admissions committees can receive and compare scores reported on the former General Test score scales to scores reported on the revised General Test score scales has resulted in various practices, including the common use of percentile ranks to compare scores in ways discouraged by GRE Program guidelines. To the extent that decision makers use concordance information recommended by Educational Testing Service (ETS) to compare scores, they do so with mixed success.

Background: Evaluations are a useful tool to learn more about the effectiveness of public measures. In the era of evidence-based policymaking, recent research suggests that quality is an important determinant of the utilisation of evaluations. Despite this claim, hardly any empirical study has investigated whether the quality of an evaluation -- measured by a meta-evaluation -- influences its perceived utilisation by decision makers. Aims and objectives: This article asks how the quality of an evaluation study is related to its perceived use, and investigates the relationship between the quality of an evaluation, assessed through a meta-evaluation, and how the evaluation is perceived and accepted by the parties concerned. Methods: The basis for the empirical analyses were 34 external evaluations, conducted from 2006 to 2014, of upper secondary schools in the canton of Zurich, as well as a standardised survey conducted among 307 representatives of these schools (teachers, administrators, members of quality development teams, and the heads of school oversight commissions). Findings: We conclude that the quality of the evaluation, as assessed in a meta-evaluation, is not particularly associated with the perception of evaluation quality and the perceived use of the evaluation. The perceived quality, however, is related to the perceived impact of an evaluation. Discussion and conclusion: These findings are relevant for evaluation research and practice, since they show that the quality of an evaluation and evaluation use do not necessarily go hand in hand.

Background: Conceptualisations of what it means to use evidence in policymaking often appear divided between two extremes. On the one side are works presenting it as the implementation of research findings - particularly evaluations of intervention effect. In contrast stand theoretically informed works exploring the multiple meanings of evidence use, political complexities, and the constructed nature of research evidence itself. The first perspective has been criticised as over-simplistic, while the latter can make it difficult to answer questions of what might be good, or improved, uses of evidence in policymaking. Methods: To further debate, this paper develops a 'programmatic approach' to evidence use, drawing on theories of institutional decision making and empirical work on evidence use within 11 National Malaria Control Programmes in Africa. We apply the programmatic approach by investigating the key goals and tasks of programme officials, recognising that these will shape the routines and logics followed affecting evidence utilisation. We then map out the forms, sources, features, and applications of evidence that serve programme officials in their goals. Findings: In the case of malaria programmes, evidence use was understood in relation to tasks including: advocacy for funding, budget allocation, regulation development, national planning, and identification of information gaps - all of which might require different evidence sources, forms, and applications. Discussion and conclusions: Ultimately the programmatic approach aims to facilitate clearer understanding of what uses of evidence are appropriate to policymakers, while also allowing critical reflection on whether such uses are 'good' from both programme and broader social perspectives.

Background: African countries are developing their monitoring and evaluation policies to systematise, structure and institutionalise evaluations and use of evaluative evidence across the government sector. The pace at which evaluations are institutionalised and systematised across African governments is progressing relatively slowly. Aims and objectives: This article offers a comparative analysis of Africa's national evaluation policy landscape. The article looks at the policies of Zimbabwe, South Africa, Nigeria, Kenya (not adopted) and Uganda. To achieve the aim we unpack the different characteristics taken by the national evaluation policies, emerging lessons for countries who wish to develop a national evaluation policy, and key challenges faced by countries with regard to evaluation policy development and implementation. The article draws on both a desktop review and action research approaches from the Centre for Learning on Evaluation and Results Anglophone Africa to build national evaluation systems across the region. The approach has included peer learning and co-creation of knowledge around public sector evaluation systems. Key conclusions: The national evaluation policies reviewed share certain common features in terms of purpose and composition. They are also struggling with common issues of institutionalising the evaluation system across the public sector. However, there are variations in the countries' guiding governance frameworks at a national level that shape the nature and content of policies, as well as the ways through which the policies themselves are expected to guide the use of evaluative evidence for decision and policymaking, and programming.

Large-scale assessment (LSA) is a tool used by education authorities for several purposes, including the promotion of teacher-based instructional change. In Canada, all 10 provinces engage in large-scale testing across several grade levels and subjects, and also have the common expectation that the results data will be used to improve instruction in classrooms. Yet despite agreement between ministries that instructional change based on LSA results is a positive development and employs data-based decision making at its heart, there remain significant differences in the kinds of incentives written into assessment policies in Canada. It is also true that implementation of the policies is less than uniform between schools and school divisions. Using mixed methods (survey data and follow-up interviews), this study examines which policy factors have the most significant impact on teacher decisions regarding the use of data. The findings indicate that highly incentivized policies correlate well to instructional change, including aspects of both teaching (to) the curriculum as well as teaching to the test. Since the latter will be examined as a neither an educationally defensible practice nor a stated policy goal, the statement that "incentives work" does not fully capture the nature of these impacts.

The purpose of this mixed-methods study was to gain insights and understandings of high school teachers' perceptions and use of formative assessment to enhance their planning, individualization of instruction, and adjustment of course content to improve student learning. The study was conducted over two years in a midwestern high school of approximately 1,000 students. Crucial to the three project teachers' understanding of formative assessment was developing and using preset curriculum road maps that tightly aligned course goals, learning objectives, activities, instructional methods, and assessment. The in-depth case studies of the sample's three teachers revealed that, when provided with specific information about formative assessment through staff development, they became more positive toward such assessment, and their implementation skills were greatly improved. The staff development had an especially positive impact on the teachers' understanding and skill sets for individualizing instructional practices. The personalization of the staff development proved to be the most beneficial when it tailored the content to the varying levels of initial proficiency of the three sample teachers. Support for formative assessment by the administrative team members was essential to creating a cultural shift from summative to formative assessment.

Based on the 2015 evaluation of response-to-intervention (RTI) efforts and our own 2 decades of experience in supporting educators' implementation of RTI efforts, four recommendations are presented to advance effective implementation of tiered systems of intervention. We suggest that by (a) assessing readiness and capacity, (b) providing content and coaching as part of professional development, (c) using evaluation data, and (d) including students with disabilities, educators can make strides to implement RTI more effectively and help to meet the needs of all students in today's schools.

With abundant access to assessments of all kinds, many high school chemistry teachers have the opportunity to gather data from their students on a daily basis. This data can serve multiple purposes, such as informing teachers of students' content difficulties and guiding instruction in a process of data-driven inquiry. In this paper, 83 resources were reviewed to provide a complete description of this process, which has not been previously done. In reviewing the literature, we found that: 1) there is very little research detailing the data-driven inquiry process in a way that can be readily implemented by teachers; 2) the research largely neglects the incorporation of disciplinary content in the data-driven inquiry process; 3) suggestions for teachers' actions provided by the research is general, limiting the impact of these suggestions; and 4) the practical considerations and fidelity of implementation of data-driven inquiry have not been examined. Implications for chemistry teachers are presented along with a call for future research on key areas, thus benefiting researchers of assessment processes. Finally, general data-driven inquiry research is described in the context of chemistry-specific examples that provide useful, practical suggestions for high school chemistry teachers.