Search

Background:Interest in using arts-informed approaches within research to increase stakeholder engagement is growing; however, there is little work describing how these approaches are operationalised across contexts. This article addresses that gap by exploring the use of arts-informed approaches across three projects. Aims and objectives:We explore how conceptualising research and evaluation as creative endeavours, particularly in arts-informed approaches to co-production, create opportunities to move knowledge into action (knowledge mobilisation). We propose an actionable configuration of context + mechanism = outcome (CMO) to understand the influence of arts-informed approaches to co-production. Methods:Multi-case design and cross-case synthesis was conducted of three studies that used arts-informed approaches. A common focus across our cases was evidence use in the K-12 education sector; however, each engaged with this focus by involving different types of evidence and sets of education stakeholders. Findings:Arts-informed approaches and co-production were influenced by a variety of contextual factors such as relationships between researchers and stakeholders, ethical issues of collaborative research activities, approaches to meaningful stakeholder engagement, co-production of knowledge, capacity-building support and resources, and communication between multi-stakeholder partners. Outcomes included new ways of thinking about research topics based on arts-informed approaches, more positive attitudes about co-production, more relevant and useful research and evaluation findings, and increased openness to future co-productive work. Discussion and conclusions:Four propositions arising from this article include: (1) arts-informed approaches address context specificity and sensitivity; (2) arts-informed approaches promote engagement; (3) arts-informed approaches enhance and intertwine skills; (4) arts-informed approaches broaden thinking about impact.

Abstract: This practice note describes the benefits of integrating mixed methods in a collaborative approach to evaluation with school districts and community partners in southwestern Ontario. We discuss the ways in which the integration of qualitative and quantitative data generated a multi-faceted perspective about a new mental health professional role as a complex educational phenomenon. The insights related to recognizing the role of context, supporting the use of mixed methods with stakeholders, and responding to evolving needs related to youth mental health and well-being in schools highlight the practical value of using mixed methods in a collaborative approach to evaluation.

In this article, the authors speak to the paradox of assessing transformative learning (TL) in higher education. TL theory, developed by Jack Mezirow, is a theory of learning to describe the process of change in how individuals view the world based on previous experiences. Recognizing that the 10 phases of Mezirow’s TL theory are fluid and intertwined, three prominent aspects resonated within the individual narratives: the importance of a disorienting dilemma, the qualities of self-reflection, and liberatory actions. By exploring the complexities, challenges, and possibilities encountered in their classrooms, the shared narratives reveal how students were engaged in TL and embedded within are holistic assessment processes the authors enacted with learners. Throughout this dialogical narrative inquiry focused on assessment, the authors underwent their own TL in the presence of each other, confessing uncertainties and vulnerabilities, thus showcasing the potential to transform understanding with and through reciprocal learning.

Abstract: Strategies for optimizing evaluator education involve ongoing discussions within the field of evaluation. There remains unrealized potential for authentic assessments to support and measure learning that demonstrates the integration of evaluation theory with practice. In this practice note, we advance collaborative evaluation designs as a useful authentic assessment for generating the type of applied learning evidence required within an evaluation-focused graduate course. First, we provide a brief review of educational literature, highlighting the value of authentic assessments and a description of the contexts where we effectively applied the collaborative evaluation design. Next, we offer details of the four instructional processes involved in the authentic assessment: planning, development, feedback, and measurement. Finally, we advance lessons learned from our instructional experiences and the learner benefits of the collaborative evaluation design as an authentic assessment.

Abstract: This practice note illustrates a situation where, as program evaluators, we crept beyond the provisional boundaries set by our Developmental Evaluation (DE) goals to facilitate learning. Our DE was initially focused on one program which was being designed for online delivery in higher education. During the DE of this program, questions and themes arose which had larger organizational applicability; we were asked to help design a strategic learning session that addressed a large group of stakeholders within which existed the tiny subset of stakeholders engaged in the original DE. This practice note describes how we negotiated the emergent purposes of the DE with the need for intentional pivots within the strategic learning session to serve our intended subset of stakeholders with their project as well as stimulate evaluative thinking within the larger stakeholder group.

The purpose of this study was to determine how resource teachers (n = 13) and classroom teachers (n = 12) experience a coaching partnership, in which both teachers work in the regular classroom to support students with special educational needs. The focus of the partnerships was to build classroom teacher capacity around inclusionary practices. Classroom teachers and resource teachers completed questionnaires about the partnerships and their impact. Results indicated that both groups of teachers found the partnerships to be valuable. The partnerships were supported by coaching principles, as well as relationship, attitudes, understanding of students, and time spent together. Benefits of the model included increased support for students; increased learning for students; learning of different approaches, for classroom teachers and resource teachers; shared responsibility for students, between classroom and resource teachers; feeling more valued, for resource teachers; and seeing a model of collaboration, for students, teachers and the community.

The goal of this study was to add to the existing literature by examining the effect of 1:1 iPad integration on the achievement of Canadian Grade 7 students. The school board we worked with...

This study aimed to demonstrate how an international experience within a teacher education program shaped student teachers as global citizens. Our cross-case study of two cohorts of student teachers who participated in an international service learning practicum demonstrates the nuanced ways that international placements influence the development of critical global citizens and the impact on their teaching. Survey and interview responses, collected 3-12 months post-practicum, demonstrate that while there is considerable evidence that participants became culturally aware global citizens, there is less evidence that they became critical global citizens, who actively respond to inequities within and outside of the classroom.

Teacher candidates are required to learn substantial fundamental and practical knowledge often within short, fast-paced initial teacher education (ITE) programs. This study examined assessment education through a case study of 35 teacher candidates enrolled in an assessment/evaluation concentration at one Canadian institution. Using a slow movement framework with Fink’s (2013) significant learning experiences taxonomy, pedagogies were analyzed that provoked slow and significant learning. Findings from multiple data sources revealed trends in significant learning across program phases and pedagogical conditions, including authentic course assessments, cycles of coursework and placements, and collaboration. Directions for future research and ITE programming are provided.

It has been almost 20 years since Shulha and Cousins (1997) published their seminal paper exploring evaluation use. The paper examined a decade, 1986 to 1996, of theory, practice, and research on evaluation use. Since that time there have been significant developments related to the phenomenon of evaluation use. Outside of evaluation a new and burgeoning field has focused on the use of research in practice and policy; in health care the term knowledge translation has been used and in social sciences knowledge mobilization. Despite the rapidly growing body of research from the knowledge field, the different terminology used in evaluation, health care, and the social sciences has created siloed bodies of knowledge, even when working on similar change processes. This may be one of the factors why the large body of literature on evaluation use has received little attention in health care and vice versa. The aim of this article is threefold: first, to examine the developments in evaluation use since Shulha and Cousins’s (1997) paper; second, to explore how the knowledge fields, focusing on knowledge translation and mobilization, can help to further refine and develop our understanding of use; and third, to imagine what future research that interweaves the knowledge field with the field of program evaluation might look like and how it has the potential to serve the contexts where this research would be conducted. Il y a presque 20 ans que Shulha et Cousins (1997) ont publie leur article phare sur l’utilisation de l’evaluation. Leurs travaux ont examine une decennie, de 1986 a 1996, de theorie, pratique et recherches sur l’utilisation de l’evaluation. Depuis lors, il y a eu d’importants developpements lies au phenomene de l’utilisation de l’evaluation. Au-dela de l’evaluation, un nouveau domaine s’est developpe sur l’utilisation de la recherche dans la pratique et les politiques; en sante, le terme transfert des connaissances a ete utilise et en sciences sociales, on parle de mobilisation du savoir. Malgre le corpus rapidement grandissant de recherches dans le domaine des connaissances, les diff erentes terminologies utilisees dans les domaines de l’evaluation, des sciences de la sante et des sciences sociales ont cree des silos, alors ce sont les memes processus de changement qui sont analyses. Ceci pourrait en partie expliquer pourquoi les ecrits sur l’utilisation de l’evaluation ont recu peu d’echo dans le domaine de la sante, et vice-versa. Cet article a trois objectifs : premierement, d’examiner les developpements en utilisation de l’evaluation depuis l’article de Shulha et Cousins (en 1997) ; deuxiemement, d’explorer la facon dont les domaines de connaissance, axes sur le transfert des connaissances et la mobilisation du savoir, peuvent nous aider a mieux comprendre l’utilisation; et troisiemement, d’imaginer les pistes de recherche a cheval sur le domaine de l’utilisation des connaissances et celui de l’evaluation de programme, et d’analyser leur interet pour les contextes dans lesquels ces travaux seraient menes.

Recently, Shulha, Whitmore, Cousins, Gilbert, and al Hudib (2015) proposed a set of evidence-based principles to guide collaboration. Our research undertakes a case study approach to explore these principles in a developmental evaluation context. Data were collected at two points in an 18-month period where an evaluation group collaborated with the program team from a national organization. This article explores the contributions of selected collaborative approaches to evaluation principles as they are applied in a developmental evaluation. The article concludes with a reflection on the implications for collaboration in theory and practice of developmental contexts. Also identified are the practical insights for implementing the principles in evaluation practice. Recemment, Shulha, Whitmore, Cousins, Gilbert et al Hudib (2015) ont propose un ensemble de principes pour guider les pratiques collaboratives. Par une etude de cas nous explorons ces principes dans un contexte d’evaluation developpementale. Des donnees ont ete recueillies a deux moments au cours d’une periode de 18 mois, lors d’une collaboration entre un groupe d’evaluateurs et l’equipe d’un programme d’une organisation nationale. L’article explore les contributions de certaines approches collaboratives au respect de ces principes dans le cadre de l’evaluation developpementale. L’article propose une reflexion sur les implications theoriques et pratiques de la collaboration dans ces contextes. Nous identifions egalement des moyens pour implanter ces principes dans la pratique evaluative.

Qualitative approaches in evaluation continue to expand, and this article focuses on the potential for the Arts to contribute to innovation in evaluation. Arts-informed inquiry is an approach that works within, augments, and extends traditional forms of qualitative approaches. This study documents an educational program evaluation that intentionally plans for, uses, and reflects upon artsinformed inquiry. Three considerations found central to arts-informed inquiry are presented: (a) working with layers of context, (b) enhancing access and engagement, and (c) supporting differentiated communication. Arts-informed inquiry emerged as a purposeful methodology that enhances the processes and forms of representation within the field of evaluation. Resume : L’approche qualitative en evaluation continue a se developper, et le present article met l’accent sur le potentiel qu’ont les arts de contribuer a l’innovation en evaluation. La recherche fondee sur les arts est une approche qui peut etre utilisee dans le cadre d’approches qualitatives traditionnelles, et qui peut elargir la porte de ces approches. La presente etude documente un programme d’evaluation qui tient sciemment compte de la recherche fondee sur les arts, tant en ce qui concerne la planification et l’utilisation que le processus de reflexion. Trois considerations jugees cruciales pour une approche fondee sur les arts sont presentees : (a) l’utilisation des couches de contexte; (b) l’amelioration de l’acces et de l’engagement; et (c) le soutien de la communication differenciee. La recherche fondee sur les arts est devenue une methodologie choisie pour ameliorer les processus et les formes de representation dans le domaine de l’evaluation.

This paper examines the impact of a school board’s one-to-one iPad initiative on equity and inclusion. Data include: questionnaires from Grade 7–9 students, teachers, and administrators; focus groups with inclusion coaches; and interviews with classroom teachers. The results show that the iPads have supported equity among students in the district; there is now less disparity in terms of access to technology on the basis of families’ socio-economic status. The results show that the iPads have also supported the academic and social inclusion of students with exceptionalities; themes that arose across the data sources include: differentiation of content, access to grade-level curriculum, the appearance of sameness, communication and collaboration among students with and without exceptionalities, and positive student affect. Negative implications included the potential for students who struggle with self-regulation to be negatively affected and the potential for the technology to be used in socially exclusionary ways.

Our paper is a conversation with the implicit and explicit expectations of the academy, namely that as doctoral students we must achieve a high degree of intellectual autonomy and generate new knowledge. We contest the implications of this view of learners and learning – that learners are autonomous intellectual agents, that knowledge is a private possession, that cognition is an individual process, and that learning is a static and singular path. We propose instead an understanding of learning as pathmaking, which suggests a creative, richly textured, open‐ended, collaborative and passionate process through which new possibilities emerge. This paper comes out of our shared experiences of doctoral studies in Education. We began with initial conversations about our learning in an Ontario Faculty of Education, which then unfolded into a decision to write a paper together. Our process oscillated between individual writing, shared discussion, and analysis. After jointly identifying six keywords, each of us wr...

There is increasing concern over disengagement of teacher candidates during on‐campus periods of pre‐service programs due to the perceived disconnect between educational theory and practice. In response, teacher education has been called to engage candidates in praxis through authentic learning. In this paper, we bring together the fields of teacher education, authentic learning, and arts‐based learning to articulate a means of enhancing pre‐service programs. Specifically, this paper documents and analyzes the process of engaging teacher candidates in an authentic learning experience of a musical theatre production. Data collected from 30 teacher candidates through rehearsal observations, a post‐production questionnaire, and focus group interviews indicated that participation created a sense of community and belonging, as well as developed skills necessary to carry forward art‐based learning activities into their teaching careers.

Given the recent movement toward standards-based education there is a heightened need for teacher competency in the area of assessment and evaluation. However, despite demands for teacher assessment literacy, there are few instances of mandatory assessment courses in preservice teacher education programmes. Further, there is generally a lack of research guiding the development of effective assessment curricula for initial teacher learning in this area. This study presents one institution's response to developing an assessment curriculum for approximately 700 preservice candidates. The paper traces the curriculum development process from its initial research to its third year of implementation. Specific attention is given to shifts in content and pedagogy over this three-year period as informed by findings from ongoing course evaluations as well as identification of the challenge facing assessment educators.

Using a cross-case analysis of three distinct participatory program evaluations, this paper contributes to current discourse on how evaluators create opportunities for learning. Although each team of evaluators intended to promote planned process use, post-evaluation discussions amongst them identified evidence where use emerged throughout the evaluations. This article describes examples of emergent process use, identifies evaluator strategies, and promotes participatory evaluators as facilitators of learning and catalysts in organizational change. The need for further research related to the integration of evaluator reflection and strategies for enabling learning are identified.