Search

The Programme for International Assessment (PISA) is an important cross-national study of 15-year olds academic achievement. Although it has traditionally been conducted using paper-and-pencil tests, the vast majority of countries will use computer-based assessment from 2015. In this paper, we consider how cross-country comparisons of children’s skills differ between paper and computer versions of the PISA mathematics test. Using data from PISA 2012, where more than 200,000 children from 32 economies completed both paper and computer versions of the mathematics assessment, we find important and interesting differences between the two sets of results. This includes a substantial drop of more than 50 PISA test points (half a standard deviation) in the average performance of children from Shanghai-China. Moreover, by considering children’s responses to particular test items, we show how differences are unlikely to be solely due to the interactive nature of certain computer test questions. The paper c...

In the main, attention to disciplinary practices has been neglected in assessment and feedback research (Coffey et al., 2011; Cowie & Moreland, 2015). Only recently, the longstanding interest in authentic assessment (e.g. Wiggins, 1989) has re-surfaced in higher education literature on authentic assessment design (Ashford-Rowe et al., 2014; Villarroel et al., 2018) and authentic feedback (Dawson et al., 2020).\ud To address this gap, in our 2019 call for papers for this special issue, we sought articles that would explore the potential of what we called ‘signature’ assessment and feedback practices. Just as signature pedagogies (Shulman, 2005) have directed attention to discipline- and profession-specific teaching practices in higher education, we used the term ‘signature’ to invite researchers and educators to consider discipline-specific assessment and feedback practices. While these signatures will be authentic to a discipline, the term implies that they will be uniquely characteristic of a particular discipline. Thus, we invited researchers and educators to dig deeply into what makes a discipline or profession special and distinct from other fields. Because attention to disciplines has the potential to connect primary and secondary with tertiary education, which is often siloed in its own journals, the call for papers also explicitly sought examples from different levels of education.\ud Two years later, this special issue contains five theoretically framed and grounded empirical papers that: a) situate particular assessment and feedback practices within a discipline; b) analyse how engagement with those assessment and feedback activities allows students to participate more fully or effectively within the disciplinary or professional community, and c) illuminate new aspects of assessment and feedback. We (Quinlan and Pitt, this issue) conclude this special issue with an article that draws on the five empirical papers to construct a taxonomy for advancing research on signature assessment and feedback practices.

Omitted prior achievement bias is pervasive in international assessment studies and precludes causal inference. For example, reported negative associations between student-oriented teaching strategies and student performance are against expectations and might actually reflect omitted prior achievement bias. Namely, that these teaching strategies are negatively correlated with unobserved prior achievement performance, because teachers offer more support to lower performing students, and not that these strategies cause lower performance. This paper examines omitted prior achievement bias in teaching effects with prior achievement data available for students in England participating in the Progress in International Reading Literacy Study. Further, it proposes an analytical approach to account for omitted prior achievement bias in international estimates. The paper argues that the bias is not simply a technical artefact, but reflects educational mechanisms unobserved in international assessment studies, which can be captured with matched assessment data-sets or with evidence from previous studies. Estimates of these mechanisms can be used to postulate scenarios of the bias across education systems and thereby adjust international estimates of teaching effects as if prior achievement were observed. Potentials and limitations of this approach for studying educational effectiveness with international assessment data are discussed. 25 1 5 27

AbtractThis article focuses on how assessment practices are used by teachers to develop and maintain their own professional standing – how assessment works for them as professionals and the work they must do with it to be successful. Reporting on an empirical study involving interviews with 12 primary Key Stage 2 (7–11 years) teachers and using an analysis focused on teachers’ accumulation of capital through their work, the paper examines how teachers construct assessment data in particular, often carefully managed, ways. This, in turn, subjectifies both pupils and the teachers themselves in ways which create tensions in practice. The paper argues that high-stakes accountability through assessment is unlikely to be helpful in two ways: firstly, it does not actually do what teachers claim in accurately measuring pupils’ progress; secondly, that it is likely to lead to pedagogy that has negative effects on them as learners.

With more constructivist approaches to learning in higher education and more value on teamwork skills, students' oracy (speaking and listening) features more prominently in curriculum, pedagogy and assessment. The paper reports on a study of two first‐year Australian university courses in disciplines with explicit industry orientations and high proportions of international students. Drawing on classroom observations and interviews with the lecturers, this paper investigates their pedagogical designs on oracy and the oracy demands of their assessment tasks. The study found that talk‐based assessment tasks (a group project and a group oral presentation) featured in both courses but the two courses treated students' oracy differently: as product or process. The contrast between the two assessment designs explicates issues around EAL (English as an additional language) student needs, authentic links to industry, the provenance of criteria used to assess performance, perceptions about the relevance of talk and...

This paper draws on semi‐structured interview data and participant observations of senior secondary Physical Education (PE) teachers and students at two school sites across 20 weeks of the school year. The data indicated that the teachers in this study made progressive judgements about students’ level of achievement across each unit of work without explicit or overt reference to the criteria and standards represented in the schools’ work programmes and in the Senior PE syllabus. The teachers’ justification for such an approach was that the criteria and standards had become for them sufficiently ‘internalised’. Determining students’ levels of achievement was for the teachers somewhat ‘intuitive’, being reliant on their memory of students’ performances, and influenced by the construct‐irrelevant affective characteristics of the students. It is argued in this paper that such construct‐irrelevance compromised the construct validity and possible inter‐rater reliability of the decisions made and advantaged some...

hk This paper focuses on reconfiguring assessment processes so that they support a learning function, in addition to the more traditional measurement function. In the first half of the paper we discuss a framework for ‘learning-oriented assessment’ derived from a project carried out in Hong Kong. We conceptualize learning-oriented assessment as containing three key components: assessment tasks as learning tasks; student involvement in assessment; and explicitly forward-looking feedback. The second half of the paper presents an action research case in which the first author implemented some of the principles of learning-oriented assessment within a module in a teacher education context. The module, focusing on multimedia and web authoring, was taught through blended learning with an emphasis on peer learning and project-based learning. A particular feature was the interplay between students’ learning experiences and the module assessments.

In this paper we explore the literacy-curriculum interface reconceptualising this relationship in terms of curriculum literacies . The reconceptualisation is an outcome of a major Australian research study that examined the literacy demands of curriculum in senior schooling. The study was multidisciplinary, multi-theoretical in nature, involving a team of international researchers with expertise in literacy and in curriculum. The paper argues the need for exploring the coherence of literacy demands that students encounter in managing their learning in different contexts and disciplines and the need to incorporate these demands explicitly in instruction and assessment. Our conclusion is that the reconceptualisation of curriculum literacies challenges current constructs of assessment and calls for the domains of assessment to be expanded to include curriculum knowledge, and epistemological domains that take account of diverse ways of working with and in semiotic systems. Student success across the years of schooling hinges on their increasing control of this combination of knowledges and ability to use these productively.

The term 'authentic assessment' has recently gained widespread use in education. This paper explores various ways in which authentic assessment is being interpreted and the relationship between these different interpretations and the original focus of authenticity in learning. The paper explores briefly the ways in which implicit and explicit beliefs about the nature of learning and knowledge formation direct the ways in which authentic assessment is interpreted and used. Educational issues that arise from some implementations of authentic assessment, identified as camouflage, simulation and abstraction, are discussed. The need for authentic assessment to be contextualised through a coherent teaching, learning and assessment domain is stressed.

This paper describes the enlargement of the initial conception of formative assessment with reference to constructivist, sociocultural and situated theories of learning and the concept of co-regula...

Drawing on Kane’s argument-based approach to validity and Toulmin’s later work on cosmopolitanism and diversity, this paper asks whose validity arguments and evidence are being presented in International Large-Scale Assessments (ILSAs), where and when. With a case study of the OECD’s PISA for Development, we demonstrate that validity arguments are assembled, negotiated and transformed by the network of actors. We claim that the challenge of ILSAs is not to establish a single authoritative argument through the displacement of plural interpretations and uses. Instead, the tasks of an argument-based approach should be to create a democratic space in which legitimately diverse arguments and intentions can be recognized, considered, assembled and displayed.

For most of the considerable period of time during which they have been objects of scholarly study, learning and assessment have indeed, as the focal paper of this special issue of Assessment in Ed...

I welcome the opportunity to respond to this paper, which covers a number of important and controversial issues relating to the quantification of educational assessment. The section on assessment f...

This article provides an analysis of the integration of assessment for learning principles in the newly revised five-year Master of Education programme at the University of Oslo, Norway, across didactic subjects, pedagogy and school practice. The analysis draws on lecture notes, student videos and student exam papers among 143 student teachers, aiming to identify (a) the operationalisation of the assessment curriculum at the university campus, and in school practice, (b) how the student teachers use assessment principles as tools in their instructional designs and (c) how they self-assess their teaching practice. Our main finding is that student teachers seem to be more concerned with assessing their students than using self-assessment to improve their instruction. Based on the findings, we argue the importance of relating the teaching and learning activities with the assessment situations used in the teacher education programme. The final version of this research has been published in Assessment in Education. © 2017 Taylor & Francis

The research set out in this paper attempts to identify whether one of three conventional IQ tests is more capable of identifying intellectual potential amongst poor children in Dar es Salaam. To this end 1857 children from 17 government schools in poor districts of Dar es Salaam were asked to complete a questionnaire and undertake a range of tests. The study included teacher, peer and self-nomination. It has been noted that static testing may not fully elicit the abilities of African children. It has been suggested that dynamic testing might provide a more fair and equitable means of assessment. Therefore 101 students took part in a control and intervention group in order to investigate. The findings show a significant correlation between IQ test scores and other test outcomes. Those with larger families and older children perform less well on IQ tests. Peer ability and self-confidence positively influence test scores.

Student examinees are key stakeholders in large-scale, high-stakes, public examination systems. How they perceive the purpose, comprehend the technical characteristics of testing and how they interpret scores influence their response to the system demands and their preparation for the examinations; this information relates to intended and unintended consequences of testing and is a component of an expanded notion of test validity. The research reported in this paper investigates examinees’ perceptions about the secondary school graduation and university-entrance national exams in Cyprus. Interviews with recent examinees reveal the versatility and complexity of their perceptions about the fairness and appropriateness of the system, which are influenced by design features of the exams and by the local context. There are important, mostly unintended, consequences on their in- and out-of-school experience, on school curricula and on instructional practices. Empirical evidence about consequential aspects of examinations contributes to the validity argument needed to support such programmes. © 2014 Taylor & Francis. 21 4 427 441 Cited By :2; Export Date: 19 July 2017

This paper explores assessment and learning in a way that blurs their boundaries. The notion of assessment as learning (AaL) is offered as an aspect of formative assessment (assessment for learning). It considers how pupils self-regulate their own learning, and in so doing make complex decisions about how they use feedback and engage with the learning priorities of the classroom. Discussion is framed from a sociocultural stance, yet challenges some of the perspectives that have widely become accepted. It offers three new views to help explore the concept of AaL: understanding feedback; understanding the learning gap; and exploring vocabularies of assessment. Pragmatically, the ideas examined suggest that teachers may need to consider less about focused and directive feedback, but more about how learners interpret and understand feedback from their self-regulatory and self-productive identities and how vocabularies for assessment can be more collaboratively shared in learning contexts. © 2014 © 2014 Taylor & Francis.

This paper presents the findings of the first phase of a three-year study investigating the efficacy of the digitisation of creative practical work as digital portfolios for the purposes of high-stakes summative assessment. At the same time the paired comparisons method of scoring was tried as an alternative to analytical rubric-based marking because we believed that it was likely that a more holistic approach to scoring would be more appropriate. Researchers created digital representations of the practical submissions of 75 Visual Arts and 82 Design students graduating from secondary school in Western Australia. These digital portfolios were scored using the two methods, with the scores compared to those officially awarded to the physical forms. It was concluded that the digital representations of the Visual Arts submissions had adequate fidelity for the purpose of awarding high-stakes scores particularly using the paired comparisons method. However, the Visual Arts teachers and students were opposed to ...

Norway has seen major changes in the field of educational assessment over the past decade, following the 2001 ‘PISA shock’ that stimulated reform of the entire primary and secondary education systems: new outcome-based curricula with cross-disciplinary basic skills were accompanied by major revision of assessment regulations, comprehensive government projects promoting formative assessment, national tests as a main component in a new national quality assessment system and new regulations for examinations and teacher reporting of overall achievement marks. The paper provides a historical context to the country’s prohibition of formal marking in primary education and the recent tensions determining how assessment criteria should be stated and used for formative and summative purposes. It is argued that Norwegian primary and secondary education is riddled with unresolved tensions as to the role of assessment criteria and national tests, sparked by incremental implementation of assessment policies and princip...

Information graphics have become increasingly important in representing, organising and analysing information in a technological age. In classroom contexts, information graphics are typically associated with graphs, maps and number lines. However, all students need to become competent with the broad range of graphics that they will encounter in mathematical situations. This paper provides a rationale for creating a test to measure students’ knowledge of graphics. This instrument can be used in mass testing and individual (in‐depth) situations. Our analysis of the utility of this instrument informs policy and practice. The results provide an appreciation of the relative difficulty of different information graphics, and provide the capacity to benchmark information about students’ knowledge of graphics. The implications for practice include the need to support the development of students’ knowledge of graphics, the existence of gender differences, the role of cross‐curriculum applications in learning about ...

This paper reports the construction of a teacher rating instrument designed to monitor the social and emotional development of school age children and young people (4–17 years). The instrument was developed by reviewing previously implemented checklists to build an extensive list of behavioural and emotional criteria and through the use of focus groups to establish the views of key stakeholders. The criteria were categorized according to three areas: conduct, emotion and learning. The initial instrument of 21 items was piloted and amended accordingly. The final trial of the instrument was carried out on a sample of 7285 pupils from a cross‐section of UK schools. A principal component factor analysis confirmed the division of the scale into three factors. The instrument was supported by trends showing that the distributions were different for different types of schools and between males and females. The final version of the instrument was amended to include 15 items, five in each category (conduct, emotion...

This paper explores ten common themes concerning assessment practice in Australian education across the six states and two territories. The themes are: (1) a strong curriculum base influencing assessment, (2) the incorporation of school‐based assessment in all certification, (3) preference for standards‐referenced assessment, (4) respect for teacher judgement, (5) increasing vocational education delivery within schooling, (6) multiple pathways to future study and careers, (7) school‐based assessment in the compulsory years of schooling, (8) moves towards outcomes‐based frameworks, (9) issues relating to national benchmark data, and (10) equity issues.