Search

Scotland, traditionally, has high levels of confidence in teachers. Fairness and justice are key concepts in policy and practice in Scottish education. For more than 100 years, the high-stakes assessment system in Scotland, with the Scottish "Higher" qualification at its heart, has been crucial to that sense of opportunity and justice. However, in 2019-2020, public confidence in high-stakes assessment in Scotland, as in other United Kingdom countries, was dented. In Scotland, the COVID-19 pandemic meant that schools were closed, teachers provided online learning opportunities for pupils working at home and, for the first time in 130 years, it was not possible to run national examinations. To ensure that learners were not further disadvantaged, alternative approaches to gathering evidence for qualifications were instigated. However, these results were challenged as socially unjust and the results that had been nationally moderated were replaced by results based on locally moderated teachers' professional judgement. As Scotland looks to qualifications beyond COVID-19, trust must be re-built. This article reports on a participative research project that sought to understand public perceptions of standards and fairness across a range of key communities following this experience. Drawing on both qualitative and quantitative data, we analyse factors which affected trust in National Qualifications under the pandemic. The evidence suggests that when considering what matters for qualifications to be trusted, technocratic solutions are likely to be rejected by stakeholders. Understanding and responding to what led to the mistrust of qualifications in Scotland will be crucial to inform its future qualifications system.

The COVID-19 pandemic suspended established practices that, in normal times, are seen as central to the functioning of education systems. For example, in England, school closures led to the cancellation of national examinations in 2020, and their attempted replacement with an algorithmic model. Following public outcry about what were seen as the unjust effects of the application of that model, there was a very public policy reversal, and examination grades were awarded on the basis of moderated teacher assessments or Centre Assessed Grades, resulting in substantial grade inflation. This paper draws on research that investigated the actors involved in examinations policy in this period and focuses especially on the sources of expertise and the kinds of knowledge that were mobilised--or not--in the decision to cancel examinations, to develop the algorithm and to revert to Centre Assessed Grades.

The incorporation of epistemic aspects of science in science education continues to be a challenge for researchers and practitioners. The paper presents an empirical study investigating how epistemic framing of scientific methods can be incorporated in science teaching, learning and summative assessment, and what impact such framing has on student learning outcomes. The study was conducted with 969 secondary students taught by 152 teachers from a national sample in England. Teaching videos and summative assessments were framed by Brandon's Matrix, a theoretical framework derived from the work of a philosopher of science and focusing on the diversity of scientific methods ranging from hypothesis testing to non-manipulative parameter measurement. The findings are discussed, including (a) the students' views on the teaching videos and summative assessments, (b) the impact of the teaching videos on students' understanding of the epistemic aspects of scientific methods and (c) students' performance on summative assessments in the context of science topics covered in high-stakes examinations in England. The findings suggest that the students' understanding of scientific methods significantly improved after watching the videos. Furthermore, the students' performance on the summative assessment items indicated a high level of accuracy in responses.

This article responds to two key concerns in science education: firstly, that policies designed to assess practical work have distorted its use as an effective pedagogical tool. Secondly, it addresses concerns about the lack of research on the assessment of practical work. The article analyses the policy trajectory for the assessment of science practical work, through the GCSE, in the English National Curriculum from 1988 to the present day. Drawing on published research and policy documents, using Bowe, Ball and Gould's policy cycle approach to policy analysis, we first identify three distinct eras which represent different ways of assessing practical work from 1988 to the present day. Second, we demonstrate that the interaction between high-stakes assessment narrows the ways practical work is conducted in schools. The interaction between curriculum policy and assessment policy and its influences on pedagogy for practical work has been influenced by the high-stakes nature of the tests. This is not a unique case to England nor to science education. Finally, we question whether we can design assessments, 'tests worth teaching to', that can withstand the implications of high-stakes testing.

This article addresses whether the introduction of end-of-course, linear General Certificate of Secondary Education (GCSE) examinations changed the socio-economic equity gap in England. The GCSE is a national examination offered in a wide range of subjects and taken by almost the entire 16-year-old age cohort. Between the years 2007 and 2014, it underwent a number of reforms to both the underlying curriculum and the examination structure. At the beginning of the period, examinations were primarily modular in structure where the course was decomposed into discrete units tested in a staged manner. By 2014, all GCSE examinations were linear and the whole course content was tested simultaneously when study was complete. These structural changes and the curriculum reforms mean that the impact of modular and linear testing on the performance of students has been the focus of recent interest. Some educational commentators suggested that modular examinations are more suitable for lower-performing students, including those with lower socio-economic status (SES). This research has been conducted to monitor the socio-economic equity gap in the light of the structural changes. It focuses on GCSE mathematics and concludes that, although there is still a clear gap in attainment between disadvantaged students and their peers, this gap does not seem to be exacerbated by the examination structure. In other words, the linearisation of GCSE mathematics is unlikely to have increased inequity between students of high and low SES.

This paper explores what underlies the recent introduction of a Higher Education Teaching Excellence Framework (TEF) in England. Related changes to the higher education landscape are discussed: the 2017 Higher Education Act and creation of a new HE regulator, the Office for Students. How TEF works and some of the consequences of TEF are outlined. As well as discussing what constitutes teaching excellence and what TEF itself is attempting to signal (which includes graduate destinations), we also analyse the underpinning ideologies and logics of choosing metrics to assess teaching excellence, albeit accompanied by peer panel evaluation of institutional written submissions, in determining Gold, Silver and Bronze TEF outcomes. We introduce the notion of TEF as an index rather than a measure. It is suggested that what underlies the English TEF is not about improving teaching but rather an endeavour to pit universities against each other in a highly marketised competitive system with an oversupply of places, in which student debt levels are rising fast. TEF policy is considered with respect to features that would be required of an intelligent accountability system for higher education teaching quality.

Seldom have comparative studies of educational assessment systems been undertaken, especially regarding their standard setting procedures. This study examines the effects of governance structures on the power relations in standard setting in the dominant school-leaving or university-entrance examinations. We present acritical analysis of the published research and policy documents, including sense-checking with senior assessment practitioners from 22 jurisdictions. The nature of standard setting systems in three cases of Ireland, the USA and India are described in detail to showcase the differences between the following three models of governance systems: nationalised, commercial market and quasi-market. The contribution of this article, then, is to describe the three models of governance systems, to classify the 22 jurisdictions using the three models, and to generate propositions inductively. Thus, the article provides aconceptual basis for extension of this work to other cases to advance the literature cumulatively by theory-building.

The theoretical literature on the meaning of qualification standards depicts a variety of definitions. Some definitions describe properties of examinees, whilst others rely on cohort-level or system-level characteristics. Different definitions can be compatible or contradictory. In this study, stakeholders' views of the meaning of qualification standards in Scotland were collected, using focus groups (82 participants) and a questionnaire (918 participants). Almost 60% of questionnaire participants responded that standards tell us about performances on the assessment (criterion-referencing) and approximately 40% responded that they tell you about an underlying ability (construct-referencing). Few participants considered that maintaining statistical grade distributions every year were important. Discrepancies in views raise questions regarding how an examination board manages the political and technical process of maintaining public confidence in standards. Based upon this Scottish case, the authors argue that social settlements regarding qualification standards are a social contract, and a solely technocratic view of standards is conceptually inadequate. [ABSTRACT FROM AUTHOR]

International large-scale assessments are on the rise, with the Programme for International Student Assessment (PISA) seen by many as having strategic prominence in education policy debates. The present article reviews PISA-related English-language peer-reviewed articles from the programme's first cycle in 2000 to its most current in 2015. Five literature bases were searched, and results were analysed with SPSS. Results map the frequency of publications according to journal, country, and scientific discipline. They also summarise major themes within three identified categories: secondary analysis, policy impact, and critiques. Findings indicated that studies based on the PISA dataset has led to progress in educational research while simultaneously pointing to the need for caution when using this research to inform educational policy.

Predicting item difficulty is highly important in education for both teachers and item writers. Despite identifying a large number of explanatory variables, predicting item difficulty remains a challenge in educational assessment with empirical attempts rarely exceeding 25% of variance explained. This paper analyses 216 science items of key stage 2 tests which are national sampling assessments administered to 11 year olds in England. Potential predictors (topic, subtopic, concept, question type, nature of stimulus, depth of knowledge and linguistic variables) were considered in the analysis. Coding frameworks employed in similar studies were adapted and employed by two coders to independently rate items. Linguistic demands were gauged using a computational linguistic facility. The stepwise regression models predicted 23% of the variance with extended constructed questions and photos being the main predictors of item difficulty. While a substantial part of unexplained variance could be attributed to the unpredictable interaction of variables, we argue that progress in this area requires improvement in the theories and the methods employed. Future research needs to be centred on improving coding frameworks as well as developing systematic training protocols for coders. These technical advances would pave the way to improved task design and reduced development costs of assessments.

This study evaluated rater accuracy with rater-monitoring data from high stakes examinations in England. Rater accuracy was estimated with cross-classified multilevel modelling. The data included face-to-face training and monitoring of 567 raters in 110 teams, across 22 examinations, giving a total of 5500 data points. Two rater-monitoring systems (Expert consensus scores and Supervisor judgement of correct scores) were utilised for all raters. Results showed significant group training (table leader) effects upon rater accuracy and these were greater in the expert consensus score monitoring system. When supervisor judgement methods of monitoring were used, differences between training teams (table leader effects) were underestimated. Supervisor-based judgements of raters' accuracies were more widely dispersed than in the Expert consensus monitoring system. Supervisors not only influenced their teams' scoring accuracies, they overestimated differences between raters' accuracies, compared with the Expert consensus system. Systems using supervisor judgements of correct scores and face-to-face rater training are, therefore, likely to underestimate table leader effects and overestimate rater effects.

Key debates within educational assessment continuously encourage us to reflect on the design, delivery and implementation of examination systems as well as their relevance to students. In more recent times, such reflections have also required a rethinking of who is authoritative about assessment issues and whose views we seek in order to better understand these perennial assessment dilemmas. This paper considers one such dilemma, predictability in high-stakes assessment, and presents students' perspectives on this issue. The context is the Irish Leaving Certificate (LC) taken by upper secondary students (aged between 16 and 18) in order (mainly) to enter tertiary-level education. The data come from 13 group interviews with 81 students across a range of schools in Ireland. Listening to students about complex, high-stakes examining problems has a limited history within the educational assessment literature. The findings from the study address this shortcoming and depict how students' insightful reflections can improve our understanding of these dilemmas. Further, students are more than able to reflect on their own situations with regard to high stakes examining contexts and have important contributions to make to our fuller understanding of those elements that will promote high quality and fair assessment. [The work presented here was from a joint project located within Oxford University Centre for Educational Assessment and Queen's University Belfast.]

In response to the commentaries to their original article, the authors thank the commentators for their remarks and note that there is some general consensus across the commentaries around some major themes: (1) the lack of articulation between assessment and learning theories, particularly in relation to large-scale testing used for accountability purposes; (2) educational assessment as a socially driven activity in which the culture shapes both assessment and what and how we learn; (3) the need to develop learning theories in a way that can be utilized in assessment, for example learning progressions; and (4) using technology in a way that will better align learning and assessment. They address some of these themes in this article. [This article is a response to the commentaries on Jo-Anne Baird et al.'s original article: "Assessment and Learning: Fields Apart" (EJ1149539).]

Educational assessments define what aspects of learning will formally be given credit and therefore have a huge impact upon teaching and learning. Although the impact of high-stakes national and international assessments on teaching and learning is considered in the literature, remarkably, there is little research on the connection between theories of learning and educational assessments. Given the voluminous assessment that takes place annually in systematic ways in most many nations, it is surprising that more has not been gained from these assessments in the development of theories of learning and "vice versa." In this article, we consider both theories of learning and assessment and draw the main message of the article, that if assessments are to serve the goals of education, then theories of learning and assessment should be developing more closely with each other. We consider fundamental aspects of assessment theory, such as constructs, unidimensionality, invariance and quantifiability, and in doing so, we distinguish between educational and psychological assessment. Second, we show how less traditionally considered cases of (a) international assessments and (b) "Assessment for Learning" affect student learning. Through these cases we illustrate the otherwise somewhat theoretical discussion in the article. We argue that if assessment is to serve the learning goals of education, then this discussion on the relationship between assessment and learning should be developed further and be at the forefront of high-stakes, large-scale educational assessments.

Entirely predictable examinations are ones for which the questions are known in advance. Some assessments are designed this way, but in public examinations, predictability is subtler. Students familiarise themselves with the requirements broadly: likely topics that will come up, question formats and how to maximise their marks. If students can predict what they have to do, they can memorise performances, such as essays, and restrict their learning to fit only with examination requirements. The danger is that this focus could undermine curriculum aims. Further, examinations that are overly predictable might produce results that do not generalise to other performances or have predictive validity. This paper presents part of a broader project investigating whether the Higher Level Irish Leaving Certificate (LC) examinations were too predictable. Here, the development of a rating scale for students' views of examination predictability is described. Data were collected from 1002 Irish LC students taking higher level examinations in biology (n = 536), English (n = 749) and geography (n = 387). Students' views on predictability of the examination could be grouped consistently across subject areas into three factors: valuable learning, predictability and narrowing of the curriculum. Belief that narrowing of the curriculum was a good examination preparation tactic had a negative relationship with examination scores and perceived learning value of examinations was positively associated with students' scores in biology and English. These findings indicate that the scoring system rewards students who believe they must study the discipline broadly.

Background: PISA results appear to have a large impact upon government policy. The phenomenon is growing, with more countries taking part in PISA testing and politicians pointing to PISA results as reasons for their reforms. Purpose: The aims of this research were to depict the policy reactions to PISA across a number of jurisdictions, to see whether they exhibited similar patterns and whether the same reforms were evident. Sources of evidence: We investigated policy and media reactions to the 2009 and 2012 PISA results in six cases: Canada, China (Shanghai), England, France, Norway and Switzerland. Cases were selected to contrast high-performing jurisdictions (Canada, China) with average performers (England, France, Norway and Switzerland). Countries that had already been well reported on in the literature were excluded (Finland, Germany). Design and methods: Policy documents, media reports and academic articles in English, French, Mandarin and Norwegian relating to each of the cases were critically evaluated. Results: A policy reaction of "scandalisation" was evident in four of the six cases; a technique used to motivate change. Five of the six cases showed "standards-based reforms" and two had reforms in line with the "ideal-governance" model. However, these are categorisations: the actual reforms had significant differences across countries. There are chronological problems with the notion that PISA results were causal with regard to policy in some instances. Countries with similar PISA results responded with different policies, reflecting their differing cultural and historical education system trajectories. Conclusions: The connection between PISA results and policy is not always obvious. The supranational spell of PISA in policy is in the way that PISA results are used as a magic wand in political rhetoric, as though they conjure particular policy choices. This serves as a distraction from the ideological basis for reforms. The same PISA results could motivate a range of different policy solutions.

The ways in which examination standards are conceptualised and operationalised differently across nations has not been given sufficient attention. The international literature on standard-setting has been dominated by the psychometrics tradition. Broader conceptualisations of examination standards have been discussed in the literature in England, which has curriculum-related examinations at the end of schooling. There has, however, been little analysis of conceptualisations of examination standards in Scotland. Different education systems and examinations operate in Scotland and England, and the stated value positions and processes relating to examination standards differ markedly. This paper critically examines policy positions on assessment standards in Scotland and England through the lens of recent theories of standard-setting. By analysing public statements on standards, the paper illuminates similarities and differences in conceptual bases and operational approaches, and examines the effects of these on outcomes for candidates. We conclude that both systems are operationalising attainment-referencing, but with different processes in Scotland and England and these practices do not fit within previous examination standards classifications. As such, the paper moves examination standards theory forward by concluding that there is at least one superordinate definitional category that draws upon more than one definitional stance.

We investigate the extent to which language versions (English, French and Arabic) of the same science test are comparable in terms of item difficulty and demands. We argue that language is an inextricable part of the scientific literacy construct, be it intended or not by the examiner. This argument has considerable implications on methodologies used to address the equivalence of multiple language versions of the same assessment, including in the context of international assessment where cross-cultural fairness is a concern. We also argue that none of the available statistical or qualitative techniques are capable of teasing out the language variable and neutralising its potential effects on item difficulty and demands. Exploring the use of automated text analysis tools at the quality control stage may be successful in addressing some of these challenges.

Disruption of the status quo caused by the pandemic in England provides a lens to view the priorities embedded in the qualifications system. The Government’s initial priority was to avert grade inflation, followed by a populist turn to teacher assessment after a widespread backlash. Subsequent Government arguments for the return to examinations as the fairest way to assess re-introduced policies which emphasised excellence agenda. Learning loss, a consequence of the pandemic, compounded pre-existing patterns of inequality. Inclusive policies to address this in the qualification system would require fundamentally different ways of thinking compared to, for example, the current adaptations that are made for young people with special educational needs because the modern inclusion agenda is broader. The excellence agenda assumes a competitive system in which educational resources must be rationed; that there will be winners and losers. We question this logic, arguing for principles that would underpin a more inclusive qualification system. Since 2015, pupils must stay in education or training until they are 18. Thus, at very least, rationing educational access before this is unwarranted. A modern approach would be more flexible, putting learners first and embrace diversity rather than standardisation as the main principle for fair assessment. [ABSTRACT FROM AUTHOR]

Much has already been written on the controversies surrounding the use of different test theories in educational assessment. Other authors have noted the prevalence of classical test theory over item response theory in practice. This Special Issue draws together articles based upon work conducted on the Reliability Programme for England's examinations regulator, Office of Qualifications and Examinations Regulation. One strand of the work was methodological, in which we noted the advantages and assumptions of different approaches to investigating reliability of public examinations. A feature of the field that has not been well documented is why psychometrics in general does not always fit the educational, cultural and political priorities of public examinations. Psychometrics has implications for test design and reliability. Public examinations are: curriculum-embedded; it is desirable that the domain is transparent; there are many curriculum changes (which makes item banking more problematical and costly); curriculum exposure affects performance; public examination questions might need to be secure until the examination is released and then, made public; pre-testing is infrequent; there is non-random syllabus representation; item independence assumptions are often broken; score distribution might not be expected to be normal; complex assessment designs compose whole qualifications; a latent trait might not be assumed; unidimensionality might not be considered important; and there might be very weakly described constructs. Implications for measurement of reliability in this context in England include less concern for internal reliability or occasion-related factors and an emphasis upon standard-setting and inter-rater reliability. Psychometricians sometimes view this approach as technically weak and behind the times, but we raise the prospect that psychometrics could seem like an answer to somebody else's problems if the curriculum, questions and washback upon learning are the main concern of the assessment designers. We agree with other authors that the field is currently under-theorised and raise the prospect that future theory needs to take account of the educational context of public examinations. (Contains 3 notes.)

Over recent years, the credibility of public examinations in England has increasingly come to the fore. Government agencies have invested time and money into researching public perceptions of the reliability and validity of examinations. Whilst such research overlaps into the conceptual domain of trust, trust in examinations remains an elusive concept. Little is known about what it "means" to trust in examinations and on what individuals base their trust judgments. This paper reports the findings of research into key stakeholders' perceptions of trust in the English A-level examination system through a series of focus groups comprising (separately) of examination board staff, students, higher education admissions tutors, teachers and the general public. Analysis of the data revealed a number of factors that are integral to the concept of trust in A-levels, including examination standards, the marking of examination papers, the syllabus, communication and provision of information, whether A-levels are fit for purpose and the mode of assessment. The fact stakeholders have different expectations and requirements of the A-level system, which are subject to change, means that it is difficult for the A-level system to engender the trust of all of its users all of the time. (Contains 1 table.)

Survey findings from 1701 Year 11 and Year 13 students across 35 English educational institutions are reported, which indicate young people's hopes and goals, and the ways in which their institutions support them in their aspirations. This research adds to the literature empirical findings using a methodological stance on the study of aspirations that is open to broader perspectives of young people's agendas for their lives. The majority of young people reported high-educational (e.g. university) and career (e.g. a professional or well-paid job) aspirations. Life satisfaction and developing relationships was also high on the agenda of many young people.

This paper describes an exploration into a reform of the A-level qualification in England in 2008; namely, the introduction of the "stretch and challenge" policy. This policy was initiated by the exams regulator and determined that exam papers should be redesigned to encourage the application of higher order thinking skills, both in the classroom and in examinations. The present study comprised two strands that explored the perceptions of students (n = 39) and teachers (n = 27) regarding the degree to which the incorporation of opportunities for stretch and challenge in the new examination papers had been achieved, and the likely effects on teaching, learning and exam preparation. On the whole, students and teachers welcomed the stretch and challenge policy and there were some indications that changes to the design of question papers could have some positive backwash effects. (Contains 2 notes and 2 tables.)

This study compares the aspirations of students in further education (FE) colleges and non-FE post-compulsory settings. Within the context of a survey about their education, 928 students across 26 institutions were asked what their "hopes, goals and plans" were, with space for five open responses. Coding of responses was based upon self-determination theory. Career aspirations were most frequently mentioned, followed by educational aspirations (approximately one quarter of responses for each). Students from FE settings mentioned career aspirations more frequently and educational aspirations less frequently than students in non-FE settings. Aspirations such as personal happiness and life satisfaction were also mentioned by students. The need for FE colleges to support students in developing high aspirations was highlighted. Furthermore, a wider framework of student aspiration for use in both research and educational practice is needed. (Contains 7 tables and 3 figures.)

This study examined rater effects on essay scoring in an operational monitoring system from England's 2008 national curriculum English writing test for 14-year-olds. We fitted two multilevel models and analyzed: (1) drift in rater severity effects over time; (2) rater central tendency effects; and (3) differences in rater severity and central tendency effects by raters' previous rating experience. We found no significant evidence of rater drift and, while raters with less experience appeared more severe than raters with more experience, this result also was not significant. However, we did find that there was a central tendency to raters' scoring. We also found that rater severity was significantly unstable over time. We discuss the theoretical and practical questions that our findings raise.

Introducing new national examinations is a complex, multi-agency policy implementation. However, there have been some high-profile problems in examination systems in recent years. This research investigated what 10 UK managers involved in the process thought were the main problems. Time pressures were recognised as a serious problem by the managers, as well as the politically driven nature of the reform. It is argued that network management explains the lack of high-profile individual leaders, the delegation issues, the apparent lack of traditional management skills, the low level of planning and monitoring, the absence of scoping, a professionalist approach, the lack of separation between policy and implementation and the fact that managers could not specify from the outset what needed to be done for a new round of examination developments. Inter-agency power relationships accounted for lack of negotiation of timescales. Recommendations are made for construction of a qualification development blueprint, scoping of resources, appraisal of the human resources shortage in the UK assessment sector and better stakeholder management of the qualification development policy community by government. (Contains 3 notes, 4 tables, and 5 figures.)

Hundreds of thousands of raters are recruited internationally to score examinations, but little research has been conducted on the selection criteria for these raters. Many countries insist upon teaching experience as a selection criterion and this has frequently become embedded in the cultural expectations surrounding the tests. Shortages in raters for some of England's national examinations has led to non-teachers being hired to score a small minority of items and changes in technology have fostered this approach. For a National Curriculum test in English taken at age 14, this study investigated whether teaching experience was a necessary selection criterion for all aspects of the examination. Fifty-seven raters with different backgrounds were trained in the normal manner and scored the same 97 students' work. Accuracy was investigated using a cross-classified multilevel model of absolute score differences with accuracy measures at level 1 and raters crossed with candidates at level 2. By comparing the scoring accuracy of graduates with a degree in English, teacher trainees, experienced teachers and experienced raters, this study found that teaching experience was not a necessary selection criterion. A rudimentary model for allocation of raters to different question types is proposed and further research to investigate the limits of necessary qualifications for scoring is suggested.

Marking reliability is purported to be produced by having an effective community of practice. No experimental research has been identified which attempts to verify empirically the aspects of a community of practice that have been observed to produce marking reliability. This research outlines what that community of practice might entail and presents two experimental studies on the effects of particular aspects of community of practice on examiners' marking reliability. In the first study, the impact of exemplar work is investigated: examiners were provided with mark schemes and some examiners were provided with exemplar scripts and given feedback about the marking of those scripts. In the second study, the effects of discussion of the mark scheme are explored: all examiners received mark schemes and exemplar scripts, but some examiners did not attend a coordination meeting. Neither procedure (use of exemplar scripts or discussion between examiners) demonstrated an improvement in marking reliability, which called into question the predictive utility of the theory of community of practice.

Presents a study that compared examiners' judgments of the grade-worthiness of candidates' A-level examination work at question paper level and at subject level. Suggests that examiners may have compensated to some extent for the different aspects of the subject tested in different question papers when they made holistic judgments. (CMK)

Advanced-level English and chemistry examinations in Britain were graded in several conditions: with or without examinees' names; with male or female names; and with "male" or "female" handwriting. No consistent evidence of gender bias was found in the marking. (SK)

Newton's article (2010) makes three main contributions to the literature. First, it is transatlantic, bringing together literatures that have been dealing with similar problems, using sometimes different methods and certainly with distinctive educational, cultural perspectives. He points out that neither of these literatures has all of the solutions, but each has something to offer. Second, Newton has moved theory forward from the 2007 book (Newton, Baird, Goldstein, Patrick, & Tymms, 2007). He classifies linking into three categories: phenomenal, causal, and predictive. Whilst the predictive validity of tests has been discussed, the author believes it is new to construe this as a linking or standards construct. However, arguably the main way in which Newton has moved the theorizing forward is by beginning to specify what might constitute causal linking. One of the issues the author has previously raised regarding what Newton terms causal linking is the legitimacy of measures to be included as controls for similar kinds of students taking the two examinations to be linked. A third way in which Newton adds to the literature in this paper is by proposing a different way of thinking about linking--linking as shared meaning of scores for a particular purpose. Thus, one test could be linked with a variety of others, with different purposes in the linking and different meanings attached to them. The author agrees with Newton's conceptualization of linking as shared meaning of scores for a particular purpose. However, the author would reserve the right to conjure different linking relationships for a particular test for different purposes.

A study that proves that computer environment can provide new ways to assess the problem solving skills of able students based on the reports of observations and interviews conducted with the school students is discussed. Both positive and negative effects of the computerised problem solving tests were observed during the study.