Search

This article asserts the importance of e-assessment. It further suggests that assessment questions and tasks will change substantially as the art of e-assessment progresses. The article then exemplifies sophisticated e-assessment tasks, notes taxonomic schemes that have attempted to classify them and seeks to identify aspects of their definition. Next, some key claims for sophisticated e-assessment tasks are summarised and evaluated. These claims are: (1) sophisticated e-assessment tasks can be used to assess novel constructs, and (2) sophisticated e-assessment tasks can be used to address summative and formative assessment purposes. In the final part of the article, issues arising from the article's findings are discussed and necessary areas for further research are noted. (Contains 2 figures and 1 note.)

Background: Education, and information about education, is highly structured: individuals are grouped into classes, which are grouped into schools, which are grouped into local authorities, which are grouped into countries. The degree of similarity among members of a group, such as a school or classroom, is a very important factor in the design and analysis of studies in education. Purpose: The aim of this article is to provide information on this degree of similarity within schools to enable those involved in carrying out surveys of schools to do so most efficiently in terms of resources and minimum disturbance of schools. Sources of data: This paper uses data from 13 studies at primary and secondary level conducted by the National Foundation for Educational Research in England and Wales. Main argument: The degree of similarity among members of a group is measured by two statistics, the intra-cluster correlation and the design effect. The study described here classifies outcomes into a number of categories and estimates intra-cluster correlation and design effect. The relevance of the results to survey design and analysis is discussed, and examples of how to use these are given. Conclusions: The main findings of this study, rather than conclusions as such, are the intra-cluster correlations for each topic category. However, the paper reaches some tentative conclusions about the degree of clustering by topic. Using Hox's convention ("Multilevel analysis: Techniques and applications," Lawrence Erlbaum, London, 2002) for the size of intra-cluster correlations, it was found that the degree of clustering of achievement was high, while ethnic and language variables were highly clustered in secondary but not primary. By contrast, attitudes towards school, educationally relevant home characteristics, and perception of school policies have quite low values of [rho] (mean less than 0.05), defined as small. (Contains 9 tables, 1 figure and 2 notes.)

There is a degree of instability in any measurement, so that if it is repeated, it is possible that a different result may be obtained. Such instability, generally described as "measurement error", may affect the conclusions drawn from an investigation, and methods exist for allowing it. It is less widely known that different disciplines, and different workers within a discipline, have different interpretations of the term "measurement error," and that these different interpretations are liable to give rise to different results. This article is an attempt to describe and systematize a number of conceptualizations of measurement error, and the associated concept of randomness, and to show how these work out in practice. Because of the importance of this topic, the aim has been to make the descriptions accessible, avoiding jargon and other technical matters such as equations as far as is feasible. (Contains 2 tables and 8 notes.)

A recent issue of "International Journal of Research & Methods in Education" (IJRME) contained a challenging article by Stephen Gorard ["The Dubious Benefits of Multi-Level Modelling," "International Journal of Research & Method in Education," Vol. 30 (2), p221-236 (2007)] in which he attacks aspects of current practice in statistical modelling, with particular focus on multilevel modelling (MLM). In this response, the authors believe there is much in his argument which is misleading or misconceived, both at a general and a detailed level, and this article explains where the difficulties arise. The authors start by a brief overview of some of the basic principles of statistical modelling. They then discuss MLM and some of the criticisms in general. Finally, they detail some of the mis-statements in the Gorard article.

This paper reports a comparison of human and computer marking of approximately 600 essays produced by 11-year-olds in the UK. Each essay script was scored by three human markers. Scripts were also scored by the "e-rater" program. There was a good agreement between human and machine marking. Scripts with highly discrepant scores were flagged and assessed blind by expert markers for characteristics considered likely to produce human-machine discrepancies. As hypothesised, essays marked higher by humans exhibited more abstract qualities such as interest and relevance, while there was little, if any, difference on more mechanical factors such as paragraph demarcation.

The Malta Primary Literacy Value-Added study was the outcome of a process of testing and matching at ages 9-10, the pupils in Maltese primary schools who were tested three years earlier, at ages 6-7. All primary schools in the island took part at both stages. Pupils were tested in both Maltese and English at both ages. In addition extensive data on pupils' home background was collected at both ages and added to the individual test records. Some 4239 pupils had complete data at both ages. This paper reports the findings of analyses of the resulting data using multilevel modelling. Individualised reports were prepared for each school, taking account of all the background data collected, and a series of one-to-one sessions were held with representatives of all schools to discuss how these could be used to help improve the education provided. An anonymised example of a feedback report is given. The authors believe that, in addition to being of interest in its own right, this study can provide a template for other states to carry out a high quality study efficiently and relatively inexpensively, using the special resources of such communities.

The work reported here is based on the second Malta National Literacy Survey, carried out in March 2002. All Year 5 pupils in Malta were involved and their literacy attainment in both Maltese and English was tested. Two parallel multi-level analyses were carried out, one for Maltese, the other for English. The background variables were; gender, age, first language, special educational needs, parents' education, parents' occupation, number of Year 5 classes, type of school and region of school. Significant effects were discovered for pupil, home and school variables.

Responds to article "Comparing School Systems to Explain Enduring Birth Date Effects". Addresses article in relation to England and Wales. Refutes argument that selective promotion is a mechanism for the continuation of the birth effect in schools. Agrees that this phenomenon is worthy of further research. (CAJ)

Handbook of Statistical Methods for Randomized Controlled Trials byKyungMann Kim, Franj Bretz, Ying Kuen Cheung and Lisa V. Hampson(Eds.), Chapman & Hall/CRC : Statistics in the Behavioural Sciences Series. 2021, 620 pp, £205 (Hardback), ISBN 9781498714624By Dougal HutchisonReported by Author [Extracted from the article]