Search

High-stakes external assessment for practical courses is fraught with problems impacting on the manageability, validity and reliability of scoring. Alternative approaches to assessment using digital technologies have the potential to address these problems. This paper describes a study that investigated the use of these technologies to create and submit digital representations of practical production work and forms of creative expression for summative high-stakes assessment. The study set out to determine the feasibility of students creating and submitting these digital representations for assessment and to identify which of analytical or comparative pairs scoring generated the more reliable scores. This paper proposes that scoring digital representations of creative practical work submitted by students is a viable alternative to traditional approaches to assessment.

This paper reports on the perceptions of teacher education student of the value of using digital tools to analyse video-based information on the practice of teachers in classrooms. This strategy was employed to address a dilemma in providing sufficient exposure to real examples of good classroom teaching to link theory with practice. The use of video of teachers in action could augment their practicum experience in schools but this required efficient and effective access to appropriate video material. For this purpose, the use of digital tools in a system called LessonLab was successfully implemented in an introductory unit focussed on learning theories at Edith Cowan University. Data collected using a survey of students and focus group discussion indicated that for almost all the students the experience was valued and the system operated effectively.

In this article we describe a three-year study that was conducted in three phases to evaluate the feasibility of assessing digitized portfolios of student creative work for high-stakes purposes. The first two phases suggested that creative work could be digitized with adequate fidelity, and that students could submit their own work from schools to an online portfolio system. These portfolios were assessed using online tools to facilitate both an analytical and comparative judgment method of scoring, with the latter appearing to be the more reliable. In the final phase this approach was evaluated to facilitate moderation and professional learning of standards with rural teachers. The study showed that comparative judgment is a viable and reliable method for assessing digitized creative products and coupled with online communication systems provides an effective and efficient approach to scoring, moderation and teacher professional development, particularly in rural locations.

Traditional moderation of student assessments is often carried out with groups of teachers working face-to-face in a specified location making judgements concerning the quality of representations of achievement. This traditional model has relied little on modern information communications technologies and has been logistically challenging. We argue that social online moderation, coupled with the use of analytical and pairwise scoring methods and technologies, can provide better moderation outcomes and highly valuable professional learning experiences improving teachers' understandings of assessment standards. This paper reports on a component of a study involving Visual Arts teachers from rural schools making comparative judgements of digitised student artworks. We report the teachers' observations of the social online moderation processes, including the quality and standard of the digitised artworks, the effectiveness of the pairwise comparison process, the functionality of the online tools, and the concept of using online scoring for moderation and standard setting purposes.

This paper reports on the outcomes of a three-year study investigating the use of digital technologies to increase the authenticity of high-stakes summative assessment in four Western Australian senior secondary courses. The study involved 82 teachers and 1015 students and a range of digital forms of assessment using computer-based exams, digital portfolios and audiovisual recordings. The results were analysed using a feasibility framework concerning manageability, technical facility, functional operation and pedagogic alignment. By the end of the study, each form of assessment that was implemented was found to be feasible once some obstacles were overcome. Two methods of marking were tried, analytical rubric-based marking and holistic comparative pairs marking, with the latter found to generate more reliable scores. With the increased use of digital technologies in schools and the expectation that children will achieve more complex performances, more use of digital forms of assessment will be required.

This article reports on an investigation to advise a teacher education institution on the feasibility of having a "Bring Your Own Digital Device" policy for students. The investigation built on components of two research projects while adding the comprehensive testing of representative potential hardware and software platforms. The initial emphasis was on identifying the digital technology demands of our teacher education courses. A representative range of potential devices was tested against these demands. At the time of testing, all laptop devices were found to be adequate but only the iPad was adequate in the mobile range. In a survey of a sample of students, nearly all owned an appropriate device but most made limited, if any, use of it at the university.

Portable computing technologies such as laptops, tablets, smartphones, wireless networking, voice/stylus input, and plug and play peripheral devices, appear to offer the means of finally realising much of the long heralded vision for computers to support learning in schools. There is the possibility for the technology to finally become a ubiquitously invisible component of the learning environment, empowering children to attempt feats well beyond their current capabilities. These technologies are finding a place in many schools, and there has now been over two decades of research conducted into their use in schools. What is now known about implementing portable computing technologies in schools? What should educational leaders take from this research before making decisions about the technologies used in schools?

This paper presents the findings of the first phase of a three-year study investigating the efficacy of the digitisation of creative practical work as digital portfolios for the purposes of high-stakes summative assessment. At the same time the paired comparisons method of scoring was tried as an alternative to analytical rubric-based marking because we believed that it was likely that a more holistic approach to scoring would be more appropriate. Researchers created digital representations of the practical submissions of 75 Visual Arts and 82 Design students graduating from secondary school in Western Australia. These digital portfolios were scored using the two methods, with the scores compared to those officially awarded to the physical forms. It was concluded that the digital representations of the Visual Arts submissions had adequate fidelity for the purpose of awarding high-stakes scores particularly using the paired comparisons method. However, the Visual Arts teachers and students were opposed to digitisation. For the Design portfolios teachers and students were supportive of digital submission, but the structure of the portfolios reduced the reliability and validity of scores, particularly from analytical marking.

Worldwide, fewer and fewer work tasks are done using paper and pen, yet most high-stakes assessment in schools continues to use this primitive technology. This paper reports on one component of a project investigating the use of digital technologies to facilitate assessment tasks for high-stakes summative purposes in senior secondary courses. It reports on how a computer-based production exam was implemented in 17 final-year secondary school classes for the Applied Information Technology course in Western Australia. Further, it compares the comparative pairs method of marking with a more traditional analytical method. The conclusion was that this digital form of assessment should replace the existing paper-based exam; however, it was less certain which method of marking should be applied.

An applied Information Technology (IT) course that is assessed using pen and paper may sound incongruous but it is symptomatic of the state of high-stakes assessment in jurisdictions such as Western Australia. Whereas technology has permeated most aspects of modern life, including schooling, and more has been demanded of education systems in terms of outcomes and participation, methods of summative assessment have changed little and are seriously out of alignment with curriculum, pedagogy and the needs of individuals and society. This paper reports on an analysis of some of the data from a component of a study into the feasibility of using digital technologies to achieve greater authenticity in summative performance assessment in the Applied Information Technology (AIT) course in Western Australian secondary schools. In the first phase of the study a sample of 115 students completed a digital portfolio and a computer-based exam that were both externally assessed using online tools and by two methods of marking, with the results analysed using Rasch modelling software. A traditional analytical method and a comparative pairs method of marking were investigated. The study found that both the digital portfolio and computer-based exam were implemented without significant technical difficulty and were well accepted by the students and teachers. The work output in digital form was readily accessed from an online repository by external markers using a standard web browser. The two methods of marking provided highly reliable scores, with those from the comparative pairs method being the more reliable. A number of questions of validity and manageability were raised and the strengths and weaknesses of the two forms of assessment revealed. It was concluded that it was feasible to implement either form of assessment for high-stakes purposes, with a resulting improvement in alignment and authenticity. (Contains 5 tables and 3 figures.)

There is a critical need for research into the use of digital technologies to support the assessment of performance on complex tasks in schools. This paper reports on a component of a pilot study aimed at investigating the use of digital forms of performance assessment, manageable within schools, with high levels of reliability and capable of being scaled up for statewide implementation. Student practical performances were represented in digital files, placed in an online repository and marked using both a standards-reference and comparative pairs method. It was found that the short performance tasks exam was readily implemented and the work easily marked. The resulting scores were shown to be highly reliable and well correlated when compared with the other case studies and suitable for ranking students. However, the limitations of the tasks, time and scaffolding of the tasks reduced the study's validity in assessing student capability in the course. Overall the results were encouraging and provide a basis for a larger and longer study that has commenced. (Contains 1 figure and 5 tables.)

For over two decades educators have hailed the possibility of harnessing the capabilities of portable computing to transform Australian schools into places where students experience powerful learning environments, relevant to the 21st Century. For some schools in Australia this journey is well into its second decade with the use of networkable portable computing devices to provide anywhere-anytime learning opportunities. While some of the potential has been realised invariably the finding has been that using the technology to create powerful learning environments in real school setting is not a trivial matter and needs to consider a range of factors. So when in 2003 the Department of Education and Training (DET) in Western Australia decided to improve a regional state school and include portable computing as part of an approach to transform the school careful planning and implementation ensued. This paper discusses a range of issues around 1:1 student to computing devices in schools and reports on some of the factors that contributed to the success of the DET initiative and the direction it provides for public schooling.

In many ways assessment drives the curriculum and often pedagogy at all levels of education. In a world where it is more important what you can do with what you know rather than just what you know, it is critical that practical performance is assessed in reliable and valid ways that are also cost-effective. Increasingly the use of digital technologies to support the assessment of performance is seen as a key aspect to achieving this aim. This paper will report on the use of an online portfolio system and database linked marking tool to support a number of ways of using digital technologies to represent student work for assessment. (Contains 4 tables.)

Mobile computing devices are increasingly finding a place in universities, putting the pressure on teacher education to consider how best to incorporate the use of these technologies. At the same time there is pressure from the requirement for teacher education students to develop skills and experience in using digital technologies to support their teaching in schools. In response to these pressures the School of Education at Edith Cowan University has involved students in two exploratory projects over the past two years concerned with the use of laptop computers. This paper reports on the implementation and results of these projects. While generally the outcomes were positive it is not clear that this is the preferred solution when considering the range of digital device options available. (Contains 3 tables and 5 figures.)

Over the past decade there has been an explosion in the provision of computing facilities in schools for student use. However, there is concern that the development of these facilities has often given little regard to the ergonomics of the design for use by children, particularly adolescents. This paper reports on a study that investigated the ergonomics in the use of computer facilities by Year 8 students attending a private secondary school in Western Australia. In particular, it considered the use of computer in number of locations in the school and the relationship between the furniture provided and the anthropometric characteristics of the students. Data collected included general physical information of the students; furniture dimensions and adjustability; and observed student behaviours. The participants all used the facilities in each of the locations so that throughout a six-month course of the study a profile of how students related to these environments emerged. Data collected supports the findings that students are less likely to engaged in physically risky bahaviour if they have some degree of control over their environment, particularly in being able to adjust furniture to suit their own characteristics. These early adolescent students exhibited a huge rage of anthropometric characteristics leading to the need to provide adjustable furniture for use with computers. A number of recommendations are made for schools to consider as they respond to this situation. (Contains 2 tables and 1 figure.)

Reports on a 1999 study that investigated student and teacher perceptions regarding the use of portable computers in a Western Australia secondary school. Compares results with a 1995 study and discusses the need for professional development, support for the development of student computer-related skills, and curriculum changes towards more learner-centered approaches. (Author/LRW)

The concerns-based adoption model (CBAM) was used to analyze results of a study examining the use of student-owned portable computers at a secondary school. CBAM was found to be very useful in explaining the actions of teachers and in providing a basis on which to develop a theoretical model for the implementation of portable computers in schools. (AEF)

The collection and scoring of student artwork, for high-stakes assessment, across a large jurisdiction such as Western Australia is challenging. An alternative approach would be to submit digital representations of the artworks online for assessing. However, to give a valid and reliable measure the representations would need to be of adequate quality. Further, judgements of artworks are necessarily subjective giving concern about the reliability of scoring. The comparative pairs method of scoring lends itself to addressing this problem and is feasible where the work is in digital form. This paper reports on one component of a three-year study to investigate the representation of student practical work in digital forms for the purpose of summative assessment. The first phase of the project involved the researchers creating digital representations of the artwork submitted at the end of secondary schooling by a sample of students. The second phase involved a sample of students creating digital representations of their own work and submitting them through an online system. The study found this process was feasible, and the results were acceptable, but it lacked support from teachers and students who wanted the original artworks to be assessed. The comparative pairs method of scoring was found to provide reliable scores and be well suited to making judgements around creative work. [ABSTRACT FROM AUTHOR]

There is little doubt that the curriculum content and pedagogy in schools is driven by the structure and forms of assessment employed, particularly for summative purposes. When most such assessment was limited to what a student could do with a pen and paper in short 'exams' this pushed the content of the curriculum towards small descriptive chunks, and the pedagogy towards memorisation and replication techniques. Over the past two decades alternative forms of assessment supported by the power of computer systems have been conceived and tried. This paper discusses progress towards various forms of computer-based exams and how these may encourage curriculum and pedagogy suited to 21st Century learning. In particular it draws attention to research I have led to investigate the feasibility of a number of forms of computer-based exams for high-stakes summative assessment in secondary education. The result was the successful development and implementation of audiovisual stimuli and response computer-based exams, and digital production exams resulting in portfolios of evidence. Each implemented form of exam was tested for feasibility to ensure a defensible balance of manageability, reliability and validity. I believe that for most highstakes summative assessment in Australian schools it is time to replace the ancient paper-based technology with computer-based technologies. [ABSTRACT FROM AUTHOR]