Search

A concern that has been raised with the transition from pen and paper examinations to electronic examinations is whether this will make cheating easier. This article investigates how teachers and students perceive the differences in ease of cheating during three types of written examination: paper exams, bring your own device e-exams and e-exams using university-owned devices. It also investigates perceptions about the effectiveness of some typical countermeasures towards cheating across these examination types. A mixed-method approach was used, combining questionnaires and interviews with students and teachers in the authors’ own university. A total of 212 students and 162 teachers participated in the questionnaire survey, and then, a more limited number were interviewed to get a deeper understanding of the results. Six-different cheating practices were considered – impersonation, forbidden aids, peeking, peer collaboration, outside assistance and student–staff collusion and seven different countermeasures were considered – proctors, biometry, mingling, shuffling, random drawing, sequencing and broadcasting. Both students and teachers perceived cheating as easier with e-exams, and especially with bring your own device. They also thought some countermeasures would be easier to implement with e-exams.

The emergence of personalised data technologies, such as learning analytics is framed as a solution to manage the needs of higher education student populations that are growing ever more diverse and larger in size. However, the current approach to learning analytics presents tensions between increasing student agency in making learning-related decisions and ‘datafying’ students in the process of collecting, analysing and interpreting data. This paper presents a study that explores staff and student experience of agency, equity, and transparency in existing data practices and expectations towards learning analytics in a UK university. The results show a number of intertwined factors that have contributed to the tensions between enhancing a learner’s control of their studies and, at the same time, diminishing their autonomy as an active agent in the process of learning analytics. This paper argues that learner empowerment should not be automatically assumed to have taken place as part of the adoption of learning analytics. Instead, the interwoven power relationships in a complex educational system and the interactions between humans and machines need to be taken into consideration when presenting learning analytics as an equitable process to enhance student agency and educational equity.

Formative assessment is widely accepted as being crucial to promoting student learning and, since 2010, the UK General Medical Council has mandated its use in workplace-based clinical training for all new doctors. As a result, the Royal College of Radiologists (RCR) instituted a range of formative workplace-based assessments including the Radiology Direct Observation of Procedural Skills (Rad-DOPS), in which supervisors appraise trainees’ performance in carrying out clinical procedures. This paper reports on the quality of the written feedback in 2,500 Rad-DOPS online feedback forms in addressing the aims of the new assessment approach. Random samples of 500 were selected from the first three years of the new assessment implementation, 2010–13, and from 2016 to 17. Using an appropriate coding frame, the feedback was analysed across the samples against key trainee attributes including stage of training and level of adjudged competence. Criteria for identifying high quality feedback were derived from the literature and a simplified form of qualitative comparative analysis was used to identify the conditions associated with high quality feedback. An average of 97% of the assessments contained written feedback but the number of instances of high quality feedback was found to be exceedingly small at around 5%. The paper offers suggestions for making the feedback process more purposeful in achieving the aims of formative assessment.

Articles discussing and analysing student peer-review activities proliferate the educational literature, typically describing one or more class exercises where students provide feedback on each other’s work. These papers usually focus on a peer-review activity designed as a scholarly study, and make conclusions about its success or otherwise. There is not one standard model for ‘peer-review’, and information on the many different assessment designs used is distributed over an increasing number of publications and websites. This paper provides a meta-review of peer-review activities as they are implemented in practice, using configuration data from over a thousand assignments conducted using an online peer-review system during an eight-year period. We present data on the wide variety of assignment designs and the parameters that comprise them, their rubrics, and comparisons between subject areas. Information on the norms and range of all decisions to be made will encourage instructors (both new to ...

Research comparing continuous assessment (e.g. coursework) with examinations generally reveals a student preference for the former. The perceived increased use of continuous assessment periodically captures media attention, with claims of greater numbers of higher degree classifications being awarded. This paper takes a case-study approach to investigate the extent to which different types of assessment allow students to effectively demonstrate and apply their learning. By considering data gathered from second-level, undergraduate students completing 60 point, online psychology modules, this paper investigates assessment effectiveness in terms of student satisfaction, pass rate and level of pass rate. Findings reveal that modules with an end of module assessment (EMA), rather than an examination, have higher completion and pass rates. Whilst students who took a seen examination also performed well, those who completed an unseen examination recorded the lowest overall pass and completion rates, des...

Feedback plays an integral role in students’ learning and development, as it is often the only personal communication that students have with tutors or lecturers about their own work. Yet, in spite of its integral role in student learning there is disagreement between how students and tutors or lecturers perceive the pedagogic purpose of feedback. Central to this disagreement is the role that feedback has to play in ensuring that students produce the ‘right’ kinds of knowledge, and become the ‘right’ kinds of knowers within their disciplines. This paper argues that in order to find common ground between students and tutors or lecturers on what feedback is for, and how to both give and use it effectively, we need to conceptualise disciplinary knowledge and knowers anew. We offer, as a useful starting point, the Specialisation dimension of Legitimation Code Theory as both practical theory and methodological tool for exploring knowledge and knowers in English Studies and Law as two illustrative cases. The paper concludes that this analysis offers lecturers and tutors a fresh understanding of the disciplinary knowledge and knower structures they work within and, relatedly, a clearer view of the work their feedback needs to do within these.

The analysis of exemplars is a potentially powerful way of acquainting students with academic standards and supporting their capacities to make informed academic judgements. This paper investigates the role of dialogue in supporting students to develop their appreciation of the nature of quality work. The research derives from a project involving nine teachers in a Faculty of Education, and uses data from a single case to analyse the dialogic use of exemplars. The findings illustrate how the teacher prioritised student talk and withheld his own evaluative judgements in the management of the discussion. A related dilemma lies in the balance between the student voice in constructing their views on the nature of quality and explicit teacher guidance. The main significance of the paper lies in its description of how exemplars dialogue can be orchestrated, and a discussion of some of the main features of the dialogue.

© 2015 Taylor & Francis. A variety of understandings of feedback exist in the literature, which can broadly be categorised as cognitivist information transmission and socio-constructivist. Understanding feedback as information transmission or ‘telling’ has until recently been dominant. However, a socio-constructivist perspective of feedback posits that feedback should be dialogic and help to develop students’ ability to monitor, evaluate and regulate their learning. This paper is positioned as part of the shift away from seeing feedback as input, to exploring feedback as a dialogical process focusing on effects, through presenting an innovative methodological approach to analysing feedback dialogues in situ. Interactional analysis adopts the premise that artefacts and technologies set up a social field, where understanding human–human and human–material activities and interactions is important. The paper suggests that this systematic approach to analysing dialogic feedback can enable insight into previously undocumented aspects of feedback, such as the interactional features that promote and sustain feedback dialogue. The paper discusses methodological issues in such analyses and implications for research on feedback.

It is clear from the literature that feedback is potentially the most powerful and potent part of the assessment cycle when it comes to improving further student learning. However, for some time, there has been a growing amount of research evidence that much feedback practice does not fulfil this potential to influence future student learning because it fails in a host of different ways. This dilemma of the disjuncture between theory and practice has been increasingly highlighted by the UK National Student Survey results. This paper uses a model of the assessment process cycle to frame understandings drawn from the literature, and argues that the problem with much current practice resides largely in a failure to effectively engage students with feedback. The paper goes on to explore how best to effectively engage students with assessment feedback, with evidenced examples of feedback strategies that have successfully overcome this problem.

Several times each year the teaching performance of academics at higher education institutions are evaluated through anonymous, online Student Evaluation of Teaching (SET) surveys. Universities use SET surveys to inform decisions about staff promotion and tenure. But low student participation levels make the surveys impractical for this use. This scoping review aims to explore student motivations, perceptions, and opinions of SET survey completion. Five EBSCO® databases were searched using key words: “student evaluation of teaching” OR “student evaluation*” OR “student rating*” OR “student satisfaction” OR “teach* evaluation” OR “teach* effective*” OR “teach* performance OR “student feedback*” OR “student survey” AND “higher*education” OR university OR college OR tertiary. Thematic analysis of a meta-synthesis of qualitative findings derived from 21 papers identified five themes: (i) The value students’ place on SET, (ii) the knowledge that SET responses are acted upon to improve teaching, (iii) assurance of survey confidentiality and anonymity, (iv) incentives for completing SET, and (v) survey design and timing of survey release. Perceptions, knowledge, and attitudes about the value of SET are essential factors in motivating students to engage and complete SETs, particularly if surveys are easy to interpret, time for completion is incentivised, and responses are valued.

When instructors assess students’ laboratory reports to appraise the underlying scientific reasoning, they disclose their own concerns, epistemological assumptions and beliefs about science. The analysis of such assessments (i.e. rubric-centred scores and corresponding justificatory comments) offer a wealth of insights that can be re-engaged in further improvements of the assessment tool and procedure, and in developments in formative assessment more generally. Such insights include concerns exceeding the rubric’s descriptions (about meaningfulness, exhaustiveness, implicitness, connectivity, true inquiry, relevance), while differences among assessors are exposed (regarding epistemic values, approaches to scoring, sensitivity). This contribution is part of a broader effort to promote students’ conducive scientific thinking and deep-learning in science and engineering education. It addresses the question(s): what does the assessors’ reasoning tell us about the ways in which formative assessment is conducted, and could ideally be? The empirical investigation connects to existing knowledge, and discusses issues of representativeness and granularity in formative assessment. The paper elaborates on the design and use of the assessment tool, and presents evidence supporting context-bound recommendations and general conclusions. It is proposed that developments in formative assessment will benefit from reconceptualisation of assessment criteria, as the result of a co-design activity that engages with the assessors’ epistemological concerns.

Within higher education student evaluations of teaching (SET) are used to inform evaluations of performance of courses and teachers. An anonymous online survey was constructed and implemented using Qualtrics. This study was situated within a more extensive study investigating the impact of narrative SET comments on teaching quality and the health and wellbeing of academic staff. This paper reports specifically on two open questions that were designed to elicit examples of non-constructive and offensive anonymous narrative feedback. Five themes were identified: allegations; insults; comments about appearance, attire and accent; projections and blame; and threats and punishment. These are represented in non-redacted form. Personally destructive, defamatory, abusive and hurtful comments were commonly reported. These kinds of comments may have adverse consequences for the well-being of teaching staff, could contribute to occupational stress and in some cases could be considered libellous. The high prevalence of offensive comments accessible to and shared by teachers may be a reflection of the anonymity afforded to respondents using internet surveys, resulting in de-individuation and enabling some respondents to give voice to ‘hate speech’ which has no place in evaluations of teaching.

This paper proposes a new way of flipping small group classes in quantitative courses by active reading and peer marking using the virtual learning environment. We aim to engage students in the learning material by attempting a problem followed by peer marking based on some given solution guideline before they are exposed to another similar problem to solve during the small group classes. We design a quasi-experiment to evaluate the effect of peer marking by introducing an incentive in one such problem set and not in the other. The solution to the class problem is to act as the ‘incentive’, to be released only to the participants of the peer assessment. Using the data of two units from two UK universities and ‘incentives’ as instrumental variables to participation, our quantitative findings reveal the effect of participating in one more peer marking as a 3% increase in final marks on average. The qualitative analysis based on focus group discussions shows that the process increases student engagement, satisfaction, confidence and overall learning responsibility. The challenges often lie to establish a clear understanding of the purpose and the process of peer marking to ensure student buy-in to the system.

Increasingly, attention is being turned toward the actions of students in relation to feedback. This paper presents the outcomes of a questionnaire study researching what undergraduate and postgraduate architecture students perceive as the purpose of feedback, how they use feedback on their coursework, and what makes them more likely to act on it. An overarching theme in their conception of feedback’s purpose was that of a guide for improving coursework and learning. Two key dimensions that facilitated use over the longer term were feedback being generative and informing their working methods. The study revealed students engaging in a range of strategies with both formative and summative feedback to prioritise their next work; these could form the basis of facilitating wider adoption of these methods across the student body. There were similarities, but also substantial differences, between how undergraduate and postgraduate students reported perceiving and utilising feedback, which may have impact on feedback provision to enhance uptake and use. Whereas undergraduate students saw feedback’s purpose in terms of directing improvement, postgraduates were more likely to see their own role in the process. Postgraduates were more likely to question feedback’s validity over the longer term.

This paper presents the findings of a four-year mainly qualitative study of peer and self-assessment in university teaching. Peer and self-assessment activities were introduced with the intention of supporting students' learning, but they also formed part of the formal grading of the course assignment. Thus, the research aimed to explore the students' experiences with these activities in this specific learning situation - which benefits they perceived from these activities, what challenges they faced, and what supported their learning. The students completed the survey after participating in the activities, but before receiving their grades and peer feedback so as to capture their authentic experiences with the activities. A total of 103 students completed the survey. The data were analysed using descriptive statistics and thematic analysis. The results indicate that despite being stressful and uncomfortable for many, peer assessment was more beneficial for the students' learning than self-assessment. The students expressed concerns related to their competence to grade and responsibility for their peers' grades. However, by addressing students' needs for autonomy, competence and relatedness, these initial worries can be transformed into drivers for learning. It may be concluded that peer assessment can play a role in supporting students to self-assess.

Teamwork is one of the most important competences for most professionals, and it is therefore imperative that it be included in university education programmes, yet its evaluation continues to pose challenges. This paper reports on part of a study into the self and peer evaluation of teamwork, research which was carried out in order to analyse teamwork dynamics among students on degree courses in different disciplines, and how these relate to the outcome of teamwork and the grades obtained by each team member. The views of 351 students studying five subjects on five different degree courses were analysed, with participants divided into 95 groups. The results show that effective teamwork is related to higher grades and higher quality products. The weighting of the grade awarded to teamwork in the final subject grade conditions students’ assessment of how well the team has functioned, as does the grade each student receives from their teammates. Those teams that view their team as functioning best distribute equal grades among their teammates. The results also show that those students who assess themselves positively do the same for their peers

Although dialogue can augment the impact of feedback on student learning, dialogic feedback is unaffordable by instructors teaching large classes. In this regard, peer feedback can offer a scalable and effective solution. However, the existing practices optimistically rely on students’ discussion about feedback and lack a systematic design approach. In this paper, we propose a theoretical framework of collaborative peer feedback which structures feedback dialogue into three distinct phases and outlines the learning processes involved in each of them. Then, we present a web-based platform, called Synergy, which is designed to facilitate collaborative peer feedback as conceptualised in the theoretical framework. To enable instructor support and facilitation during the feedback practice, we propose a learning analytics support integrated into Synergy. The consolidated model of learning analytics, which concerns three critical pieces for creating impactful learning analytics practices, theory, design and data science, was employed to build the analytics support. The learning analytics support aims to guide instructors’ class-wide actions toward improving students’ learning experiences during the three phases of peer feedback. The actionable insights that the learning analytics support offers are discussed with examples.

The literature on improving student engagement with assessment and feedback has a tendency to treat all students as if they are the same. Students with lower levels of attainment are generally underrepresented within empirical studies and their feedback behaviours are less well understood. The recent drive to improve student assessment and feedback literacy and the move from ‘feedback’ being information about a task to being a process of understanding and using performance information is a larger conceptual leap for some students than others. In this paper, we consider issues surrounding the transition to new modes of feedback, focusing on what is needed for those who find study difficult and persistently are disappointed by their levels of attainment, to benefit from and take advantage of our feedback pedagogies. We examine literature advocating strategies such as increasing agency, using praise, developing feedback literacy and cultivating a growth mind-set. We argue that students who underachieve may benefit from strong relationships with educators and peers; exposure to feedback rich, low stakes environments, which permit repeated integrations of practice and feedback and building feedback literacy through peer assessment activities.

This paper reports on findings from a large Australian research project that explored the relationship between contract cheating and assessment design. Using survey responses from 14,086 students and 1147 educators at eight universities, a multivariate analysis examined the influence of a range of factors on the likelihood that different assessment types would prompt considerations of contract cheating in students. Perceptions of likelihood were highest among students who speak a language other than English at home. Perceptions of likelihood were also higher among students who reported there to be lots of opportunities to cheat, and amongst students who were dissatisfied with the teaching and learning environment. Perceptions of likelihood for certain assessment types were also higher in commerce and engineering than in any other discipline. Overall, four assessment types were perceived by students to be the least likely to be outsourced, however these are also the least likely to be set by educators. The analysis indicates that educators are more likely to use these assessment tasks when they report positively on organisational support for teaching and learning Refereed/Peer-reviewed

A key challenge for feedback practice involves promoting student uptake through the closing of feedback loops. This paper investigates feedback loops by using the concepts of single and double-loop...

This paper describes the teaching of psychological report writing at Karolinska Institutet and the efforts to improve it through the introduction of formative assessment applied as peer and teacher...

This paper presents an analysis of the response of student satisfaction scores to online delivery for courses at a large Australian university. Taking data from 2653 courses delivered since 2011 we employ a difference-in-differences estimator approach to evaluate the impact of a transition from face-to-face learning to online learning of courses on student satisfaction. We estimate that, on a five-point scale, conversion to online learning lowers course satisfaction by about 0.2 points and instructor satisfaction by about 0.15 points. These correspond to shifts relative to the underlying distributions of about 25–30% of a standard deviation. Some implications of the (slight) relative unpopularity of online learning are discussed.

Participatory assessment is increasingly employed in higher education worldwide as a formative mechanism to support students’ active learning. But do students in an increasingly relationally diverse environment perceive that peer assessment of individuals’ contributions to group-work tasks enhances their learning? Recognising the impact of students’ conceptions on the quality of their learning, this study considers students’ perspectives of peer assessment of group-work contributions at a South African university. Questionnaires elicited students’ perspectives of and general attitudes towards assessment of and by their peers. A growing measure of discontent with the process of assessing peer contributions to group tasks emerged, including actual and perceived racial and gender stereotyping, and related rejection-sensitivity. These initial findings were checked against the students’ experiences in a report-and-respond process that enabled probing discussions of the interpretations. This paper examines and explores the implications of such identifications and receptions for learning engagement and group-work curriculum development in the context of a rapidly transforming higher education sector.

Feedback is an important but challenging aspect of higher education pedagogy. In addition to providing quality feedback, teachers are expected to develop students’ skills and awareness for effective feedback processes. This case study addresses both processes and products of a Chinese university English teacher’s feedback enabling practice by involving students in peer feedback on oral presentations. Data from classroom observations and interviews reveal various strategies of cognitive scaffolding and social-affective support in the teacher’s feedback enabling processes, as well as skill development of generating peer feedback and awareness enhancement in accepting critical feedback with a positive attitude. Our analysis revolves around generic and contextually grounded strategies in cognitive scaffolding and social-affective support, as well as how verbal peer feedback can be utilised as a productive solution to mitigate resource constraints. The paper concludes by highlighting the centrality of ...

In this paper, we present an in-depth case study of a single student who failed an online module which formed part of a master’s programme in Professional Education and Leadership. We use this case study to examine assessment practices in higher education in the online environment. In taking this approach, we go against the current predilection for Big Data which has given rise to ‘learning analytics’, a data-intensive approach to monitoring learning. In particular, we draw attention to the model of the learner produced by learning analytics and to issues of ‘dataveillance’ in online learning. We also use the case to examine assessment in higher education more broadly, exploring the tensions between the requirements for certification and the need for learning. We conclude that assessment practices in higher education may have more to do with ‘quality assurance’ and regulatory frameworks than with ‘enhancing the student experience’ and inculcating the qualities that mark out higher education as an ...

Assessment in higher education has focused on the performance of individual students. This focus has been a practical as well as an epistemic one: methods of assessment are constrained by the technology of the day, and in the past they required the completion by individuals under controlled conditions of set-piece academic exercises. Recent advances in learning analytics, drawing upon vast sets of digitally stored student activity data, open new practical and epistemic possibilities for assessment, and carry the potential to transform higher education. It is becoming practicable to assess the individual and collective performance of team members working on complex projects that closely simulate the professional contexts that graduates will encounter. In addition to academic knowledge, this authentic assessment can include a diverse range of personal qualities and dispositions that are key to the computer-supported cooperative working of professionals in the knowledge economy. This paper explores t...

In this paper, we analyse 53 Dutch accreditation reports in the field of information technology to assess the mechanisms of the reported involvement of the professional field in the undergraduate programmes of universities of applied sciences. The results of qualitative content analysis reveal a coupling effect in reporting on mechanisms of interaction. Although the involvement of the professional field is tightly coupled with the undergraduate programmes at universities of applied sciences at the strategic level, there is an under-representation of university-industry interaction on an operational level, which suggests the need to explore the actual interaction taking place between the professional field and the programmes. Simultaneously, our results indicate that accreditation reports are not able to provide a holistic picture of professional field engagement in the curriculum of undergraduate programmes at the operational level, which questions their role in acknowledging the role of industry in shaping and achieving intended learning outcomes. Perhaps policymakers should consider introducing other tools or standards for addressing the outcome of the engagement and the responsiveness of the programmes at universities of applied sciences to the professional field.

The research informing this paper set out to investigate the qualitative experiences of students, supervisors and tutors involved in a summative collaborative assessment of placement learning on an undergraduate professional qualifying programme in the UK. Analysis of data gathered through semi-structured interviews provides valuable insights into its use. In particular, engagement in a collaborative assessment brings a degree of exposure for all concerned. Participants need to feel ‘able’ to participate, and ability to participate is linked to notions of agency and autonomy. Collaborative assessment design needs to acknowledge power relations and include strategies to reduce imbalances. The importance of recognising and working with the complexities of assessment should not be underestimated. Criterion referencing offers a guide to support consistency but can also falsely present assessment as mechanistic and instrumental. The research shows that it is in the ‘doing’ of assessment that a ‘shared language...

This paper is focused on an innovation which involved students being assessed on short written responses to an issue to be addressed in the following classroom session. The innovation was evaluated through a student survey, individual and focus-group interviews with participants, and the analysis of a critical friend. Positive findings included promoting student preparation and reflection, and enabling the teacher to understand students’ prior knowledge before the next session. Challenges included some student misperceptions about the assessment innovation, and the difficulty of assessing the short responses efficiently and reliably. The analysis suggests that starting small is a useful principle for assessment change. The discussion also brings out some wider issues of incentives and barriers for assessment innovation, and sketches some related future research directions.

Existing research provides evidence at the module level of systematic differences in patterns of assessment, marks achieved and distributions of marks between different disciplines. This paper examines those issues at the degree course level, and suggests reasons for the presence or absence of those module-level relationships at this higher level. The analysis finds that both discipline and institution have large and roughly equal impact on the balance between assessment types. However, contrary to the suggestions in the literature, that balance has virtually no independent impact on degree outcomes. The analysis also discovers that, while there is only a small independent impact of disciplines on average degree marks compared to the institutional impact, disciplines do have a larger relative impact on the distribution of those marks.