Search

In calibration tasks students assess exemplar texts using criteria against which their own work will be assessed. Typically, these tasks are used in the context of training for peer assessment. Little research has been conducted on the benefits of calibration tasks, such as benchmarking, as learning opportunities in their own right. This paper examines a dataset from a long-running benchmarking task (~500 students per semester, for four semesters). We investigate the relationship of benchmarking performance to other student outcomes, including ability to self-assess accurately. We show that students who complete the benchmarking perform better, that there is a relationship between benchmarking performance and self-assessment performance, and that students appreciate the support for learning that benchmarking tasks provide. We discuss implications for teaching and learning flagging the potential of calibration tasks as an under-explored tool.

Summative assessment is often considered a motivator that drives students' learning. Higher education has a responsibility in promoting lifelong learning and assessment plays an important role in supporting students' capability to make evaluative judgements about their work and that of others. However, as research often focuses on formal pedagogical design, it is unclear what behaviours summative assessment prompts, thus the relationship between summative assessment, learning and evaluative judgement requires further investigation. Drawing on a small-scale ethnography-informed study, this paper adopts a practice theory approach to explore how undergraduate physics students from three year levels make evaluative judgements in the context of summative assessment tasks. The contexts explored through observations and interviews include a graded in-class tutorial, an out-of-class study group for an in-semester assignment, and individual preparation for examinations. The findings suggest that while summative assessment is a crucial aspect of students' learning context, it does not fully shape students' practices. Instead, students engage in incidental conversations about the quality of their work and how to do things in their studies. By focusing on what students "actually do," this study integrates formal and informal aspects of students' learning, highlighting the tensions between undergraduate practices and intended learning outcomes.

If feedback is to be conducted effectively, then there needs to be clarity about what is involved and what is necessary for teachers to be able to undertake it well. While much attention has recently been devoted to student feedback literacy, less has been given to what is required of teaching staff in their various roles in feedback processes. This paper seeks to elucidate teacher feedback literacy through an analysis of the accounts of those who do feedback well. An inductive analysis was undertaken of conversations about feedback with 62 university teachers from five Australian universities using a dataset of transcripts of interviews and focus groups from two earlier research studies. Through an iterative process a teacher feedback literacy competency framework was developed which represents the competencies required of university teachers able to design and enact effective feedback processes. The paper discusses the different competencies required of those with different levels of responsibility, from overall course design to commenting on students' work. It concludes by considering implications for the professional development of university teachers in the area of feedback.

As a form of assessment, examinations are designed to determine whether students have met learning outcomes. However, students with disabilities report avoiding examinations, selecting units of study where the assessments align with their strengths. To ensure examinations do not contribute to the systematic exclusion of students with disabilities, it is important to explore their experiences. In this paper, we use a sociomaterial frame to analyse how examination arrangements construct inclusion in examinations. Interviews with 40 students were conducted across two universities. Inclusion or exclusion was variably constituted for students through emergent combinations of social and material arrangements. COVID-19 pandemic related social distancing related changes such as shifting examinations online, using technology, increasing time limits and moving to open-book examinations contributed to increased inclusion for most students, who were able to use familiar equipment in spaces they had adapted to their own needs. Staff acceptance and implementation of access requirements and assessment flexibility also contributed. While the attitudes and actions of staff involved in examinations can facilitate inclusion, reducing the need for adjustments through assessment design is important. This requires consideration of how time, technology, equipment and materials contribute to inclusion or exclusion, which may have benefits for many students.

Work-integrated learning (WIL) is a feature of university courses, both in professional areas, where it is commonplace, but also across many different disciplines. Assessment of WIL can be complex as it involves parties and settings external to the university, and it can be problematic because of difficulties in aligning learning activities during placements with what is or can be assessed by the university. This paper explores the relationship between students' placement experiences and accompanying assessments in contexts where activities are tightly coupled with the curriculum, and in those where it is not. It draws on a qualitative analysis of student interviews and drawings by the interviewees of their WIL experiences, supplemented with analysis of unit guides. Our findings highlight that students' perceptions of authenticity of assessment were undermined by misalignments between the student, university and industry. Assessment authenticity was perceived by students as based on alignment between their current and future selves in the assessment process, involvement of industry supervisors and relevance of placement activities to assessment activities. The paper discusses the complexity of coordination of educational activities with external partners, especially when one party drives assessment. It then suggests a reframing of WIL assessment to promote alignment and authenticity.

The use of assessment rubrics in the higher education sector is now widespread in a number of disciplines. Typically, these rubrics are constructed by teachers who also tend to be the main users of the rubrics throughout the grading process. In recent years, questions have been raised about this teacher-directed approach and some educators have begun to explore an alternate approach to rubric construction; that is, engaging students in collaboration with their teachers to co-construct assessment rubrics. This paper outlines the processes employed in a project that investigated the co-construction of rubrics within six different contexts. The project aimed to engage students in collaboration with their teachers to co-construct rubrics which are co-owned by teacher and student. A mixed methods approach was utilized to explore the effectiveness of the strategy. Questionnaires, interviews and focus groups were utilized to gather data from both the teacher-participants and student-participants regarding their experiences of being involved in the study. Findings are presented from the perspectives of both students and teachers, relating their views of rubrics and the activity of rubric co-construction. The paper concludes with recommendations for practical approaches to rubric co-construction and future research directions.

The growing trend for massification in higher education delivery has contributed to challenges in student engagement and experience. This has been further compounded by the global pandemic, with the sector experiencing reduced financial, staffing and resourcing budgets. The authors identified the delivery of meaningful feedback at scale as a critical emerging challenge. Although models of feedback exist within education, there are few discipline-agnostic frameworks for providing feedback that accounts for first-year education in the context of massification. With a focus on feedback within large-scale teaching and the first-year experience, and drawing on the authors' lived experiences, the paper proposes a conceptual non-disciplinary framework to scaffold the delivery of timely feedback in three stages. The proposed 'strategic framework for feedback at scale' promotes deeper first-year undergraduate students' learning and engagement across multiple teaching contexts through the feed-forward assessment design of automated ongoing feedback, peer-led staged feedback and teacher-led staged feedback. Examples of activities are discussed as well as steps for implementation applicable to tailoring to any discipline setting. Limitations and areas for future research are then discussed, calling for empirical research into the practice and effectiveness of the conceptual framework.

Feedback is justified when it has a positive influence on students' subsequent performance. Opportunities for student action need therefore to be consciously designed if feedback is to influence learning. In this paper, we discuss how ipsative design of feedback processes, i.e. involving comparison of a student's current performance with a previous similar task, can promote action from feedback. This design-based research was conducted with English for Academic Purposes students. A formative e-portfolio was implemented to facilitate students' access to their past work and feedback comments. Its aim was to explore how students exploit the ipsative design of feedback to build on both their own and teacher's actions. Analysis of student artefacts and interviews indicated that ipsative design prompts student comparison processes. Students were motivated to revisit their goals, make comparisons with previous work, and review peers' work. The effects of the design included motivation to engage with feedback beyond the immediate task and the revisiting of past work and action to improve it. Implications for teachers and curriculum designers are discussed including the need to create conditions for student action from feedback by including interconnected tasks and explicit comparison activities in the courses.

It is incumbent upon universities to deliver quality degree programs that produce employable graduates with discipline-specific knowledge and a well-developed set of generic skills known as graduate attributes. To safeguard the achievement of these outcomes, this paper argues for a holistic, longitudinal approach to degree planning known as programmatic assessment. The status of this relatively new phenomenon in the Australian university landscape is examined through an analysis of assessment-related policies from twenty-two of Australia's top-performing universities. Discourse analysis was employed to determine how programmatic assessment is depicted in policy, how key players are positioned in this space, and the discursive practices used to imbue policy with hortatory intent. The results of this analysis are outlined and indicate that policy constructions of a program-level approach to assessment are inconsistent across different universities and very few policies have specific guidelines about how programmatic assessment should be implemented.

Consensus moderation, where collaboration and discussion take place to reach an agreement on mark allocation, is a frequently used approach to quality assurance in higher education. Unit coordinators play a vital role in facilitating consensus moderation, yet limited research has focused on their role in moderation practices. This study explored unit coordinators' perceptions and experiences of consensus moderation in Australian higher education through five focus groups. Using Foucauldian discourse analysis, data analysis identified three discursive constructions of consensus moderation situated in the wider discourse of the neoliberal university: a truly collaborative process, an illusion, and a process to manage markers. Unit coordinators in this study were positioned as either supportive, compliant but ineffective, or powerful. In contrast, markers were positioned as helpful and compliant although inexperienced and needing support; uncooperative, resistant, troublesome and demanding; or inexperienced and malleable. This paper has identified varied knowledge and understanding of consensus moderation processes and practice. These findings can inform moderation policy and practices and unit coordinator professional development.

Anonymous student evaluation of teaching (SET) is a universal practice in higher education. We conducted a mixed-methods approach to investigate the nature and impact of anonymous SET commentary in the Australian higher education sector. Respondents shared a range of detailed SET exemplars, which revealed the extent of hurtful, defamatory and abusive commentary made by students. This paper reports the self-perceived impact of these on the health and wellbeing of academics. The majority of respondents reported that anonymous narrative comments contributed to workplace stress. There were no significant differences for gender. Younger academics were more likely to report the process of SET as stressful. Four themes were identified from the narrative responses: stress, distress, disorder and coping. These themes highlight the mental distress and impacts on well-being from repeated exposure to uncivil commentary made in SET by students. This distress was exacerbated by the failure of many employing universities to take substantial action to remedy or limit exposure to uncivil behaviour. The current system of anonymous SET has little validity and instead may operate as a vehicle for unfettered incivility directed towards teaching staff. The mental health impacts are significant for some and may impact the recruitment, retention and renewal of academic teaching staff into the future.

The analysis of exemplars of different quality is a potentially powerful tool in enabling students to understand assessment expectations and appreciate academic standards. Through a systematic review methodology, this paper synthesises exemplar-based research designs, exemplar implementation and the educational effects of exemplars. The review of 40 empirical exemplar-based studies indicated frequent use of post-intervention evaluation designs. The integrated use of exemplars and strategies such as rubrics, peer or self-assessment was a major pedagogic feature to enhance students' understanding of standards and aid their generation of internal feedback for self-monitoring. There was evidence suggesting that the combined use of exemplars with these strategies could advance students' academic performance and development of self-regulated learning. The review results set future directions for exemplar-based studies: quantitative research would benefit from controlled manipulation of variables to disentangle the effects of exemplars and those of other strategies; qualitative research could profitably use self-reflective diaries, think aloud protocols or classroom observations to develop deeper understandings of exemplar implementation. The key pedagogical implication recommends students' production of an assignment draft prior to exposure to exemplars so that they can compare their own work with exemplars and then make improvements independently of teachers.

Within higher education student evaluations of teaching (SET) are used to inform evaluations of performance of courses and teachers. An anonymous online survey was constructed and implemented using Qualtrics. This study was situated within a more extensive study investigating the impact of narrative SET comments on teaching quality and the health and wellbeing of academic staff. This paper reports specifically on two open questions that were designed to elicit examples of non-constructive and offensive anonymous narrative feedback. Five themes were identified: allegations; insults; comments about appearance, attire and accent; projections and blame; and threats and punishment. These are represented in non-redacted form. Personally destructive, defamatory, abusive and hurtful comments were commonly reported. These kinds of comments may have adverse consequences for the well-being of teaching staff, could contribute to occupational stress and in some cases could be considered libellous. The high prevalence of offensive comments accessible to and shared by teachers may be a reflection of the anonymity afforded to respondents using internet surveys, resulting in de-individuation and enabling some respondents to give voice to 'hate speech' which has no place in evaluations of teaching.

Rubrics have been suggested as a means to foster students' evaluative judgement, the capacity to appraise their own work and that of others; however, empirical evidence of rubrics' effectiveness is still emerging. This paper contributes findings from a randomised controlled experiment on the effect of rubrics on evaluative judgement. Participants were randomly assigned to one of two groups: a control group which evaluated peer-authored learning resources without the use of a rubric and an experiment group which carried out the evaluation using a three-item rubric based on (1) alignment with course content, (2) accuracy and (3) clarity. Both groups were asked to rate their confidence and provide comments to justify their scoring. The results showed a small effect size in increasing average agreement on the quality of learning resources in the experiment group. Analysis of comments reveals that criteria in and beyond the rubric guided participants' ratings of quality. The study provides evidence of the impact of rubrics on students' evaluative judgement and an example of how data-driven approaches and learning analytics can inform actionable design choices for embedding pedagogically supported strategies derived from the literature into actively operating educational technologies.

Australian universities have a long history of use of student satisfaction surveys. Their use has expanded and purpose changed over time. The surveys are often viewed as distorted by external influences such as discipline context, class size and year level of participants. This paper reports on the results of a large-scale investigation interrogating the influence of class size on student satisfaction ratings. The investigation was conducted at a large, comprehensive, research-intensive Australian university. It drew on the data from a survey administered to all students in all units of undergraduate and postgraduate study conducted across the university over four semesters. Data were collated into four class size categories. This categorised data were subjected to statistical analyses. This paper discusses the approach taken, the results of the analysis and their implications for student evaluation of teaching and learning.

Despite compelling evidence of its potential effectiveness, uptake of self and peer assessment in higher education has been slower than expected. As with other assessment practices, self and peer assessment is ultimately enabled, or inhibited, by the actions of individual academics. This paper explores what academics see as the benefits and challenges of implementing self and peer assessment, through the analysis of interviews with 13 Australian academics. Thematic analysis of our qualitative data identified seven themes of benefits and five challenges. Our academics showed strong belief in the power of self and peer assessment as formative assessment, contrary to past literature which has focussed on the accuracy of students' marking. This paper therefore brings insights as to not only what academics value about self and peer assessment but also identifies potential inhibitors in practice. Recommendations are made about improving the design and implementation of self and peer assessment in higher education.

Despite widespread recognition of the need to improve assessment in higher education, assessment tasks in individual courses are too often dominated by conventional methods. While changing assessment depends on many factors, improvements to assessment ultimately depend on the decisions and actions of individual educators. This paper considers research within the "heuristics and biases" tradition in the field of decision-making and judgement which has identified unconscious factors with the potential to limit capacity for such change. The paper focuses on issues that may compromise the process of improving assessment by supporting a reluctance to change existing tasks, by limiting the time allocated to develop alternative assessment tasks, by underestimating the degree of change needed or by an unwarranted overconfidence in assessment design decisions. The paper proposes countering these unconscious limitations to change by requiring justification for changing, or not changing, assessment tasks, and by informal and formal peer review of assessment task design. Finally, an agenda for research on heuristics and biases in assessment design is suggested in order to establish their presence and help counter their influence.

Indigenous Australians represent 2.2% of the working age population, yet account for only 1.4% of all university enrolments. In relation to higher degree research students, Indigenous Australians account for 1.1% of enrolments, but only 0.8% of all higher degree research completions. This paper reports on findings that emerged from an Australian Research Council-funded study which aimed to establish a model of best practice for the supervision of Indigenous doctoral students. The project identified the dissertation examination as one of the critical factors underpinning the success of doctoral candidates. Whilst research into the examination process for doctoral students is limited, the research that specifically explores the examination of dissertations submitted by Indigenous students is entirely inadequate. Our research identified key epistemological concerns that impact approaches to the examination process, to demonstrate how the dominance of Western methods of research impact the examination process for Indigenous doctoral students. This paper explores the experiences of 50 successful Indigenous Australian doctors with a specific focus on their examiner preference and disciplinary requirements. It highlights the limitations that some Indigenous students and their supervisors experience in finding an appropriate examiner.

The extent and reach of commercial cheating opportunities is ever present; thousands of websites promote differing business models offering assignments in multiple languages and currencies. In addition to commercial companies, students are known to outsource their assignments from friends and family. Assignment outsourcing and contract cheating are not a new problem, yet research to-date has been conducted utilising different survey tools and in different locations, thereby alluding accurate comparisons. This paper reports on a large international research project which utilised the same survey tool across multiple countries. Respondents most commonly reported outsourcing assignments from friends and family, and peer-sharing sites, as compared to essay mills. Large differences were found in the self-reported outsourcing behaviours between countries. Due to the differing outsourcing methods used, a new definition is offered for these behaviours: assignment outsourcing.

Research on contract cheating usually considers most forms of outsourcing combined, although studies have found that students more commonly obtain work from those known to them, rather than from commercial sites; yet little research exists which explores which variables and motivators may be associated with this type of outsourcing. Strategies employed to reduce student interaction with essay mills are unlikely to have an impact on reducing outsourcing from friends or family. Implementation of practical responses requires knowledge unique to the different methods of outsourcing. This paper presents statistical analysis undertaken on an international research population to consider which variables were statistically able to predict university student's engagement in assignment outsourcing from either friends, family or other students. Situational factors, such as knowledge of other student's cheating, and believing cheating is acceptable, were found to have a correlation to self-reported outsourcing behaviours. Individual factors did not show statistical significance in predicting student engagement with outsourcing from those known to them.

Authentic assessment has theorised benefits for student outcomes in higher education. More needs to be done, however, to empirically test relationships between the critical elements of authenticity and student outcomes, particularly in online learning environments. In this paper we examine whether an online simulation-based learning and assessment design, developed using the tools and functionality within the available university online ecosystem, has measurable 'authenticity' in a postgraduate course. In addition, we assess whether the simulation is associated with improvements in student learning behaviours and outcomes. We analyse qualitative data from lecturer and student interviews and student work submitted online, as well as data analytics from the learning management system in both control and experimental conditions. Seven critical elements of authentic assessment were identified including new aspects of authenticity relating to simulation-specific affordances (e.g. emotional content, temporal dimensions and unpredictability). Students watched lecture videos and participated in online discussion more frequently in the online simulation than in the control condition and indicated perceived mastery of, and confidence in, the learning outcomes. These findings make important contributions to authentic assessment theory and have practical implications for simulations built in online ecosystems available in higher education.

An increasingly regulated higher education sector is renewing its attention to those activities referred to as "moderation" in its efforts to ensure that judgements of student achievement are based on appropriate standards. Moderation practices conducted throughout the assessment process can result in purposes identified as equity, justification, accountability and community building. This paper draws on the limited studies of moderation and wider relevant research on judgement, standards and professional learning to test commonly used moderation practices against these identified purposes. The paper concludes with recommendations for maximising the potential of moderation practices to establish and maintain achievement standards.

As higher education systems and institutions expand, more energy is being invested in ensuring that sufficient learning has been achieved to warrant the award of a qualification. Many commonly used assessment approaches do not scale well, and there remains a pressing need for reform. This paper distils insights from international investigations of student learning outcomes assessment, using this analysis to chart frontiers for innovation. This paper sets out principles for guiding change in this field, presents an evaluation of progress via a review of signature assessment initiatives, reviews likely facilitators and blockers and, through these analyses, derives a strategy for spurring development.

Employers want students who are able to work effectively as members of a team, and expect universities to develop this ability in their graduates. This paper proposes a framework for a collaborative writing assignment that specifically develops students' ability to work in teams. The framework has been tested using two iterations of an action research project, with this paper focusing on the second iteration. The paper provides detailed information on how the framework was implemented, and then reports on the students' perceptions of their learning about teamwork.

Since the early 2010s the literature has shifted to view feedback as a process that students do where they make sense of information about work they have done, and use it to improve the quality of their subsequent work. In this view, effective feedback needs to demonstrate effects. However, it is unclear if educators and students share this understanding of feedback. This paper reports a qualitative investigation of what educators and students think the purpose of feedback is, and what they think makes feedback effective. We administered a survey on feedback that was completed by 406 staff and 4514 students from two Australian universities. Inductive thematic analysis was conducted on data from a sample of 323 staff with assessment responsibilities and 400 students. Staff and students largely thought the purpose of feedback was improvement. With respect to what makes feedback effective, staff mostly discussed feedback design matters like timing, modalities and connected tasks. In contrast, students mostly wrote that high-quality feedback comments make feedback effective--especially comments that are usable, detailed, considerate of affect and personalised to the student's own work. This study may assist researchers, educators and academic developers in refocusing their efforts in improving feedback.

This paper reports on findings from a large Australian research project that explored the relationship between contract cheating and assessment design. Using survey responses from 14,086 students and 1147 educators at eight universities, a multivariate analysis examined the influence of a range of factors on the likelihood that different assessment types would prompt considerations of contract cheating in students. Perceptions of likelihood were highest among students who speak a language other than English at home. Perceptions of likelihood were also higher among students who reported there to be lots of opportunities to cheat, and amongst students who were dissatisfied with the teaching and learning environment. Perceptions of likelihood for certain assessment types were also higher in commerce and engineering than in any other discipline. Overall, four assessment types were perceived by students to be the least likely to be outsourced, however these are also the least likely to be set by educators. The analysis indicates that educators are more likely to use these assessment tasks when they report positively on organisational support for teaching and learning.

Contract cheating sites advertise that they provide high quality, undetectable, bespoke work delivered in a timely manner to students purchasing their assignments. This paper tests contract cheating sites' promises about the products they sell. We built on previous reported research examining contract cheating sites' persuasive features which were mapped into three major dimensions. In this study, we explore how those dimensions are realised in practice. By purchasing 54 assignments from 18 different contract cheating sites across a range of disciplines, we found contract cheating sites' promises flawed. Many sites do not deliver assignments on time, or at all; they provide variable quality assignments (including fail grade work), and do not necessarily respond rapidly to user queries. When markers graded work, 52% of cheated tasks failed to meet the university pass standard. Furthermore, many contract cheating sites retain the right to share personal details with third parties under their privacy clauses and require levels of personal identification that leave users vulnerable. Students need to be aware that contract cheating sites' slick advertising is not necessarily borne out in reality. Universities can draw on this study's findings for student awareness and deterrence campaigns pointing out the risks of using contract cheating sites.

Assessment feedback is one of the most important components of the learning process. However, student and educator dissatisfaction with feedback practices continues to remain a significant problem in higher education. To better understand the barriers to effective feedback, the present study explores feedback challenges identified by 3807 students and 281 educators from two Australian universities. Open-response data were analysed using an inductively derived coding framework and thematic analysis. The findings reveal three major themes relating to the challenges of feedback from the perspective of students and educators: feedback practices, contextual constraints and individual capacity. In addition to confirming the persistence of many previously reported challenges in the literature, this paper also offers insights into challenges that are particular to feedback, especially relating to the production of useful comments and the perceived hurdles of both student and educator attitudes and capabilities. The findings also highlight the constrained nature of educators' work, particularly in relation to time. While further research is needed, we propose that feedback needs to be understood as an interaction between practices, context and individuals.

This paper investigates the use of new media technologies, in particular wikis, for the compiling and grading of group assessment tasks. Wikis are open web pages that can be viewed and modified by anyone with internet access and are well known for their collaborative nature. Wikis are also transparent, which means that any edit/modification is recorded and attributed to a specific user. Such transparency has immense implications for the assessment of group work, as one of the major criticisms of group assessment tasks made by students relates to fairness in the distribution of grades. Therefore, the aims of this paper are to examine whether a wiki can help make the grading of group work fairer and whether the wiki environment encourages students to engage more directly with the contributions of other group members, thereby making it a truly collaborative experience.(Contains 3 figures.)

For the past 30?years, beginning with the seminal work of Herbert Marsh in Australia and New Zealand, institutions of higher education have developed internal practices and procedures to collect and analyse student evaluations of teaching and learning. However, the question remains: has this development resulted in the achievement of institutional goals that maximise learning across all teaching contexts? As is the case in many other countries, a recent review of Australian national student evaluation data failed to conclude that student evaluations have improved overall teaching. However, these data have made student perceptions of teaching and learning more salient in the minds of tertiary educators. Certainly, teaching staff are aware of the impacts of student evaluations on informed decision-making, such as the continuation or discontinuation of courses, and on their promotion processes. This paper will review student evaluation practices according to criteria used in change theories, such as the transtheoretical change model (TTM). TTM construes organisational change as a process involving progress through a series of five stages: precontemplation, contemplation, preparation, action and maintenance. This paper will focus on organisational and behavioural outcomes that can be linked to the use of student data. It will recommend strategies for better aligning evaluation results to the stages of change. (Contains 2 tables.)

Assessment tasks can focus student efforts in what they choose to study. This paper describes the development and initial application of an assessment matrix to support student learning in postgraduate research studies at the University of Adelaide. Students and supervisors in the multi-disciplinary Master programmes also benefit from the clear descriptors to clarify expectations and standardise grading. This reflective paper explores the capacity of a matrix to improve constructive alignment between research course objectives and learning outcomes when it is an integral part of feedback loops. (Contains 2 tables.)

A fundamental aspect of work integrated learning (WIL) is the development of professional competence, the ability of students to perform in the workplace. Alignment theory therefore suggests that the assessment of WIL should include an assessment of students' demonstration of professional competence in the workplace. The assessment of professional competence in WIL is, however, problematic. It may be impractical for the academic supervisor to directly assess professional competence if there are a large number of students in external placements. If evidence of professional competence is provided by the student, the student's ability to articulate his or her own capabilities will interfere with the validity of the assessment. If evidence of professional competency is provided by the supervisor then the assessment is heavily dependent on the individual supervisor and may be unreliable. This paper will examine the literature relating to the assessment of professional competence in WIL. The paper will be informed by the author's experience in coordinating a WIL subject in an undergraduate law course. It will recommend that a mix of evidence provided by the student, the workplace supervisor and the academic supervisor should be used to assess professional competence in WIL.

Conventional student-led evaluation is now an orthodox feature of the North American, UK and Australian higher education landscape. Increasingly, it is guiding major institutional decisions around educational quality, academic promotion and more recently institutional funding by government. Yet significant research around student-led evaluation has demonstrated that this form of student opinion-based evaluation remains highly fragile and susceptible to multiple forms of influence. This paper argues that student-led evaluation of itself is not sufficiently robust to appropriately inform notions of educational quality or pedagogical capability of academics. An alternative approach to evaluation in higher education is proposed, broadly drawing on the conception of "fourth generation evaluation." The model foregrounds academic collaboration in shaping a professional evaluative framework at an activity level (i.e. programme or sub-discipline level) based on an ongoing dialogue between peers and with students, as well as qualitative evaluation of student learning. The model attempts to make tensions and conflicts in the teaching and learning context more explicit and the active focus of evaluative efforts to devise appropriate and sustainable responses to improve the quality of learning and professional practice. This paper presents some preliminary research on the effectiveness of this model in its initial piloting in postgraduate programmes in an Australian university. (Contains 1 table.)

This paper explores the attitudes and perceptions of three cohorts of Australian humanities and social science undergraduate students towards peer assessment. It seeks to gain a deeper understanding of local dissatisfaction with peer assessment through an in-depth analysis of students' contributions to focus group discussions. Responding to education theorists' claims that peer and other forms of innovative assessment empower students, the paper concentrates on themes of power evident within students' narratives. The interpretive framework coheres around four interrelated conceptions of power: sovereign, epistemological, disciplinary and structural. Students were found to support the notion of peer assessment as a formative exercise, yet were highly critical of it as a summative practice. Significantly, some of the students' dissatisfaction stemmed from broader changes across the higher education sector more generally. The paper draws attention to the need for this to be taken into account when considering assessment as a technology of power and empowerment.

Student engagement has rapidly developed a central place in the quality agenda of Australian universities since the introduction of the Australasian Survey of Student Engagement (AUSSE). The AUSSE is based on one developed in the USA. The main arguments given for adopting this survey in Australia are that it provides a valid instrument for measuring engagement and that it enables international comparisons. However, the survey instrument and scales have been adopted with little scrutiny of these arguments. This paper examines these arguments by considering different perspectives of engagement, examining the importance of contextual differences and evaluating the AUSSE engagement scales in the light of both. The paper concludes that the AUSSE results should be used by universities and policy-makers with caution. (Contains 1 table.)

Programme evaluation is essential for quality assurance in education. In this paper, we describe our evaluation strategy for the first year of a new medical programme. Although we used multiple methods in the evaluation, the use of the focus group method was core. This paper reports our experiences of focus groups for this purpose. We describe the implementation of the strategy including the development of topic guides, interviewer training, respondent recruitment, record keeping, template production, transcription, analysis and reporting of results. Using interviewer notes and individual interviews with focus group interviewers and participants, we evaluated focus groups as a method in educational programme evaluation. Focus groups are the most important evaluation method in helping us measure the extent to which we have met the goals of the first year of our programme. Benefits include generating detailed information in a relatively short time. Focus groups are largely valued by participants and appear to be contributing to the development of an "evaluation culture" in our new medical school. They are also building our capacity and capability in qualitative research methods. Challenges include the human resource intensive nature of scheduling, transcribing, analysing and report writing. (Contains 2 tables.)

This paper examines the participatory approach used by a group of academic support staff in evaluating an academic professional development resource designed to support e-learning and teaching. The resource, titled Designing Electronic Learning and Teaching Approaches (DELTA), showcases examples of electronic learning and teaching approaches developed at Monash University, Australia. The evaluation included individual and collective reflection, dialogue and action, drawing on the features of participatory action research. This paper explores the value of this critically reflective, participatory approach for evaluation to improve the use of new learning technologies, demonstrating how it provided a clear decision-making framework for iterative improvement of the DELTA site by identifying consensus items for action and recording other items for later consideration, while also contributing to team members' own professional development. (Contains 1 figure and 3 tables.)

One issue universities face is the need to demonstrate excellence in postgraduate research supervision at the individual, faculty and university level. While poor supervision might become obvious over time, with grievances, withdrawals and poor completion times and rates, this paper focuses specifically on identifying and demonstrating supervisory excellence. Currently, the amount and range of evidence used to support claims of supervisory excellence tends to be limited, leaving supervisors, faculties and institutions in a position where demonstrating excellence remains difficult. This paper proposes two inter-dependent ideas which, considered together, help to redress this problem. The first is a "map" for the collection and use of evidence of supervisory excellence. The second is a "template" for a "supervisory excellence report". The "map" details the organisational elements, uses of data, and data types which can be considered. The "report" explains one simple and potent way to organise and present these data for multiple purposes. Together they constitute a much-needed framework for promoting and recognising excellence in the supervision of research students. (Contains 3 tables and 1 figure.)

The plentiful and steadily increasing literature on teaching and learning in higher education has produced a number of helpful frameworks and guidelines that can be applied to the development and communication of assessment practice. The continued prevalence of much imprecise, unclear and otherwise confusing terminology around the discussion of assessment tasks, however, indicates the lack of a framework that facilitates understanding of individual task components. This paper describes an approach to addressing this gap in the literature through the development of an assessment task design framework drawn from systemic functional linguistics. To date, application of the framework has included the support of academics in "designing out" opportunities for plagiarism and the succinct description of assessment innovations. The paper concludes that the framework has the potential not only to underpin more purposeful design and analysis of assessment tasks but also to introduce academics to a vast but generally unfamiliar literature on the systematic development of academic communication skills and to encourage the critique of assessment as cultural practice. (Contains 2 figures.)

While there has been a good deal of discussion about the scholarship of teaching and learning, and models have been developed to understand its scope, the effects on students' learning of academics engaging in the scholarship of teaching and learning are unclear. In the context of initiatives to develop the scholarship of teaching and learning in a large research-intensive university in Australia, this paper discusses the relationship between faculty performance on a set of scholarly accomplishments in relation to teaching and learning from 2002 to 2004, and changes in students' course experiences from 2001 to 2005. The paper provides evidence of the relationship between the scholarship of teaching and learning and students' course experiences and demonstrates the effectiveness of institutional strategies to encourage the scholarship of teaching and learning. (Contains 1 table and 2 figures.)

This paper reports on seven calibrated scales of student engagement emerging from a large-scale study of first year undergraduate students in Australian universities. The analysis presents insights into contemporary undergraduate student engagement, including online, self-managed, peer and student-staff engagement. The results point to the imperative for developing a broader understanding of engagement as a process with several dimensions. These must be acknowledged in any measurement and monitoring of this construct in higher education. The paper calls for a more robust theorising of the engagement concept that encompasses both quantitative and qualitative measures. It considers implications for pedagogy and institutional policy in support of enhancing the quality of the student experience. (Contains 3 tables.)

This paper presents an analysis of the response of student satisfaction scores to online delivery for courses at a large Australian university. Taking data from 2653 courses delivered since 2011 we employ a difference-in-differences estimator approach to evaluate the impact of a transition from face-to-face learning to online learning of courses on student satisfaction. We estimate that, on a five-point scale, conversion to online learning lowers course satisfaction by about 0.2 points and instructor satisfaction by about 0.15 points. These correspond to shifts relative to the underlying distributions of about 25-30% of a standard deviation. Some implications of the (slight) relative unpopularity of online learning are discussed.

Self and peer-assessment are becoming central aspects of student-centred assessment processes in higher education. Despite increasing evidence that both forms of assessment are helpful for developing key capabilities in students, such as taking more responsibility for their own learning, developing a better understanding of the subject matter, assessment criteria and their own values and judgements, and developing critical reflection skills, both forms of assessment are still not the norm at universities. This paper provides the findings of a two-year study of formative self and peer-assessment at an Australian university. The study supports other research showing that students tend to regard formative self and peer-assessment as beneficial for gaining more insights about the assessment process and for improving their own work. We argue that self and peer-assessment requires careful design and implementation for it to be an effective tool for formative assessment processes; and that the development of students' capacities for giving feedback, and the continuous and timely involvement of the teacher, are central aspects for successful self and peer-assessment. The move to self and peer-assessment is not simple for teachers and students but is worthwhile and necessary for twenty-first century higher education.

Assessment feedback allows students to obtain valuable information about how they can improve their future performance and learning strategies. However, research indicates that students are more likely to reject or ignore comments if they evoke negative emotional responses. Despite the importance of this issue, there is a lack of research exploring if certain types of students are more likely to experience negative emotional responses than others. This study builds on extant qualitative studies through a quantitative examination of two previously identified student variables: different citizenship backgrounds (domestic and international) and different grade expectations (higher or lower than expected). The participants were 4514 students from two Australian universities. Analysis of survey data revealed that, regardless of language experience, international students were more likely than domestic students to find feedback comments to be discouraging, upsetting and too critical. Students who received grades lower than they expected on a particular assessment task were more likely than students who received grades higher than they expected to feel sad, shameful and angry as a result of the feedback comments. This paper concludes with several recommendations, including the need to modify assessment feedback practices in order to be sensitive to different student cohorts.

External referencing of assessment and students' achievement standards is a growing priority area within higher education, which is being pressured by government requirements to evidence outcome attainment. External referencing benefits stakeholders connected to higher education by helping to assure that assessments and standards within courses are appropriate and comparable among institutions. External referencing takes many forms, which have different resourcing requirements, outcomes, and operational strengths and challenges. This paper describes the External Referencing of Standards (ERoS) approach developed, tested and adopted by a university consortium. ERoS draws on the strengths of existing methodologies to produce an evolved model that is effective, efficient, transparent and open, capability building and sustainable. The model enables participants to communicate directly and construct peer relationships, and findings suggest this is a significant design strength. The process facilitates capability building, such that participants garnered insights valuable to enact quality assurance and enhancement of existing courses, and fosters connections that facilitate collaboration and peer-learning. ERoS successfully used open source collaborative tools to review work samples, which can be used to benchmark costed systems.

Contract cheating is the purchasing of custom-made university assignments with the intention of submitting them. Websites providing contract cheating services often claim this form of cheating is undetectable, and no published research has examined this claim. This paper documents a pilot study where markers were paid to mark a mixture of real student work and contract cheating assignments, to establish their accuracy at detecting contract cheating. Seven experienced markers individually blind marked the same bundle of 20 second-year psychology assignments, which included 6 that were purchased from contract cheating websites. Sensitivity analyses showed markers detected contract cheating 62% of the time. Specificity analyses showed markers correctly identified real student work 96% of the time. Our results contrast with contract cheating sites' claims that contract cheating is undetectable. However, they should be taken with caution as they are from one course unit in one discipline.

This paper reports the outcomes of a School of Law project undertaken in response to issues arising during the implementation of an institutional criterion-referenced assessment policy. The project involved the development of a set of generic assessment criteria and standards, or rubric, which could be customized to the requirements of individual law subjects. Of significance are the key decisions that shaped the generic resource and additional outcomes or byproducts of the project which include the professional learning experienced by individual members of the project working party and the identification of mutually reinforcing relationships between criterion-referenced assessment practice and other institutional policies and priorities. The paper concludes that analysis of project processes can produce findings whose value and significance are of equal interest to those resulting from implementation studies. (Contains 2 tables and 1 figure.)

This paper reflects on the ongoing debate surrounding the usefulness (or otherwise) of multiple-choice questions (MCQ) as an assessment instrument. The context is a graduate school of business in Australia where an experiment was conducted to investigate the use of assertion-reason questions (ARQ), a sophisticated form of MCQ that aims to encourage higher-order thinking on the part of the student. It builds on the work of Connelly (2004) which produced a quantitative analysis of the use of ARQ testing in two economics course units in a flexibly-delivered Master of Business Administration (MBA) program. Connelly's main findings were that ARQ tests were good substitutes for the more conventional type of multiple-choice/short-answer type questions and, perhaps more significantly, ARQ test performance was a good predictor of student performance in essays--the assessment instrument most widely favoured as an indicator of deeper learning. The main focus of this paper is the validity of the second of these findings, analysis of questionnaire data casting some doubt over whether student performance in ARQ tests can, indeed, be looked upon as a sound indicator of deeper learning--student reactions and opinions suggesting instead that performance might have more to do with one's proficiency in the English language.

Wikipedia is an open educational resource that connects writers and editors to diverse discourse communities around the world. Unwarranted stigma is attached to the use of Wikipedia in higher education due to fears that students will not pursue rigorous research practices because of the easy access to information that Wikipedia facilitates. In studies referred to in this paper, undergraduate writing students are taught about the need to interrogate any information they find on Wikipedia just as they would other online source material. They are inducted into fact checking, editing and creating Wikipedia articles as a means to analyse source material critically and to advance their research, writing and digital literacy. Meanwhile, in a postgraduate course in magazine studies, instead of writing essays, students are promoting Australian magazines and print culture by writing Wikipedia entries about Antipodean magazines and their editors. These courses experiment with new approaches to formative and summative assessment; promote group research, collaborative and participatory writing, writing across networks and negotiating discourse communities; and challenge students' perceptions about peer review and the legitimacy of Wikipedia.