Search

This paper reports on an interdisciplinary pedagogical research project involving academic literacy experts and lecturers at a School of Nursing. Specifically, the paper focusses on the development of a data-driven analytical rubric to teach and assess critical reflections in year-one nursing. The purpose of the project was to support the teaching and evaluating of critical reflections of nursing students after their initial clinical placement. Rather than focusing on inter-rater reliability or on pedagogical uses of rubrics, this paper is concerned with the ways the criteria that constitute these rubrics were devised. The data involved 200 student assignments, the existing marking criteria and two focus groups with the nursing lecturers. We analysed the data using aspects of a linguistic theory, systemic functional linguistics, and elements of a sociological framework, Legitimation Code Theory, to understand what constitutes 'deep reflection' in clinical nursing practice. Our findings led to a revised analytical rubric which makes visible what is highly valued in nursing reflection tasks. We conclude with a data-driven analytical rubric design framework which involves the analysis of student assignments at various levels of achievement to reveal academic literacy and knowledge practices requirements of reflection tasks.

This is the second of two papers based on a study of how digital and paper diaries contribute to students' understanding of the processes of their learning within their academic disciplines. The purpose of the study was to use diary writing as a vehicle by which we would try and comprehend how students both make sense of assessment feedback and how this impacts on their emotional and motivational attitudes toward improving subsequent work. In this paper, samples are provided of diary entries for both digital and paper forms that illustrate the distinctive approaches to prospective but often highly self-critical and impromptu writing, as often characterized by the digital diary entries, when compared with retrospective and deeply reflective writing, which was common to much of the paper diary writing. The students who preferred the paper diaries seemed to have less mutable self-concepts, using their entries to craft and re-script ideas (often quite negative and potentially damaging ones) about themselves and their work. Conversely, students preferring the digital diaries were more willing to engage in reflexive criticism, and entertain the possibility of feeling very differently about themselves and their work on a day-to-day basis. This study ultimately does two things: it illuminates the complexity of students' feelings about learning and being assessed, stressing the importance of time and interaction in assisting students to contextualize and interpret the interrelation of learner identity, learner change and learner achievement; and second, it illustrates, within this small-scale study, how different types of diaries become important vehicles for expressing the variety of emotions and reflections that characterize the student learning experience. [ABSTRACT FROM AUTHOR]

This paper proposes a new way of flipping small group classes in quantitative courses by active reading and peer marking using the virtual learning environment. We aim to engage students in the learning material by attempting a problem followed by peer marking based on some given solution guideline before they are exposed to another similar problem to solve during the small group classes. We design a quasi-experiment to evaluate the effect of peer marking by introducing an incentive in one such problem set and not in the other. The solution to the class problem is to act as the 'incentive', to be released only to the participants of the peer assessment. Using the data of two units from two UK universities and 'incentives' as instrumental variables to participation, our quantitative findings reveal the effect of participating in one more peer marking as a 3% increase in final marks on average. The qualitative analysis based on focus group discussions shows that the process increases student engagement, satisfaction, confidence and overall learning responsibility. The challenges often lie to establish a clear understanding of the purpose and the process of peer marking to ensure student buy-in to the system.

The analysis of exemplars of different quality is a potentially powerful tool in enabling students to understand assessment expectations and appreciate academic standards. Through a systematic review methodology, this paper synthesises exemplar-based research designs, exemplar implementation and the educational effects of exemplars. The review of 40 empirical exemplar-based studies indicated frequent use of post-intervention evaluation designs. The integrated use of exemplars and strategies such as rubrics, peer or self-assessment was a major pedagogic feature to enhance students' understanding of standards and aid their generation of internal feedback for self-monitoring. There was evidence suggesting that the combined use of exemplars with these strategies could advance students' academic performance and development of self-regulated learning. The review results set future directions for exemplar-based studies: quantitative research would benefit from controlled manipulation of variables to disentangle the effects of exemplars and those of other strategies; qualitative research could profitably use self-reflective diaries, think aloud protocols or classroom observations to develop deeper understandings of exemplar implementation. The key pedagogical implication recommends students' production of an assignment draft prior to exposure to exemplars so that they can compare their own work with exemplars and then make improvements independently of teachers.

A successful first year is of vital importance to academic achievement in higher education. How can we identify those students at risk at a very early stage, i.e. even before they have entered university? This study focuses on three pre-university factors, namely academic self-efficacy, self-regulated learning, in terms of autonomous motivation and time-management, and beyond classroom engagement. 2355 Dutch prospective first-year students in the fields of science, humanities and social sciences filled in our survey. Data on first year academic achievement in terms of grade point average (GPA) was obtained at the end of the first year. Prospective first-year students scored high on all of the variables; we found gender-related differences, favouring female students, as well as discipline-related differences. Time management and autonomous motivation were found to be positive predictors of achievement; beyond classroom engagement was a negative predictor of achievement. This paper contributes to the research on first-year academic achievement and emphasises the importance of providing students with personalised trajectories from the moment they enter university.

An increasingly regulated higher education sector is renewing its attention to those activities referred to as "moderation" in its efforts to ensure that judgements of student achievement are based on appropriate standards. Moderation practices conducted throughout the assessment process can result in purposes identified as equity, justification, accountability and community building. This paper draws on the limited studies of moderation and wider relevant research on judgement, standards and professional learning to test commonly used moderation practices against these identified purposes. The paper concludes with recommendations for maximising the potential of moderation practices to establish and maintain achievement standards.

As higher education systems and institutions expand, more energy is being invested in ensuring that sufficient learning has been achieved to warrant the award of a qualification. Many commonly used assessment approaches do not scale well, and there remains a pressing need for reform. This paper distils insights from international investigations of student learning outcomes assessment, using this analysis to chart frontiers for innovation. This paper sets out principles for guiding change in this field, presents an evaluation of progress via a review of signature assessment initiatives, reviews likely facilitators and blockers and, through these analyses, derives a strategy for spurring development.

Since the turn of the century there has been increasing interest in educational assessment and the ways in which it can encourage or discourage effective learning processes. Much of this interest has centered around formative assessment and those practices which can promote student learning through the act of being assessed. Societies that have a Confucian heritage culture are widely believed to prioritize summative assessment at the expense of formative assessment. This will inevitably impact the ways in which students from these countries approach and engage with assessment tasks. The present study seeks to understand the dispositions of students from one Confucian heritage culture though the analysis of quantitative and qualitative data. The conceptions and experiences of university students in Japan were studied in order to provide direction for the development of formative assessment practice. Two data collection instruments were used: a self-report survey and a narrative frame task. The results indicate that students from Japan approach assessment tasks in a way that is at odds with some popular Confucian categories, such as intense competition, a desire to please the family, and the priority of book knowledge above practical skill. The paper concludes with pedagogical implications for teachers.

The paper reports the results of a phenomenographic study on academics' conceptions of e-assessment. A cohort of 21 academics from 17 disciplines participated in semi-structured interviews exploring their experiences of using web-based technologies for formative and summative assessment purposes. Through iterative analysis of the interview transcripts, the study identified four qualitatively different ways in which academic teachers understand e-assessment. It was seen as a means of: (a) efficiently managing and streamlining the assessment process; (b) facilitating dialogue and student engagement; (c) enhancing student learning; and (d) developing (digital) identity and the community. Six interrelated dimensions of variation were also established: the benefit of e-assessment, the role of the assessing teacher, the role of the assessed student, the role of the medium, the purpose, quality and level of collaboration, and, finally, the relationship between e-assessment and teaching and learning. The results thematise how university teachers relate to technology-enabled assessment and represent incrementally expanding levels of agency within relatively recent, often hybrid assessment milieus. More importantly, the reported dimensions of variation can be utilised to inform which aspects of e-assessment warrant further attention for the improvement of formative and summative assessment design and practice.

In higher education, student evaluation of teaching is widely used as a measure of an academic's teaching performance despite considerable disagreement as to its value. This paper begins by examining the merit of teaching evaluations with reference to the factors influencing the accuracy of the teaching evaluation process. One of the central assumptions on which student evaluation of teaching is based is that there is a relationship between student achievement and student rating of teachers. However, the findings of the majority of studies do not support this assumption. The absence of a strong link between student achievement and teaching evaluations suggests that there is scope for examining other approaches to measuring effective classroom dynamics. This paper presents such an approach based on the notion of transformational classroom leadership. (Contains 1 table and 1 note.)

A flexible approach to assessment may promote students' engagement and academic achievement by allowing them to personalise their learning experience, even in the context of large undergraduate classes. However, studies reporting flexible assessment strategies and their impact are limited. In this paper, I present a feasible and effective approach to flexible assessment, describe choices made by 2016 students in 12 sections of two different courses using this approach, and explore associations between students' choices and academic achievement. Students decided at the beginning of the term whether they would use the assessment scheme proposed by the instructor or modify it by selecting (from within ranges provided) which assessments they would complete and the value of each in calculating their final grade. Most students (62%) made some change, but many (38%) opted to use the suggested values. Students did not choose to minimise their workload by selecting the fewest assessments possible. Opting out of a large assignment was the most common change, but the majority (69%) completed the assignment and almost all (95%) included quizzes based on readings. Different choices were not associated with notable differences in achievement: midterm score was the most important predictor of performance on a cumulative final examination.

External referencing of assessment and students' achievement standards is a growing priority area within higher education, which is being pressured by government requirements to evidence outcome attainment. External referencing benefits stakeholders connected to higher education by helping to assure that assessments and standards within courses are appropriate and comparable among institutions. External referencing takes many forms, which have different resourcing requirements, outcomes, and operational strengths and challenges. This paper describes the External Referencing of Standards (ERoS) approach developed, tested and adopted by a university consortium. ERoS draws on the strengths of existing methodologies to produce an evolved model that is effective, efficient, transparent and open, capability building and sustainable. The model enables participants to communicate directly and construct peer relationships, and findings suggest this is a significant design strength. The process facilitates capability building, such that participants garnered insights valuable to enact quality assurance and enhancement of existing courses, and fosters connections that facilitate collaboration and peer-learning. ERoS successfully used open source collaborative tools to review work samples, which can be used to benchmark costed systems.

Analysis of three years of student records revealed a marginal disadvantage in academic performance for students entering the University of Newcastle via three different non-traditional enabling programs (findings differed by program), but a positive effect on academic performance of mature age on entry. (EV)

Sustainable assessment has been proposed as an idea that focused on the contribution of assessment to learning beyond the timescale of a given course. It was identified as an assessment that meets the needs of the present in terms of the demands of formative and summative assessment, but which also prepares students to meet their own future learning needs. This paper reviews the value of such a notion for assessment; how it has been taken up over the past 15 years in higher education and why it might still be needed. It identifies how it has been a successful intervention in assessment discourse. It explores what more is needed to locate assessment as an intervention to focus on learning for the longer term. It shows how sustainable assessment can help bridge the gap between assessment and learning, and link to ideas such as self-regulation, students' making judgements about their own work and course-wide assessment.

The goal of this paper is to examine the relationship of student motivation and achievement in low-stakes assessment contexts. Using Pearson product-moment correlations and hierarchical linear regression modelling to analyse data on 794 tertiary students who undertook a low-stakes engineering mechanics assessment (along with the questionnaire of current motivation and the "Effort Thermometer"), we find that different measures of student motivation (effort, interest, sense of challenge, expectancy of success and anxiety to fail) showed atypical correlation patterns. The nature of the correlations further varies depending on the type of test booklet used by students. The difficulty of the early items in the assessment were positively correlated with "anxiety" and "success", but negatively correlated with "interest". In the light of our findings, we suggest that future research should systematically explore (taking into account testing conditions like test booklet design and test-item format) the implications of student motivation for achievement in low-stakes assessment contexts. Until the consequences of these processes are better understood, the validity of assessment data generated in low-stake conditions in the higher education sector will continue to be questioned. With a greater understanding of these processes, steps could be taken to correct for student motivation in such settings, thus increasing the utility of such assessments.

When students fail an examination at the end of their first year, they are offered a resit examination, which they merely need to pass to progress into the second year. These resits anecdotally provide a dual purpose of testing that students have achieved the required level of attainment to progress, and incentivising additional effort. This paper uses regression discontinuity design to attempt to estimate the effect of resits in first year statistics with econometrics examinations on future outcomes. Whilst resits alone appear to make zero significant effect on outcomes, students who perform well in the resit examination perform 0.7 standard deviations better in second-year microeconomics than similar students who do not receive resit examinations. These effects, if replicated more widely, could be worth up to £48,000 across the lifetime of each student.

This paper is an empirical field study of whether college students' preferences for assessment type correspond to their performance in assessment that tests that particular strength. For example, if students say they prefer assessment that tests their creativity, do they actually perform better on assessment tasks requiring the use of creativity? Seventy-eight students in three different courses were surveyed to determine their preferences in four types of assessment: memorisation, analysis, creativity and practical application. These preferences were then compared to student grades on corresponding forms of assessment to see if the preferences corresponded to actual performance. The study found that, while students had a clear preference for memorisation, they were not likely to deliver their best performances on memorisation tasks. There was no relationship at all between student preferences in assessment type and their performance in the respective assessments. These results indicate that, while in theory assessing students based on their preferences is reasonable for improved learning, we were not able to find evidence that it actually leads to higher performance.

Increasingly, student assessments of courses are being conducted online as opposed to administered in class. A growing body of research compares response rates and course ratings of courses evaluated online versus on paper. The present study extends this research by comparing student course assessments before and after the University of South Florida made online evaluations mandatory for all courses. This change only directly affected courses taught on-campus, as online courses were already being assessed online. However, we examine the effect of this change on courses taught on-campus and online, because we expect this change in policy to have differential effects. We hypothesise that by making online assessments mandatory for all courses, online assessments went from a novel method of evaluation to the norm; and, therefore, increased response rates for online courses, but had the opposite effect for on-campus courses. We find mixed support for our hypothesis. [ABSTRACT FROM AUTHOR]

If feedback is to be conducted effectively, then there needs to be clarity about what is involved and what is necessary for teachers to be able to undertake it well. While much attention has recently been devoted to student feedback literacy, less has been given to what is required of teaching staff in their various roles in feedback processes. This paper seeks to elucidate teacher feedback literacy through an analysis of the accounts of those who do feedback well. An inductive analysis was undertaken of conversations about feedback with 62 university teachers from five Australian universities using a dataset of transcripts of interviews and focus groups from two earlier research studies. Through an iterative process a teacher feedback literacy competency framework was developed which represents the competencies required of university teachers able to design and enact effective feedback processes. The paper discusses the different competencies required of those with different levels of responsibility, from overall course design to commenting on students' work. It concludes by considering implications for the professional development of university teachers in the area of feedback. [ABSTRACT FROM AUTHOR]

The aim of this paper is to report on an ongoing experiment in an enquiry-based approach to feedback. Over the course of four years, almost 1800 students have studied a final-year undergraduate module involving a mid-term assignment and end of module examination. Feedback on the assignment is delivered through a process which involves the provision of feedback resources and action planning rather than any individualised comments on the piece of work. The results of the experiment to date suggests that this approach to feedback can have a positive effect on student performance as they make the transition from formative assignment to summative examination but that the main beneficiaries of that performance improvement are students who did well in the assignment rather that students who had average or poor performance.

One of the implicit aims of higher education is to enable students to become better judges of their own work. This paper examines whether students who voluntarily engage in self-assessment improve in their capacity to make those judgements. The study utilises data from a web-based marking system that provides students with the opportunity to assess themselves on each criterion for each assessment task throughout a programme of study. Student marks were compared with those from tutors to plot changes over time. The findings suggest that overall students' judgements do converge with those of tutors, but that there is considerable variation across achievement levels, with weaker students showing little improvement. Whilst the study is limited by the exigencies of voluntary participation and thus consequential gaps in the data set, it shows how judgement over time can be demonstrated and points to the potential for more systematic interventions to improve students' judgements. It also illustrates the use of the web-based marking and feedback software (ReView) that has considerable utility in aiding self-assessment research.

This paper critically reviews the empirical research on the use of rubrics at the post-secondary level, identifies gaps in the literature and proposes areas in need of research. Studies of rubrics in higher education have been undertaken in a wide range of disciplines and for multiple purposes, including increasing student achievement, improving instruction and evaluating programmes. While, student perceptions of rubrics are generally positive and some authors report positive responses to rubric use by instructors, others noted a tendency for instructors to resist using them. Two studies suggested that rubric use was associated with improved academic performance, while one did not. The potential of rubrics to identify the need for improvements in courses and programmes has been demonstrated. Studies of the validity of rubrics have shown that clarity and appropriateness of language is a central concern. Studies of rater reliability tend to show that rubrics can lead to a relatively common interpretation of student performance. Suggestions for future research include the use of more rigorous research methods, more attention to validity and reliability, a closer focus on learning and research on rubric use in diverse educational contexts. (Contains 1 table.)

Graduate attributes have received considerable attention in recent years as universities seek to renew and articulate their purposes. Though many claims are made with respect to the implementation of graduate attributes, there is growing acceptance of the proposition that the strongest evidence of their achievement is their explicit embedding in assessment. While university policies espouse the assessment of graduate attributes, studies have suggested that the effectiveness of this undertaking is often limited by the influence of a range of diverse but interrelated factors. This paper discusses these influential factors and argues that the assessment of graduate attributes is a complex issue that is unlikely to be resolved unless these factors are addressed at a systemic level. (Contains 1 figure and 1 note.)

Introduction: Medical student concern that the submission of named examination scripts to examiners could cause bias initiated a study on the effect of identified and de-identified scripts on assessment outcome. Methods: Data were collected from a convenience examination sample of Year 1 (n = 88 students; n = 29 questions) and Year 2 scripts (n = 75 students; n = 27 questions). Scripts were randomized for presentation to examiners with or without identification for all of a given student's work. Assessment outcomes, by year and marking condition, were a non-normal distribution. Results: Non-parametric analysis determined that there were no systematic differences in assessment outcome under the two marking conditions (MW less than 0.05). Conclusion: We continue for a range of pedagogical reasons to present identified papers to examiners. Importantly, this study also demonstrates a research-led approach to resolving student queries. (Cotains 3 tables.)

With its multidisciplinary and applied foci, team-working skills are seen as especially critical in business courses in general and in business information systems courses in particular, and are specifically incorporated into desired graduate attributes by many universities. Past research has focused on the benefits of group working but little work has been conducted to examine the relative contribution of group formation choices on academic performance. This paper reports on a study that develops and validates a data-collection instrument and identifies four underlying factors that influence formation of groups: convenience, social cohesion, task management and technical skills/knowledge. The results thus suggest that academic performance is influenced by the skills and knowledge of individual members, potential social cohesion among the group members and the task management ability considered at the group formation stage. The study notes that while how a group is formed influences ultimate academic performance, how the group operates is also an important factor. (Contains 7 tables.)

Increasingly, institutions of higher education are required to evaluate student progress and programme effectiveness through implementation of performance assessment practices. Faculty members frequently resist performance assessment because of concerns that assessment activities will increase workloads, reduce time for scholarly activities, eliminate professional autonomy, and reduce faculty work into component parts or discrete technical competences. This paper describes how curriculum-embedded performance assessment can be used to evaluate student and programme effectiveness without placing an undue burden on faculty. Examples of the use of curriculum-embedded performance assessment strategies in a graduate-level educational psychology programme are provided.

Among the academic community, there is a perception that there is an upward shift in grade point average over an extended period of time without a corresponding increase in achievement. This trend has become an alarming topic among educators, industry and the general public. Some attribute increases in GPA to improvements in student quality while others point to the emergence of a consumer-based perception of education that unjustly awards high grades. The objective of this paper is to review various opinions regarding grade inflation, investigate whether grade inflation exists in engineering curricula, and analyze the related factors. Using eight years of detailed course data from a college of engineering, we seek insight into recent grading practices within each engineering discipline. The results indicate an upward trend in grade point average and an increasing prevalence of "A" grades. However, both trends match increases in student achievement potential as measured by ACT composite score. The results also show different grade outcomes among the engineering departments and the importance of class size as a predictor of grades. (Contains 4 tables and 8 figures.)

The analysis of exemplars of different quality is a potentially powerful tool in enabling students to understand assessment expectations and appreciate academic standards. Through a systematic review methodology, this paper synthesises exemplar-based research designs, exemplar implementation and the educational effects of exemplars. The review of 40 empirical exemplar-based studies indicated frequent use of post-intervention evaluation designs. The integrated use of exemplars and strategies such as rubrics, peer or self-assessment was a major pedagogic feature to enhance students' understanding of standards and aid their generation of internal feedback for self-monitoring. There was evidence suggesting that the combined use of exemplars with these strategies could advance students' academic performance and development of self-regulated learning. The review results set future directions for exemplar-based studies: quantitative research would benefit from controlled manipulation of variables to disentangle the effects of exemplars and those of other strategies; qualitative research could profitably use self-reflective diaries, think aloud protocols or classroom observations to develop deeper understandings of exemplar implementation. The key pedagogical implication recommends students' production of an assignment draft prior to exposure to exemplars so that they can compare their own work with exemplars and then make improvements independently of teachers. [ABSTRACT FROM AUTHOR]

In this paper the relationship between academic load (the number of modules attempted) and academic performance is investigated in a Scottish and an Australian university. An engagement approach to academic integration is employed, in which there is feedback between load and performance, and in which there is scope for diminishing returns to the study of additional modules once loads become high. The results indicate that full-time students reduced module load in response to information on academic performance. At the Scottish business school many non-traditional students had taken up opportunities to enter university under the UK government's drive to widen participation. In that school load reduction was undertaken at twice the rate of the Australian business school. For women, reductions from full-time loads by one or two modules appear rational in that better average marks result. There are indications that status as a widening participation entrant, the learning and assessment environment, the funding regime and rest-of-life demands have influences on load reduction and on academic performance. (Contains 5 notes, 3 tables, and 3 figures.)

This paper reports on a longitudinal study designed to assess the progress made by UK foreign language undergraduate students during the residence abroad (RA) component of their degree. The article first of all places the study within the international research context, and then describes the data collation procedures and methodology employed. It reports findings concerning proficiency gains and the predictive validity of factors such as gender, IQ, length of stay, prior levels of achievement and previous methodological exposure. The findings confirm significant linguistic gains in general for RA. They also appear to show that individuals vary greatly in their rates of progress during RA on holistic measures of proficiency but that previous methodological exposure is a key factor in determining gains on discrete measures. There is also evidence that longer stays do not necessarily lead to greater proportionate proficiency gains. Findings concerning progress, prediction of progress and individual performance variation are compared and contrasted with those of American and other British studies. (Contains 13 tables, 2 figures, and 6 notes.)

The term "grade descriptors" has commonly been used to refer to the practice of describing for students characteristic work that would merit different grades. This paper reports the results of a phenomenographic study on the different ways that academic staff understood and practised grade descriptors as forms of standards-based assessment. Four qualitatively different conceptions of grade descriptors were identified. Firstly, grade descriptors were described as "generic descriptors" as they depict achievement levels as descriptions of standards for generic purposes. Secondly, grade descriptors were understood as "grade distributors" as they focus on how students' work can be understood in terms of how they are distributed amongst different levels of achievement. Thirdly, grade descriptors were labelled as "grade indicators" since they indicate to staff and students what a piece of student's work might mean in terms of specific criteria. Finally, grade descriptors were labelled as grade interpreters since they are perceived as authentic bodies of intrinsic meaning as to what actual achievement levels are. Through its findings, this study seeks to provide a basis for identifying and resolving different expectations for understanding and practising grade descriptors as well as clarifying the place of standards and criteria in assessment. Each of the conceptions is discussed in terms of providing a form of standards-based assessment. Suggestions for enhancing the use of grade descriptors as standards-based assessment are then made. (Contains 2 tables.)

Higher education institutions have difficulty providing stakeholders with reliable, valid information about student achievement. High stakes assessment data are ambiguous in communicating such information. It is important to distinguish criterion-referenced assessment from norm-referenced assessment. This paper proposes a system that goes beyond high stakes assessment and describes the design of reliable assessment systems. (SM)

A system has been developed for measuring the performance of faculty members at Middle East Technical University (METU) in 1998. This system has been designed to provide feedback for improving the quality of academic work. It involves a separate set of criteria and measures for each college within the university. The criteria used for the College of Engineering include publications, editorial work and translation, professional and other research activities, educational activities, memberships and awards, and other activities. Faculty members are recruited based on these criteria. They report their activities and works annually as required by this system, and they can access their measurement results through the university Intranet. These results play a significant role in promotion of the faculty members. Awards are given to faculty members who demonstrate superior performance. This system has also been utilised to measure the efficiency of engineering departments, and guide and support departments in improving their efficiency. Since the beginning of implementation of this system, significant changes have been observed in preferences and activities of faculty members, and their performance. The paper discusses impact of this system on performance of the Engineering College. [ABSTRACT FROM AUTHOR]

It is incumbent upon universities to deliver quality degree programs that produce employable graduates with discipline-specific knowledge and a well-developed set of generic skills known as graduate attributes. To safeguard the achievement of these outcomes, this paper argues for a holistic, longitudinal approach to degree planning known as programmatic assessment. The status of this relatively new phenomenon in the Australian university landscape is examined through an analysis of assessment-related policies from twenty-two of Australia's top-performing universities. Discourse analysis was employed to determine how programmatic assessment is depicted in policy, how key players are positioned in this space, and the discursive practices used to imbue policy with hortatory intent. The results of this analysis are outlined and indicate that policy constructions of a program-level approach to assessment are inconsistent across different universities and very few policies have specific guidelines about how programmatic assessment should be implemented. [ABSTRACT FROM AUTHOR]

Assessment in higher education has focused on the performance of individual students. This focus has been a practical as well as an epistemic one: methods of assessment are constrained by the technology of the day, and in the past they required the completion by individuals under controlled conditions of set-piece academic exercises. Recent advances in learning analytics, drawing upon vast sets of digitally stored student activity data, open new practical and epistemic possibilities for assessment, and carry the potential to transform higher education. It is becoming practicable to assess the individual and collective performance of team members working on complex projects that closely simulate the professional contexts that graduates will encounter. In addition to academic knowledge, this authentic assessment can include a diverse range of personal qualities and dispositions that are key to the computer-supported cooperative working of professionals in the knowledge economy. This paper explores the implications of such opportunities for the purpose and practices of assessment in higher education, as universities adapt their institutional missions to address twenty-first century needs. The paper concludes with a strong recommendation for university leaders to deploy analytics to support and evaluate the collaborative learning of students working in realistic contexts. [ABSTRACT FROM AUTHOR]

Using additional learning materials positively influences student performance. The literature focuses mainly on learning outcomes, whereas the present study investigates whether two intermediate (take-home) tests are a successful tool to increase student use of online formative assessments (OFAs). This study is a between-subjects quasi-experiment, administered in an undergraduate accounting course. Measures are used to capture both the intensity and the timing of OFA use. Results show that more students begin using OFA when intermediate tests are introduced. In addition, students use OFA more intensively and indications are present that students also begin their OFA use earlier. This paper contributes to the higher education literature by illustrating how online take-home tests can prevent procrastination. Its advantage lies in stimulating the use of online formative assessment and providing students with immediate feedback. Results are discussed for both blended and online learning. [ABSTRACT FROM AUTHOR]

A successful first year is of vital importance to academic achievement in higher education. How can we identify those students at risk at a very early stage, i.e. even before they have entered university? This study focuses on three pre-university factors, namely academic self-efficacy, self-regulated learning, in terms of autonomous motivation and time-management, and beyond classroom engagement. 2355 Dutch prospective first-year students in the fields of science, humanities and social sciences filled in our survey. Data on first year academic achievement in terms of grade point average (GPA) was obtained at the end of the first year. Prospective first-year students scored high on all of the variables; we found gender-related differences, favouring female students, as well as discipline-related differences. Time management and autonomous motivation were found to be positive predictors of achievement; beyond classroom engagement was a negative predictor of achievement. This paper contributes to the research on first-year academic achievement and emphasises the importance of providing students with personalised trajectories from the moment they enter university. [ABSTRACT FROM AUTHOR]

An increasingly regulated higher education sector is renewing its attention to those activities referred to as ‘moderation’ in its efforts to ensure that judgements of student achievement are based on appropriate standards. Moderation practices conducted throughout the assessment process can result in purposes identified as equity, justification, accountability and community building. This paper draws on the limited studies of moderation and wider relevant research on judgement, standards and professional learning to test commonly used moderation practices against these identified purposes. The paper concludes with recommendations for maximising the potential of moderation practices to establish and maintain achievement standards. [ABSTRACT FROM PUBLISHER]

Conceptualising peer assessment as a dialogue is expected to bring many benefits for learners. However, open-ended dialogue still faces many challenges in learning effectiveness because superficial feedback contents, resistance to critical thinking and distrust of peers' assessments may negatively affect learners' performance in their interactions. This study created regulation scripts for dialogic peer assessment, which not only engaged learners in meaningful dialogues through prompt questions, but also guided them to establish shared goals and monitor their collaboration processes. To verify their effectiveness, a quasi-experimental study was performed with 34 undergraduate students in an authentic learning environment. Learners in the experimental groups were required to exchange feedback information in stages involving case evaluation, group dialogic feedback and individual reflection with the support of regulation scripts, while learners in the control groups completed the same experimental process with general scripts. The quantitative data showed that the experimental groups achieved better development in climate of trust and critical thinking. In terms of feedback quality, however, there was no significant difference between the two groups. [ABSTRACT FROM AUTHOR]

Authentic assessment has theorised benefits for student outcomes in higher education. More needs to be done, however, to empirically test relationships between the critical elements of authenticity and student outcomes, particularly in online learning environments. In this paper we examine whether an online simulation-based learning and assessment design, developed using the tools and functionality within the available university online ecosystem, has measurable 'authenticity' in a postgraduate course. In addition, we assess whether the simulation is associated with improvements in student learning behaviours and outcomes. We analyse qualitative data from lecturer and student interviews and student work submitted online, as well as data analytics from the learning management system in both control and experimental conditions. Seven critical elements of authentic assessment were identified including new aspects of authenticity relating to simulation-specific affordances (e.g. emotional content, temporal dimensions and unpredictability). Students watched lecture videos and participated in online discussion more frequently in the online simulation than in the control condition and indicated perceived mastery of, and confidence in, the learning outcomes. These findings make important contributions to authentic assessment theory and have practical implications for simulations built in online ecosystems available in higher education. [ABSTRACT FROM AUTHOR]

A modified essay is an open-ended question about clinical presentations, with the goal to test application of basic science principles to novel scenarios. Due to high reliability and validity, multiple-choice question (MCQ) examinations are more commonly used. We have employed a combination of MCQ and modified essay examinations in medical school. The correlation between MCQ and modified essay grades is positive but small, suggesting they may be testing different cognitive levels/skills. In this paper, we evaluate the impact of a non-graded mock modified essay on student's performance and satisfaction, and outline our best practices. We have found that it is important to involve faculty from multiple disciplines to construct robust cases, and to review examinations and rubrics. The mock modified essay resulted in higher performance and student satisfaction, and fewer students' comments related to subjectivity, frustration and anxiety. Yet, there was not an improvement in confidence. Moreover, there was no correlation between grades in the mock and graded modified essays, suggesting that the mock modified essay may not be useful to predict academic performance. Yet, the incorporation of a mock modified essay was useful to clarify expectations and provide guidance, potentially reducing the anxiety induced by an unfamiliar assessment. [ABSTRACT FROM AUTHOR]

This paper critically reviews the empirical research on the use of rubrics at the post-secondary level, identifies gaps in the literature and proposes areas in need of research. Studies of rubrics in higher education have been undertaken in a wide range of disciplines and for multiple purposes, including increasing student achievement, improving instruction and evaluating programmes. While, student perceptions of rubrics are generally positive and some authors report positive responses to rubric use by instructors, others noted a tendency for instructors to resist using them. Two studies suggested that rubric use was associated with improved academic performance, while one did not. The potential of rubrics to identify the need for improvements in courses and programmes has been demonstrated. Studies of the validity of rubrics have shown that clarity and appropriateness of language is a central concern. Studies of rater reliability tend to show that rubrics can lead to a relatively common interpretation of student performance. Suggestions for future research include the use of more rigorous research methods, more attention to validity and reliability, a closer focus on learning and research on rubric use in diverse educational contexts. [ABSTRACT FROM AUTHOR]

Graduate attributes have received considerable attention in recent years as universities seek to renew and articulate their purposes. Though many claims are made with respect to the implementation of graduate attributes, there is growing acceptance of the proposition that the strongest evidence of their achievement is their explicit embedding in assessment. While university policies espouse the assessment of graduate attributes, studies have suggested that the effectiveness of this undertaking is often limited by the influence of a range of diverse but interrelated factors. This paper discusses these influential factors and argues that the assessment of graduate attributes is a complex issue that is unlikely to be resolved unless these factors are addressed at a systemic level. [ABSTRACT FROM AUTHOR]

The literature on improving student engagement with assessment and feedback has a tendency to treat all students as if they are the same. Students with lower levels of attainment are generally under-represented within empirical studies and their feedback behaviours are less well understood. The recent drive to improve student assessment and feedback literacy and the move from 'feedback' being information about a task to being a process of understanding and using performance information is a larger conceptual leap for some students than others. In this paper, we consider issues surrounding the transition to new modes of feedback, focussing on what is needed for those who find study difficult and persistently are disappointed by their levels of attainment, to benefit from and take advantage of our feedback pedagogies. We examine literature advocating strategies such as increasing agency, using praise, developing feedback literacy and cultivating a growth mind-set. We argue that students who underachieve may benefit from strong relationships with educators and peers, exposure to feedback rich, low stakes environments, which permit repeated integrations of practice and feedback, and building feedback literacy through peer assessment activities. [ABSTRACT FROM AUTHOR]

This paper reports on seven calibrated scales of student engagement emerging from a large-scale study of first year undergraduate students in Australian universities. The analysis presents insights into contemporary undergraduate student engagement, including online, self-managed, peer and student-staff engagement. The results point to the imperative for developing a broader understanding of engagement as a process with several dimensions. These must be acknowledged in any measurement and monitoring of this construct in higher education. The paper calls for a more robust theorising of the engagement concept that encompasses both quantitative and qualitative measures. It considers implications for pedagogy and institutional policy in support of enhancing the quality of the student experience. [ABSTRACT FROM AUTHOR]

In higher education, student evaluation of teaching is widely used as a measure of an academic's teaching performance despite considerable disagreement as to its value. This paper begins by examining the merit of teaching evaluations with reference to the factors influencing the accuracy of the teaching evaluation process. One of the central assumptions on which student evaluation of teaching is based is that there is a relationship between student achievement and student rating of teachers. However, the findings of the majority of studies do not support this assumption. The absence of a strong link between student achievement and teaching evaluations suggests that there is scope for examining other approaches to measuring effective classroom dynamics. This paper presents such an approach based on the notion of transformational classroom leadership. [ABSTRACT FROM AUTHOR]

Decisions by markers about quality in student work remain confusing to most students and markers. This may in part be due to the relatively subjective nature of what constitutes a quality response to an assessment task. This paper reports on an experiment that documented the process of decision-making by multiple markers at a university who assessed the same written student assessment responses. The paper analyses the professional conversations between those markers around their conceptions of quality in the student assessment responses. It was found that the markers appeared to share common understandings of quality in the context of the marking criteria and standards across the achievement levels awarded. However, despite these apparently shared notions of quality, in some cases different levels of achievement were awarded to the same student assessment responses. This suggests that that there is a clear need for explicitly stated standard descriptors for each level of achievement and that this must be interpreted through substantive professional conversations in the context of real student work. The key driver is the student work, and conversations amongst markers about what constitutes 'quality' in the context of the written and explicit criteria and standards of achievement that are available to students and markers alike are a necessity. [ABSTRACT FROM AUTHOR]

The term 'feedforward' is increasingly employed in higher education, and this paper focuses on the way in which it fits into contemporary debates about feedback and its impact on practitioners. Semi-structured interviews were used to investigate the practices academics associate with feedforward and the ways in which their intentions and understandings varied. The term resulted in practices being framed as a process and prompted academics to consider the points in the future when students were expected to deploy information and improve. Three future horizons were identified: the 'within-module' future horizon dominated, while the challenges of 'beyond-module' and 'beyond-programme' horizons became apparent. Written comments, guidance, formative assessment and design practices were associated with feedforward, comprising both transmission and student-focused approaches. It is concluded that the modular system restricts longer-term future horizons and sustainable practices, and more must be done to address the divide between theoretical and practitioners' perspectives. [ABSTRACT FROM AUTHOR]

This paper reports the effect of continuous summative assessment on the behaviour and learning environment of students. Assessing the performance of students is considered to be the most important thing a teacher can do for their students and it can have a profound effect on their learning. Continuous summative assessment was introduced as a form of assessment on a module within an undergraduate degree at a UK university. Student perceptions of the process were sought via a questionnaire and interviews. The paper describes the effect the assessment had on student motivation, their approach to learning and the change to their learning environment. The conclusion reached is that while continuous summative assessment may be time‐consuming to administer, the rewards of an enhanced learning environment for students outweigh the additional burden on staff. The results should be of interest to those academics who are concerned with assessment and its impact on behaviour. [ABSTRACT FROM AUTHOR]

In today's market-driven educational culture, universities are coming under increasing pressure to justify funding through the disclosure of measurable outcomes in education and research. One educational objective that receives particular attention is critical thinking, regarded as an essential skill in both academic and work environments. The assessment of critical thinking has become a significant enterprise, with a number of standardised tests available for both individuals and organisations. While these tests are based on well-known taxonomies of critical thinking, this paper argues that institutions should be wary of using them as a means to measure educational outcomes. First, they fail to take into account fundamentally contested issues within conceptions of critical thinking. They also have significant weaknesses in terms of validity and reliability. Finally, and most importantly, they provide only a limited assessment of critical thinking, failing to evaluate the skills exercised in real-life academic tasks. A more effective approach to critical thinking testing would be one implemented at a faculty level, with assessments carried out on coursework integral to the curricula of specific academic disciplines. [ABSTRACT FROM AUTHOR]