Search

2017 The Australian Psychological SocietyObjective: Clinical psychology trainees with problems of professional competence (PPC) continue to be a challenge for courses. Despite the rapid development of competency-based training models, the impact of this shift to the identification and management of professional competency problems is unclear. This project aims to describe how clinical psychology trainees with PPC are identified and managed within the Australian and New Zealand context. Method: An online survey was distributed through Australian and New Zealand universities offering clinical psychology training programmes. Questions addressed approaches to monitoring progress on placements, identification and management of trainees determined to be underperforming on placements, and the perceived usefulness of a range of strategies such as the use of standardised-rating tools. Results: Thirty one responses were received, representing 40 clinical psychology training courses in 22 institutions across Australia and New Zealand. In all cases, at least one trainee with a PPC had been detected in the previous 5 years, most commonly attributed to psychological, behavioural, and developmental issues. Respondents reported the use of a range of preventive and remedial strategies, including the use of psychometrically validated competency evaluation rating forms to assist in the grading of placements. Conclusion: Trainees with PPC occur on a fairly regular basis in clinical psychology training courses in Australian and New Zealand. While some processes involved in the identification and management of these students have been refined and systematised, some opportunities to facilitate early identification and remediation may yet need further enhancement.

Supervisors' ratings of psychology trainees' competence in field settings are a critical component of training assessment. There has been little systematic research regarding the validity of these assessments, but the available evidence suggests we have a problem! Supervisors' judgments may be affected by systemic biases that pose a serious threat to assessment credibility. The current study is part of a research collaboration among six universities that endeavors to develop and evaluate a new method the use of vignettes against outcomes derived from a conventional rating scale. Individual vignettes were designed and subjected to a rigorous process of peer-review and revisions, before final vignettes were assigned calibration scores by a group of experts. A catalogue of vignettes (n = 41) that represent various domains of competence across several developmental stages was compiled. University and field supervisors used the conventional rating scale and the vignette-matching procedure (VMP) to evaluate competencies at end-placement. Data from a pilot (n = 20) and a follow-up study (n = 57) suggest that compared with a conventional rating scale, the VMP reduced leniency and halo biases. The VMP has the potential to improve outcomes of competency assessments in field placements and merits further research and development.

Converging evidence from several health disciplines suggests that field supervisor ratings of student competencies may be affected by systematic leniency and halo biases. Characteristics of the rating scales, rather than unskilled assessors, may be primarily responsible for these unsatisfactory assessment outcomes. The vignette method has yielded positive preliminary results. In the vignette approach, the supervisor is provided with pre-designed vignettes and asked to choose a vignette that best matches the student‘s performance. The current study is a multi-site initiative involving six universities in Australia and funded by an Australian Learning and Teaching Council (ALTC) grant. A key aim is to design, standardise, and evaluate a catalogue of vignettes to assess clinical psychology competencies in the field, and to compare outcomes from the vignette and rating-scale approaches. Preliminary results from the study are reported. The vignettes were designed following a rigorous process: (i) A group of experts drafted vignettes (V1) for each of 4 performance levels across 9 broad competency domains (36 vignettes), (ii) V1-vignettes were scrutinised within small group discussions or assigned to an expert in the psychology domain before revised V2-vignettes were produced, (iii) V2-vignettes were evaluated either within a small group of experts or by two blinded experts who independently revised the vignettes. Expert comments were considered before revised V3-vignettes were prepared for pilot testing, (iv) a web-based survey program was used to present the V3-vignettes to a group of experts (n=12) for evaluation and calibration. The experts reviewed each vignette independently, rated the vignettes in terms of their adequacy and efficacy, and calibrated each vignette using a visual analogue scale that ranged from 1 (unskilled) to 10 (competent), (v) The V3-vignettes were field tested by having a group of field supervisors (n=25) use the vignettes and the conventional ra

Objectives: Provoke scrutiny and reflection about assessment of competencies in field placements Enhance awareness of biases that may affect supervisor ratings of students on placements Discuss issues and solutions Outline a multi-site, ALTC-funded initiative that seeks to address problems

Health professional education invariably includes clinical field placements as an integral component of professional training in an effort to ensure that knowledge, skills, and behaviour learned about in an academic context are integrated and applied to clinical practice. Significant weight is usually attached to the assessment of competence by supervisors, which typically consists of a structured report rating performance on a number of areas. Such reports are subject to systematic rating biases, particularly leniency and halo errors. Strikingly similar results are observed across disciplines including psychology, social work, nursing, pharmacy and medicine. The leniency bias is typically evident in the reluctance of field supervisors to assign low and fail grades to students on placement. The halo bias is reflected in an overall positive impression of a student who has done well in one competency domain, reducing judgement of true differences between competency domains. Factors contributing this phenomenon may include the small numbers of students on placement at any given time, leaving the supervisor without an anchor', and leading to a greater propensity for uncertainty and bias. Secondly, the active face-to-face engagement and intensive supervisor-student interactions over an extended period of time in the field may lead to a relatively close supervisor-student interpersonal relationship that serves to systematically bias supervisor ratings. This study set out to determine the extent of the problems with reliability and validity of supervisor judgments of competency in postgraduate clinical psychology training.