Search

Supervisor interview guide. (DOC 45Â kb)

The identification and specification of competency based standards in speech language pathology has provided practitioners, educators, employers and government regulators with information and guidance. This paper reports the outcomes of workshops that provided familiarisation with the new competency based assessment tool, COMPASS®, which was introduced for the assessment of speech-language pathology (SLP) students across all 13 SLP professional preparation programs in Australia during 2007. An anonymous evaluation was administered before and after the first 8 familiarisation workshops held nationally, involving 240 clinical educators. Quantitative data were analysed descriptively, and qualitative data were entered into NVivo qualitative analysis software for content analysis. Post workshop, results indicated partial or full uptake of the main concepts involved in the new approach to assessment. Least uptake was observed for the need for direct observation of competence in workplace performance. Qualitatively, post workshop, formative assessment was more apparent within student goals formulated in response to a hypothetical scenario. A possible contributor to this outcome is suggested to be the alignment between the tool and the professional community of practice, due to the collaborative process of its development. Research into the longer term impact of the new assessment in the context of everyday practice is suggested.

In Australia, the discipline of speech pathology has been engaged in competency based education in a sustained manner for 15 years. Speech pathologists are university educated professionals who work in the health and education sectors, with people experiencing communication and/or swallowing difficulties. Speech pathology programs have, from their inception, combined university and workplace based education (practicum) strategies. University courses are accredited by Speech Pathology Australia (SPA), so that their graduates are deemed ‘eligible’ for membership of SPA. This accreditation process has been guided by the CBOS Competency Based Occupational Standards for Speech Pathologists - Entry level (SPAA, 2001) (known as CBOS, pronounced cee-bos). The CBOS were developed through a national collaborative project (Dawson, 1993a, 1993b) and subsequently revised in 2001. CBOS identifies concepts and descriptions of professional competence, and makes statements about what level of performance is required for entry into practice (Ferguson, 2006)., Adelaide, SA

Ensuring that speech pathology students are sufficiently competent to practise their profession is of critical importance to the speech pathology profession, students, their future employers, and clients/patients. This thesis describes the development and validation of a competency based assessment of speech pathology students’ performance in the workplace and their readiness to enter the profession. Development involved an extensive literature review regarding the nature of competency and its relationship to professional practice, the purpose and nature of assessment, and the validation of performance assessments. An online and hard copy assessment tool (paper) was designed through integrating multiple sources of information regarding speech pathology and assessment of workplace performance. Sources included research, theory, expert opinion, current practice, and focus group consultations with clinical educators and speech pathology students. The resulting assessment tool and resource material included four generic components of competency (clinical reasoning, professional communication, lifelong learning, and professional role) and seven occupational competencies previously developed by the speech pathology profession. The tool comprised an assessment format, either in a booklet or online, for clinical educators to rate students’ performances on the competencies at mid and end placement using a visual analogue scale. Behavioural descriptors and an assessment resource booklet informed and supported clinical educators’ judgement. The validity of the assessment tool was evaluated through a national field trial and using Messick’s six interrelated validity criteria which address content, substantive, structural, generalisability, external, and consequential aspects of validity (Messick, 1996). The validity of the assessment tool and its use with speech pathology students was evaluated through Rasch analysis, parametric statistical evaluation of relationships existing between information yielded by the Rasch analysis and other factors, and student and clinical educator feedback. The assessment tool was found to have strong validity characteristics across all validity components. Item Fit statistics generated through Rasch analysis ranged from .81 to 1.17 strongly upholding that the assessment items sampled a unidimensional construct of workplace competency for speech pathology students and confirming that generic and occupational competencies are both necessary for competent practice of speech pathology. High Item and Person Reliabilities (analogous to Cronbach’s alpha) were found (.98 and .97 respectively) and a wide range of person measures (-14.2 to 13.1) were generated. This indicated that a large spread of ability and a clear hierarchy of development on the construct was identified and that the assessment tool was highly reliable. This was further confirmed by high Intra Class Correlation coefficients for a small group of paired clinical educators rating the same student in the same workplace (.87) or in different workplaces concurrently (.82). Rasch analysis of the visual analogue scale used to rate student performance on 11 items of competence identified that clinical educators were able to reliably discriminate 7 categories or levels of student performance. This, in combination with careful calibration procedures, has resulted in an assessment tool that Australian Speech Pathology pre-professional preparation programs can use with confidence to place their students’ level of workplace competence into 7 zones of competency, with the seventh representing sufficient competence to enter the profession. The assessment tool also showed strong potential for identifying marginal students and for future use in promoting quality teaching and learning of professional competence. Limitations to the research and the tool validity were discussed, and recommendations made regarding future research. First, the clinical educator, who has dual and possibly conflicting roles as facilitator and assessor of student learning, made the assessment. Second, situating the assessment in the real workplace limits the students’ opportunities to demonstrate competence to those that naturally arise in the workplace. Paradoxically, both these factors also contributed to the validity of the assessment tool. It was recommended that the assessment tool be revised on the basis of the information gathered from the field trial, that further data be collected to ensure a broader proportional representation of speech pathology programs, to investigate possible threats to validity as well as those areas for which the tool showed promise. This research developed the first prototype of a validated assessment of entry level speech pathology competence that is grounded in a unified theoretical conception of entry level competence to the profession of speech pathology and the developmental progression required to reach this competence. This research will assist the profession of speech pathology by ensuring that speech pathologists enter the workplace well equipped to provide quality care to their future clients, the ultimate goal of any professional preparation program. Messick, S. (1996). Validity of performance assessments. In G. W. Phillips (Ed.), Technical Issues in Large-Scale Performance Assessment (pp. 1-18). Washington: National Centre for Education Statistics.