Search

Purpose: The percutaneous renal access (PRA) is the most critical step of percutaneous renal surgery (PRS). For the training of PRA in the lab, a novel non-biological bench model was developed and set for validation test. Materials and Methods: Experts in PRS (> 60 cases) and novices were included to perform fluoroscopy guided PRA on the model. Overall time, X-ray exposure time and puncture attempts were recorded to establish construct validity. After accomplishment, the experts rated the model using a standardized questionnaire for face and content validity based on a 5-point Likert scale, with 1 denoting very bad and 5 as excellent. Baseline and post-training data of novices were analyzed for skill acquisition. Results: 9 experts and 30 novices were finally included. The overall appraisal was 4 by the experts, and consensus of all experts was reached for the model as an excellent training tool. Significant difference between experts and novices was detected with the experts using less total time 183.11 ± 29.40 vs. 278.00 ± 50.30 seconds (P < 0.001), shorter X-ray exposure time 109.22 ± 19.93 vs. 183.13 ± 38.83 seconds (P < 0.001), and fewer attempts 1.28 ± 0.44 vs. 2.35 ± 0.65 (P < 0.001). After training, the novices demonstrated significant skill improvement in total and fluoroscopy time, and number of attempts (P < 0.001). Conclusions: Our non-biological model provides a new method for PRA training. The face, content and construct validity were demonstrated. This model allows contact with PRA skills and could be applied to the first step in the learning curve.

Objective The drive towards valid and reliable assessment methods for health professions’ training is becoming increasingly focused towards authentic models of workplace performance assessment. This study investigates the validity of such a method, longitudinal evaluation of performance (LEP), which has been implemented in the assessment of postgraduate dental trainees in Scotland. Although it is similar in format to the mini-CEX (mini clinical evaluation exercise) and other tools that use global ratings for assessing performance in the workplace, a number of differences exist in the way in which the LEP has been implemented. These include the use of a reference point for evaluators’ judgement that represents the standard expected upon completion of the training, flexibility, a greater range of cases assessed and the use of frequency scores within feedback to identify trainees’ progress over time. Methods A range of qualitative and quantitative data were collected and analysed from 2 consecutive cohorts of trainees in Scotland (2002–03 and 2003–04). Results There is rich evidence supporting the validity, educational impact and feasibility of the LEP. In particular, a great deal of support was given by trainers for the use of a fixed reference point for judgements, despite initial concerns that this might be demotivating to trainees. Trainers were highly positive about this approach and considered it useful in identifying trainees’ progress and helping to drive learning. Conclusions The LEP has been successful in combining a strong formative approach to continuous assessment with the collection of evidence on performance within the workplace that (alongside other tools within an assessment system) can contribute towards a summative decision regarding competence. [ABSTRACT FROM AUTHOR]

Context Self-assessment promotes reflective practice, helps students identify gaps in their learning and is used in curricular evaluations. Currently, there is a dearth of validated self-assessment tools in rheumatology. We present a new musculoskeletal self-assessment tool (MSAT) that allows students to assess their confidence in their skills in and knowledge of knee and shoulder examination. Objectives We aimed to validate the 15-item MSAT, addressing its construct validity, internal consistency, responsiveness, repeatability and relationship with competence. Methods Participants were 241 Year 3 students in Newcastle upon Tyne and 113 Year 3 students at University College London, who were starting their musculoskeletal skills placement. Factor analysis explored the construct validity of the MSAT; Cronbach's α assessed its internal consistency; standardised response mean (SRM) evaluated its responsiveness, and test-retest, before and after a pathology lecture, assessed its repeatability. Its relationship with competence was explored by evaluating its correlation with shoulder and knee objective structured clinical examinations (OSCEs). Results The MSAT was valid in distinguishing the 5 domains it intended to measure: clinical examination of the knee; clinical examination of the shoulder; clinical anatomy of the knee and shoulder; history taking, and generic musculoskeletal anatomical and clinical terms. It was internally consistent (α = 0.93), responsive (SRM 0.6 in Newcastle and 2.2 in London) and repeatable (intraclass correlation coefficient 0.97). Correlations between MSAT scores and OSCE scores were weak ( r < 0.2). Conclusions The MSAT has strong psychometric properties, thereby offering a valid approach to evaluating the self-assessment of confidence in examination skills by students. Confidence does not necessarily reflect competence; future research should clarify what underpins confidence. [ABSTRACT FROM AUTHOR]

Background Distance learning has been advocated increasingly as a modern efficient method of teaching surgery. Efficiency of knowledge transfer and validity of web-based courses have not been subjected to rigorous study to date. Methods An entirely web-based surgical 5-week lecture course was designed. Fifty per cent of the lectures were prepared as HTML slides with voice-over while the other group was presented in the text-only form. Only written material presented was examined. The lectures were presented via an educational web module. The lecture series was balanced specifically to reduce the pre-existent knowledge bias. Web usage was estimated utilising surrogates, including the number of hits as well as log-on timing. Face validity was assessed by a standardised questionnaire. Results Eighty-eight students took part in the lecture series and subsequent examination and questionnaire. Median multiple choice questionnaire (MCQ) marks were significantly higher in the aural lecture-derived stems versus the non-aural ( P = 0.012, Mann–Whitney U-test). There was widespread approval of web-based learning as an adjunct to conventional teaching. Usage rates were augmented significantly in the final week when compared to the previous 4 weeks (mean total hits weeks 1–4 ± SEM: 100.9 ± 9.7 and mean total hits week 5: 152.1 ± 13.1; P < 0.001, Kruskal–Wallis). However, total hits did not correlate with overall examination results ( r2 = 0.16). The aural lectures demonstrated higher face validity than the non-aural for content and presentation ( P < 0.05, Kruskal–Wallis). Conclusions The addition of aural files to the novel web-based lecture series is face valid and results in significantly increased examination performance. [ABSTRACT FROM AUTHOR]

Context Admissions interviews are unreliable and have poor predictive validity, yet are the sole measures of non-cognitive skills used by most medical school admissions departments. The low reliability may be due in part to variation in conditional reliability across the rating scale. Objectives To describe an empirically derived estimate of conditional reliability and use it to improve the predictive validity of interview ratings. Methods A set of medical school interview ratings was compared to a Monte Carlo simulated set to estimate conditional reliability controlling for range restriction, response scale bias and other artefacts. This estimate was used as a weighting function to improve the predictive validity of a second set of interview ratings for predicting non-cognitive measures (USMLE Step II residuals from Step I scores). Results Compared with the simulated set, both observed sets showed more reliability at low and high rating levels than at moderate levels. Raw interview scores did not predict USMLE Step II scores after controlling for Step I performance (additional r2 = 0.001, not significant). Weighting interview ratings by estimated conditional reliability improved predictive validity (additional r2 = 0.121, P < 0.01). Conclusions Conditional reliability is important for understanding the psychometric properties of subjective rating scales. Weighting these measures during the admissions process would improve admissions decisions. [ABSTRACT FROM AUTHOR]

Context Cognitive and learning styles research is limited by the lack of evidence supporting valid interpretations of style assessment scores. We sought evidence to support the validity of scores from 4 instruments: the Index of Learning Styles (ILS); the Learning Style Inventory (LSI); the Cognitive Styles Analysis (CSA), and the Learning Style Type Indicator (LSTI). The ILS assesses 4 domains: sensing−intuitive (SensInt), active−reflective (ActRefl), sequential−global (SeqGlob) and visual−verbal (VisVerb), each of which parallel a similar domain in at least 1 of the other instruments. Methods We administered the ILS, LSI and CSA to family medicine and internal medicine residents and Year 1 and 3 medical students and applied the multitrait−multimethod matrix to evaluate convergence and discrimination. After 3 months participants repeated the ILS and completed the LSTI. Results A total of 89 residents and medical students participated. Multitrait−multimethod analysis showed evidence of both convergence and discrimination for ActRefl (ILS, LSI and LSTI) and SensInt (ILS and LSTI) scores. ILS SeqGlob and SensInt scores showed unanticipated correlation. No other domains met the criteria for convergence or discrimination. Test-retest reliabilities for ILS scores were 0.856 for SensInt, 0.809 for ActRefl, 0.703 for SeqGlob and 0.684 for VisVerb. Cronbach's α values were ≥ 0.810 for LSI and 0.237–0.758 for LSTI. At least 9 participants misinterpreted the LSI instructions. Conclusions These data support the validity of ILS active−reflective and sensing−intuitive scores, LSI active−reflective scores and LSTI sensing−intuitive scores for determining learning styles in this population. Cognitive style and learning style scores may not be interchangeable, even for constructs with similar definitions. [ABSTRACT FROM AUTHOR]