Search

To determine the inter-rater reliability (IRR) of a procedure-specific checklist scored in a binary fashion for the evaluation of surgical skill and whether it meets a minimum level of agreement (≥0.8 between 2 raters) required for high-stakes assessment.In a prospective randomized and blinded fashion, and after detailed assessment training, 10 Arthroscopy Association of North America Master/Associate Master faculty arthroscopic surgeons (in 5 pairs) with an average of 21 years of surgical experience assessed the video-recorded 3-anchor arthroscopic Bankart repair performance of 44 postgraduate year 4 or 5 residents from 21 Accreditation Council for Graduate Medical Education orthopaedic residency training programs from across the United States.No paired scores of resident surgeon performance evaluated by the 5 teams of faculty assessors dropped below the 0.8 IRR level (mean = 0.93; range 0.84-0.99; standard deviation = 0.035). A comparison between the 5 assessor groups with 1 factor analysis of variance showed that there was no significant difference between the groups (P = .205). Pearson's product-moment correlation coefficient revealed a strong and statistically significant negative correlation, that is, -0.856 (P.000), indicating that as intra-operative error rate scores increased, the IRR decreased.Arthroscopy Association of North America shoulder faculty raters from across the United States showed high levels of IRR in the assessment of an arthroscopic 3-anchor Bankart repair procedure. All paired assessments were above the 0.8 level and the mean IRR of all resident assessments was 0.93, indicating that they could be used for high-stakes decisions.With the move toward outcomes-based performance evaluation for graduate medical education, high-stakes assessments of surgical skill will require robust, reliable measurement tools that are able to withstand challenge. Surgical checklists employing metrics scored in a binary fashion meet the need and can show a high (80%) IRR.

Purpose To investigate the impact of a proficiency-based progression (PBP) curriculum employed to teach trainees in the skills needed to demonstrate proficiency for an arthroscopic Bankart repair (ABR) and an arthroscopic rotator cuff repair (ARCR) by objectively comparing pre- and immediate postcourse performances. Methods In a prospective study, 16 arthroscopy/sports medicine fellows and 2 senior residents (complete group: N = 18) were randomly assigned to perform a precourse cadaveric ABR (Bankart subgroup: N = 6), ARCR (cuff subgroup: N = 6), or basic skills on a shoulder simulator (N = 6). After completing a PBP training curriculum, all 18 registrants performed both an ABR and ARCR scored in real time by trained raters using previously validated metrics. Results The Bankart subgroup made 58% fewer objectively assessed errors at the completion of the course than at baseline (P = .004, confidence interval –1.449 to –0.281), and performance variability was substantially reduced (standard deviation = 5.89 vs 2.81). The cuff subgroup also made 58% fewer errors (P = .001, confidence interval –1.376 to 0.382) and showed a similar reduction in performance variability (standard deviation = 5.42 vs 2.1). Only one subject’s precourse baseline performance met the proficiency benchmark compared with 89% and 83% of the all registrants on the final ABR and ARCR cadaveric assessments, respectively. Conclusions The results of this study reject the null hypothesis. They demonstrate that the implementation of a PBP simulation curriculum to train the skills necessary to perform arthroscopic Bankart and rotator cuff repairs results in a large and statistically significant improvement in the trainee’s ability to meet the 2 related performance benchmarks. Proficiency was demonstrated by 89% and 83% of the trainees for an ABR and an ARCR, respectively, in a two- and one-half day course. Clinical Relevance Surgical training employing a PBP curriculum is efficient, effective, and has the potential to improve patient safety.

Using a switching stick from the posterior portal is a frequently employed method of identifying the location for the subscapularis split for a shoulder arthroscopic Latarjet. However, in a recent cadaveric study, in none of the 20 specimens studied did this technique result in the split being placed in the preferred position-the junction of the superior two-thirds and the inferior one-third of the subscapularis. As a result, if the position of the coracoid graft is too superior, the anteroinferior extension of the glenoid arc and the vector of the conjoined sling may be suboptimal and fail to restrain humeral head translation. There is thus a need for a more reliable method for establishing the subscapularis split in patients with shoulder instability and significant glenoid bone loss. Moreover, in comparison to the open Latarjet surgical technique, an arthroscopic Latarjet procedure is notably more complex, technically challenging, and involves a demanding learning curve. For any procedure, the morbidity curve is inversely related to the learning curve-the lower one is on the learning curve, the greater the expected complications. The area beneath the morbidity curve is related to increased costs to manage complications and suboptimal outcomes. Study, practice, acquisition, and refinement of the necessary skills needed to perform a complex procedure should be done primarily in the cadaver lab. It is imperative that each of us critically evaluate our own experience and clinical practice to determine if we have or expect to generate the critical volume necessary to successfully master the arthroscopic Latarjet techniques and to retain the expertise needed for optimal patient outcomes.

Surgical simulation will play an increasingly important role in helping to train surgical skills. Tools of varying fidelity including benchtop models and virtual reality computer-generated simulations will be needed to most effectively train for accomplishing specific tasks and techniques. We must search for and identify how best to employ simulation to enable the trainee to master the requisite skills. It is imperative that in our investigation, validation methodologies, rating scales, and statistical tools be rigorously and accurately employed lest we be led astray by acceptance and assumption too freely given. The measure of our effectiveness will be whether or not patients enjoy improved outcomes and experience a reduction in surgical errors and complications.

Purpose To create and determine face validity and content validity of arthroscopic rotator cuff repair (ARCR) performance metrics, to confirm construct validity of the metrics coupled with a cadaveric shoulder, and to establish a performance benchmark for the procedure on a cadaveric shoulder. Methods Five experienced arthroscopic shoulder surgeons created step, error, and sentinel error metrics for an ARCR. Fourteen shoulder arthroscopy faculty members from the Arthroscopy Association of North America formed the modified Delphi panel to assess face and content validity. Eight Arthroscopy Association of North America shoulder arthroscopy faculty members (experienced group) were compared with 9 postgraduate year 4 or 5 orthopaedic residents (novice group) in their ability to perform an ARCR. Instructions were given to perform a diagnostic arthroscopy and a 2-anchor, 4–simple suture repair of a 2-cm supraspinatus tear. The procedure was videotaped in its entirety and independently scored in blinded fashion by trained, paired reviewers. Results Delphi panel consensus for 42 steps and 66 potential errors was obtained. Overall performance assessment showed a mean inter-rater reliability of 0.93. Novice surgeons completed 17% fewer steps (32.1 vs 37.5, P = .001) and enacted 2.5 times more errors than the experienced group (6.21 vs 2.5, P = .012). Fifty percent of the experienced group members and none of the novice group members achieved the proficiency benchmark of a minimum of 37 steps completed with 3 or fewer errors. Conclusions Face validity and content validity for the ARCR metrics, along with construct validity for the metrics and cadaveric shoulder, were verified. A proficiency benchmark was established based on the mean performance of an experienced group of arthroscopic shoulder surgeons. Clinical Relevance Validated procedural metrics combined with the use of a cadaveric shoulder can be used to accurately assess the performance of an ARCR.

Purpose: To determine if previously validated performance metrics for an arthroscopic Bankart repair (ABR) coupled with a cadaveric shoulder are a valid assessment tool with the ability to discriminate between the performances of experienced and novice surgeons and to establish a proficiency benchmark for an ABR using a cadaveric shoulder. Methods: Ten master/associate master faculty from an Arthroscopy Association of North America Resident Course (experienced group) were compared with 12 postgraduate year 4 and postgraduate year 5 orthopaedic residents (novice group). Each group was instructed to perform a diagnostic arthroscopy and a 3 suture anchor Bankart repair on a cadaveric shoulder. The procedure was videotaped in its entirety and independently scored in blinded fashion by a pair of trained reviewers. Scoring was based on defined and previously validated metrics for an ABR and included steps, errors, “sentinel” (more serious) errors, and time. Results: The inter-rater reliability was 0.92. Novice surgeons made 50% more errors (5.86 v 2.95, P ¼ .013), showed more performance variability (SD, 1.86 v 0.55), and took longer to perform the procedure (45.5 minutes v 25.9 minutes, P < .001). The greatest difference in errors related to suture delivery and management (exclusive of knot tying) (1.95 v 0.45, P ¼ .024). Conclusions: The assessment tool composed of validated arthroscopic Bankart metrics coupled with a cadaveric shoulder accurately distinguishes the performance of experienced from novice orthopaedic surgeons. A benchmark based on the mean performance of the experienced group includes completion of a 3-anchor Bankart repair, and enacting no more than 3 total errors and 1 sentinel error. Clinical Relevance: Validated procedural metrics combined with the use of a cadaveric shoulder can be used to assess the performance of an ABR. The methodology used may serve as a template for outcomes-based procedural skills training in general.

Purpose To determine if a dry shoulder model simulator coupled with previously validated performance metrics for an arthroscopic Bankart repair (ABR) would be a valid tool with the ability to discriminate between the performance of experienced and novice surgeons, and to establish a proficiency benchmark for an ABR using a model simulator. Methods We compared an experienced group of arthroscopic shoulder surgeons (Arthroscopy Association of North America faculty) (n = 12) with a novice group (n = 7) (postgraduate year 4 or 5 orthopaedic residents). All surgeons were instructed to perform a diagnostic arthroscopy and a 3 suture anchor Bankart repair on a dry shoulder model. Each procedure was videotaped in its entirety and scored in blinded fashion independently by 2 trained reviewers. Scoring used previously validated metrics for an ABR and included steps, errors, and "sentinel" (more serious) errors. Results The inter-rater reliability among pairs of raters averaged 0.93. The experienced group made 63% fewer errors, committed 79% fewer sentinel errors, and performed the procedure in 42% less time than the novice group (all significant differences). The greatest difference in errors between the groups involved anchor preparation and insertion, suture delivery and management, and knot tying. Conclusions The tool comprised by validated ABR metrics coupled with a dry shoulder model simulator is able to accurately distinguish between the performance of experienced and novice orthopaedic surgeons. A performance benchmark based on the mean performance of the experienced group includes completion of a 3 anchor Bankart repair, enacting no more than 4 total errors and 1 sentinel error. Clinical Relevance The combination of performance metrics and an arthroscopic shoulder model simulator can be used to improve the effectiveness of surgical skills training for an ABR. The methodology used may serve as a template for outcomes-based procedural skills training in general.

The value of video gaming in relation to endoscopic surgical skills is explored in this article. Subjects who had more experience and performed better on 3-dimensional games also performed better on an arthroscopic simulator during tasks requiring manipulation of tools. Although the correlation is established, no data are presented to support causation. Video gaming holds promise in assisting the trainee attempting to develop components of endoscopic skill. Further investigation is required to demonstrate that improvements in performance can be expected to result.