Search

Task-specific metrics facilitate the assessment of surgeon performance. This 3-phased study was designed to (1) develop task-specific metrics for stapled small bowel anastomosis, (2) obtain expert consensus on the appropriateness of the developed metrics, and (3) establish its discriminant validity.In Phase I, a hierarchical task analysis was used to develop the metrics. In Phase II, a survey of expert colorectal surgeons established the importance of the developed metrics. In Phase III, to establish discriminant validity, surgical trainees and surgeons, divided into novice and experienced groups, constructed a side-to-side anastomosis on porcine small bowel using a linear cutting stapler. The participants' performances were videotaped and rated by 2 independent observers. Partial least squares regression was used to compute the weights for the task-specific metrics to obtain weighted total score.In Phase II, a total of 45 colorectal surgeons were surveyed: 28 with more than 15 years, 13 with 5 to 15 years, and 4 with less than 5 years of experience. The consensus was obtained on all the task-specific metrics in the more experienced groups. In Phase III, 20 subjects participated equally in both groups. The experienced group performed better than the novice group regardless of the rating scale used: global rating scale (p = 0.009) and the task-specific metrics (p = 0.012). After partial least squares regression, the weighted task-specific metric score continued to show that the experienced group performed better (plt; 0.001).Task-specific metric items were developed based on expert consensus and showed good discriminant validity compared with a global rating scale between experienced and novice operators. These items can be used for evaluating technical skills in a stapled small bowel anastomosis model.

Objective To develop and evaluate the performance of artificial intelligence (AI) models that can identify safe and dangerous zones of dissection, and anatomical landmarks during laparoscopic cholecystectomy (LC). Summary background data Many adverse events during surgery occur due to errors in visual perception and judgment leading to misinterpretation of anatomy. Deep learning, a subfield of AI, can potentially be used to provide real-time guidance intraoperatively. Methods Deep learning models were developed and trained to identify safe (Go) and dangerous (No-Go) zones of dissection, liver, gallbladder, and hepatocystic triangle during LC. Annotations were performed by four high-volume surgeons. AI predictions were evaluated using 10-fold cross-validation against annotations by expert surgeons. Primary outcomes were intersection-over-union (IOU) and F1 score (validated spatial correlation indices), and secondary outcomes were pixel-wise accuracy, sensitivity, specificity, ± standard deviation. Results AI models were trained on 2627 random frames from 290 LC videos, procured from 37 countries, 136 institutions and 153 surgeons. Mean IOU, F1 score, accuracy, sensitivity, and specificity for the AI to identify Go zones were 0.53 (±0.24), 0.70 (±0.28), 0.94 (±0.05), 0.69 (±0.20) and 0.94 (±0.03) respectively. For No-Go zones, these metrics were 0.71 (±0.29), 0.83 (±0.31), 0.95 (±0.06), 0.80 (±0.21) and 0.98 (±0.05), respectively. Mean IOU for identification of the liver, gallbladder and hepatocystic triangle were: 0.86 (±0.12), 0.72 (±0.19) and 0.65 (±0.22), respectively. Conclusions AI can be used to identify anatomy within the surgical field. This technology may eventually be used to provide real-time guidance and minimize the risk of adverse events.

We report for the first time the use of the Operating Room Black Box (ORBB) to track checklist compliance, engagement, and quality.Implementation of operative checklists is associated with improved outcomes. Compliance is difficult to monitor. Most studies report either no assessment of checklist compliance or deployed in-person short-term assessment. The ORBB a novel artificially intelligence-driven data analytic platform affords the opportunity to assess checklist compliance without disrupting surgical workflow.This was a retrospective review of prospectively collected ORBB data. Operative cases included elective surgery at a quaternary referral center. Cases were analyzed as prepolicy change (first 9 months) or as a postpolicy change (last 9 months). Measures of checklist compliance, engagement, and quality were assessed.There were 3879 cases that were performed and monitored for checklist compliance between August 15, 2020, and February 20, 2022. The overall scores for compliance, engagement, and quality were 81%, 84%, and 67% respectively. When broken down by phase, the scores for time-out were compliance 100%, engagement 98%, and quality 61%. Scores for the debrief phase were 81% for compliance, 98% for engagement, and 66% for quality. After a hospital policy change, the debrief scores improved significantly (85%; P0.001 for compliance, 88%; P0.001 for engagement and 71%; P0.001 for quality).ORBB provides the unprecedented ability to assess not only compliance with surgical safety checklists but also engagement and quality. Utilization of this technology allows the assessment of compliance in near real time and to accurately address safety threats that may arise from noncompliance.

BACKGROUND: The complexity of laparoscopy requires special training and assessment. Analyzing the streaming videos during the surgery can potentially improve surgical education. The tedium and cost of such an analysis can be dramatically reduced using an automated tool detection system, among other things. We propose a new multilabel classifier, called LapTool-Net to detect the presence of surgical tools in each frame of a laparoscopic video. METHODS: The novelty of LapTool-Net is the exploitation of the correlations among the usage of different tools and, the tools and tasks - i.e., the context of the tools’ usage. Towards this goal, the pattern in the co-occurrence of the tools is utilized for designing a decision policy for the multilabel classifier based on a Recurrent Convolutional Neural Network (RCNN), which is trained in an end-to-end manner. In the post-processing step, the predictions are corrected by modeling the long-term tasks’ order with an RNN. RESULTS: LapTool-Net was trained using publicly available datasets of laparoscopic cholecystectomy, viz., M2CAI16 and Cholec80. For M2CAI16, our exact match accuracy (when all the tools in one frame are predicted correctly) in online and offline modes were 80.95% and 81.84% with per-class F1-score of 88.29% and 90.53%. For Cholec80, the accuracies were 85.77% and 91.92% with F1-scores if 93.10% and 96.11% for online and offline respectively. CONCLUSIONS: The results show LapTool-Net outperformed state-of-the-art methods significantly, even while using fewer training samples and a shallower architecture. Our context-aware model does not require expert’s domain-specific knowledge and the simple architecture can potentially improve all existing methods.

Early introduction and distributed learning have been shown to improve student comfort with basic requisite suturing skills. The need for more frequent and directed feedback, however, remains an enduring concern for both remote and in-person training. A previous in-person curriculum for our second-year medical students transitioning to clerkships was adapted to an at-home video-based assessment model due to the social distancing implications of COVID-19. We aimed to develop an Artificial Intelligence (AI) model to perform video-based assessment.Second-year medical students were asked to submit a video of a simple interrupted knot on a penrose drain with instrument tying technique after self-training to proficiency. Proficiency was defined as performing the task under two minutes with no critical errors. All the videos were first manually rated with a pass-fail rating and then subsequently underwent task segmentation. We developed and trained two AI models based on convolutional neural networks to identify errors (instrument holding and knot-tying) and provide automated ratings.A total of 229 medical student videos were reviewed (150 pass, 79 fail). Of those who failed, the critical error distribution was 15 knot-tying, 47 instrument-holding, and 17 multiple. A total of 216 videos were used to train the models after excluding the low-quality videos. A k-fold cross-validation (k = 10) was used. The accuracy of the instrument holding model was 89% with an F-1 score of 74%. For the knot-tying model, the accuracy was 91% with an F-1 score of 54%.Medical students require assessment and directed feedback to better acquire surgical skill, but this is often time-consuming and inadequately done. AI techniques can instead be employed to perform automated surgical video analysis. Future work will optimize the current model to identify discrete errors in order to supplement video-based rating with specific feedback.