Search

Contains fulltext : 305368.pdf (Publisher’s version ) (Closed access), Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. In biomedical image analysis, chosen performance metrics often do not reflect the domain interest, and thus fail to adequately measure scientific progress and hinder translation of ML techniques into practice. To overcome this, we created Metrics Reloaded, a comprehensive framework guiding researchers in the problem-aware selection of metrics. Developed by a large international consortium in a multistage Delphi process, it is based on the novel concept of a problem fingerprint-a structured representation of the given problem that captures all aspects that are relevant for metric selection, from the domain interest to the properties of the target structure(s), dataset and algorithm output. On the basis of the problem fingerprint, users are guided through the process of choosing and applying appropriate validation metrics while being made aware of potential pitfalls. Metrics Reloaded targets image analysis problems that can be interpreted as classification tasks at image, object or pixel level, namely image-level classification, object detection, semantic segmentation and instance segmentation tasks. To improve the user experience, we implemented the framework in the Metrics Reloaded online tool. Following the convergence of ML methodology across application domains, Metrics Reloaded fosters the convergence of validation methodology. Its applicability is demonstrated for various biomedical use cases.

Contains fulltext : 305370.pdf (Publisher’s version ) (Closed access), Validation metrics are key for tracking scientific progress and bridging the current chasm between artificial intelligence research and its translation into practice. However, increasing evidence shows that, particularly in image analysis, metrics are often chosen inadequately. Although taking into account the individual strengths, weaknesses and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multistage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides a reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Although focused on biomedical image analysis, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. The work serves to enhance global comprehension of a key topic in image analysis validation.

Introduction: Robot-assisted surgery is often performed by experienced laparoscopic surgeons. However, this technique requires a different set of technical skills and surgeons are expected to alternate between these approaches. The aim of this study is to investigate the crossover effects when switching between laparoscopic and robot-assisted surgery. Methods: An international multicentre crossover study was conducted. Trainees with distinctly different levels of experience were divided into three groups (novice, intermediate, expert). Each trainee performed six trials of a standardized suturing task using a laparoscopic box trainer and six trials using the da Vinci surgical robot. Both systems were equipped with the ForceSense system, measuring five force-based parameters for objective assessment of tissue handling skills. Statistical comparison was done between the sixth and seventh trial to identify transition effects. Unexpected changes in parameter outcomes after the seventh trial were further investigated. Results: A total of 720 trials, performed by 60 participants, were analysed. The expert group increased their tissue handling forces with 46% (maximum impulse 11.5 N/s to 16.8 N/s, p = 0.05), when switching from robot-assisted surgery to laparoscopy. When switching from laparoscopy to robot-assisted surgery, intermediates and experts significantly decreased in motion efficiency (time (sec), resp. 68 vs. 100, p = 0.05, and 44 vs. 84, p = 0.05). Further investigation between the seventh and ninth trial showed that the intermediate group increased their force exertion with 78% (5.1 N vs. 9.1 N, p = 0.04), when switching to robot-assisted surgery. Conclusion: The crossover effects in technical skills between laparoscopic and robot-assisted surgery are highly depended on the prior experience with laparoscopic surgery. Where experts can alternate between approaches without impairment of technical skills, novices and intermediates should be aware of decay in eff, Medical Instruments & Bio-Inspired Technology

Introduction: Although robotic-assisted surgery is increasingly performed, objective assessment of technical skills is lacking. The aim of this study is to provide validity evidence for objective assessment of technical skills for robotic-assisted surgery. Methods: An international multicenter study was conducted with participants from the academic hospitals Heidelberg University Hospital (Germany, Heidelberg) and the Amsterdam University Medical Centers (The Netherlands, Amsterdam). Trainees with distinctly different levels of robotic surgery experience were divided into three groups (novice, intermediate, expert) and enrolled in a training curriculum. Each trainee performed six trials of a standardized suturing task using the da Vinci Surgical System. Using the ForceSense system, five force-based parameters were analyzed, for objective assessment of tissue handling skills. Mann–Whitney U test and linear regression were used to analyze performance differences and the Wilcoxon signed-rank test to analyze skills progression. Results: A total of 360 trials, performed by 60 participants, were analyzed. Significant differences between the novices, intermediates and experts were observed regarding the total completion time (41 s vs 29 s vs 22 s p = 0.003), mean non zero force (29 N vs 33 N vs 19 N p = 0.032), maximum impulse (40 Ns vs 31 Ns vs 20 Ns p = 0.001) and force volume (38 N3 vs 32 N3 vs 22 N3p = 0.018). Furthermore, the experts showed better results in mean non-zero force (22 N vs 13 N p = 0.015), maximum impulse (24 Ns vs 17 Ns p = 0.043) and force volume (25 N3 vs 16 N3p = 0.025) compared to the intermediates (p ≤ 0.05). Lastly, learning curve improvement was observed for the total task completion time, mean non-zero force, maximum impulse and force volume (p ≤ 0.05). Conclusion: Construct validity for force-based assessment of tissue handling skills in robot-assisted surgery is established. It is advis, Medical Instruments & Bio-Inspired Technology

The complexity of handheld steerable laparoscopic instruments (SLI) may impair the learning curve compared to conventional instruments when first utilized. This study aimed to provide the current state of interest in the use of SLI, the current use of these in daily practice and the type of training which is conducted before using SLI in the operating room (OR) on real patients. An online survey was distributed by European Association of Endoscopic Surgery (EAES) Executive Office to all active members, between January 4th and February 3rd, 2020. The survey consisted of 14 questions regarding the usage and training of steerable laparoscopic instruments. A total of 83 members responded, coming from 33 different countries. Twenty three percent of the respondents using SLI, were using the instruments routinely and of these 21% had not received any formal training in advance of using the instruments in real patients. Of all responding EAES members, 41% considered the instruments to potentially compromise patient safety due to their complexity, learning curve and the inexperience of the surgeons. The respondents reported the three most important aspects of a possible steerable laparoscopic instruments training curriculum to be: hands-on training, safe tissue handling and suturing practice. Finally, a major part of the respondents consider force/pressure feedback data to be of significant importance for implementation of training and assessment of safe laparoscopic and robotic surgery. Training and assessment of skills regarding safe implementation of steerable laparoscopic instruments is lacking. The respondents stressed the need for specific hands-on training during which feedback and assessment of skills should be guaranteed before operating on real patients., Medical Instruments & Bio-Inspired Technology, Electrical Engineering, Mathematics and Computer Science, Team Sergio Grammatico

The complexity of handheld steerable laparoscopic instruments (SLI) may impair the learning curve compared to conventional instruments when first utilized. This study aimed to provide the current state of interest in the use of SLI, the current use of these in daily practice and the type of training which is conducted before using SLI in the operating room (OR) on real patients. An online survey was distributed by European Association of Endoscopic Surgery (EAES) Executive Office to all active members, between January 4th and February 3rd, 2020. The survey consisted of 14 questions regarding the usage and training of steerable laparoscopic instruments. A total of 83 members responded, coming from 33 different countries. Twenty three percent of the respondents using SLI, were using the instruments routinely and of these 21% had not received any formal training in advance of using the instruments in real patients. Of all responding EAES members, 41% considered the instruments to potentially compromise patient safety due to their complexity, learning curve and the inexperience of the surgeons. The respondents reported the three most important aspects of a possible steerable laparoscopic instruments training curriculum to be: hands-on training, safe tissue handling and suturing practice. Finally, a major part of the respondents consider force/pressure feedback data to be of significant importance for implementation of training and assessment of safe laparoscopic and robotic surgery. Training and assessment of skills regarding safe implementation of steerable laparoscopic instruments is lacking. The respondents stressed the need for specific hands-on training during which feedback and assessment of skills should be guaranteed before operating on real patients., Medical Instruments & Bio-Inspired Technology, Electrical Engineering, Mathematics and Computer Science, Team Sergio Grammatico

Objective: To optimize anastomotic technique and gastric conduit perfusion with hyperspectral imaging (HSI) for total minimally invasive esophagectomy (MIE) with linear stapled anastomosis. Summary Background Data: Esophagectomy is the mainstay of esophageal cancer treatment but anastomotic insufficiency related morbidity and mortality remain challenging for patient outcome. Methods: A live porcine model (n=50) for MIE was used with gastric conduit formation and linear stapled side-to-side esophagogastrostomy. Four main experimental groups differed in stapling length (3 vs. 6 cm) and anastomotic position on the conduit (cranial vs. caudal). Tissue oxygenation around the anastomotic site was evaluated using HSI and was validated with histopathology. Results: The tissue oxygenation (ΔStO2) after the anastomosis remained constant only for the short stapler in caudal position (-0.4±4.4%, n.s.) while it dropped markedly in the other groups (short-cranial: -15.6±11.5%, p=0.0002; long-cranial: -20.4±7.6%, p=0.0126; long-caudal: -16.1±9.4%, p<0.0001) Tissue samples from deoxygenated stomach as measured by HSI showed correspondent eosinophilic pre-necrotic changes in 35.7±9.7% of the surface area. Conclusions: Tissue oxygenation at the anastomotic site of the gastric conduit during MIE is influenced by stapling technique. Optimal oxygenation was achieved with a short stapler (3 cm) and sufficient distance of the anastomosis to the cranial end of the gastric conduit. HSI tissue deoxygenation corresponded to histopathologic necrotic tissue changes. These findings allow for optimization of gastric conduit perfusion and anastomotic technique in MIE.

Background: Robotic-assisted surgery (RAS) potentially reduces workload and shortens the surgical learning curve compared to conventional laparoscopy (CL). The present study aimed to compare robotic-assisted cholecystectomy (RAC) to laparoscopic cholecystectomy (LC) in the initial learning phase for novices. Methods: In a randomized crossover study, medical students (n = 40) in their clinical years performed both LC and RAC on a cadaveric porcine model. After standardized instructions and basic skill training, group 1 started with RAC and then performed LC, while group 2 started with LC and then performed RAC. The primary endpoint was surgical performance measured with Objective Structured Assessment of Technical Skills (OSATS) score, secondary endpoints included operating time, complications (liver damage, gallbladder perforations, vessel damage), force applied to tissue, and subjective workload assessment. Results: Surgical performance was better for RAC than for LC for total OSATS (RAC = 77.4 ± 7.9 vs. LC = 73.8 ± 9.4; p = 0.025, global OSATS (RAC = 27.2 ± 1.0 vs. LC = 26.5 ± 1.6; p = 0.012, and task specific OSATS score (RAC = 50.5 ± 7.5 vs. LC = 47.1 ± 8.5; p = 0.037). There were less complications with RAC than with LC (10 (25.6%) vs. 26 (65.0%), p = 0.006) but no difference in operating times (RAC = 77.0 ± 15.3 vs. LC = 75.5 ± 15.3 min; p = 0.517). Force applied to tissue was similar. Students found RAC less physical demanding and less frustrating than LC. Conclusions: Novices performed their first cholecystectomies with better performance and less complications with RAS than with CL, while operating time showed no differences. Students perceived less subjective workload for RAS than for CL. Unlike our expectations, the lack of haptic feedback on the robotic system did not lead to higher force application during RAC than LC and did not increase tissue damage. These results show potential advantages for RAS over CL for surgical novices while performing thei, Medical Instruments & Bio-Inspired Technology

Background: Robotic-assisted surgery (RAS) potentially reduces workload and shortens the surgical learning curve compared to conventional laparoscopy (CL). The present study aimed to compare robotic-assisted cholecystectomy (RAC) to laparoscopic cholecystectomy (LC) in the initial learning phase for novices. Methods: In a randomized crossover study, medical students (n = 40) in their clinical years performed both LC and RAC on a cadaveric porcine model. After standardized instructions and basic skill training, group 1 started with RAC and then performed LC, while group 2 started with LC and then performed RAC. The primary endpoint was surgical performance measured with Objective Structured Assessment of Technical Skills (OSATS) score, secondary endpoints included operating time, complications (liver damage, gallbladder perforations, vessel damage), force applied to tissue, and subjective workload assessment. Results: Surgical performance was better for RAC than for LC for total OSATS (RAC = 77.4 ± 7.9 vs. LC = 73.8 ± 9.4; p = 0.025, global OSATS (RAC = 27.2 ± 1.0 vs. LC = 26.5 ± 1.6; p = 0.012, and task specific OSATS score (RAC = 50.5 ± 7.5 vs. LC = 47.1 ± 8.5; p = 0.037). There were less complications with RAC than with LC (10 (25.6%) vs. 26 (65.0%), p = 0.006) but no difference in operating times (RAC = 77.0 ± 15.3 vs. LC = 75.5 ± 15.3 min; p = 0.517). Force applied to tissue was similar. Students found RAC less physical demanding and less frustrating than LC. Conclusions: Novices performed their first cholecystectomies with better performance and less complications with RAS than with CL, while operating time showed no differences. Students perceived less subjective workload for RAS than for CL. Unlike our expectations, the lack of haptic feedback on the robotic system did not lead to higher force application during RAC than LC and did not increase tissue damage. These results show potential advantages for RAS over CL for surgical novices while performing thei, Medical Instruments & Bio-Inspired Technology

Background: Chest tube insertion is a standard intervention for management of various injuries of the thorax. Efficient clinical training of this and similar bed-side procedures is equally demanded and improvable. Here, we propose a nouveau means of assessment and feedback using an Objective Structured Assessment of Technical Skills (OSATS) tool. The modified OSATS for chest drain insertion is evaluated in a pilot trial focusing on chest drain insertion.Methods: Participants in the pilot trial were medical students (3rd-6th year of studies, n=9), junior residents (1st-3rd post-graduate year, n=12), senior residents (4th-6th post-graduate year, n=14), and attending surgeons (n=6) from Heidelberg University. Chest drain insertions on a cadaveric porcine model were rated by experts with the modified OSATS score. Participants' performances were videotaped and subsequently rated by two remote experts (video rating). Primary aim was to assess criterion validity of the OSATS to distinguish experience levels. Results: Kruskal-Wallis test showed significant differences between means of scores between four groups stratified by previous experience in chest tube insertion (level 0: 22.1±3.2 vs. level 1: 26.8±2.8 vs. level 2: 35.4±2.2 vs. level 3: 41.0±2.0; p=0.002; p1,3=0.049, p0,3=0.005). However, if groups were stratified by formal professional level, no statistically significant distinction could be made using OSATS. Hence, the OSATS tool showed criterion validity for differentiation between experience levels.Conclusion: In the pilot study, the modified OSATS for chest tube insertion was apt to standardize expert rating and could be used to measure skill and to depict different experience levels. The OSATS will help facilitate training and assessment of chest drain insertion and could therefore improve surgical training for trauma situations. According to our data, the OSATS might be integrated into modern curricula., Hintergrund: Die Thoraxdrainage ist ein etabliertes Standardverfahren zur Versorgung von Verletzungen des Thorax und der Lunge. Die effektive und zügige Anlage einer Thoraxdrainage setzt profunde Kenntnisse des Eingriffs voraus. Ein regelmäßiges Training sowie ein standardisiertes, objektives Feedback sind essentielle Voraussetzungen zur Sicherstellung einer optimalen Thoraxdrainagenanlage, auch unter Stresssituationen. Eine Möglichkeit zur Objektivierung und Strukturierung von Feedback besteht in der Anwendung von Assessment-Instrumenten wie dem Objective Structured Assessment of Technical Skill (OSATS). Ziel der vorliegenden Pilotstudie war die Evaluation eines modifizierten OSATS zur Thoraxdrainagenanlage hinsichtlich der Kriteriumsvalidität.Material und Methodik: In die vorliegende Pilotstudie wurden 41 Probanden eingeschlossen (Medizinstudierende (3.-6. Studienjahr) n=9, Assistenzärzte (1.-3. Ausbildungsjahr) n=12, Assistenzärzte (4.-6. Ausbildungsjahr) n=14, Fachärzte n=6. Die Stratifizierung der Probanden erfolgte nach formalem Ausbildungsstand sowie nach individueller Vorerfahrung im Bereich der Thoraxdrainagenanlage in jeweils 4 Gruppen (Level 0-3). Die Thoraxdrainagenanlage erfolgte an Schweinekadavern unter standardisierten Bedingungen. Die Bewertung der Trainingsperformance erfolgte mittels Videoaufzeichnungen des Eingriffs (Videorating) durch zwei unabhängige Rater anhand des modifiziertem OSATS zur Thoraxdrainagenanlage.Ergebnisse: Bei Stratifizierung nach Vorerfahrung der Probanden zeigten sich signifikante Unterschiede zwischen den Leistungen der vier Gruppen (Level 0: 22,1±3,2 vs. Level 1: 26,8±2,8 vs. Level 2: 35,4±2,2 vs. Level 3: 41.0±2,0; p=0.002; p0,3=0.005, p1,3=0.049,). Bei Stratifizierung nach formalem Ausbildungsstand zeigten sich hingegen keine signifikanten Unterschiede.Schlussfolgerung: Die Ergebnisse der Pilotstudie zeigen die Kriteriumsvalidität des OSATS aufgrund dessen Fähigkeit zur Diskriminierung verschiedener Erfahrungslevel. Weit