Search

Purpose: Micro-electrode recordings (MERs) are a key intra-operative modality used during deep brain stimulation (DBS) electrode implantation, which allow for a trained neurophysiologist to infer the anatomy in which the electrode is placed. As DBS targets are small, such inference is necessary to confirm that the electrode is correctly positioned. Recently, machine learning techniques have been used to augment the neurophysiologist's capability. The goal of this paper is to investigate the generalisability of these methods with respect to different clinical centres and training paradigms., Methods: Five deep learning algorithms for binary classification of MER signals have been implemented. Three databases from two different clinical centres have also been collected with differing size, acquisition hardware, and annotation protocol. Each algorithm has initially been trained on the largest database, then either directly tested or fine-tuned on the smaller databases in order to estimate their generalisability. As a reference, they have also been trained from scratch on the smaller databases as well in order to estimate the effect of the differing database sizes and annotation systems., Results: Each network shows significantly reduced performance (on the order of a 6.5% to 16.0% reduction in balanced accuracy) when applied out-of-distribution. This reduction can be ameliorated through fine-tuning the network on the new database through transfer learning. Although, even for these small databases, it appears that retraining from scratch may still offer equivalent performance as fine-tuning with transfer learning. However, this is at the expense of significantly longer training times., Conclusion: Generalisability is an important criterion for the success of machine learning algorithms in clinic. We have demonstrated that a variety of recent machine learning algorithms for MER classification are negatively affected by domain shift, but that this can be quickly ameliorated through simple transfer learning procedures that can be readily performed for new centres., Competing Interests: Declarations. Conflict of interest: The authors have no conflicts of interest to declare. Ethics approval: The London database was approved by the Research Ethics Board at Western University, Canada (REB # 109045). Both Rennes databases were approved by the Rennes University Hospital Centre ethics committee (Ethical authorisation declaration n2205295)., (© 2024. CARS.)

Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. In biomedical image analysis, chosen performance metrics often do not reflect the domain interest, and thus fail to adequately measure scientific progress and hinder translation of ML techniques into practice. To overcome this, we created Metrics Reloaded, a comprehensive framework guiding researchers in the problem-aware selection of metrics. Developed by a large international consortium in a multistage Delphi process, it is based on the novel concept of a problem fingerprint-a structured representation of the given problem that captures all aspects that are relevant for metric selection, from the domain interest to the properties of the target structure(s), dataset and algorithm output. On the basis of the problem fingerprint, users are guided through the process of choosing and applying appropriate validation metrics while being made aware of potential pitfalls. Metrics Reloaded targets image analysis problems that can be interpreted as classification tasks at image, object or pixel level, namely image-level classification, object detection, semantic segmentation and instance segmentation tasks. To improve the user experience, we implemented the framework in the Metrics Reloaded online tool. Following the convergence of ML methodology across application domains, Metrics Reloaded fosters the convergence of validation methodology. Its applicability is demonstrated for various biomedical use cases., (© 2024. Springer Nature America, Inc.)

Background and Objective: In order to be context-aware, computer-assisted surgical systems require accurate, real-time automatic surgical workflow recognition. In the past several years, surgical video has been the most commonly-used modality for surgical workflow recognition. But with the democratization of robot-assisted surgery, new modalities, such as kinematics, are now accessible. Some previous methods use these new modalities as input for their models, but their added value has rarely been studied. This paper presents the design and results of the "PEg TRAnsfer Workflow recognition" (PETRAW) challenge with the objective of developing surgical workflow recognition methods based on one or more modalities and studying their added value., Methods: The PETRAW challenge included a data set of 150 peg transfer sequences performed on a virtual simulator. This data set included videos, kinematic data, semantic segmentation data, and annotations, which described the workflow at three levels of granularity: phase, step, and activity. Five tasks were proposed to the participants: three were related to the recognition at all granularities simultaneously using a single modality, and two addressed the recognition using multiple modalities. The mean application-dependent balanced accuracy (AD-Accuracy) was used as an evaluation metric to take into account class balance and is more clinically relevant than a frame-by-frame score., Results: Seven teams participated in at least one task with four participating in every task. The best results were obtained by combining video and kinematic data (AD-Accuracy of between 93% and 90% for the four teams that participated in all tasks)., Conclusion: The improvement of surgical workflow recognition methods using multiple modalities compared with unimodal methods was significant for all teams. However, the longer execution time required for video/kinematic-based methods(compared to only kinematic-based methods) must be considered. Indeed, one must ask if it is wise to increase computing time by 2000 to 20,000% only to increase accuracy by 3%. The PETRAW data set is publicly available at www.synapse.org/PETRAW to encourage further research in surgical workflow recognition., Competing Interests: Declaration of Competing Interest The authors declare that they have no conflict of interest., (Copyright © 2023. Published by Elsevier B.V.)

Purpose: Deep brain stimulation (DBS) is a common treatment for a variety of neurological disorders which involves the precise placement of electrodes at particular subcortical locations such as the subthalamic nucleus. This placement is often guided by auditory analysis of micro-electrode recordings (MERs) which informs the clinical team as to the anatomic region in which the electrode is currently positioned. Recent automation attempts have lacked flexibility in terms of the amount of signal recorded, not allowing them to collect more signal when higher certainty is needed or less when the anatomy is unambiguous., Methods: We have addressed this problem by evaluating a simple algorithm that allows for MER signal collection to terminate once the underlying model has sufficient confidence. We have parameterized this approach and explored its performance using three underlying models composed of one neural network and two Bayesian extensions of said network., Results: We have shown that one particular configuration, a Bayesian model of the underlying network's certainty, outperforms the others and is relatively insensitive to parameterization. Further investigation shows that this model also allows for signals to be classified earlier without increasing the error rate., Conclusion: We have presented a simple algorithm that records the confidence of an underlying neural network, thus allowing for MER data collection to be terminated early when sufficient confidence is reached. This has the potential to improve the efficiency of DBS electrode implantation by reducing the time required to identify anatomical structures using MERs., (© 2021. CARS.)

PURPOSE : Evaluating the quality of surgical procedures is a major concern in minimally invasive surgeries. We propose a bottom-up approach based on the study of Sleeve Gastrectomy procedures, for which we analyze what we assume to be an important indicator of the surgical expertise: the exposure of the surgical scene. We first aim at predicting this indicator with features extracted from the laparoscopic video feed, and second to analyze how the extracted features describing the surgical practice influence this indicator. METHOD : Twenty-nine patients underwent Sleeve Gastrectomy performed by two confirmed surgeons in a monocentric study. Features were extracted from spatial and procedural annotations of the videos, and an expert surgeon evaluated the quality of the surgical exposure at specific instants. The features were used as input of a classifier (linear discriminant analysis followed by a support vector machine) to predict the expertise indicator. Features selected in different configurations of the algorithm were compared to understand their relationships with the surgical exposure and the surgeon's practice. RESULTS : The optimized algorithm giving the best performance used spatial features as input ([Formula: see text]). It also predicted equally the two classes of the indicator, despite their strong imbalance. Analyzing the selection of input features in the algorithm allowed a comparison of different configurations of the algorithm and showed a link between the surgical exposure and the surgeon's practice. CONCLUSION : This preliminary study validates that a prediction of the surgical exposure from spatial features is possible. The analysis of the clusters of feature selected by the algorithm also shows encouraging results and potential clinical interpretations.

Objective: Conventional surgical assistance and skill analysis for suturing mostly focus on the motions of the tools. As the quality of the suturing is determined by needle motions relative to the tissues, having knowledge of the needle motion would be useful for surgical assistance and skill analysis. As the first step toward demonstrating the usefulness of the knowledge of the needle motion, we developed a needle detection algorithm., Methods: Owing to the small needle size, attaching sensors to it is difficult. Therefore, we developed a real-time video-based needle detection algorithm using a region-based convolutional neural network., Results: Our method successfully detected the needle with an average precision of 89.2%. The needle was robustly detected even when the needle was heavily occluded by the tools and/or the blood vessels during microvascular anastomosis. However, there were some incorrect detections, including partial detection., Conclusion: To the best of our knowledge, this is the first time deep neural networks have been applied to real-time needle detection. In the future, we will develop a needle pose estimation algorithm using the predicted needle location toward computer-aided surgical assistance and surgical skill analysis.

Purpose: Lack of annotated training data hinders automatic recognition and prediction of surgical activities necessary for situation-aware operating rooms. We propose using knowledge transfer to compensate for data deficit and improve prediction., Methods: We used two approaches to extract and transfer surgical process knowledge. First, we encoded semantic information about surgical terms using word embedding. Secondly, we passed knowledge between different clinical datasets of neurosurgical procedures using transfer learning., Results: The combination of two methods provided 22% improvement of activity prediction. We also made several pertinent observations about surgical practices based on the results of the performed transfer., Conclusion: Word embedding boosts learning process. Transfer learning was shown to be more effective than a simple combination of data, especially for less similar procedures.

Purpose: With the intention of extending the perception and action of surgical staff inside the operating room, the medical community has expressed a growing interest towards context-aware systems. Requiring an accurate identification of the surgical workflow, such systems make use of data from a diverse set of available sensors. In this paper, we propose a fully data-driven and real-time method for segmentation and recognition of surgical phases using a combination of video data and instrument usage signals, exploiting no prior knowledge. We also introduce new validation metrics for assessment of workflow detection., Methods: The segmentation and recognition are based on a four-stage process. Firstly, during the learning time, a Surgical Process Model is automatically constructed from data annotations to guide the following process. Secondly, data samples are described using a combination of low-level visual cues and instrument information. Then, in the third stage, these descriptions are employed to train a set of AdaBoost classifiers capable of distinguishing one surgical phase from others. Finally, AdaBoost responses are used as input to a Hidden semi-Markov Model in order to obtain a final decision., Results: On the MICCAI EndoVis challenge laparoscopic dataset we achieved a precision and a recall of 91 % in classification of 7 phases., Conclusion: Compared to the analysis based on one data type only, a combination of visual features and instrument signals allows better segmentation, reduction of the detection delay and discovery of the correct phase order.

Purpose: Surgical process models (SPMs) have recently been created for situation-aware computer-assisted systems in the operating room. One important challenge in this area is the automatic acquisition of SPMs. The purpose of this study is to present a new method for the automatic detection of low-level surgical tasks, that is, the sequence of activities in a surgical procedure, from microscope video images only. The level of granularity that we addressed in this work is symbolized by activities formalized by triplets  ., Methods: Using the results of our latest work on the recognition of surgical phases in cataract surgeries, and based on the hypothesis that most activities occur in one or two phases only, we created a light-weight ontology, formalized as a hierarchical decomposition into phases and activities. Information concerning the surgical tools, the areas where tools are used and three other visual cues were detected through an image-based approach and combined with the information of the current surgical phase within a knowledge-based recognition system. Knowing the surgical phases before the activity, recognition allows supervised classification to be adapted to the phase. Multiclass Support Vector Machines were chosen as a classification algorithm., Results: Using a dataset of 20 cataract surgeries, and identifying 25 possible pairs of activities, a frame-by-frame recognition rate of 64.5 % was achieved with the proposed system., Conclusions: The addition of human knowledge to traditional bottom-up approaches based on image analysis appears to be promising for low-level task detection. The results of this work could be used for the automatic indexation of post-operative videos.

In the creation of new computer-assisted intervention systems, Surgical Process Models (SPMs) are an emerging concept used for analyzing and assessing surgical interventions. SPMs represent Surgical Processes (SPs) which are formalized as symbolic structured descriptions of surgical interventions using a pre-defined level of granularity and a dedicated terminology. In this context, one major challenge is the creation of new metrics for the comparison and the evaluation of SPs. Thus, correlations between these metrics and pre-operative data are used to classify surgeries and highlight specific information on the surgery itself and on the surgeon, such as his/her level of expertise. In this paper, we explore the automatic classification of a set of SPs based on the Dynamic Time Warping (DTW) algorithm. DTW is used to compute a similarity measure between two SPs that focuses on the different types of activities performed during surgery and their sequencing, by minimizing time differences. Indeed, it turns out to be a complementary approach to the classical methods that only focus on differences in the time and the number of activities. Experiments were carried out on 24 lumbar disk herniation surgeries to discriminate the surgeons level of expertise according to a prior classification of SPs. Supervised and unsupervised classification experiments have shown that this approach was able to automatically identify groups of surgeons according to their level of expertise (senior and junior), and opens many perspectives for the creation of new metrics for comparing and evaluating surgeries., (Copyright © 2011. Published by Elsevier Inc.)

Unlabelled: Detecting perfusion interhemispheric asymmetry in neurologic nuclear medicine imaging is an interesting approach to epilepsy., Methods: This study compared 4 methods that detect interhemispheric asymmetries of brain perfusion in SPECT. The first (M1) was conventional side-by-side expert-based visual interpretation of SPECT. The second (M2) was visual interpretation assisted by an interhemispheric difference (IHD) volume. The last 2 were automatic methods: unsupervised analysis using volumes of interest (M3) and unsupervised analysis of the IHD volume (M4). Use of these methods to detect possible perfusion asymmetry was compared on 60 simulated SPECT datasets by controlling the presence and location of asymmetries. From the detection results, localization receiver operating characteristic curves were generated and areas under curves were estimated and compared. Finally, the methods were applied to analyze interictal SPECT datasets to localize the epileptogenic focus in temporal lobe epilepsies., Results: This study showed an improvement in asymmetry detection on SPECT images with the methods using IHD volume (M2 and M4), in comparison with the other methods (M1 and M3). However, the most useful method for analyzing clinical SPECT datasets appeared to be visual inspection assisted by the IHD volume, since the automatic method using the IHD volume was less specific., Conclusion: The use of quantitative methods can improve performance in detection of perfusion asymmetry over visual inspection alone.

Technetium-99m HMPAO and technetium-99m ECD single photon emission computed tomography (SPECT) imaging is commonly used to highlight brain regions with altered perfusion. It is particularly useful in the investigation of intractable partial epilepsy. However, SPECT suffers from poor spatial resolution that makes interpretation difficult. In this context, we propose an unsupervised voxel neighbourhood based method to assist the detection of significant functional inter-hemispheric asymmetries in brain SPECT, using anatomical information from MRI. For each MRI voxel, the anatomically homologous voxel in the contralateral hemisphere is identified. Both homologous voxel coordinates are then mapped into the SPECT volume using SPECT-MRI registration. Neighbourhoods are then defined around each SPECT voxel and compared to obtain a volume of inter-hemispheric differences. A volume including only the statistically significant inter-hemispheric differences is deduced from this volume using a non-parametric approach. The method was validated using realistic analytical simulated SPECT data including known asymmetries (in size and amplitude) as ground truth (gold standard). Detection performance was assessed using an ROC (receiver operating characteristic) approach based on the measures of the overlap between known and detected asymmetries. Validation with computer-simulated data demonstrates the ability to detect asymmetric zones with relatively small extension and amplitude. The registration of these detected functional asymmetries on the MRI enables good anatomical localization to be achieved.