Search

Objectives To assess a new application of artificial intelligence for real-time detection of laryngeal squamous cell carcinoma (LSCC) in both white light (WL) and narrow-band imaging (NBI) videolaryngoscopies based on the You-Only-Look-Once (YOLO) deep learning convolutional neural network (CNN). Study design Experimental study with retrospective data. Methods Recorded videos of LSCC were retrospectively collected from in-office transnasal videoendoscopies and intraoperative rigid endoscopies. LSCC videoframes were extracted for training, validation, and testing of various YOLO models. Different techniques were used to enhance the image analysis: contrast limited adaptive histogram equalization, data augmentation techniques, and test time augmentation (TTA). The best-performing model was used to assess the automatic detection of LSCC in six videolaryngoscopies. Results Two hundred and nineteen patients were retrospectively enrolled. A total of 624 LSCC videoframes were extracted. The YOLO models were trained after random distribution of images into a training set (82.6%), validation set (8.2%), and testing set (9.2%). Among the various models, the ensemble algorithm (YOLOv5s with YOLOv5m-TTA) achieved the best LSCC detection results, with performance metrics in par with the results reported by other state-of-the-art detection models: 0.66 Precision (positive predicted value), 0.62 Recall (sensitivity), and 0.63 mean Average Precision at 0.5 intersection over union. Tests on the six videolaryngoscopies demonstrated an average computation time per videoframe of 0.026 seconds. Three demonstration videos are provided. Conclusion This study identified a suitable CNN model for LSCC detection in WL and NBI videolaryngoscopies. Detection performances are highly promising. The limited complexity and quick computational times for LSCC detection make this model ideal for real-time processing. Level of evidence 3 Laryngoscope, 2021.

In the last decades, new technological devices and instruments have been developed to overcome the technical limits of transoral laser microsurgery. The recent introduction of 3D endoscopy seems to be a promising tool in the field of diagnostic and operative laryngology as an alternative to the traditional microlaryngoscopy. Our work aims to present a novel transoral microsurgical setting that expands the use of exoscopic systems (in this case the VITOM® 3D-HD) as an alternative to the standard operating microscope. A customized support arm and an adaptor to firmly connect the VITOM® 3D-HD camera to the laser micromanipulator were specially designed. This setup was used as an alternative to the standard operating microscope in a cohort of 17 patients affected by suspicious early to intermediate pharyngo-laryngeal neoplasms. A historical cohort of patients treated with the traditional setting and matching the same inclusion criteria was used as a reference for the duration of surgical procedures. The surgical procedures comprised 7 cordectomies, 2 endoscopic partial supraglottic laryngectomies, 4 tongue base resections, and 4 lateral oropharyngectomies or hypopharyngectomies. In 6 cases (35%), a simultaneous neck dissection was performed. The low rate of positive deep (6%) or superficial (12%) margins reinforced the safety of this platform, and the results obtained in terms of operating time were comparable to the control group (p > 0.05), which confirms the feasibility of the system. Our surgical setting setup is a convincing alternative to traditional transoral laser microsurgery for early to intermediate pharyngo-laryngeal neoplasms. The main advantages of this system are comfortable ergonomics for the first surgeon and a potential benefit in terms of teaching if applied in university hospitals, since the entire surgical team can view the same surgical 3D-HD view of the first operator. Further work is still needed to objectively compare the traditional and new technique, and to validate our preliminary clinical findings.

Human-induced pluripotent stem cells (hiPSCs) have provided new methods to study neurodegenerative diseases. In addition to their wide application in neuronal disorders, hiPSCs technology can also encompass specific conditions, such as inherited retinal dystrophies. The possibility of evaluating alterations related to retinal disorders in 3D organoids increases the truthfulness of in vitro models. Moreover, both Alzheimer's (AD) and Parkinson's disease (PD) have been described as causing early retinal alterations, generating beta-amyloid protein accumulation, or affecting dopaminergic amacrine cells. This review addresses recent advances and future perspectives obtained from in vitro modeling of retinal diseases, focusing on retinitis pigmentosa (RP). Additionally, we depicted the possibility of evaluating changes related to AD and PD in retinal organoids obtained from potential patients long before the onset of the disease, constituting a valuable tool in early diagnosis. With this, we pointed out prospects in the study of retinal dystrophies and early diagnosis of AD and PD.

In a recent advance in surgery, a computer-assisted laser microsurgery system has demonstrated its suitability for transoral operations. Thanks to its motorized micromanipulator setup, surgeons can perform delicate operations on lesions in the larynx using an intuitive user interface. The major hurdle to ensure surgical quality is to guarantee $\mu$ m-scale precision in the control of the laser beam (spot diameter 110–250 $\mu$ m), because the mechanism includes inherent discontinuous nonlinearities such as Coulomb friction and stiction and unavoidable modeling uncertainties. To this end, in this article, a precise position controller directly creating a voltage command is newly proposed by amalgamating two robust control concepts—a generalized super-twisting algorithm (GSTA) and a model-free compensation method called time-delay estimation (TDE). Fast convergence in finite time is proved in the Lyapunov sense. The experimental results verify that the proposed controller can satisfy the precision requirement of 55 $\mu$ m, while the efficacy is validated by comparisons with respective GSTA- and TDE-based controllers. As a practical performance–safety guideline for users, we benchmark the tracking precision of the proposed controller with respect to operation speeds.

IntroductionNarrow Band Imaging (NBI) is an endoscopic visualization technique useful for upper aero-digestive tract (UADT) cancer detection and margins evaluation. However, NBI analysis is strongly operator-dependent and requires high expertise, thus limiting its wider implementation. Recently, artificial intelligence (AI) has demonstrated potential for applications in UADT videoendoscopy. Among AI methods, deep learning algorithms, and especially convolutional neural networks (CNNs), are particularly suitable for delineating cancers on videoendoscopy. This study is aimed to develop a CNN for automatic semantic segmentation of UADT cancer on endoscopic images.Materials and MethodsA dataset of white light and NBI videoframes of laryngeal squamous cell carcinoma (LSCC) was collected and manually annotated. A novel DL segmentation model (SegMENT) was designed. SegMENT relies on DeepLabV3+ CNN architecture, modified using Xception as a backbone and incorporating ensemble features from other CNNs. The performance of SegMENT was compared to state-of-the-art CNNs (UNet, ResUNet, and DeepLabv3). SegMENT was then validated on two external datasets of NBI images of oropharyngeal (OPSCC) and oral cavity SCC (OSCC) obtained from a previously published study. The impact of in-domain transfer learning through an ensemble technique was evaluated on the external datasets.Results219 LSCC patients were retrospectively included in the study. A total of 683 videoframes composed the LSCC dataset, while the external validation cohorts of OPSCC and OCSCC contained 116 and 102 images. On the LSCC dataset, SegMENT outperformed the other DL models, obtaining the following median values: 0.68 intersection over union (IoU), 0.81 dice similarity coefficient (DSC), 0.95 recall, 0.78 precision, 0.97 accuracy. For the OCSCC and OPSCC datasets, results were superior compared to previously published data: the median performance metrics were, respectively, improved as follows: DSC=10.3% and 11.9%, recall=15.0% and 5.1%, precision=17.0% and 14.7%, accuracy=4.1% and 10.3%.ConclusionSegMENT achieved promising performances, showing that automatic tumor segmentation in endoscopic images is feasible even within the highly heterogeneous and complex UADT environment. SegMENT outperformed the previously published results on the external validation cohorts. The model demonstrated potential for improved detection of early tumors, more precise biopsies, and better selection of resection margins.

Biomedical robotic systems continue to hold unlimited potential for surgical procedures. Robotized laser endoscopic tools provide surgeons with increased accuracy in the laser ablation of tissue and tumors. The research here catalogs the design and implementation of a new laser endoscopic tool for tissue ablation. A novel feature of this new device is the inclusion of a feedback loop that measures the position of the laser beam via a photodetector sensor. The scale of this new device was governed by the dimensions of the photodetector sensor. The tip of the laser's fiber optic cable is controlled by the torque interaction between permanent magnet rings surrounding the fiber-optic and the custom-designed solenoid coils. Prior to building the physical test-bed, the system was modeled and simulated using comsol software. In preclinical trials, the physical experimental results showed that the designed prototype laser scanner system accurately tracks different ablation patterns and gives a consistent output position for the laser beam; however, the heat diffusion into the tissue around the desired line of the geometric shape would give wider ablation margins than was desirable.

Optical Coherence Tomography (OCT) is an optical imaging technology occupying a unique position in the resolution vs. imaging depth spectrum. It is already well established in the field of ophthalmology, and its application in other fields of medicine is growing. This is motivated by the fact that OCT is a real-time sensing technology with high sensitivity to precancerous lesions in epithelial tissues, which can be exploited to provide valuable information to clinicians. In the prospective case of OCT-guided endoscopic laser surgery, these real-time data will be used to assist surgeons in challenging endoscopic procedures in which high-power lasers are used to eradicate diseases. The combination of OCT and laser is expected to enhance the detection of tumors, the identification of tumor margins, and ensure total disease eradication while avoiding damage to healthy tissue and critical anatomical structures. Therefore, OCT-guided endoscopic laser surgery is an important nascent research area. This paper aims to contribute to this field with a comprehensive review of state-of-the-art technologies that may be exploited as the building blocks for achieving such a system. The paper begins with a review of the principles and technical details of endoscopic OCT, highlighting challenges and proposed solutions. Then, once the state of the art of the base imaging technology is outlined, the new OCT-guided endoscopic laser surgery frontier is reviewed. Finally, the paper concludes with a discussion on the constraints, benefits and open challenges associated with this new type of surgical technology.

International audience; Laser microsurgery is the current gold standard surgical technique for the treatment of selected diseases in delicate organs such as the larynx. However, the operations require large surgical expertise and dexterity, and face significant limitations imposed by available technology, such as the requirement for direct line of sight to the surgical field, restricted access, and direct manual control of the surgical instruments. To change this status quo, the European project μRALP pioneered research towards a complete redesign of current laser microsurgery systems, focusing on the development of robotic micro-technologies to enable endoscopic operations. This has fostered awareness and interest in this field, which presents a unique set of needs, requirements and constraints, leading to research and technological developments beyond μRALP and its research consortium. This paper reviews the achievements and key contributions of such research, providing an overview of the current state of the art in robot-assisted endoscopic laser microsurgery. The primary target application considered is phonomicrosurgery, which is a representative use case involving highly challenging microsurgical techniques for the treatment of glottic diseases. The paper starts by presenting the motivations and rationale for endoscopic laser microsurgery, which leads to the introduction of robotics as an enabling technology for improved surgical field accessibility, visualization and management. Then, research goals, achievements, and current state of different technologies that can build-up to an effective robotic system for endoscopic laser microsurgery are presented. This includes research in micro-robotic laser steering, flexible robotic endoscopes, augmented imaging, assistive surgeon-robot interfaces, and cognitive surgical systems. Innovations in each of these areas are shown to provide sizable progress towards more precise, safer and higher quality endoscopic laser microsurgeries. Yet, major impact is really expected from the full integration of such individual contributions into a complete clinical surgical robotic system, as illustrated in the end of this paper with a description of preliminary cadaver trials conducted with the integrated μRALP system. Overall, the contribution of this paper lays in outlining the current state of the art and open challenges in the area of robot-assisted endoscopic laser microsurgery, which has important clinical applications even beyond laryngology.

Peripheral intravenous catheterization (PIVC) is pervasively needed in hospitals. However, given the levels of precision and controllability needed for PIVC, this operation suffers from very low success rates. For young patients, about half of the first insertions fail. Robotic systems have great potential to effectively assist the operation and improve the success rates, which has led to the recent development of different robots to automate PIVC. These robots are equipped with various sensors and actuators, resulting in expensive, complex, and grounded machines. Yet, fully automating the operation is neither needed nor desired, as current clinical preference is oriented toward keeping the practitioner involved and in control of the operation. Therefore, in this study we proposed an innovative smart hand-held robotic device, named CathBot, that enhances intra-operative control during PIVC with automatic features that guarantee very high success rates. It exploits an electrical impedance sensor to detect the venipuncture and a crank-slider mechanism to automate the subsequent cannula advancement and needle retraction. Here, CathBot is first characterized through engineering experiments that demonstrate its capability to successfully perform the whole PIVC operation on a realistic baby arm phantom without human involvement. Subsequent experiments evaluate the device with naive subjects on the same realistic pediatric PIVC scenario. The results demonstrate that CathBot can significantly improve the PIVC performance. Naive subjects achieved an average 86% success rate, and 80% of the subjects succeeded in their first attempt. These results demonstrate the technology has potential to greatly improve both the clinician's and the patient's PIVC experience.

Background The inability of users to directly and intuitively control their state-of-the-art commercial prosthesis contributes to a low device acceptance rate. Since Electromyography (EMG)-based control has the potential to address those inabilities, research has flourished on investigating its incorporation in microprocessor-controlled lower limb prostheses (MLLPs). However, despite the proposed benefits of doing so, there is no clear explanation regarding the absence of a commercial product, in contrast to their upper limb counterparts. Objective and methodologies This manuscript aims to provide a comparative overview of EMG-driven control methods for MLLPs, to identify their prospects and limitations, and to formulate suggestions on future research and development. This is done by systematically reviewing academical studies on EMG MLLPs. In particular, this review is structured by considering four major topics: (1) type of neuro-control, which discusses methods that allow the nervous system to control prosthetic devices through the muscles; (2) type of EMG-driven controllers, which defines the different classes of EMG controllers proposed in the literature; (3) type of neural input and processing, which describes how EMG-driven controllers are implemented; (4) type of performance assessment, which reports the performance of the current state of the art controllers. Results and conclusions The obtained results show that the lack of quantitative and standardized measures hinders the possibility to analytically compare the performances of different EMG-driven controllers. In relation to this issue, the real efficacy of EMG-driven controllers for MLLPs have yet to be validated. Nevertheless, in anticipation of the development of a standardized approach for validating EMG MLLPs, the literature suggests that combining multiple neuro-controller types has the potential to develop a more seamless and reliable EMG-driven control. This solution has the promise to retain the high performance of the currently employed non-EMG-driven controllers for rhythmic activities such as walking, whilst improving the performance of volitional activities such as task switching or non-repetitive movements. Although EMG-driven controllers suffer from many drawbacks, such as high sensitivity to noise, recent progress in invasive neural interfaces for prosthetic control (bionics) will allow to build a more reliable connection between the user and the MLLPs. Therefore, advancements in powered MLLPs with integrated EMG-driven control have the potential to strongly reduce the effects of psychosomatic conditions and musculoskeletal degenerative pathologies that are currently affecting lower limb amputees.

Intuitive interaction is the cornerstone of accurate and effective performance in remote robotic teleoperation. It requires high-fidelity in control actions as well as perception (vision, haptic, and other sensory feedback) of the remote environment. This paper presentsVicarios, a Virtual Reality (VR) based interface with the aim of facilitating intuitive real-time remote teleoperation, while utilizing the inherent benefits of VR, including immersive visualization, freedom of user viewpoint selection, and fluidity of interaction through natural action interfaces.Vicariosaims to enhance the situational awareness, using the concept ofviewpoint-independent mappingbetween the operator and the remote scene, thereby giving the operator better control in the perception-action loop. The article describes the overall system ofVicarios, with its software, hardware, and communication framework. A comparative user study quantifies the impact of the interface and its features, including immersion and instantaneous user viewpoint changes, termed “teleporting”, on users’ performance. The results show that users’ performance with the VR-based interface was either similar to or better than the baseline condition of traditional stereo video feedback, approving the realistic nature of theVicariosinterface. Furthermore, including the teleporting feature in VR significantly improved participants’ performance and their appreciation for it, which was evident in the post-questionnaire results.Vicarioscapitalizes on the intuitiveness and flexibility of VR to improve accuracy in remote teleoperation.

When planning an operation, surgeons usually rely on traditional 2D imaging. Moreover, colon neoplastic lesions are not always easy to locate macroscopically, even during surgery. A 3D virtual model may allow surgeons to localize lesions with more precision and to better visualize the anatomy. In this study, we primary analyzed and discussed the clinical impact of using such 3D models in colorectal surgery. This is a monocentric prospective observational pilot study that includes 14 consecutive patients who presented colorectal lesions with indication for surgical therapy. A staging computed tomography (CT)/magnetic resonance imaging (MRI) scan and a colonoscopy were performed on each patient. The information gained from them was provided to obtain a 3D rendering. The 2D images were shown to the surgeon performing the operation, while the 3D reconstructions were shown to a second surgeon. Both of them had to locate the lesion and describe which procedure they would have performed; we then compared their answers with one another and with the intraoperative and histopathological findings. The lesion localizations based on the 3D models were accurate in 100% of cases, in contrast to conventional 2D CT scans, which could not detect the lesion in two patients (in these cases, lesion localization was based on colonoscopy). The 3D model reconstruction allowed an excellent concordance correlation between the estimated and the actual location of the lesion, allowing the surgeon to correctly plan the procedure with excellent results. Larger clinical studies are certainly required.

Objective: Current limitations in Electromyography (EMG)-driven Neuromusculoskeletal (NMS) modeling for control of wearable robotics are the requirement of both Motion Capture for both an indoor system and numerous EMG electrodes. These limitations make the technology unsuitable for amputees with only proximal muscles, who need optimal prosthetic device control during everyday activities. Therefore, we developed a novel Machine Learning (ML)driven NMS model able to predict lower limb joint torque only from wearable sensors than can be embedded in a prosthetic device. Methods: After the NMS model calibration of a single healthy subject (OpenSim® software and Calibrated EMGInformed Neuromusculoskeletal Modelling CEINMS Toolbox), an additional ML layer (Gaussian Mixture Regressors) was added to the model to replace the MoCap-derived dependent variables with estimations obtained only from wearable sensors. An on-line open-loop Forward Dynamic (FD) simulation of the knee joint is computed and torque trajectories are compared to experimental ones. Results: Estimations of the novel ML-driven Musculoskeletal model were comparable with experimental knee joint torque during typical locomotion tasks. Accuracy results were comparable to standard EMG-driven MS models and errors are below the threshold of Normalized Root Mean Square Deviation $\le 0.30$ recognized in literature. Conclusions: We developed the first concept of completely wearable and subject-specific EMG-driven NMS model control for lower limb prostheses. The possibility to use this NMS model for FD simulations and the estimation of torque reference control avoids the use of current heuristic and overly complex standard controllers for lower limb prostheses. This research, in fact, represents a key step for the definition of a novel human-machine interface able to create a seamless interconnection between human native control and future wearable robotics.

Background and Objectives During Twin-to-Twin Transfusion Syndrome (TTTS), abnormal vascular anastomoses in the monochorionic placenta can produce uneven blood flow between the fetuses. In the current practice, this syndrome is surgically treated by closing the abnormal connections using laser ablation. Surgeons commonly use the inter-fetal membrane as a reference. Limited field of view, low fetoscopic image quality and high inter-subject variability make the membrane identification a challenging task. However, currently available tools are not optimal for automatic membrane segmentation in fetoscopic videos, due to membrane texture homogeneity and high illumination variability. Methods To tackle these challenges, we present a new deep-learning framework for inter-fetal membrane segmentation on in-vivo fetoscopic videos. The framework enhances existing architectures by (i) encoding a novel (instance-normalized) dense block, invariant to illumination changes, that extracts spatio-temporal features to enforce pixel connectivity in time, and (ii) relying on an adversarial training, which constrains macro appearance. Results We performed a comprehensive validation using 20 different videos (2000 frames) from 20 different surgeries, achieving a mean Dice Similarity Coefficient of 0.8780 ± 0.1383 . Conclusions The proposed framework has great potential to positively impact the actual surgical practice for TTTS treatment, allowing the implementation of surgical guidance systems that can enhance context awareness and potentially lower the duration of the surgeries.

Background Unsafe surgical care has emerged as a significant public health concern, motivated by a high percentage of major complications happening during surgery, attributed to surgeons' skills and experience, and determined to be preventable. Methods This article presents APSurg, an Abdominal Positioning Surgical system designed to improve awareness and safety during laparoscopic surgery. The proposed system behaves like a GPS, offering an additional dynamic virtual reality view of the surgical field. Results This work presents an evaluation study in terms of accuracy, effectiveness, and usability. Tests were conducted performing a localization task on an abdomen phantom in a simulated scenario. Results show a navigation accuracy below 5 mm. The task execution time was reduced by a 15% and the performed incision dimension was reduced by a 46%, with respect to a standard setup. A custom questionnaire showed a significant positive impact in exploiting APSurg during the surgical task execution.

This work presents an implementation of Error-related Potential (ErrP) detection to produce progressive adaptation of a motor imagery task classifier. The main contribution is in the evaluation of the effect of vibrotactile feedback on both ErrP and motor imagery detection. Results confirm the potential of self-adaptive techniques to improve motor imagery classification, and support the design of vibratory and in general tactile feedback into Brain-Computer Interfaces to improve both static and adaptive performance.

Patients with Locked-In Syndrome such as people with Amyotrophic Lateral Sclerosis (ALS) rely on technology for basic communication. However, available Augmentative and Alternative Communication (AAC) tools such as gaze-controlled keyboards have limited abilities. In particular, they do not allow for expression of emotions in addition to words. In this paper we propose a novel gaze-based speaking tool that enable locked-in syndrome patients to express emotions as well as sentences. It also features patient-controlled emotionally modulated speech synthesis. Additionally, an emotional 3D avatar can be controlled by the patient to represent emotional facial-expressions. The systems were tested with 36 people without disabilities separated into an affective group - full control of emotional voice, avatar facial expressions and laugh - and a control group - no emotional tools. The study proved the system’s capacity to enhance communication for both the patient and the interlocutor. The emotions embedded in the synthesized voices were found recognizable at 80% on the first trial and 90% on the second trial. The conversation was perceived as more natural when using the affective tool. The subjects felt it was easier to express and identify emotions using this system. The emotional voice and the emotional avatar were found to help the conversation. This highlights the needs for more affective-driven communicative solutions for locked-in patients.

This paper presents the integration of electrical bio-impedance sensing technology into a bipolar surgical forceps for soft tissue identification during a robotic assisted procedure. The EBI sensing is done by pressing the forceps on the target tissue with a controlled pressing depth and a controlled jaw opening distance. The impact of these 2 parameters are characterized by finite element simulation. Subsequently, an experiment is conducted with 4 types of ex-vivo tissues including liver, kidney, lung and muscle. The experimental results demonstrate that the proposed EBI sensing method can identify these 4 tissue types with an accuracy higher than 92.82%.

Microsurgical procedures, such as petroclival meningioma resection, require careful surgical actions in order to remove tumor tissue, while avoiding brain and vessel damaging. Such procedures are currently performed under microscope magnification. Robotic tools are emerging in order to filter surgeons' unintended movements and prevent tools from entering forbidden regions such as vascular structures. The present work investigates the use of a handheld robotic tool (Micron) to automate vessel avoidance in microsurgery. In particular, we focused on vessel segmentation, implementing a deep-learning-based segmentation strategy in microscopy images, and its integration with a feature-based passive 3D reconstruction algorithm to obtain accurate and robust vessel position. We then implemented a virtual-fixture-based strategy to control the handheld robotic tool and perform vessel avoidance. Clay vascular phantoms, lying on a background obtained from microscopy images recorded during petroclival meningioma surgery, were used for testing the segmentation and control algorithms. When testing the segmentation algorithm on 100 different phantom images, a median Dice similarity coefficient equal to 0.96 was achieved. A set of 25 Micron trials of 80 s in duration, each involving the interaction of Micron with a different vascular phantom, were recorded, with a safety distance equal to 2 mm, which was comparable to the median vessel diameter. Micron's tip entered the forbidden region 24% of the time when the control algorithm was active. However, the median penetration depth was 16.9 μm, which was two orders of magnitude lower than median vessel diameter. Results suggest the system can assist surgeons in performing safe vessel avoidance during neurosurgical procedures.

Fast and accurate graft hepatic steatosis (HS) assessment is of primary importance for lowering liver dysfunction risks after transplantation. Histopathological analysis of biopsied liver is the gold standard for assessing HS, despite being invasive and time consuming. Due to the short time availability between liver procurement and transplantation, surgeons perform HS assessment through clinical evaluation (medical history, blood tests) and liver texture visual analysis. Despite visual analysis being recognized as challenging in the clinical literature, few efforts have been invested to develop computer-assisted solutions for HS assessment. The objective of this paper is to investigate the automatic analysis of liver texture with machine learning algorithms to automate the HS assessment process and offer support for the surgeon decision process. Forty RGB images of forty different donors were analyzed. The images were captured with an RGB smartphone camera in the operating room (OR). Twenty images refer to livers that were accepted and 20 to discarded livers. Fifteen randomly selected liver patches were extracted from each image. Patch size was $$100\times 100$$ . This way, a balanced dataset of 600 patches was obtained. Intensity-based features (INT), histogram of local binary pattern ( $$H_{{\mathrm{LBP}}_{riu2}}$$ ), and gray-level co-occurrence matrix ( $$F_{\mathrm{GLCM}}$$ ) were investigated. Blood-sample features (Blo) were included in the analysis, too. Supervised and semisupervised learning approaches were investigated for feature classification. The leave-one-patient-out cross-validation was performed to estimate the classification performance. With the best-performing feature set ( $$H_{{\mathrm{LBP}}_{riu2}}+\hbox {INT}+\hbox {Blo}$$ ) and semisupervised learning, the achieved classification sensitivity, specificity, and accuracy were 95, 81, and 88%, respectively. This research represents the first attempt to use machine learning and automatic texture analysis of RGB images from ubiquitous smartphone cameras for the task of graft HS assessment. The results suggest that is a promising strategy to develop a fully automatic solution to assist surgeons in HS assessment inside the OR.

Background Blood vessel segmentation is a topic of high interest in medical image analysis since the analysis of vessels is crucial for diagnosis, treatment planning and execution, and evaluation of clinical outcomes in different fields, including laryngology, neurosurgery and ophthalmology. Automatic or semi-automatic vessel segmentation can support clinicians in performing these tasks. Different medical imaging techniques are currently used in clinical practice and an appropriate choice of the segmentation algorithm is mandatory to deal with the adopted imaging technique characteristics (e.g. resolution, noise and vessel contrast). Objective This paper aims at reviewing the most recent and innovative blood vessel segmentation algorithms. Among the algorithms and approaches considered, we deeply investigated the most novel blood vessel segmentation including machine learning, deformable model, and tracking-based approaches. Methods This paper analyzes more than 100 articles focused on blood vessel segmentation methods. For each analyzed approach, summary tables are presented reporting imaging technique used, anatomical region and performance measures employed. Benefits and disadvantages of each method are highlighted. Discussion Despite the constant progress and efforts addressed in the field, several issues still need to be overcome. A relevant limitation consists in the segmentation of pathological vessels. Unfortunately, not consistent research effort has been addressed to this issue yet. Research is needed since some of the main assumptions made for healthy vessels (such as linearity and circular cross-section) do not hold in pathological tissues, which on the other hand require new vessel model formulations. Moreover, image intensity drops, noise and low contrast still represent an important obstacle for the achievement of a high-quality enhancement. This is particularly true for optical imaging, where the image quality is usually lower in terms of noise and contrast with respect to magnetic resonance and computer tomography angiography. Conclusion No single segmentation approach is suitable for all the different anatomical region or imaging modalities, thus the primary goal of this review was to provide an up to date source of information about the state of the art of the vessel segmentation algorithms so that the most suitable methods can be chosen according to the specific task.

Peripheral intravenous catheterization (PIVC) is frequently required for various medical treatments. Over 1 billion PIVC operations are performed per year in the United States alone. However, this operation is characterized by a very low success rate, especially amongst pediatric patients. Statistics show that only 53% of first PIVC attempts are successful in pediatric patients. Since their veins are small and readily rupture, multiple attempts are commonly required before successfully inserting the catheter into the vein. This article presents and evaluates a novel venous entry detection method based on measuring the electrical bio-impedance of the contacting tissue at the tip of a concentric electrode needle (CEN). This detection method is then implemented in the design of a clinical device called smart venous entry indicator (SVEI), which lights up a LED to indicate the venous entry when the measured value is within the range of blood. To verify this detection method, two experiments are conducted. In the first experiment, we measured the bio-impedance during the insertion of a CEN into a rat's tail vein with different excitation frequencies. Then three classifiers are tested to discriminate blood from surrounding tissues. The experimental results indicate that with 100 kHz excitation frequency the blood bio-impedance can be identified with accuracy nearly 100%, demonstrating the feasibility and reliability of the proposed method for venous entry detection. The second experiment aims to assess the impact of SVEI on PIVC performance. Ten naive subjects were invited to catheterize a realistic baby arm phantom. The subjects are equally divided into two groups, where one group does PIVC with SVEI and the other group uses an ordinary IV catheter. The results show that subjects using SVEI can achieve much higher success rates (86%) than those performing PIVC in a conventional way (12%). Also, all subjects assisted by SVEI succeeded in their first trials while no one succeed in their first attempt using the conventional unassisted system. These results demonstrate the proposed detection method has great potential to improve pediatric PIVC performance, especially for non-expert clinicians. This supports further investment towards clinical validation of the technology.

Despite the benefits introduced by robotic systems in abdominal Minimally Invasive Surgery (MIS), major complications can still affect the outcome of the procedure, such as intra-operative bleeding. One of the causes is attributed to accidental damages to arteries or veins by the surgical tools, and some of the possible risk factors are related to the lack of sub-surface visibilty. Assistive tools guiding the surgical gestures to prevent these kind of injuries would represent a relevant step towards safer clinical procedures. However, it is still challenging to develop computer vision systems able to fulfill the main requirements: (i) long term robustness, (ii) adaptation to environment/object variation and (iii) real time processing. The purpose of this paper is to develop computer vision algorithms to robustly track soft tissue areas (Safety Area, SA), defined intra-operatively by the surgeon based on the real-time endoscopic images, or registered from a pre-operative surgical plan. We propose a framework to combine an optical flow algorithm with a tracking-by-detection approach in order to be robust against failures caused by: (i) partial occlusion, (ii) total occlusion, (iii) SA out of the field of view, (iv) deformation, (v) illumination changes, (vi) abrupt camera motion, (vii), blur and (viii) smoke. A Bayesian inference-based approach is used to detect the failure of the tracker, based on online context information. A Model Update Strategy (MUpS) is also proposed to improve the SA re-detection after failures, taking into account the changes of appearance of the SA model due to contact with instruments or image noise. The performance of the algorithm was assessed on two datasets, representing ex-vivo organs and in-vivo surgical scenarios. Results show that the proposed framework, enhanced with MUpS, is capable of maintain high tracking performance for extended periods of time ( ≃ 4 min - containing the aforementioned events) with high precision (0.7) and recall (0.8) values, and with a recovery time after a failure between 1 and 8 frames in the worst case.

Robotic operations in the real world rely on bearing-only sensors, e.g., optical, acoustic, etc., to sense physical phenomena. Bearing-only sensors are limited because they do not provide range information. To overcome this limitation in estimating source locations, the most common solutions involve making multiple measurements from different locations, either through multiple sensors in the field or a single moving sensor, and then applying triangulation or filtering. In unknown environments with single motion-capable sensors (e.g., mobile robots with on-board sensors), planned motion of the sensor can allow accurate and efficient source position estimation. This paper presents a novel approach in estimating the locations of stationary sources, using a motion-capable sensor. The proposed method combines the concepts of Extended Kalman Filter (EKF) and Mutual Information (MI) from information theory to estimate the Next Best View (NBV) pose to which the sensor should be moved. A utility function, that accounts for the movement cost, the characteristics of the sensor, and the MI and EKF information, facilitates efficient estimation. The proposed algorithm has been implemented in the realworld for Sound-source Position Estimation (SPE), using an acoustic sensor mounted at the end of a 6 degrees-of-freedom (DOF) robotic manipulator. The algorithm, termed as NBV-SPE, proves its utility and performance through preliminary indoor and outdoor experiments for sound sources in 3D space.

Patient outcomes in the medical field are improving through the use and incorporation of robotics technology and laser physics, e.g., the use of optical fibers and lasers in micro-surgery. This paper describes the design and implementation of a new optical fiber laser micro-surgery system, one that is the first to use closed-loop feedback control. In this computerassisted laser scanning tool the laser beam is controlled by four magnetic actuators. After attaching permanent magnets to the free end of the optical fiber, it is the control of these four magnets that produces an accurate laser scanning system, one suited to micro-surgery applications. The interaction between the electromagnetic fields generated by the external magnetic coils and the flux of the internal permanent magnets, produces the control torques required to produce the desired movement of the optic fiber. The tracking error of the optic fiber is compensated for by using a photo-detector sensor as the feedback transducer in the control system. The magnetic torque bends the optical fiber and the feedback from the photodetector gives automated control and high-speed laser scanning of the fiber tip. The simulation and the experimental results are accurate and are co-related., Since some of the experimental results from the paper should be modified, one of the authors asked me to withdrawal the paper until we modify the paper with new results

Endoscopic laser surgery is a minimally invasive procedure in which a fiber laser tool is used to perform precise incisions in soft tissue. Although the precision of such incisions depends on the proper focusing of the laser, endoscopic laser tools use no optics at all, due to the limited space in the endoscopic system. Instead, they rely on placing the tip of the fiber in direct contact with the tissue, which often leads to tissue carbonization. To solve this problem, we developed a compact auto-focusing system based on a MEMS varifocal mirror. The proposed system is able to ensure the focusing of the laser by controlling the deflection of the varifocal mirror using hydraulic actuation. Validation experiments showed that the system is able to keep the variation of the laser spot diameter under 3% for a distance range between 12.15 and 52.15 mm.

This paper introduces a new generation of the motorized laser micromanipulator: The computer-assisted laser microsurgery system (CALM). The system has a motorized laser micromanipulator and a graphics tablet for intuitive laser position control. The CALM system is proposed to be used in Transoral Laser Microsurgery (TLM) for accurate resections at the surgical site. This paper focuses on the technical design and control of the new motorized micromanipulator, based on a 2 degrees-of-freedom spherical orienting mechanism to actuate the beam-splitter mirror. The design takes into account the constraints of the surgical setup and the mechanical space requirements in TLM. The CALM prototype has been validated through preliminary comparative user trials demonstrating that the laser positioning accuracy improves by 60% over the traditional manual system. CALM provides an intuitive mechanism and interface for improved surgical accuracy and surgical outcomes.

Laser tattoo removal is a standard non-invasive method for removing color pigments on the skin. Increasing number of tattooed people who want to remove their tattoo has driven the medical laser market to develop new technologies for painless, scar-free and complete tattoo removal. However, manual use of such laser systems creates post-operative complications since they do not guarantee (i) protection on non-tattooed skin from laser exposure, nor (ii) precise control of the laser focus during the operations for best performance. This paper introduces deTattoo, a robotic system to improve tattoo removal operations. A robotic arm is equipped with a RGB-D camera and a visible laser, in eye-in-end configuration. A hybrid visual servoing control is proposed to guarantee the correct pose of the laser with respect to the tattooed tissue while compensating body motions. 2D features tracked with a mass-spring-damper deformable mesh model are combined with the 3D reconstruction retrieved from a RGB-D camera in order to build the control law. Several experiments were conducted to evaluate the performance of the system with a fixed or moving tattooed surface, at different inclinations. Results showed that the proposed framework is able to fulfil the laser-based tattoo removal requirements, providing high positioning accuracy $(\lt 1 mm)$ orientation $(\lt 0. 2^{\circ})$ and body motion compensation.

Minimally invasive surgical (MIS) applications are among the most challenging scenarios where an intelligent and autonomous surgical robotics system (SRS) could operate. Current human-operated SRS integrates mainly endoscopic video feedback, but more advanced sensing modalities are required to support the execution of complex surgical actions, especially by an autonomous system. In this work, we present the application of an additional sensing modality based on a compact electric bio-impedance (EBI) measurement device that can be integrated into existing surgical instruments with minimum modifications. Reliable soft tissue EBI measurements are hard to be obtained due to the sensitivity of this sensing modality to acquisition pa- rameters. Thus, we propose a robotically assisted EBI acquisition system (REAS) to obtain stable EBI measurements of soft tissue in a userdefined region. We demonstrate the feasibility of the REAS in an ex-vivo experiment using the da Vinci Research Kit (dVRK). The results confirm the capabilities of the proposed method in performing robust EBI data acquisition.

Emotion, mood, and stress recognition (EMSR) has been studied in laboratory settings for decades. In particular, physiological signals are widely used to detect and classify affective states in lab conditions. However, physiological reactions to emotional stimuli have been found to differ in laboratory and natural settings. Thanks to recent technological progress (e.g., in wearables) the creation of EMSR systems for a large number of consumers during their everyday activities is increasingly possible. Therefore, datasets created in the wild are needed to insure the validity and the exploitability of EMSR models for real-life applications. In this paper, we initially present common techniques used in laboratory settings to induce emotions for the purpose of physiological dataset creation. Next, advantages and challenges of data collection in the wild are discussed. To assess the applicability of existing datasets to real-life applications, we propose a set of categories to guide and compare at a glance different methodologies used by researchers to collect such data. For this purpose, we also introduce a visual tool called Graphical Assessment of Real-life Application-Focused Emotional Dataset (GARAFED). In the last part of the paper, we apply the proposed tool to compare existing physiological datasets for EMSR in the wild and to show possible improvements and future directions of research. We wish for this paper and GARAFED to be used as guidelines for researchers and developers who aim at collecting affect-related data for real-life EMSR-based applications.

Numerous studies on emotion recognition from physiological signals have been conducted in laboratory settings. However, differences in the data on emotions elicited in the lab and in the wild have been observed. Thus, there is a need for systems collecting and labelling emotion-related physiological data in ecological settings. This paper proposes a new solution to collect and label such data: an open-source mobile application (app) based on the appraisal theory. Our approach exploits a commercially available wearable physiological sensor connected to a smartphone. The app detects relevant events from the physiological data, and prompts the users to report their emotions using a questionnaire based on the Ortony, Clore and Collins (OCC) Model. We believe that the app can be used to collect emotional and physiological data in ecological settings and to ensure high quality of ground truth labels.

Nowadays, motor-impaired people with Locked-In Syndrome (LIS) use eye-tracking technology to write and to access the web through commands generated by the detection of gaze movements and fixations on a graphical user interface (GUI). However, novel GUI concepts must be designed to further ease the control offered by eye-tracking, making it more responsive and less tiresome. This paper presents and evaluates an open-source standalone GUI design, SightWeb, that fully and easily controls a web browser through gaze movements without any page analysis. Its temporary menu centered in the area of action allows for reduced eye saccade and mental workload. SightWeb was evaluated against a state-of-the-art eye tracking based web interface (The Grid 3 by Thinksmartbox). The results (gaze movements, execution times, user experience questionnaire scores) suggest that SightWeb is quicker to learn and to use, more appreciated and less tiring than a traditional assistive GUI. This shows that the system can be, not only an improved interface for patients, but also an instrumental piece of software for researchers working in related areas.

The integration of intra-operative sensors into surgical robots is a hot research topic since this can significantly facilitate complex surgical procedures by enhancing surgical awareness with real-time tissue information. However, currently available intra-operative sensing technologies are mainly based on image processing and force feedback, which normally require heavy computation or complicated hardware modifications of existing surgical tools. This paper presents the design and integration of electrical bio-impedance sensing into a commercial surgical robot tool, leading to the creation of a novel smart instrument that allows the identification of tissues by simply touching them. In addition, an advanced user interface is designed to provide guidance during the use of the system and to allow augmented-reality visualization of the tissue identification results. The proposed system imposes minor hardware modifications to an existing surgical tool, but adds the capability to provide a wealth of data about the tissue being manipulated. This has great potential to allow the surgeon (or an autonomous robotic system) to better understand the surgical environment. To evaluate the system, a series of ex-vivo experiments were conducted. The experimental results demonstrate that the proposed sensing system can successfully identify different tissue types with 100% classification accuracy. In addition, the user interface was shown to effectively and intuitively guide the user to measure the electrical impedance of the target tissue, presenting the identification results as augmented-reality markers for simple and immediate recognition.

Trans-oral laser microsurgery (TLM) is a surgical procedure for removing malignancies (e.g., cysts, polyps, tumors) of the laryngeal region through laser ablation. Intraoperative microsurgical forceps (i.e., microforceps) are used for tissue manipulation. The microforceps are rigid, single degree-of-freedom (DOF) devices (open–close) with precurved jaws to access different parts of the curved cylindrical laryngeal region. These microforceps are manually handled and are subject to hand tremors, poor reachability, and nonergonomic use, resulting in poor efficacy and efficiency in the surgery. A novel 3DOF motorized microforceps device is presented here, integrated with a 6DOF serial robotic manipulator. The device, referred to as RMF-3, offers three motorized DOFs: (i) open–close forceps jaw; (ii) tool rotation; and (iii) tool-tip articulation. It is designed to be compliant with TLM spatial constraints. The manual handling is replaced by tele-operation device, the omega.7. The design of the RMF-3 is characterized through theoretical and experimental analysis. The device shows a maximum articulation of 38 deg and tool rotation of 100 deg. Its performance is further evaluated through user trials using the ring-in-loop setup. The user trials demonstrate benefits of the 3DOF workspace of the device along with its teleoperation control. RMF-3 offers an improved workspace and reachability within the laryngeal region. Surgeons, in their preliminary evaluation of the device, appreciated the ability to articulate the tip, along with rotation, for hard-to-reach parts of the surgical site. RMF-3 offers an ergonomic robotic teleoperation control interface which overcomes hand tremors and extreme wrist excursion which leads to surgeon pain and discomfort.

State-of-the-art surgical laser systems achieve high-quality tissue ablations with minimal thermal damage to the tissue by providing high-speed laser scanning. However, current technology is not applicable to difficult-to-reach anatomical sites for endoscopic surgery. In these cases, flexible laser fibers are used, but the resulting ablation quality is limited. Therefore, there is a need to improve flexible laser surgery systems. This paper proposes a solution to this issue based on a compact laser scanner and a focusing module designed to be placed at the distal end of a flexible endoscopic system. The device uses magnetic actuation to bend a laser fiber; thus, allowing precise 2-D position control and high-speed scanning of a high-power surgical laser. The design, implementation, and control of such a system are described in this paper, with special focus on its dynamic modeling and controller design to provide real-time control and fast automated execution of surgeon-defined trajectories. Validation experiments performed with different trajectories and scanning frequencies attest the performance of the system, which demonstrates trajectory tracing with 90 μm (1.4 mrad) accuracy for scanning frequencies up to 15 Hz. This is a significant result that promises to bring the benefits of free-beam laser scanning systems to endoscopic microsurgeries.

The development of the narrow-band imaging (NBI) has been increasing the interest of medical specialists in the study of laryngeal microvascular network to establish diagnosis without biopsy and pathological examination. A possible solution to this challenging problem is presented in this paper, which proposes an automatic method based on anisotropic filtering and matched filter to extract the lesion area and segment blood vessels. Lesion classification is then performed based on a statistical analysis of the blood vessels' characteristics, such as thickness, tortuosity, and density. Here, the presented algorithm is applied to 50 NBI endoscopic images of laryngeal diseases and the segmentation and classification accuracies are investigated. The experimental results show the proposed algorithm provides reliable results, reaching an overall classification accuracy rating of 84.3%. This is a highly motivating preliminary result that proves the feasibility of the new method and supports the investment in further research and development to translate this study into clinical practice. Furthermore, to our best knowledge, this is the first time image processing is used to automatically classify laryngeal tumors in endoscopic videos based on tumor vascularization characteristics. Therefore, the introduced system represents an innovation in biomedical and health informatics.

Blastocyst microinjections, like many other biomanipulation tasks, involve delicate and precise manual control of micropipettes under a microscope. The operations are carried out by highly trained operators who spend hours peering through the binoculars of microscopes. As a result, the skills of the operator and the taxing working conditions have a significant impact on the results of biomanipulations. This has led to the development of robotic systems to automate the operations, which have been shown to significantly improve their consistency and efficiency. However, the flexibility of direct operator control is often desired, especially in research environments developing or testing new protocols. In such cases, robotics can still play a significant role to improve biomanipulations when used in connection with an intuitive teleoperation interface. This is shown here for the case of blastocyst microinjection, for which we have developed a new teleoperation system offering improved ergonomics and easy and precise control over the biomanipulation equipment. Here, the performance of expert and naive operators with this system is assessed and compared with results from traditional manual microinjections conducted by experts. The results show that the robotic system allows even naive operators to outperform experts and achieve very high success rates (greater than 80%). Furthermore, the quality of the microinjections tends to improve with the teleoperated robotic system, as birth rate data demonstrates. These results offer evidence that robotics and teleoperation have the potential to significantly improve biomanipulation efficiency while maintaining the flexibility of the operations and eliminating the need for extensive training of microinjection personnel.

Endoscopic laser tools allow using the precision of surgical lasers in minimally invasive procedures. However due to the difficulty of adjusting the focal length of the laser during the procedure, ablation is typically performed with the fiber laser in contact with the tissue, which often leads to tissue carbonization. To overcome this problem, MEMS varifocal mirrors can be used. This paper presents a large-stroke MEMS varifocal mirror for an endoscopic laser tool, composed of a 4 mm circular silicon nitride membrane. Using hydraulic actuation, large optical power can be achieved while maintaining good optical quality. Experimental characterization shows that the mirror can achieve an optical power range of more than 64 m−1with 9000 Pa of actuation.

Deformable membrane mirrors allow designing compact optical systems with variable focal length. This is useful for many applications, including endoscopic laser surgery, in which a high-power laser beam is used to perform precise incisions. However, most membrane mirrors proposed so far are not suitable for high-power laser, for being too small or having limited deflection. Hydraulic actuation allows achieving large deflection, regardless of the mirror size. In this work, we present a new deformable membrane mirror designed for hydraulic actuation and high-power laser compatibility.

Motor Imagery (MI) is one of the most promising paradigms of electroencephalographic (EEG) brain-computer interfaces (BCIs) for people with severe motor impairments. Since MI requires imagining the motion of a body part, this approach can be further improved through somatosensory feedback strategies. This paper introduces a finger-based kinesthetic-tactile feedback mimicking the click event in a computer mouse. The click-like feedback follows the classification of a MI activity trial during a training and a telepresence robot control session as a validation scenario for a pilot study. The robot navigation is controlled by means of a hybrid EEG-BCI and eye-tracking paradigm (EyeBCI) in which the person explores the telepresence scene with spontaneous gazing and trigger the robot motion only when the MI command is produced. EyeBCI allows a transparent control that does not require looking outside the camera view to trigger robot movements. The effect of the click-like feedback on the MI-based hybrid EyeBCI has been compared to the feedback-less condition. In addition, the performance of the user in telepresence has been evaluated in three conditions:(i) only eye-tracker control;(ii) hybrid control without feedback;(iii) hybrid control with click-like feedback. The proposed solution promotes an easier generation of different mental states for the click-like feedback condition than for the feedback-less condition, disclosing the potential of the novel feedback design to improve the adoption of MI in real-world applications.

This paper describes the implementation and the preliminary evaluation of a novel interaction solution for people with severe motor impairments, and particularly for those affected by Amyotrophic Laterals Sclerosis (ALS) in Locked-In State (LIS): patients able to control only the ocular muscles. The proposed communication approach allows the person to control a laser beam and interact with the environment or interlocutors in an intuitive way by pointing at objects through their gaze. For this, a tip-tilt laser control device was developed and mounted on a small robot equipped with a high definition video camera. In addition, a gaze-tracking user interface was developed to control of both systems, i.e., laser and robot. This new platform was then preliminary assessed through trials with 14 subjects without disabilities, who performed tasks of robot navigation and pointing at objects along its path. User experience was evaluated with encouraging results, indicating the proposed technology has real potential to assist communication and interaction acts of people with ALS.