Search

In breast-conserving surgery (BCS), the tumour is removed with the goal of preserving as much healthy breast tissue as possible. Breast conservation comes with a risk of positive resection margins, an independent predictor of ipsilateral tumour recurrence, necessitating reoperation1. Contemporary data from the UK Get it Right First Time1 suggest high average reoperation rates of around 19 %. Current tumour localization techniques can only guide surgeons to the tumour epicentre, but fail to provide identification of the boundary between tumour and normal tissue. Imaging techniques, such as intraoperative ultrasonography (IOUS), intraoperative MRI (iMRI) or fluorescence-guided surgery (FGS), enable visualization of the tumour in its entirety and may provide improved operative precision2–5.

Gastrointestinal endoscopy is a subjective procedure that frequently requires tissue samples for diagnosis. Contact optical biopsy (OB) techniques have the aim of providing direct diagnosis of endoscopic areas without excising tissue samples but lack the wide-area coverage required for locating and resecting lesions. This article presents a large-area robotically deployed OB imaging platform for endoscopic detection of colorectal cancer as an add-on for conventional endoscopes. In vitro, in silicon colon phantoms, the platform achieves an optical resolution of 0.5 line pairs per millimeter, while resolving simulated cancer lesions down to 0.75 mm diameter across large-area images (55-103 cm2). Large-area OB images were generated in an ex vivo porcine colon. The platform allows centimeter-sized large-area OB imaging in vitro and ex vivo with submillimeter resolution, including automatic data segmentation of simulated cancer areas. The ability for robotic actuation and spectrum collection is also shown for ex vivo animal colon. If successful, this technology could widen access to user-independent high-quality endoscopy and early detection of gastrointestinal cancers.

The standard-of-care for the detection of laryngeal pathologies involves distinguishing suspicious lesions from surrounding healthy tissue via contrasts in colour and texture captured by white-light endoscopy. However, the technique is insufficiently sensitive and thus leads to unsatisfactory rates of false negatives. Here, we show that laryngeal lesions can be better detected in real time by taking advantage of differences in the light-polarization properties of cancer and healthy tissues. By measuring differences in polarized-light reflectance, the technique, which we named ‘surgical polarimetric endoscopy’ (SPE), generates about one-order-of-magnitude greater contrast than white-light endoscopy, and hence allows for the better discrimination of cancerous lesions, as we show with patients diagnosed with squamous cell carcinoma. Polarimetric imaging of excised and stained slices of laryngeal tissue with SPE indicated that changes in the retardance of polarized light can be largely attributed to architectural features of the tissue. We also assessed SPE to aid routine transoral laser surgery for the removal of a cancerous lesion, indicating that SPE can complement white-light endoscopy for the detection of laryngeal cancer.

Depth estimation from a stereo image pair has become one of the most explored applications in computer vision, with most previous methods relying on fully supervised learning settings. However, due to the difficulty in acquiring accurate and scalable ground truth data, the training of fully supervised methods is challenging. As an alternative, self-supervised methods are becoming more popular to mitigate this challenge. In this paper, we introduce the H-Net, a deep-learning framework for unsupervised stereo depth estimation that leverages epipolar geometry to refine stereo matching. For the first time, a Siamese autoencoder architecture is used for depth estimation which allows mutual information between rectified stereo images to be extracted. To enforce the epipolar constraint, the mutual epipolar attention mechanism has been designed which gives more emphasis to correspondences of features that lie on the same epipolar line while learning mutual information between the input stereo pair. Stereo correspondences are further enhanced by incorporating semantic information to the proposed attention mechanism. More specifically, the optimal transport algorithm is used to suppress attention and eliminate outliers in areas not visible in both cameras. Extensive experiments on KITTI2015 and Cityscapes show that the proposed modules are able to improve the performance of the unsupervised stereo depth estimation methods while closing the gap with the fully supervised approaches.

We present a novel self-supervised training framework with 3D displacement (3DD) module for accurately estimating per-pixel depth maps from single laparoscopic images. Recently, several self-supervised learning based monocular depth estimation models have achieved good results on the KITTI dataset, under the hypothesis that the camera is dynamic and the objects are stationary, however this hypothesis is often reversed in the surgical setting (laparoscope is stationary, the surgical instruments and tissues are dynamic). Therefore, a 3DD module is proposed to establish the relation between frames instead of ego-motion estimation. In the 3DD module, a convolutional neural network (CNN) analyses source and target frames to predict the 3D displacement of a 3D point cloud from a target frame to a source frame in the coordinates of the camera. Since it is difficult to constrain the depth displacement from two 2D images, a novel depth consistency module is proposed to maintain depth consistency between displacement-updated depth and model-estimated depth to constrain 3D displacement effectively. Our proposed method achieves remarkable performance for monocular depth estimation on the Hamlyn surgical dataset and acquired ground truth depth maps, outperforming monodepth, monodepth2 and packnet models.

Introduction Conventional methods for axillary sentinel lymph node biopsy (SLNB) are fraught with complications such as allergic reactions, skin tattooing, radiation, and limitations on infrastructure. A novel technique has been developed for lymphatic mapping utilizing fluorescence imaging. This meta-analysis aims to compare the gold standard blue dye and radioisotope (BD-RI) technique with fluorescence-guided SLNB using indocyanine green (ICG). Methods This study was registered with PROSPERO (CRD42019129224). The MEDLINE, EMBASE, Scopus, and Web of Science databases were searched using the Medical Subject Heading (MESH) terms ‘Surgery’ AND ‘Lymph node’ AND ‘Near infrared fluorescence’ AND ‘Indocyanine green’. Studies containing raw data on the sentinel node identification rate in breast cancer surgery were included. A heterogeneity test (using Cochran’s Q) determined the use of fixed- or random-effects models for pooled odds ratios (OR). Results Overall, 1748 studies were screened, of which 10 met the inclusion criteria for meta-analysis. ICG was equivalent to radioisotope (RI) at sentinel node identification (OR 2.58, 95% confidence interval [CI] 0.35–19.08, p p p Conclusion Fluorescence imaging for axillary sentinel node identification with ICG is equivalent to the single technique using RI, and superior to the dual technique (RI-BD) and single technique with BD. Hospitals using RI and/or BD could consider changing their practice to ICG given the comparable efficacy and improved safety profile, as well as the lesser burden on hospital infrastructure.

Significance: Diffuse reflectance spectroscopy (DRS) allows discrimination of tissue type. Its application is limited by the inability to mark the scanned tissue and the lack of real-time measurements. Aim: This study aimed to develop a real-time tracking system to enable localization of a DRS probe to aid the classification of tumor and non-tumor tissue. Approach: A green-colored marker attached to the DRS probe was detected using hue-saturation-value (HSV) segmentation. A live, augmented view of tracked optical biopsy sites was recorded in real time. Supervised classifiers were evaluated in terms of sensitivity, specificity, and overall accuracy. A developed software was used for data collection, processing, and statistical analysis. Results: The measured root mean square error (RMSE) of DRS probe tip tracking was 1.18±0.58 mm and 1.05±0.28 mm for the x and y dimensions, respectively. The diagnostic accuracy of the system to classify tumor and non-tumor tissue in real time was 94% for stomach and 96% for the esophagus. Conclusions: We have successfully developed a real-time tracking and classification system for a DRS probe. When used on stomach and esophageal tissue for tumor detection, the accuracy derived demonstrates the strength and clinical value of the technique to aid margin assessment in cancer resection surgery.

Smoke generated during surgery affects tissue visibility and degrades image quality, affecting surgical decisions and limiting further image processing and analysis. Polarization is a fundamental property of light and polarization-resolved imaging has been studied and applied to general visibility restoration scenarios such as for smog or mist removal or in underwater environments. However, there is no related research or application for surgical smoke removal. Due to differences between surgical smoke and general haze scenarios, we propose an alternative imaging degradation model by redefining the form of the transmission parameters. The analysis of the propagation of polarized light interacting with the mixed medium of smoke and tissue is proposed to realize polarization-based smoke removal (visibility restoration). Theoretical analysis and observation of experimental data shows that the cross-polarized channel data generated by multiple scattering is less affected by smoke compared to the co-polarized channel. The polarization difference calculation for different color channels can estimate the model transmission parameters and reconstruct the image with restored visibility. Qualitative and quantitative comparison with alternative methods show that the polarization-based image smoke-removal method can effectively reduce the degradation of biomedical images caused by surgical smoke and partially restore the original degree of polarization of the samples.

Background From traditional open surgery to laparoscopic surgery and robot-assisted surgery, advances in robotics, machine learning, and imaging are pushing the surgical approach to-wards better clinical outcomes. Pre-clinical and clinical evidence suggests that automation may standardise techniques, increase efficiency, and reduce clinical complications. Methods A PRISMA-guided search was conducted across PubMed and OVID. Results Of the 89 screened articles, 51 met the inclusion criteria, with 10 included in the final review. Automatic data segmentation, trajectory planning, intra-operative registration, trajectory drilling, and soft tissue robotic surgery were discussed. Conclusion Although automated surgical systems remain conceptual, several research groups have developed supervised autonomous robotic surgical systems with increasing consideration for ethico-legal issues for automation. Automation paves the way for precision surgery and improved safety and opens new possibilities for deploying more robust artificial intelligence models, better imaging modalities and robotics to improve clinical outcomes.

Aim Cancers of the upper gastrointestinal (GI) tract remain a major contributor to the global cancer risk. Surgery aims to completely resect tumour with clear margins, whilst preserving as much surrounding tissue. Diffuse reflectance spectroscopy (DRS) is a novel technique that presents a promising advancement in cancer diagnosis. We have developed a novel DRS system with tracking capability. Our aim is to classify tumour and non-tumour GI tissue in real-time using this device to aid intra-operative analysis of resection margins. Method An ex-vivo study was undertaken in which data was collected from consecutive patients undergoing upper GI cancer resection surgery between August 2020- January 2021. A hand-held DRS probe and tracking system was used on normal and cancerous tissue to obtain spectral information. After acquisition of all spectra, a classification system using histopathology results was created. A user interface was developed using Python 3.6 and Qt5. A support vector machine was used to classify the results. Results The data included 4974 normal spectra and 2108 tumour spectra. The overall accuracy of the DRS probe in differentiating normal versus tumour tissue was 88.08% for the stomach (sensitivity 84.8%, specificity 89.3%), and 91.42% for the oesophagus (sensitivity 76.3%, specificity 98.9%). Conclusion We have developed a successful DRS system with tracking capability, able to process thousands of spectra in a small timeframe, which can be used in real-time to distinguish tumour and non-tumour tissue. This can be used for intra-operative decision-making during upper GI cancer surgery to help select the best resection plane.

Dense depth estimation and 3D reconstruction of a surgical scene are crucial steps in computer assisted surgery. Recent work has shown that depth estimation from a stereo image pair could be solved with convolutional neural networks. However, most recent depth estimation models were trained on datasets with per-pixel ground truth. Such data is especially rare for laparoscopic imaging, making it hard to apply supervised depth estimation to real surgical applications. To overcome this limitation, we propose SADepth, a new self-supervised depth estimation method based on Generative Adversarial Networks. It consists of an encoder-decoder generator and a discriminator to incorporate geometry constraints during training. Multi-scale outputs from the generator help to solve the local minima caused by the photometric reprojection loss, while the adversarial learning improves the framework generation quality. Extensive experiments on two public datasets show that SADepth outperforms recent state-of-the-art unsupervised methods by a large margin, and reduces the gap between supervised and unsupervised depth estimation in laparoscopic images.

Aim Gastrointestinal (GI) cancers account for 26% of global cancer incidence with prevalence projected to rise exponentially due to the ageing population and lifestyle choices. Surgical resection is the mainstay of treatment to remove the cancer in its entirety to achieve an R0 resection. Positive margins, when cancerous tissue has been left in situ, is associated with increased morbidity and mortality. Current margin assessment involves histopathological analysis, after resection of the specimen. Diffuse Reflectance Spectroscopy (DRS) and Hyperspectral Imaging (HSI) are novel imaging techniques that have the potential to provide real-time assessment of cancer margins intra-operatively to reduce the incidence of positive resection margins and improve patient outcomes. The aim of this review is to assess the current state of evidence for the use of novel imaging techniques in GI cancer margin assessment. Method A literature review was conducted of studies using DRS and HSI in GI cancers in adult patients, published from inception to October 2020. Results A total of 15 studies were analysed, nine of which used DRS and six used HSI and the majority of studies were performed ex-vivo. Current image acquisition techniques and processing algorithms vary greatly. The sensitivity and specificity of DRS ranged from 0.90-0.98 and 0.88-0.95 respectively and for HSI 0.63-0.98 and 0.69-0.98, respectively across five types of GI cancers. Conclusions DRS and HSI are novel imaging techniques, currently in their infancy but the outlook is promising. With further research focused on standardising methodology and in-vivo settings, DRS and HSI could transform intra-operative margin assessment in GI cancers.

ImportanceCancers of the upper gastrointestinal tract remain a major contributor to the global cancer burden. The accurate mapping of tumor margins is of particular importance for curative cancer resection and improvement in overall survival. Current mapping techniques preclude a full resection margin assessment in real time.ObjectiveTo evaluate whether diffuse reflectance spectroscopy (DRS) on gastric and esophageal cancer specimens can differentiate tissue types and provide real-time feedback to the operator.Design, Setting, and ParticipantsThis was a prospective ex vivo validation study. Patients undergoing esophageal or gastric cancer resection were prospectively recruited into the study between July 2020 and July 2021 at Hammersmith Hospital in London, United Kingdom. Tissue specimens were included for patients undergoing elective surgery for either esophageal carcinoma (adenocarcinoma or squamous cell carcinoma) or gastric adenocarcinoma.ExposuresA handheld DRS probe and tracking system was used on freshly resected ex vivo tissue to obtain spectral data. Binary classification, following histopathological validation, was performed using 4 supervised machine learning classifiers.Main Outcomes and MeasuresData were divided into training and testing sets using a stratified 5-fold cross-validation method. Machine learning classifiers were evaluated in terms of sensitivity, specificity, overall accuracy, and the area under the curve.ResultsOf 34 included patients, 22 (65%) were male, and the median (range) age was 68 (35-89) years. A total of 14 097 mean spectra for normal and cancerous tissue were collected. For normal vs cancer tissue, the machine learning classifier achieved a mean (SD) overall diagnostic accuracy of 93.86% (0.66) for stomach tissue and 96.22% (0.50) for esophageal tissue and achieved a mean (SD) sensitivity and specificity of 91.31% (1.5) and 95.13% (0.8), respectively, for stomach tissue and of 94.60% (0.9) and 97.28% (0.6) for esophagus tissue. Real-time tissue tracking and classification was achieved and presented live on screen.Conclusions and RelevanceThis study provides ex vivo validation of the DRS technology for real-time differentiation of gastric and esophageal cancer from healthy tissue using machine learning with high accuracy. As such, it is a step toward the development of a real-time in vivo tumor mapping tool for esophageal and gastric cancers that can aid decision-making of resection margins intraoperatively.

Purpose Intra-operative measurement of tissue oxygen saturation (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\hbox {StO}}_2$$\end{document}StO2) is important in detection of ischaemia, monitoring perfusion and identifying disease. Hyperspectral imaging (HSI) measures the optical reflectance spectrum of the tissue and uses this information to quantify its composition, including \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\hbox {StO}}_2$$\end{document}StO2. However, real-time monitoring is difficult due to capture rate and data processing time. Methods An endoscopic system based on a multi-fibre probe was previously developed to sparsely capture HSI data (sHSI). These were combined with RGB images, via a deep neural network, to generate high-resolution hypercubes and calculate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\hbox {StO}}_2$$\end{document}StO2. To improve accuracy and processing speed, we propose a dual-input conditional generative adversarial network, Dual2StO2, to directly estimate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\hbox {StO}}_2$$\end{document}StO2 by fusing features from both RGB and sHSI. Results Validation experiments were carried out on in vivo porcine bowel data, where the ground truth \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\hbox {StO}}_2$$\end{document}StO2 was generated from the HSI camera. Performance was also compared to our previous super-spectral-resolution network, SSRNet in terms of mean \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\hbox {StO}}_2$$\end{document}StO2 prediction accuracy and structural similarity metrics. Dual2StO2 was also tested using simulated probe data with varying fibre number. Conclusions \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\hbox {StO}}_2$$\end{document}StO2 estimation by Dual2StO2 is visually closer to ground truth in general structure and achieves higher prediction accuracy and faster processing speed than SSRNet. Simulations showed that results improved when a greater number of fibres are used in the probe. Future work will include refinement of the network architecture, hardware optimization based on simulation results, and evaluation of the technique in clinical applications beyond \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\hbox {StO}}_2$$\end{document}StO2 estimation.

Although new cancer therapeutics are discovered at a rapid pace, lack of effective means of delivery and cancer chemoresistance thwart many of the promising therapeutics. We demonstrate a method that confronts both of these issues with the light-activated delivery of a Bcl-2 functional converting peptide, NuBCP-9, using hollow gold nanoshells. This approach has shown not only to increase the efficacy of the peptide 30-fold in vitro but also has shown to reduce paclitaxel resistant H460 lung xenograft tumor growth by 56.4%.

Background On average, 21% of women in the USA treated with Breast Conserving Surgery (BCS) undergo a second operation because of close positive margins. Tumor identification with fluorescence imaging could improve positive margin rates through demarcating location, size, and invasiveness of tumors. We investigated the technique’s diagnostic accuracy in detecting tumors during BCS using intravenous indocyanine green (ICG) and a custom-built fluorescence camera system. Methods In this single-center prospective clinical study, 40 recruited BCS patients were sub-categorized into two cohorts. In the first ‘enhanced permeability and retention’ (EPR) cohort, 0.25 mg/kg ICG was injected ~ 25 min prior to tumor excision, and in the second ‘angiography’ cohort, ~ 5 min prior to tumor excision. Subsequently, an in-house imaging system was used to image the tumor in situ prior to resection, ex vivo following resection, the resection bed, and during grossing in the histopathology laboratory to compare the technique’s diagnostic accuracy between the cohorts. Results The two cohorts were matched in patient and tumor characteristics. The majority of patients had invasive ductal carcinoma with concomitant ductal carcinoma in situ. Tumor-to-background ratio (TBR) in the angiography cohort was superior to the EPR cohort (TBR = 3.18 ± 1.74 vs 2.10 ± 0.92 respectively, p = 0.023). Tumor detection reached sensitivity and specificity scores of 0.82 and 0.93 for the angiography cohort and 0.66 and 0.90 for the EPR cohort, respectively (p = 0.1051 and p = 0.9099). Discussion ICG administration timing during the angiography phase compared with the EPR phase improved TBR and diagnostic accuracy. Future work will focus on image pattern analysis and adaptation of the camera system to targeting fluorophores specific to breast cancer.

An improved smoke removal model for surgery is introduced with transmission parameters related to a medium’s optical depth rather than scene distance. Theoretical analysis and observation of experimental data shows that cross-polarized signals generated by multiple scattering are less affected by smoke compared to co-polarized signals. We analyze the transmission process of linearly polarized light interacting with different media, and then use polarization difference imaging and color channel information to detect smoke and estimate the transmission parameters. Several further processing procedures including parameter compensation and image smoothing are implemented to recover tissue visibility from surgical images.

We propose an all multicore fiber transmission system removing fan-in/fan-out devices from inline amplifiers. The cable capacity can be improved by > 100% over a single-core fiber transmission system under a 24 fiber pair limit.

Fast spectral tracking routine, using simultaneous analysis of color and monochrome images, was developed and tested in phantoms. This routine could improve the efficiency of fluorescence hyperspectral imaging for breast conserving surgery guidance. © 2021 The Author(s)

Graded index (GRIN) lenses focus light through a radially symmetric refractive index profile. It is not widely appreciated that the ion-exchange process that creates the index profile also causes a radially symmetric birefringence variation. This property is usually considered a nuisance, such that manufacturing processes are optimized to keep it to a minimum. Here, a new Mueller matrix (MM) polarimeter based on a spatially engineered polarization state generating array and GRIN lens cascade for measuring the MM of a region of a sample in a single-shot is presented. We explore using the GRIN lens cascade for a functional analyzer to calculate multiple Stokes vectors and the MM of the target in a snapshot. A designed validation sample is used to test the reliability of this polarimeter. To understand more potential biomedical applications, human breast ductal carcinoma slides at two pathological progression stages are detected by this polarimeter. The MM polar decomposition parameters then can be calculated from the measured MMs, and quantitatively compared with the equivalent data sampled by a MM microscope. The results indicate that the polarimeter and the measured polarization parameters are capable of differentiating the healthy and carcinoma status of human breast tissue efficiently. It has potential to act as a polarization detected fiber-based probe to assist further minimally invasive clinical diagnosis.

Transoral laser phonosurgery is a commonly used surgical procedure in which a laser beam is used to perform incision, ablation or photocoagulation of laryngeal tissues. Two techniques are commonly practiced: free beam and fiber delivery . For free beam delivery, a laser scanner is integrated into a surgical microscope to provide an accurate laser scanning pattern. This approach can only be used under direct line of sight, which may cause increased postoperative pain to the patient and injury, is uncomfortable for the surgeon during prolonged operations, the manipulability is poor and extensive training is required. In contrast, in the fiber delivery technique, a flexible fiber is used to transmit the laser beam and therefore does not require direct line of sight. However, this can only achieve manual level accuracy, repeatability and velocity, and does not allow for pattern scanning. Robotic systems have been developed to overcome the limitations of both techniques. However, these systems offer limited workspace and degrees-of-freedom (DoF), limiting their clinical applicability. This work presents the LaryngoTORS, a robotic system that aims at overcoming the limitations of the two techniques, by using a cable-driven parallel mechanism (CDPM) attached at the end of a curved laryngeal blade for controlling the end tip of the laser fiber. The system allows autonomous generation of scanning patterns or user-driven free-path scanning. Path scan validation demonstrated errors as low as 0.054 $\pm$ 0.028 mm and high repeatability of 0.027 $\pm$ 0.020 mm (6 × 2 mm arc line). Ex vivo tests on chicken tissue have been carried out. The results show the ability of the system to overcome limitations of current methods with high accuracy and repeatability using the superior fiber delivery approach.

Since the beginning of medicine, optical imaging has formed a central pillar for the diagnosis and treatment of disease. The 20th century saw the development of many other diagnostic imaging methods – CT, MRI, nuclear methods, etc. – but optical imaging remains of paramount importance. The minimally invasive surgery revolution has been enabled by the detection of the surgical field with a color camera system that is either mounted on the proximal end of an endoscope or in more recent times miniaturized and placed at the tip. The color responses of the red–green–blue image data are well matched to the human eye and can be presented on a color display to the surgeon in the operating theater for direct visual guidance of the intervention. However, while this provides a visually recognizable picture of the tissue to the surgeon, it misses the potential of light to reveal otherwise invisible tissue information. Within the last few decades alternative optical imaging methods have started to become more widely used and it is the potential of biophotonics techniques to guide interventions that will be explored in this chapter. Descriptions will be provided to explain how these additional signals may be collected, understood, and applied to different diseases.

There is an unmet need for accurate tumour localization in vivo during breast conserving surgery. Herein a novel snapshot hyperspectral system is presented for accurately detecting the intrinsic fluorescence signal in real-time fluorescence guided surgery.

LaryngoTORS, a transoral laryngeal surgery robot, can manipulate instruments accurately. Confocal imaging has potentials in laryngeal cancer diagnosis but suffer from high scanning requirement. This work studies using LaryngoTORS to assist confocal imaging of larynx.

Medical robotics encompasses surgical, therapeutic and rehabilitative devices that are changing medicine and healthcare. Although the field of medical robotics predates Intuitive’s da Vinci by more than a decade, it was the clinical and commercial achievements of that system that brought medical robotics to widespread patient and public attention. It is now more than 15 years since the robot began to be used for laparoscopic prostatectomy.1 Since then, research in the field has advanced tremendously due to various technological breakthroughs. Over the last few years, there has been a surge in commercial activities in medical robotics, led both by traditional medical device and technology companies as well as new start-ups. This special issue has been commissioned to capture some of the latest research being carried out by these multidisciplinary bioengineering teams and to showcase how some of these advances can impact clinical care.

Surgical guidance and decision making could be improved with accurate and real-time measurement of intra-operative data including shape and spectral information of the tissue surface. In this work, a dual-modality endoscopic system has been proposed to enable tissue surface shape reconstruction and hyperspectral imaging (HSI). This system centers around a probe comprised of an incoherent fiber bundle, whose fiber arrangement is different at the two ends, and miniature imaging optics. For 3D reconstruction with structured light (SL), a light pattern formed of randomly distributed spots with different colors is projected onto the tissue surface, creating artificial texture. Pattern decoding with a Convolutional Neural Network (CNN) model and a customized feature descriptor enables real-time 3D surface reconstruction at approximately 12 frames per second (FPS). In HSI mode, spatially sparse hyperspectral signals from the tissue surface can be captured with a slit hyperspectral imager in a single snapshot. A CNN based super-resolution model, namely “super-spectral-resolution” network (SSRNet), has also been developed to estimate pixel-level dense hypercubes from the endoscope cameras standard RGB images and the sparse hyperspectral signals, at approximately 2 FPS. The probe, with a 2.1 mm diameter, enables the system to be used with endoscope working channels. Furthermore, since data acquisition in both modes can be accomplished in one snapshot, operation of this system in clinical applications is minimally affected by tissue surface movement and deformation. The whole apparatus has been validated on phantoms and tissue (ex vivo and in vivo), while initial measurements on patients during laryngeal surgery show its potential in real-world clinical applications.

The first version of this article neglected to mention that this work was additionally supported by ERC award 637960. This has now been corrected online. The authors would like to apologise for any inconvenience caused.

The IEEE Transactions on Medical Robotics and Bionics (T-MRB) is an initiative shared by the two IEEE Societies of Robotics and Automation—RAS—and Engineering in Medicine and Biology—EMBS.

The first example of an octadentate gadolinium unit based on DO3A (hydration number q = 1) with a dithiocarbamate tether has been designed and attached to the surface of gold nanoparticles (around 4.4 nm in diameter). In addition to the superior robustness of this attachment, the restricted rotation of the Gd complex on the nanoparticle surface leads to a dramatic increase in relaxivity (r1) from 4.0 mM‐1 s‐1 in unbound form to 34.3 mM‐1 s‐1 (at 10 MHz, 37 °C) and 22 ± 2 mM‐1s‐1 (at 63.87 MHz, 25 °C) when immobilised on the surface. The ‘one‐pot’ synthetic route provides a straightforward and versatile way of preparing a range of multifunctional gold nanoparticles. The incorporation of additional surface units improving biocompatibility (PEG and thioglucose units) and targeting (folic acid) lead to little detrimental effect on the high relaxivity observed for these non‐toxic multifunctional materials. In addition to the passive targeting attributed to gold nanoparticles, the inclusion of a unit capable of targeting the folate receptors overexpressed by cancer cells, such as HeLa cells, illustrates the potential of these assemblies.

INTRODUCTION AND OBJECTIVES:In prostate cancer, radioguided surgery (RGS) is used for sentinel node biopsy (SNB) and holds potential for 99mTc-PSMA-guided surgery of lymph node metastases. Rigid la...

A two-camera fluorescence system for indocyanine green (ICG) signal detection has been developed and tested in a clinical feasibility trial of ten patients, with a resolution in the submillimetre scale. Immediately after systemic ICG injection, the two-camera system can detect ICG signals in vivo (∼2.5 mg l − 1 or 3.2 × 10 − 6 M). Qualitative assessment has shown that the fluorescence signal does not always correlate with the cancer location in the surgical scene. Conversely, fluorescence image texture metrics when used with the logistic regression model yields good accuracy scores in detecting cancer. We have demonstrated that intraoperative fluorescence imaging for resection guidance is a feasible solution to tackle the current challenge of positive resection margins in breast conserving surgery.

The diffusion of minimally invasive surgery has thrived in recent years, providing substantial benefits over traditional techniques for a number of surgical interventions. This rapid growth has been possible due to significant advancements in medical technology, which partly solved some of the technical and clinical challenges associated with minimally invasive techniques. The issues that still limit its widespread adoption for some applications include the limited field of view; reduced manoeuvrability of the tools; lack of haptic feedback; loss of depth perception; extended learning curve; prolonged operative times and higher financial costs. The present review discusses some of the main recent technological advancements that fuelled the uptake of minimally invasive surgery, focussing especially on the areas of imaging, instrumentation, cameras and robotics. The current limitations of state-of-the-art technology are identified and addressed, proposing future research directions necessary to overcome them.

Background: Lung cancer is one of the commonest malignancies with a worldwide incidence of 1.6 million cases each year. Although the main aetiological factor has been identified (cigarette smoking), the progression of lung cancer from early changes such as dysplasia through to cancer is still not fully understood. Furthermore, current research techniques are reliant on obtaining tissue biopsies, a process that alters the natural history of the very process under investigation. Hence, there is a need for developing optical biopsy techniques. Objectives: To prospectively evaluate the feasibility of endocytoscopy and confocal endomicroscopy in the detection of malignant and pre-malignant changes in the airways. Methods: Findings with endocytoscopy and endomicroscopy were compared with conventional biopsies obtained from the same areas in 25 patients undergoing bronchoscopy for evaluation of endobronchial abnormalities and in 5 healthy control subjects. Results: Endocytoscopy was technically more difficult, and interpretable images were only obtained in 21 of the patients evaluated, and hence, complete information including histopathological information was available in 21 patients. Endocytoscopy appeared to correlate with the histopathological findings on tissue biopsy, and was able to distinguish normal epithelium from dysplasia and carcinoma. Confocal endomicroscopy was a more reliable technique with adequate visual information obtained in all patients examined but was unable to distinguish between dysplasia and carcinoma. Conclusion: This feasibility study suggests that endocytoscopy may have the potential to fulfil the role of optical biopsy in the evaluation of the pathogenesis of lung cancer.

Purpose Multispectral imaging can provide reflectance measurements at multiple spectral bands for each image pixel. These measurements can be used for estimation of important physiological parameters, such as oxygenation, which can provide indicators for the success of surgical treatment or the presence of abnormal tissue. The goal of this work was to develop a method to estimate physiological parameters in an accurate and rapid manner suited for modern high-resolution laparoscopic images. Methods While previous methods for oxygenation estimation are based on either simple linear methods or complex model-based approaches exclusively suited for off-line processing, we propose a new approach that combines the high accuracy of model-based approaches with the speed and robustness of modern machine learning methods. Our concept is based on training random forest regressors using reflectance spectra generated with Monte Carlo simulations. Results According to extensive in silico and in vivo experiments, the method features higher accuracy and robustness than state-of-the-art online methods and is orders of magnitude faster than other nonlinear regression based methods. Conclusion Our current implementation allows for near real-time oxygenation estimation from megapixel multispectral images and is thus well suited for online tissue analysis.

Gold nanoparticles are chemically fabricated and tuned to strongly absorb near infrared (NIR) light, enabling deep optical penetration and therapy within human tissues, where sufficient heating induces tumour necrosis. In our studies we aim to establish the optimal gold nanorod (GNR) concentration and laser power for inducing hyperthermic effects in tissues and test this photothermal effect on ex vivo human oesophagogastric adenocarcinoma. The ideal GNR concentration and NIR laser power that would elicit sufficient hyperthermia for tumour necrosis was pre-determined on porcine oesophageal tissues. Human ex vivo oesophageal and gastric adenocarcinoma tissues were incubated with GNR solutions and a GNR-free control solution with corresponding healthy tissues for comparison, then irradiated with NIR light for 10 minutes. Temperature rise was found to vary linearly with both the concentration of GNRs and the laser power. Human ex vivo oesophageal and gastric tissues consistently demonstrated a significant temperature rise when incubated in an optimally concentrated GNR solution (3 x 10(10) GNRs/ml) prior to NIR irradiation delivered at an optimal power (2 W/cm2). A mean temperature rise of 27 degrees C was observed in tissues incubated with GNRs, whereas only a modest 2 degrees C rise in tissues not exposed to any GNRs. This study evaluates the photothermal effects of GNRs on oesophagogastric tissue examines their application in the minimally invasive therapeutics of oesophageal and gastric adenocarcinomas. This could potentially be an effective method of clinically inducing irreversible oesophagogastric tumour photodestruction, with minimal collateral damage expected in (healthy) tissues free from GNRs.

Background Current optical diagnostic techniques for malignancies are limited in their diagnostic accuracy and lack the ability to further characterise disease, leading to the rapidly increasing development of novel imaging methods within urology. This systematic review critically appraises the literature for novel imagining modalities, in the detection and staging of urological cancer and assesses their effectiveness via their utility and accuracy. Methods A systematic literature search utilising MEDLINE, EMBASE and Cochrane Library Database was conducted from 1970 to September 2018 by two independent reviewers. Studies were included if they assessed real-time imaging modalities not already approved in guidelines, in vivo and in humans. Outcome measures included diagnostic accuracy and utility parameters, including feasibility and cost. Results Of 5475 articles identified from screening, a final 46 were included. Imaging modalities for bladder cancer included optical coherence tomography (OCT), confocal laser endomicroscopy, autofluorescence and spectroscopic techniques. OCT was the most widely investigated, with 12 studies demonstrating improvements in overall diagnostic accuracy (sensitivity 74.5–100% and specificity 60–98.5%). Upper urinary tract malignancy diagnosis was assessed using photodynamic diagnosis (PDD), narrow band imaging, optical coherence tomography and confocal laser endomicroscopy. Only PDD demonstrated consistent improvements in overall diagnostic accuracy in five trials (sensitivity 94–96% and specificity 96.6–100%). Limited evidence for optical coherence tomography in percutaneous renal biopsy was identified, with anecdotal evidence for any modality in penile cancer. Conclusions Evidence supporting the efficacy for identified novel imaging modalities remains limited at present. However, OCT for bladder cancer and PDD in upper tract malignancy demonstrate the best potential for improvement in overall diagnostic accuracy. OCT may additionally aid intraoperative decision making via real-time staging of disease. Both modalities require ongoing investigation through larger, well-conducted clinical trials to assess their diagnostic accuracy, use as an intraoperative staging aid and how to best utilise them within clinical practice.

Background: At present, there is no system available that permits objective assessment of vocal cord dysfunction (VCD) or exercise-induced laryngeal obstruction (EILO). The reproducibility of the currently available subjective methods is poor, impacting treatment outcome surveillance and clinical decision making. The aim of this work is to describe the development of a computerized analysis system that provides robust, objective means of characterizing endoscopic video of laryngeal movement (including during exercise). Methods: We developed a pipeline of convolutional neural networks and singular spectrum analysis to process laryngoscopic videos, i.e. to delineate and quantify the movement of laryngeal structures. The system was trained and validated with image frames acquired from ~100 patients using different endoscopic systems. Results: The method was able to robustly analyse video at a speed of at least 16 frames per second, running on a desktop computer with a high-end graphics processing unit. A promising clinical showcase of the analysis of an exercise laryngoscopy test, performed by an individual with EILO is shown in fig. 1. Conclusion: The proposed method guarantees a robust and fast processing of laryngoscopic video recordings and could potentially replace subjective and manual methods for analysis of laryngeal obstruction. Subsequent work will focus on validating the method for real-world clinical use.

Objective: At present, there are no objective techniques to quantify and describe laryngeal obstruction, and the reproducibility of subjective manual quantification methods is insufficient, resulting in diagnostic inaccuracy and a poor signal-to-noise ratio in medical research. In this work, a workflow is proposed to quantify laryngeal movements from laryngoscopic videos, to facilitate the diagnosis procedure. Methods: The proposed method analyses laryngoscopic videos, and delineates glottic opening, vocal folds, and supraglottic structures, using a convolutional neural networks (CNNs) based algorithm. The segmentation is divided into two steps: A bounding box which indicates the region of interest (RoI) is found, followed by segmentation using fully convolutional networks (FCNs). The segmentation results are statistically quantified along the temporal dimension and processed using singular spectrum analysis (SSA), to extract clear objective information that can be used by the clinicians in diagnosis. Results: The segmentation was validated on 400 images from 20 videos acquired using different endoscopic systems from different patients. The results indicated significant improvements over using FCN only in terms of both processing speed (16 FPS vs. 8 FPS) and segmentation result statistics. Five clinical cases on patients have also been provided to showcase the quantitative analysis results using the proposed method. Conclusion: The proposed method guarantees a robust and fast processing of laryngoscopic videos. Measurements of glottic angles and supraglottic index showed distinctive patterns in the provided clinical cases. Significance: The proposed automated and objective method extracts important temporal laryngeal movement information, which can be used to aid laryngeal closure diagnosis.