1. Surgical spectral imaging
- Author
-
Lena Maier-Hein, Daniel S. Elson, Danail Stoyanov, Geoffrey Jones, and Neil T. Clancy
- Subjects
CNN, Convolutional neural network ,EMCCD, Electron-multiplying charge-coupled device ,Hyperspectral imaging ,MRI, Magnetic resonance imaging ,Computer science ,Multispectral image ,GI, Gastrointestinal ,SVM, Support vector machine ,computer.software_genre ,09 Engineering ,030218 nuclear medicine & medical imaging ,VOF, Variable optical filter ,Multispectral imaging ,Machine Learning ,0302 clinical medicine ,Image Processing, Computer-Assisted ,NBI, Narrowband imaging ,11 Medical and Health Sciences ,AOTF, Acousto-optic tuneable filter ,DPF, Differential pathlength factor ,Radiological and Ultrasound Technology ,Minimally-invasive surgery ,DMD, Digital micromirror device ,MSI, Multispectral imaging ,Computer Graphics and Computer-Aided Design ,RGB, Red, green, blue ,INN, Invertible neural network ,Nuclear Medicine & Medical Imaging ,LOOCV, Leave-one-out cross validation ,CT, Computed tomography ,Computer Vision and Pattern Recognition ,FIGS, Fluorescence image-guided surgery ,Diagnostic Imaging ,medicine.medical_specialty ,FWHM, Full-width at half-maximum ,ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION ,Health Informatics ,LED, Light emitting diode ,LCTF, Liquid crystal tuneable filter ,Machine learning ,Computational imaging ,Imaging phantom ,Article ,HSI, Hyperspectral imaging ,03 medical and health sciences ,Artificial Intelligence ,White light ,medicine ,Humans ,Radiology, Nuclear Medicine and imaging ,MIS, Minimally-invasive surgery ,Modalities ,business.industry ,Deep learning ,AI, Artificial intelligence ,SNR, Signal-to-noise ratio ,NIR, Near infrared ,OEM, Original equipment manufacturer ,SFDI, Spatial frequency domain imaging ,Spectral imaging ,Visualization ,SSI, Surgical spectral imaging ,sCMOS, Scientific complementary metal-oxide-semiconductor ,Artificial intelligence ,business ,computer ,030217 neurology & neurosurgery - Abstract
Highlights • Wider sensor availability and miniaturisation are pushing speed/resolution limits. • Small surgical datasets exist in many specialities but no standard format. • Data-driven analysis avoids modelling, improves speed, addresses uncertainty. • RGB-based functional imaging could exploit existing cameras, chip-on-tip devices. • Clinical validation with standardised devices and data needed for translation., Recent technological developments have resulted in the availability of miniaturised spectral imaging sensors capable of operating in the multi- (MSI) and hyperspectral imaging (HSI) regimes. Simultaneous advances in image-processing techniques and artificial intelligence (AI), especially in machine learning and deep learning, have made these data-rich modalities highly attractive as a means of extracting biological information non-destructively. Surgery in particular is poised to benefit from this, as spectrally-resolved tissue optical properties can offer enhanced contrast as well as diagnostic and guidance information during interventions. This is particularly relevant for procedures where inherent contrast is low under standard white light visualisation. This review summarises recent work in surgical spectral imaging (SSI) techniques, taken from Pubmed, Google Scholar and arXiv searches spanning the period 2013–2019. New hardware, optimised for use in both open and minimally-invasive surgery (MIS), is described, and recent commercial activity is summarised. Computational approaches to extract spectral information from conventional colour images are reviewed, as tip-mounted cameras become more commonplace in MIS. Model-based and machine learning methods of data analysis are discussed in addition to simulation, phantom and clinical validation experiments. A wide variety of surgical pilot studies are reported but it is apparent that further work is needed to quantify the clinical value of MSI/HSI. The current trend toward data-driven analysis emphasises the importance of widely-available, standardised spectral imaging datasets, which will aid understanding of variability across organs and patients, and drive clinical translation., Graphical abstract Image, graphical abstract
- Published
- 2020