Search

During minimally-invasive surgery, endoscopes and other surgical tools enter the body of the patient through small openings, allowing the operations to be performed with less post-operative pain and faster recovery for the patient, as well as less wound complications. Although great advantages are achieved from the patient’s point-of-view, several difficulties have still to be handled by the surgeons. First, the limited field of view of the endoscope makes target-area localizations and related identifications complex. Second, surgeons should still match mentally the medical images for the surgical planning with the current patient anatomy. These difficulties can be addressed by using computer vision technologies for guiding surgical procedures. More specifically, this thesis aims at the following three points, the first two for neurosurgery, and the third for perfusion assessment during surgery. The challenges for this dissertation are partitioned in two primary aspects. The first point aims at the fusion of medical images on the endoscopic view, in order to develop an augmented reality system. The second point is based on hyperspectral imaging (HSI), which is compared with diffuse reflectance spectroscopy (DRS), to improve brain tissue classification for neurosurgery. The third point exploits the PPG imaging (iPPG) technique and its potential use is evaluated for peripheral arterial disease (PAD) and organ perfusion assessment during surgery. To address the first point, a new neurosurgical application is implemented on the already existing Philips Augmented Reality (AR) surgical navigation system, designed for spinal surgery. This navigation system incorporates an optical tracking system (OTS) with four video cameras embedded in the flat detector of the motorized C-arm. A hand-eye camera calibration algorithm for the fusion of medical images on the endoscopic view is implemented and integrated into the AR system. This technology is validated for endo-nasal surgery, a neurosurgical procedure for the removal of tumors located at the skull base, such as pituitary tumors. Intra-operative Cone Beam Computed Tomography (CBCT) images are fused with the view of the surgical field obtained by the endoscope camera. The accuracy of CBCT image co-registration is tested, using a custom-made grid with incorporated 3D spheres. The system achieves a sub-millimeter accuracy of image overlay of 0.55 mm, measured as mean target registration error (TRE), with a standard deviation of 0.24 mm. Afterwards, an anatomically realistic head phantom is developed, with materials chosen to achieve both X-ray attenuation and mechanical properties, similar to the real tissue. Using the phantom, a proof of concept of the skull-base surgical simulation is provided, then the accuracy and efficacy of the AR system are evaluated for the insertion of biopsy needles, which is a common neurosurgical procedure that requires high precision. Several 2-mm spherical biopsy targets are inserted inside the brain of the brain phantom. The obtained mean accuracy of the biopsy needle insertions (n=30) is 0.8±0.43 mm, with a mean device insertion time of 155±43 seconds. These experiments demonstrate that a high accuracy is obtained during neurosurgery with the proposed methods and one phantom. For the second point, a near-infrared (NIR) hyperspectral imaging (HSI) sensor is mounted on an endoscope, to explore contactless brain-tissue classification and HSI is compared with diffuse reflectance spectroscopy (DRS), which is an alternative optical technique that requires a probe in contact with the tissue. The classification is performed on ex-vivo porcine brain tissue, which is analyzed and classified in white and gray matter. The HSI reaches a sensitivity of 95% and specificity of 93%, whereas DRS reaches sensitivity and specificity of 96%. The results show that the spectral signature of the tissue in the NIR range contains sufficient information to discriminate brain tissue in white/ gray matter. Further investigation on ex-vivo tumor sample data is required prior to the clinical validation of HSI. The third part concentrates on exploring PPG imaging for extracting perfusion information on tissue. By using an off-the-shelf camera and a light source, the dynamic changes in blood volume are remotely detected beneath the skin and a map is derived correlated to the blood perfusion. After evaluating PPG imaging for local and temporal perfusion-change detections, it is employed for Peripheral Arterial Diseases (PAD) assessment. Reduced blood flow is simulated on 21 volunteers and iPPG is compared with ultrasound and Laser Speckle Contrast Analysis. These experiments show that iPPG can detect reduced perfusion levels and correlates well with the other measurement systems. Finally, this technology is deployed for organ perfusion assessment during intestine surgery. The experiments demonstrate that PPG imaging can be successfully used for extracting perfusion maps from the organ surface, even for detecting perturbations and perfusion changes during several stages of the surgery. The results of this dissertation contribute with novel techniques and approaches to add value to endoscopic and navigation technology systems, as well as to tissue classification and perfusion monitoring, during minimally-invasive surgery. The three main explored research points and their proposed techniques, namely image fusion with the endoscopic view, tissue classification via HSI and PPG imaging for perfusion assessment, can be potentially implemented and combined into a single endoscopic platform, resulting into a new multi-modal endoscopic system. Moreover, the thesis shows that the proposed techniques and algorithms increase the quality of the decision-making process and have the potential to improve the patient surgical outcome.

During minimally-invasive surgery, endoscopes and other surgical tools enter the body of the patient through small openings, allowing the operations to be performed with less post-operative pain and faster recovery for the patient, as well as less wound complications. Although great advantages are achieved from the patient’s point-of-view, several difficulties have still to be handled by the surgeons. First, the limited field of view of the endoscope makes target-area localizations and related identifications complex. Second, surgeons should still match mentally the medical images for the surgical planning with the current patient anatomy. These difficulties can be addressed by using computer vision technologies for guiding surgical procedures. More specifically, this thesis aims at the following three points, the first two for neurosurgery, and the third for perfusion assessment during surgery. The challenges for this dissertation are partitioned in two primary aspects. The first point aims at the fusion of medical images on the endoscopic view, in order to develop an augmented reality system. The second point is based on hyperspectral imaging (HSI), which is compared with diffuse reflectance spectroscopy (DRS), to improve brain tissue classification for neurosurgery. The third point exploits the PPG imaging (iPPG) technique and its potential use is evaluated for peripheral arterial disease (PAD) and organ perfusion assessment during surgery. To address the first point, a new neurosurgical application is implemented on the already existing Philips Augmented Reality (AR) surgical navigation system, designed for spinal surgery. This navigation system incorporates an optical tracking system (OTS) with four video cameras embedded in the flat detector of the motorized C-arm. A hand-eye camera calibration algorithm for the fusion of medical images on the endoscopic view is implemented and integrated into the AR system. This technology is validated for endo-nasal surgery, a neurosurgical procedure for the removal of tumors located at the skull base, such as pituitary tumors. Intra-operative Cone Beam Computed Tomography (CBCT) images are fused with the view of the surgical field obtained by the endoscope camera. The accuracy of CBCT image co-registration is tested, using a custom-made grid with incorporated 3D spheres. The system achieves a sub-millimeter accuracy of image overlay of 0.55 mm, measured as mean target registration error (TRE), with a standard deviation of 0.24 mm. Afterwards, an anatomically realistic head phantom is developed, with materials chosen to achieve both X-ray attenuation and mechanical properties, similar to the real tissue. Using the phantom, a proof of concept of the skull-base surgical simulation is provided, then the accuracy and efficacy of the AR system are evaluated for the insertion of biopsy needles, which is a common neurosurgical procedure that requires high precision. Several 2-mm spherical biopsy targets are inserted inside the brain of the brain phantom. The obtained mean accuracy of the biopsy needle insertions (n=30) is 0.8±0.43 mm, with a mean device insertion time of 155±43 seconds. These experiments demonstrate that a high accuracy is obtained during neurosurgery with the proposed methods and one phantom. For the second point, a near-infrared (NIR) hyperspectral imaging (HSI) sensor is mounted on an endoscope, to explore contactless brain-tissue classification and HSI is compared with diffuse reflectance spectroscopy (DRS), which is an alternative optical technique that requires a probe in contact with the tissue. The classification is performed on ex-vivo porcine brain tissue, which is analyzed and classified in white and gray matter. The HSI reaches a sensitivity of 95% and specificity of 93%, whereas DRS reaches sensitivity and specificity of 96%. The results show that the spectral signature of the tissue in the NIR range contains sufficient information to discriminate brain tissue in white/ gray matter. Further investigation on ex-vivo tumor sample data is required prior to the clinical validation of HSI. The third part concentrates on exploring PPG imaging for extracting perfusion information on tissue. By using an off-the-shelf camera and a light source, the dynamic changes in blood volume are remotely detected beneath the skin and a map is derived correlated to the blood perfusion. After evaluating PPG imaging for local and temporal perfusion-change detections, it is employed for Peripheral Arterial Diseases (PAD) assessment. Reduced blood flow is simulated on 21 volunteers and iPPG is compared with ultrasound and Laser Speckle Contrast Analysis. These experiments show that iPPG can detect reduced perfusion levels and correlates well with the other measurement systems. Finally, this technology is deployed for organ perfusion assessment during intestine surgery. The experiments demonstrate that PPG imaging can be successfully used for extracting perfusion maps from the organ surface, even for detecting perturbations and perfusion changes during several stages of the surgery. The results of this dissertation contribute with novel techniques and approaches to add value to endoscopic and navigation technology systems, as well as to tissue classification and perfusion monitoring, during minimally-invasive surgery. The three main explored research points and their proposed techniques, namely image fusion with the endoscopic view, tissue classification via HSI and PPG imaging for perfusion assessment, can be potentially implemented and combined into a single endoscopic platform, resulting into a new multi-modal endoscopic system. Moreover, the thesis shows that the proposed techniques and algorithms increase the quality of the decision-making process and have the potential to improve the patient surgical outcome.

During minimally-invasive surgery, endoscopes and other surgical tools enter the body of the patient through small openings, allowing the operations to be performed with less post-operative pain and faster recovery for the patient, as well as less wound complications. Although great advantages are achieved from the patient’s point-of-view, several difficulties have still to be handled by the surgeons. First, the limited field of view of the endoscope makes target-area localizations and related identifications complex. Second, surgeons should still match mentally the medical images for the surgical planning with the current patient anatomy. These difficulties can be addressed by using computer vision technologies for guiding surgical procedures. More specifically, this thesis aims at the following three points, the first two for neurosurgery, and the third for perfusion assessment during surgery. The challenges for this dissertation are partitioned in two primary aspects. The first point aims at the fusion of medical images on the endoscopic view, in order to develop an augmented reality system. The second point is based on hyperspectral imaging (HSI), which is compared with diffuse reflectance spectroscopy (DRS), to improve brain tissue classification for neurosurgery. The third point exploits the PPG imaging (iPPG) technique and its potential use is evaluated for peripheral arterial disease (PAD) and organ perfusion assessment during surgery. To address the first point, a new neurosurgical application is implemented on the already existing Philips Augmented Reality (AR) surgical navigation system, designed for spinal surgery. This navigation system incorporates an optical tracking system (OTS) with four video cameras embedded in the flat detector of the motorized C-arm. A hand-eye camera calibration algorithm for the fusion of medical images on the endoscopic view is implemented and integrated into the AR system. This technology is validated for endo-nasal surgery, a neurosurgical procedure for the removal of tumors located at the skull base, such as pituitary tumors. Intra-operative Cone Beam Computed Tomography (CBCT) images are fused with the view of the surgical field obtained by the endoscope camera. The accuracy of CBCT image co-registration is tested, using a custom-made grid with incorporated 3D spheres. The system achieves a sub-millimeter accuracy of image overlay of 0.55 mm, measured as mean target registration error (TRE), with a standard deviation of 0.24 mm. Afterwards, an anatomically realistic head phantom is developed, with materials chosen to achieve both X-ray attenuation and mechanical properties, similar to the real tissue. Using the phantom, a proof of concept of the skull-base surgical simulation is provided, then the accuracy and efficacy of the AR system are evaluated for the insertion of biopsy needles, which is a common neurosurgical procedure that requires high precision. Several 2-mm spherical biopsy targets are inserted inside the brain of the brain phantom. The obtained mean accuracy of the biopsy needle insertions (n=30) is 0.8±0.43 mm, with a mean device insertion time of 155±43 seconds. These experiments demonstrate that a high accuracy is obtained during neurosurgery with the proposed methods and one phantom. For the second point, a near-infrared (NIR) hyperspectral imaging (HSI) sensor is mounted on an endoscope, to explore contactless brain-tissue classification and HSI is compared with diffuse reflectance spectroscopy (DRS), which is an alternative optical technique that requires a probe in contact with the tissue. The classification is performed on ex-vivo porcine brain tissue, which is analyzed and classified in white and gray matter. The HSI reaches a sensitivity of 95% and specificity of 93%, whereas DRS reaches sensitivity and specificity of 96%. The results show that the spectral signature of the tissue in the NIR range contains sufficient information to discriminate brain tissue in white/ gray matter. Further investigation on ex-vivo tumor sample data is required prior to the clinical validation of HSI. The third part concentrates on exploring PPG imaging for extracting perfusion information on tissue. By using an off-the-shelf camera and a light source, the dynamic changes in blood volume are remotely detected beneath the skin and a map is derived correlated to the blood perfusion. After evaluating PPG imaging for local and temporal perfusion-change detections, it is employed for Peripheral Arterial Diseases (PAD) assessment. Reduced blood flow is simulated on 21 volunteers and iPPG is compared with ultrasound and Laser Speckle Contrast Analysis. These experiments show that iPPG can detect reduced perfusion levels and correlates well with the other measurement systems. Finally, this technology is deployed for organ perfusion assessment during intestine surgery. The experiments demonstrate that PPG imaging can be successfully used for extracting perfusion maps from the organ surface, even for detecting perturbations and perfusion changes during several stages of the surgery. The results of this dissertation contribute with novel techniques and approaches to add value to endoscopic and navigation technology systems, as well as to tissue classification and perfusion monitoring, during minimally-invasive surgery. The three main explored research points and their proposed techniques, namely image fusion with the endoscopic view, tissue classification via HSI and PPG imaging for perfusion assessment, can be potentially implemented and combined into a single endoscopic platform, resulting into a new multi-modal endoscopic system. Moreover, the thesis shows that the proposed techniques and algorithms increase the quality of the decision-making process and have the potential to improve the patient surgical outcome.

Peripheral Artery Diseases (PAD) are caused by the occlusions of arteries in the peripheral locations of the circulatory system. The severity of PAD is usually assessed using the Ankle Brachial Index (ABI) and the Ultrasound Doppler. Non-contact Photoplethysmography (PPG) imaging is a recent emerging technology capable of monitoring skin perfusion. Using an off-The-shelf camera and a light source, is possible to remotely detect the dynamic changes in blood volume in the skin and derive a map correlated to the blood perfusion. The aim of this study is the evaluation of a PPG imaging system (iPPG) for the assessment of Peripheral Arterial Diseases. Reduced blood flow is simulated on 21 volunteers by increasing the pressure in a pressure cuff. For each volunteer, measurements with iPPG, ultrasound, Laser Speckle Contrast Analysis (LASCA) and ABI were acquired. Our experiments show that iPPG can detect reduced perfusion levels, and correlates well with the other measurement systems.

The flow of blood, or perfusion, of the skin can be indicative of the local but also systemic health of an individual. The noncontact Photoplethysmography (PPG) imaging is a recently emerging technology able to monitor skin perfusion. Using an off-the-shelf camera and a light source, it is possible to remotely detect the dynamic changes in blood volume beneath the skin and derive a map correlated to the blood perfusion. In this paper we empirically investigate perfusion monitoring by camera-based PPG imaging. Laser Speckle Contrast Analysis (LASCA), a well-known technique for perfusion monitoring, is used as reference. We design an experimental setup that allows simultaneous PPG imaging and laser speckle measurements. We conduct experiments with different local and temporary perfusion perturbation of the skin and show that camera-based PPG imaging can detect the perfusion changes in the tissue. Results correlate well with the laser speckle measurements, suggesting that non-contact PPG imaging is capable of providing a realistic map of skin perfusion.

This review provides an overview of unobtrusive monitoring techniques that could be used to monitor some of the human vital signs (i.e., heart activity, breathing activity, temperature and potentially oxygen saturation) in a car seat. It will be shown that many techniques actually measure mechanical displacement, either on the body surface and/or inside the body. However, there are also techniques like capacitive electrocardiogram or bioimpedance that reflect electrical activity or passive electrical properties or thermal properties (infrared thermography). In addition, photopleythysmographic methods depend on optical properties (like scattering and absorption) of biological tissues and&mdash, mainly&mdash, blood. As all unobtrusive sensing modalities are always fragile and at risk of being contaminated by disturbances (like motion, rapidly changing environmental conditions, triboelectricity), the scope of the paper includes a survey on redundant sensor arrangements. Finally, this review also provides an overview of automotive demonstrators for vital sign monitoring.

Remotely measuring the arterial blood oxygen saturation (SpO2) in visible light (Vis) involves different probing depths, which may compromise calibratibility. This paper assesses the feasibility of calibrating camera-based SpO2 (SpO2,cam) using red and green light. Camera-based photoplethysmographic (PPG) signals were measured at 46 healthy adults at center wavelengths of 580 nm (green), 675 nm (red), and 840 nm (near-infrared; NIR). Subjects had their faces recorded during normoxia and hypoxia and under gradual cooling. SpO2,cam estimates in Vis were based on the normalized ratio of camera-based PPG amplitudes in red over green light (RoG). SpO2,cam in Vis was validated against contact SpO2 (reference) and compared with SpO2,cam estimated using red-NIR wavelengths. An RoG-based calibration curve for SpO2 was determined based on data with a SpO2 range of 85–100%. We found an $$A^{*}_{rms}$$ error of 2.9% (higher than the $$A^{*}_{rms}$$ for SpO2,cam in red-NIR). Additional measurements on normoxic subjects under temperature cooling (from $$21\,^{\circ }{\text{C}}$$ to $$