Search

Gone are the days when medical imaging was used primarily to visualize anatomic structures. The emergence of molecular imaging (MI), championed by radiolabeled (18)F-FDG PET, has expanded the information content derived from imaging to include pathophysiologic and molecular processes. Cancer imaging, in particular, has leveraged advances in MI agents and technology to improve the accuracy of tumor detection, interrogate tumor heterogeneity, monitor treatment response, focus surgical resection, and enable image-guided biopsy. Surgeons are actively latching on to the incredible opportunities provided by medical imaging for preoperative planning, intraoperative guidance, and postoperative monitoring. From label-free techniques to enabling cancer-selective imaging agents, image-guided surgery provides surgical oncologists and interventional radiologists both macroscopic and microscopic views of cancer in the operating room. This review highlights the current state of MI and sensing approaches available for surgical guidance. Salient features of nuclear, optical, and multimodal approaches will be discussed, including their strengths, limitations, and clinical applications. To address the increasing complexity and diversity of methods available today, this review provides a framework to identify a contrast mechanism, suitable modality, and device. Emerging low-cost, portable, and user-friendly imaging systems make the case for adopting some of these technologies as the global standard of care in surgical practice.

Crucial transitions in cancer-including tumor initiation, local expansion, metastasis, and therapeutic resistance-involve complex interactions between cells within the dynamic tumor ecosystem. Transformative single-cell genomics technologies and spatial multiplex in situ methods now provide an opportunity to interrogate this complexity at unprecedented resolution. The Human Tumor Atlas Network (HTAN), part of the National Cancer Institute (NCI) Cancer Moonshot Initiative, will establish a clinical, experimental, computational, and organizational framework to generate informative and accessible three-dimensional atlases of cancer transitions for a diverse set of tumor types. This effort complements both ongoing efforts to map healthy organs and previous large-scale cancer genomics approaches focused on bulk sequencing at a single point in time. Generating single-cell, multiparametric, longitudinal atlases and integrating them with clinical outcomes should help identify novel predictive biomarkers and features as well as therapeutically relevant cell types, cell states, and cellular interactions across transitions. The resulting tumor atlases should have a profound impact on our understanding of cancer biology and have the potential to improve cancer detection, prevention, and therapeutic discovery for better precision-medicine treatments of cancer patients and those at risk for cancer.

The heterogeneity and continuous genetic adaptation of tumours complicate their detection and treatment via the targeting of genetic mutations. However, hallmarks of cancer such as aberrant protein phosphorylation and calcium-mediated cell signalling provide broadly conserved molecular targets. Here, we show that, for a range of solid tumours, a cyclic octapeptide labelled with a near-infrared dye selectively binds to phosphorylated Annexin A2 (pANXA2), with high affinity at high levels of calcium. Because of cancer-cell-induced pANXA2 expression in tumour-associated stromal cells, the octapeptide preferentially binds to the invasive edges of tumours and then traffics within macrophages to the tumour's necrotic core. As proof-of-concept applications, we used the octapeptide to detect tumour xenografts and metastatic lesions, and to perform fluorescence-guided surgical tumour resection, in mice. Our findings suggest that high levels of pANXA2 in association with elevated calcium are present in the microenvironment of most solid cancers. The octapeptide might be broadly useful for selective tumour imaging and for delivering drugs to the edges and to the core of solid tumours.

Quantifying solid tumor margins with fluorescence-guided surgery approaches is a challenge, particularly when using near infrared (NIR) wavelengths due to increased penetration depths. An NIR dual wavelength excitation fluorescence (DWEF) approach was developed that capitalizes on the wavelength-dependent attenuation of light in tissue to determine fluorophore depth. A portable dual wavelength excitation fluorescence imaging system was built and tested in parallel with an NIR tumor-targeting fluorophore in tissue mimicking phantoms, chicken tissue, and in vivo mouse models of breast cancer. The system showed high accuracy in all experiments. The low cost and simplicity of this approach make it ideal for clinical use.

Dynamic thermal imaging has improved bulk tissue characterization, but fails to capture subtle thermal property differences in heterogeneous systems. We present focal dynamic thermal imaging (FDTI), a simple, label-free, and high-resolution technology for delineating tissue heterogeneity. Stimulation of focal regions of absorptive materials with a narrow beam, low power, and low cost 405 nm laser locally perturbs the thermal equilibrium. Measurement of phantoms, ex vivo tissue, and in vivo animal models of cancer reveals finite structures of materials and delineates diseased from healthy tissue. Portable FDTI holds promise to capture the heterogeneous nature of malignant tissue.

LS301 is a promising NIR fluorescence probe for targeting brain glioma cancer. The co-localization of LS31 and 5-ALA induced PpIX in vitro and in vivo enables LS301 to guide PpIX fluorescence image surgery. Moreover, LS301 showed no negative impact on 5-ALA-PpIX PDT outcome in brain glioma cells and mouse tumor model. Therefore, the implementation of LS301 and PpIX has potential to improve tumor resection surgery and PDT treatment for better tumor outcomes.

Molecular imaging (MI) plays a critical role in many aspects of the patient care continuum in modern medicine. With increasingly high-resolution systems, multimodal imaging platforms, and large datasets generated by modern MI methods, it has become imperative to develop new approaches to store, process, and visualize information. Adaptation of virtual reality (VR) and augmented reality (AR) to MI is an emerging trend for visualizing MI information in an intuitive format that can accommodate the growing complexity and volume of multimodal molecular information. This chapter reviews VR and AR technologies currently used to visualize medical images. It begins with a brief overview of VR and AR technologies, followed by a description of available hardware and image visualization approaches for these platforms. The chapter further describes the clinical applications of VR and AR in medical imaging and highlights existing challenges for wider adoption of these technologies before concluding with anticipated future directions.

Evolution from static to dynamic label-free thermal imaging has improved bulk tissue characterization, but fails to capture subtle thermal properties in heterogeneous systems. Here, we report a label-free, high speed, and high-resolution platform technology, focal dynamic thermal imaging (FDTI), for delineating material patterns and tissue heterogeneity. Stimulation of focal regions of thermally responsive systems with a narrow beam, low power, and low cost 405 nm laser perturbs the thermal equilibrium. Capturing the dynamic response of 3D printed phantoms, ex vivo biological tissue, and in vivo mouse and rat models of cancer with a thermal camera reveals material heterogeneity and delineates diseased from healthy tissue. The intuitive and non-contact FDTI method allows for rapid interrogation of suspicious lesions and longitudinal changes in tissue heterogeneity with high-resolution and large field of view. Portable FDTI holds promise as a clinical tool for capturing subtle differences in heterogeneity between malignant, benign, and inflamed tissue.

Antibody-based proteomics is an enabling technology that has significant implications for cancer biomarker discovery, diagnostic screening, prognostic and pharmacodynamic evaluation of disease state, and targeted therapeutics. Quantum dot based fluoro-immunoconjugates possess promising features toward realization of this goal such as high photostability, brightness, and multispectral tunability. However, current strategies to generate such conjugates are riddled with complications such as improper orientation of antigen binding sites of the antibody, aggregation, and stability issues. We report a facile yet effective strategy to conjugate anti-epidermal growth factor receptor (EGFR) antibody to quantum dots using copper-free click reaction, and compared them to similar constructs prepared using traditional strategies such as succinimidyl-4-(N-maleimidomethyl) cyclohexane-1-carboxylate (SMCC) and biotin-streptavidin schemes. The Fc and Fab regions of the conjugates retain their binding potential, compared to those generated through the traditional schemes. We further applied the conjugates in testing a novel microsphere array device designed to carry out sensitive detection of cancer biomarkers through fluoroimmunoassays. Using purified EGFR, we determined the limit of detection of the microscopy centric system to be 12.5 ng/mL. The biological assay, in silico, was successfully tested and validated by using tumor cell lysates, as well as human serum from breast cancer patients, and the results were compared to normal serum. A pattern consistent with established clinical data was observed, which further validates the effectiveness of the developed conjugates and its successful implementation both in vitro as well as in silico fluoroimmunoassays. The results suggest the potential development of a high throughput in silico paradigm for predicting the class of patient cancer based on EGFR expression levels relative to normal reference levels in blood.

The inability to visualize the patient and surgical site directly, limits the use of current near infrared fluorescence-guided surgery systems for real-time sentinel lymph node biopsy and tumor margin assessment. We evaluated an optical see-through goggle augmented imaging and navigation system (GAINS) for near-infrared, fluorescence-guided surgery. Tumor-bearing mice injected with a near infrared cancer-targeting agent underwent fluorescence-guided, tumor resection. Female Yorkshire pigs received hind leg intradermal indocyanine green injection and underwent fluorescence-guided, popliteal lymph node resection. Four breast cancer patients received 99mTc-sulfur colloid and indocyanine green retroareolarly before undergoing sentinel lymph node biopsy using radioactive tracking and fluorescence imaging. Three other breast cancer patients received indocyanine green retroareolarly before undergoing standard-of-care partial mastectomy, followed by fluorescence imaging of resected tumor and tumor cavity for margin assessment. Using near-infrared fluorescence from the dyes, the optical see-through GAINS accurately identified all mouse tumors, pig lymphatics, and four pig popliteal lymph nodes with high signal-to-background ratio. In 4 human breast cancer patients, 11 sentinel lymph nodes were identified with a detection sensitivity of 86.67 ± 0.27% for radioactive tracking and 100% for GAINS. Tumor margin status was accurately predicted by GAINS in all three patients, including clear margins in patients 1 and 2 and positive margins in patient 3 as confirmed by paraffin-embedded section histopathology. The optical see-through GAINS prototype enhances near infrared fluorescence-guided surgery for sentinel lymph node biopsy and tumor margin assessment in breast cancer patients without disrupting the surgical workflow in the operating room.

A 110 × 64 pixel CMOS image sensor integrated with pixelated spectral interference filters is used for image guided surgery. The sensor operates with short exposure times for capturing visible light using blue, green, and red pixelated interference filters, and long exposure time for capturing near-infrared fluorescence with a co-located, pixelated NIR filter. The sensor was used to successfully detect the fluorescence of ICG accumulation in sentinel lymph node on patients with breast cancer.

Inspired by the visual system of the morpho butterfly, we have designed, fabricated, tested and clinically translated an ultra-sensitive, light weight and compact imaging sensor capable of simultaneously capturing near infrared (NIR) and visible spectrum information. The visual system of the morpho butterfly combines photosensitive cells with spectral filters at the receptor level. The spectral filters are realized by alternating layers of high and low dielectric constant, such as air and cytoplasm. We have successfully mimicked this concept by integrating pixelated spectral filters, realized by alternating silicon dioxide and silicon nitrate layers, with an array of CCD detectors. There are four different types of pixelated spectral filters in the imaging plane: red, green, blue and NIR. The high optical density (OD) of all spectral filters (OD>4) allow for efficient rejections of photons from unwanted bands. The single imaging chip weighs 20 grams with form factor of 5mm by 5mm. The imaging camera is integrated with a goggle display system. A tumor targeted agent, LS301, is used to identify all spontaneous tumors in a transgenic PyMT murine model of breast cancer. The imaging system achieved sensitivity of 98% and selectivity of 95%. We also used our imaging sensor to locate sentinel lymph nodes (SLNs) in patients with breast cancer using indocyanine green tracer. The surgeon was able to identify 100% of SLNs when using our bio-inspired imaging system, compared to 93% when using information from the lymphotropic dye and 96% when using information from the radioactive tracer.

Image-guided surgery can enhance cancer treatment by decreasing, and ideally eliminating, positive tumor margins and iatrogenic damage to healthy tissue. Current state-of-the-art near-infrared fluorescence imaging systems are bulky and costly, lack sensitivity under surgical illumination, and lack co-registration accuracy between multimodal images. As a result, an overwhelming majority of physicians still rely on their unaided eyes and palpation as the primary sensing modalities for distinguishing cancerous from healthy tissue. Here we introduce an innovative design, comprising an artificial multispectral sensor inspired by the Morpho butterfly’s compound eye, which can significantly improve image-guided surgery. By monolithically integrating spectral tapetal filters with photodetectors, we have realized a single-chip multispectral imager with 1000 × higher sensitivity and 7 × better spatial co-registration accuracy compared to clinical imaging systems in current use. Preclinical and clinical data demonstrate that this technology seamlessly integrates into the surgical workflow while providing surgeons with real-time information on the location of cancerous tissue and sentinel lymph nodes. Due to its low manufacturing cost, our bio-inspired sensor will provide resource-limited hospitals with much-needed technology to enable more accurate value-based health care.

A wearable all-plastic imaging system for real-time fluorescence image-guided surgery is presented. The compact size of the system is especially suitable for applications in the operating room. The system consists of a dual-mode imaging system, see-through goggle, autofocusing, and auto-contrast tuning modules. The paper will discuss the system design and demonstrate the system performance.

The inability to identify microscopic tumors and assess surgical margins in real-time during oncologic surgery leads to incomplete tumor removal, increases the chances of tumor recurrence and necessitates costly repeat surgery. To overcome these challenges, we have developed a wearable goggle augmented imaging and navigation system (GAINS) that can provide accurate intraoperative visualization of tumors and sentinel lymph nodes in real-time without disrupting normal surgical workflow. GAINS projects both near-infrared fluorescence from tumors and the natural color images of tissue onto a head-mounted display without latency. Aided by tumor-targeted contrast agents, the system detected tumors in subcutaneous and metastatic mouse models with high accuracy (sensitivity = 100%, specificity = 98% ± 5% standard deviation). Human pilot studies in breast cancer and melanoma patients using a near-infrared dye show that the GAINS detected sentinel lymph nodes with 100% sensitivity. Clinical use of the GAINS to guide tumor resection and sentinel lymph node mapping promises to improve surgical outcomes, reduce rates of repeat surgery and improve the accuracy of cancer staging.

Near infra-red (NIR) fluorescence imaging offers high sensitivity and specificity for cancer detection and tumor margin localization. In this paper, a compact and wearable NIR fluorescence imaging system (FIS) for image guided surgery is presented. The proposed FIS captures 27 frames per second of both NIR and visible spectrum data, allowing for minimum detection of 10nM of ICG concentration. The design tradeoffs of the complete system are carefully analyzed. The performance of the proposed FIS is evaluated under both sensitivity test and experimental mouse study. The imaging system is currently used in clinical trials for imaging sentinel lymph nodes in breast cancer and melanoma patients.

Near-infrared (NIR) fluorescent imaging system has been widely used for intraoperative image-guided application. In this paper, we present performance comparison from three compact NIR fluorescence imaging system prototypes with goggle display that we developed for intraoperative guidance: threshold detection based two camera system, feature matching based three cameras system and miniature beam-splitter single camera system. Their performance is evaluated according to sensitivity regarding different ICG concentrations, accuracy of image overlay between NIR-visible channels, compactness and practicability in intraoperative use. The comparison results show great potentials of using these NIR fluorescence imaging systems to improve user experience and surgical outcomes in intraoperative use.

Near infrared (NIR) fluorescence imaging has shown great potential for various clinical procedures, including intraoperative image guidance. However, existing NIR fluorescence imaging systems either have a large footprint or are handheld, which limits their usage in intraoperative applications. We present a compact NIR fluorescence imaging system (NFIS) with an image overlay solution based on threshold detection, which can be easily integrated with a goggle display system for intraoperative guidance. The proposed NFIS achieves compactness, light weight, hands-free operation, high-precision superimposition, and a real-time frame rate. In addition, the miniature and ultra-lightweight light-emitting diode tracking pod is easy to incorporate with NIR fluorescence imaging. Based on experimental evaluation, the proposed NFIS solution has a lower detection limit of 25 nM of indocyanine green at 27 fps and realizes a highly precise image overlay of NIR and visible images of mice in vivo. The overlay error is limited within a 2-mm scale at a 65-cm working distance, which is highly reliable for clinical study and surgical use.

We have developed a wearable, fluorescence goggle based system for intraoperative imaging of tumors and image guidance in oncologic surgery. Our system can detect fluorescence from cancer selective near infra-red (NIR) contrast agent, facilitating intraoperative visualization of surgical margins and tumors otherwise not apparent to the surgeon. The fluorescence information is displayed directly to the head mounted display (HMD) of the surgeon in real time, allowing unhindered surgical procedure under image guidance. This system has the potential of improving surgical outcomes in oncologic surgery and reduce the chances of cancer recurrence.

Medical imaging plays a critical role in cancer diagnosis and planning. Many of these patients rely on surgical intervention for curative outcomes. This requires a careful identification of the primary and microscopic tumors, and the complete removal of cancer. Although there have been efforts to adapt traditional imaging modalities for intraoperative image guidance, they suffer from several constraints such as large hardware footprint, high operation cost, and disruption of the surgical workflow. Because of the ease of image acquisition, relatively low cost devices and intuitive operation, optical imaging methods have received tremendous interests for use in real-time image-guided surgery. To improve imaging depth under low interference by tissue autofluorescence, many of these applications utilize light in the near-infra red (NIR) wavelengths, which is invisible to human eyes. With the availability of a wide selection of tumor-avid contrast agents, advancements in imaging sensors, electronic and optical designs, surgeons are able to combine different attributes of NIR optical imaging techniques to improve treatment outcomes. The emergence of diverse commercial and experimental image guidance systems, which are in various stages of clinical translation, attests to the potential high impact of intraoperative optical imaging methods to improve speed of oncologic surgery with high accuracy and minimal margin positivity.

Optical imaging enables real-time visualization of intrinsic and exogenous contrast within biological tissues. Applications in human medicine have demonstrated the power of fluorescence imaging to enhance visualization in dermatology, endoscopic procedures, and open surgery. Although few optical contrast agents are available for human medicine at this time, fluorescence imaging is proving to be a powerful tool in guiding medical procedures. Recently, intraoperative detection of fluorescent molecular probes that target cell-surface receptors has been reported for improvement in oncologic surgery in humans. We have developed a novel system, optical projection of acquired luminescence (OPAL), to further enhance real-time guidance of open oncologic surgery. In this method, collected fluorescence intensity maps are projected onto the imaged surface rather than via wall-mounted display monitor. To demonstrate proof-of-principle for OPAL applications in oncologic surgery, lymphatic transport of indocyanine green was visualized in live mice for intraoperative identification of sentinel lymph nodes. Subsequently, peritoneal tumors in a murine model of breast cancer metastasis were identified using OPAL after systemic administration of a tumor-selective fluorescent molecular probe. These initial results clearly show that OPAL can enhance adoption and ease-of-use of fluorescence imaging in oncologic procedures relative to existing state-of-the-art intraoperative imaging systems.

We have developed a near-infrared (NIR) fluorescence goggle system based on the complementary metal–oxide–semiconductor active pixel sensor imaging and see-through display technologies. The fluorescence goggle system is a compact wearable intraoperative fluorescence imaging and display system that can guide surgery in real time. The goggle is capable of detecting fluorescence of indocyanine green solution in the picomolar range. Aided by NIR quantum dots, we successfully used the fluorescence goggle to guide sentinel lymph node mapping in a rat model. We further demonstrated the feasibility of using the fluorescence goggle in guiding surgical resection of breast cancer metastases in the liver in conjunction with NIR fluorescent probes. These results illustrate the diverse potential use of the goggle system in surgical procedures.

Real-time image guidance in the operating room is needed to improve instantaneous surgical decisions. Toward this goal, we utilized a new fluorescence goggle system and a near-infrared fluorescent dye approved for human use, indocyanine green, to demonstrate the feasibility of detecting liver tumors intraoperatively. The fluorescence goggle provided successful imaging of multifocal breast cancer metastases in mouse liver. Diffused tumor deposits as small as 0.8 mm in diameter were detected, which were not obvious without the fluorescence goggle. A combination of surface-weighted fluorescence imaging and deep tissue-sensitive ultrasound imaging allowed comprehensive image guidance with the fluorescence goggle system for tumor resection in a rabbit VX2 liver metastasis model. This multimodal detection and guided surgical intervention strategy using ultrasonic imaging and real-time intraoperative fluorescence guidance is a promising and innovative technology platform for improving surgical outcome of human patients with primary or metastatic liver cancer.

We have developed a fluorescence goggle device for intraoperative oncologic imaging. With our system design, the surgeon can directly visualize the fluorescence information from the eyepieces in real time without any additional monitor, which can improve one's coordination and surgical accuracy. In conjunction with targeting fluorescent dyes, the goggle device can successfully detect tumor margins and small nodules that are not obvious to naked eye. This can potentially decrease the incidence of incomplete resection.

The near-infrared (NIR) fluorescence signal in the 700 to 900 nm from molecular probes used in fluorescence image-guided surgery (FIGS) is usually weak compared to the NIR component from white light-emitting diode surgical light, which is typically switched off during FIGS to enhance the molecular fluorescence contrast of the image. We propose a simple solution to this critical issue in FIGS by removing NIR light from surgical light with a low cost commercial 3M cool mirror film 330.

In this Letter, we present a novel imaging concept that a single imaging system can image different spectral bands with different aperture sizes. It is achieved by using a filter with different transmitted spectral bands in different annular rings as the aperture stop. This concept will enable more efficient system configurations and practical clinical applications. We have demonstrated this concept with a dual-mode near-infrared fluorescence image guided surgical system.