Search

Although accurate detection of breast cancer still poses significant challenges, deep learning (DL) can support more accurate image interpretation. In this study, we develop a highly robust DL model based on combined B-mode ultrasound (B-mode) and strain elastography ultrasound (SE) images for classifying benign and malignant breast tumors. This study retrospectively included 85 patients, including 42 with benign lesions and 43 with malignancies, all confirmed by biopsy. Two deep neural network models, AlexNet and ResNet, were separately trained on combined 205 B-mode and 205 SE images (80% for training and 20% for validation) from 67 patients with benign and malignant lesions. These two models were then configured to work as an ensemble using both image-wise and layer-wise and tested on a dataset of 56 images from the remaining 18 patients. The ensemble model captures the diverse features present in the B-mode and SE images and also combines semantic features from AlexNet and ResNet models to classify the benign from the malignant tumors. The experimental results demonstrate that the accuracy of the proposed ensemble model is 90%, which is better than the individual models and the model trained using B-mode or SE images alone. Moreover, some patients that were misclassified by the traditional methods were correctly classified by the proposed ensemble method. The proposed ensemble DL model will enable radiologists to achieve superior detection efficiency owing to enhance classification accuracy for breast cancers in ultrasound (US) images.

Significance Multimodal imaging based on optics and ultrasound can provide guide images and complementary structural and functional information, thus improving the accuracy of medical diagnosis and treatment monitoring. However, because conventional ultrasound transducers are opaque, in multimodal imaging with optics, the optical devices must be placed off-axis from the ultrasound transducer. This off-axis arrangement is prone to misalignment, adds complexity and bulk to the system, and can result in a low signal-to-noise-ratio. Here, we present a transparent ultrasound transducer at the heart of a quadruple fusion imaging system that seamlessly integrates ultrasound imaging, photoacoustic imaging, optical coherence tomography, and fluorescence imaging, and we demonstrate the system’s use in imaging responses to both ophthalmologic injuries and oncologic diseases., Ultrasound and optical imagers are used widely in a variety of biological and medical applications. In particular, multimodal implementations combining light and sound have been actively investigated to improve imaging quality. However, the integration of optical sensors with opaque ultrasound transducers suffers from low signal-to-noise ratios, high complexity, and bulky form factors, significantly limiting its applications. Here, we demonstrate a quadruple fusion imaging system using a spherically focused transparent ultrasound transducer that enables seamless integration of ultrasound imaging with photoacoustic imaging, optical coherence tomography, and fluorescence imaging. As a first application, we comprehensively monitored multiparametric responses to chemical and suture injuries in rats’ eyes in vivo, such as corneal neovascularization, structural changes, cataracts, and inflammation. As a second application, we successfully performed multimodal imaging of tumors in vivo, visualizing melanomas without using labels and visualizing 4T1 mammary carcinomas using PEGylated gold nanorods. We strongly believe that the seamlessly integrated multimodal system can be used not only in ophthalmology and oncology but also in other healthcare applications with broad impact and interest.

Multi-modal imaging technique has significantly spotlighted since it can provide a variety of information by combining the complementary merits of several single-modal imaging. In particular, in order to compensate for each shortcoming and improve image quality, an integrated optical and ultrasonic imaging system is being actively researched. However, the non-transparency of the ultrasound transducer made it difficult to integrate the optical and ultrasound imaging system. In previous study, we introduced the transparent ultrasound transducer (TUT) and dual-modal photoacoustic imaging (PAI)/ultrasound imaging (USI) system using the TUT. In this study, we present the multi-modal imaging system integrated with PA, US and optical coherence tomography (OCT). OCT has the advantage of acquiring anatomical information at optical resolution under subsurface and transparent media. To explore the usefulness of the multi-modal imaging system, we have successfully performed in vivo animal experiments: 1) eye imaging experiments and 2) subcutaneous melanoma imaging. In PAI, blood vessels and melanoma are clearly visualized. In OCT, the morphological information in shallow depth are observed in detail. In USI, the melanoma boundary and surrounding tissues are clearly confirmed. These results show that TUT based multi-modal imaging system can serve as a comprehensive in various applications.

Pressure ulcer (PU) formation is a common problem among patients confined to bed or restricted to wheelchairs. The ulcer forms when the affected skin and underlying tissues go through repeated cycles of ischemia and reperfusion, leading to inflammation. This theory is evident by intravital imaging studies performed in immune cell‐specific, fluorescent reporter mouse skin with induced ischemia‐reperfusion (I‐R) injuries. However, traditional confocal or multi‐photon microscopy cannot accurately monitor the progression of vascular reperfusion by contrast agents, which leaks into the interstitium under inflammatory conditions. Here, we develop a dual‐wavelength micro electro mechanical system (MEMS) scanning based optical resolution photoacoustic microscopy (OR‐PAM) system for continuous label‐free functional imaging of vascular reperfusion in an IR mouse model. This MEMS‐OR‐PAM system provides fast scanning speed for concurrent dual‐wavelength imaging, which enables continuous monitoring of the reperfusion process. During reperfusion, the revascularization of blood vessels and the oxygen saturation (sO_2) changes in both arteries and Illustration of the vascular reperfusion of the ear skin. veins are recorded, from which the local oxygen extraction ratios of the ischemic tissue and the unaffected tissue can be quantified. Our MEMS‐OR‐PAM system provides novel perspectives to understand the I‐R injuries. It solves the problem of dynamic label‐free functional monitoring of the vascular reperfusion at high spatial resolution.

The 2019 novel coronavirus (COVID-19) has spread rapidly all over the world. The standard test for screening COVID-19 patients is the polymerase chain reaction test. As this method is time consuming, as an alternative, chest X-rays may be considered for quick screening. However, specialization is required to read COVID-19 chest X-ray images as they vary in features. To address this, we present a multi-channel pre-trained ResNet architecture to facilitate the diagnosis of COVID-19 chest X-ray. Three ResNet-based models were retrained to classify X-rays in a one-against-all basis from (a) normal or diseased, (b) pneumonia or non-pneumonia, and (c) COVID-19 or non-COVID19 individuals. Finally, these three models were ensembled and fine-tuned using X-rays from 1579 normal, 4245 pneumonia, and 184 COVID-19 individuals to classify normal, pneumonia, and COVID-19 cases in a one-against-one framework. Our results show that the ensemble model is more accurate than the single model as it extracts more relevant semantic features for each class. The method provides a precision of 94% and a recall of 100%. It could potentially help clinicians in screening patients for COVID-19, thus facilitating immediate triaging and treatment for better outcomes.

We present a water-proof Microelectromechanical systems (MEMS) based scanning optical resolution Photoacoustic Microscopy (OR-PAM) system and its application in glioma tumor mouse model study. The presented OR-PAM system has high optical resolution (~3 μm) and high scanning speed (up to 50 kHz A-scan rate), which is ideal for cerebral vascular imaging. In this study, the mice with glioma tumor are treated with vascular disrupting agent (VDA). OR-PAM system is utilized to image the cerebral with the whole skull intact before and after the injection of VDA. By image registration, the response of every single blood vessel can be traced. This will provide us deeper understanding of the drug effect.

Vulnerable plaque is the one of the leading causes of cardiovascular disease occurrence. However, conventional intravascular imaging techniques suffer from difficulty in finding vulnerable plaque due to limitation such as lack of physiological information, imaging depth, and depth sensitivity. Therefore, new techniques are needed to help determine the vulnerability of plaque, Thermal strain imaging (TSI) is an imaging technique based on ultrasound (US) wave propagation speed that varies with temperature of medium. During temperature increase, strain occurs in the medium and its variation tendency is depending on the type of tissue, which makes it possible to use for tissue differentiation. Here, we demonstrate laser-induced photo-thermal strain imaging (pTSI) to differentiate tissue using an intravascular ultrasound (IVUS) catheter and a 1210-nm continuous-wave laser for heating lipids intensively. During heating, consecutive US images were obtained from a custom-made phantom made of porcine fat and gelatin. A cross correlation-based speckle-tracking algorithm was then applied to calculate the strain of US images. In the strain images, the positive strain produced in lipids (porcine fat) was clearly differentiated from water-bearing tissue (gelatin). This result shows that laser-induced pTSI could be a new method to distinguish lipids in the plaque and can help to differentiate vulnerability of plaque.

Visualizing ocular vasculature is important in clinical ophthalmology because ocular circulation abnormalities are early signs of ocular diseases. Photoacoustic microscopy (PAM) images the ocular vasculature without using exogenous contrast agents, avoiding associated side effects. Moreover, 3D PAM images can be useful in understanding vessel-related eye disease. However, the complex structure of the multi-layered vessels still present challenges in evaluating ocular vasculature. In this study, we demonstrate a new method to evaluate blood circulation in the eye by combining in vivo PAM imaging and an ocular surface estimation method based on a machine learning algorithm: a random sample consensus algorithm. By using the developed estimation method, we were able to visualize the PA ocular vascular image intuitively and demonstrate layer-by-layer analysis of injured ocular vasculature. We believe that our method can provide more accurate evaluations of the eye circulation in ophthalmic applications.

Optical resolution photoacoustic microscopy (OR-PAM) is a non-invasive, label-free method of in vivo imaging with microscopic resolution and high optical contrast. Based on intrinsic contrasts, OR-PAM has expanded to include in vivo vessel imaging, flow cytometry, physiological parameter analysis, and single-cell characterization. However, since conventional OR-PAM systems have a fixed tabletop configuration, a large system size, and slow imaging speed, their use in preclinical and clinical studies remains limited. In this study, using microelectromechanical systems (MEMS) technology, we developed a handheld PAM probe with a high signal-to-noise ratio and image rate. To enable broader application of the OR-PAM system, we reduced its size and combined its fast scanning capabilities into a small handheld probe that uses a 2-axis waterproof MEMS scanner (2A-WP-MEMS scanner). All acoustical, optical, and mechanical components are integrated into a single probe with a diameter of 17 mm and a weight of 162 g. This study shows phantom and in vivo images of various samples acquired with the probe, including carbon fibers, electrospun microfibers, and the ear, iris, and brain of a living mouse. In particular, this study investigated the possibility of clinical applications for melanoma diagnosis by imaging the boundaries and morphology of a human mole.

Ocular chemical damage may induce limbal vessel ischemia and neovascularization, but the pathophysiology of the disease is not completely known. To observe changes in blood vessels after alkaline burn, we monitored the anterior segment and choroidal vasculature using a photoacoustic microscope (OR-PAM). We were able to observe not only the iris blood vessels but also the choroidal vessels under the sclera, which were difficult to be observed with conventional photographs. After alkali burning, we observed neovascularization and limbal ischemia and successfully tracked changes in vasculature during the 7-day healing process. We also used the RANdom SAmple Consensus (RANSAC) method to segment the abnormally generated blood vessels in the cornea by detecting the eyeball surface and successfully visualize the distance from each PA signal to the center of the eye. We believe that photoacoustic imaging has an important potential to reveal the pathophysiology of limb ischemia and neovascularization.

The use of an optical resolution photoacoustic microscopy (OR-PAM) system to evaluate the vascular disruptive effect of combretastatin A4 Phosphate (CA4P) on a murine orthotopic glioma with intact skull is described here. Second generation optical-resolution photoacoustic microscopy scanner with a 532 nm pulsed diode-pumped solid-state laser that specifically matches the absorption maximum of hemoglobin in tissues was used to image orthotopic glioma inoculated in mouse brain. Two-dimensional maps of brain vasculature with a lateral resolution of 5 μm and a depth of 700 μm at a field of view 5 × 4 mm were acquired on normal brain and glioma brain. Longitudinal imaging of the brain pre- and post-administration of CA4P, a FDA approved drug for solid tumors, enabled the monitoring of hemodynamic changes in tumor vasculature revealing the well documented vascular shutdown and recovery associated with this drug. Our study marks the beginning of potential prospects of this technology as an imaging tool for preclinical and clinical study of pathologies characterized by changes in the vasculature.

We propose an improved synthetic aperture focusing technique based on a delaymultiply- and-sum for acoustic-resolution photoacoustic microscopy. A lateral resolution in out-offocus regions is much improved, thereby extending the imaging focal region.

Vulnerable plaques are the major cause of cardiovascular disease, but they are difficult to detect with conventional intravascular imaging techniques. Techniques are needed to identify plaque vulnerability based on the presence of lipids in plaque. Thermal strain imaging (TSI) is an imaging technique based on ultrasound (US) wave propagation speed, which varies with the medium temperature. In TSI, the strain that occurs during tissue temperature change can be used for lipid detection because it has a different tendency depending on the type of tissue. Here, we demonstrate photothermal strain imaging (pTSI) using an intravascular ultrasound catheter. pTSI is performed by slightly and selectively heating lipid using a relatively inexpensive continuous laser source. We applied a speckle-tracking algorithm to US B-mode images for strain calculations. As a result, the strain produced in porcine fat was different from the strain produced in water-bearing gelatin phantom, which made it possible to distinguish the two. This suggests that pTSI could potentially be a way of differentiating lipids in coronary artery.