Search

Due to its specificity, fluorescence microscopy has become a quintessential imaging tool in cell biology. However, photobleaching, phototoxicity, and related artifacts continue to limit fluorescence microscopy’s utility. Recently, it has been shown that artificial intelligence (AI) can transform one form of contrast into another. We present phase imaging with computational specificity (PICS), a combination of quantitative phase imaging and AI, which provides information about unlabeled live cells with high specificity. Our imaging system allows for automatic training, while inference is built into the acquisition software and runs in real-time. Applying the computed fluorescence maps back to the quantitative phase imaging (QPI) data, we measured the growth of both nuclei and cytoplasm independently, over many days, without loss of viability. Using a QPI method that suppresses multiple scattering, we measured the dry mass content of individual cell nuclei within spheroids. In its current implementation, PICS offers a versatile quantitative technique for continuous simultaneous monitoring of individual cellular components in biological applications where long-term label-free imaging is desirable., Quantitative phase imaging suffers from a lack of specificity in label-free imaging. Here, the authors introduce Phase Imaging with Computational Specificity (PICS), a method that combines phase imaging with machine learning techniques to provide specificity in unlabeled live cells with automatic training.

Quantitative phase imaging (QPI) has been widely applied in characterizing cells and tissues. Spatial light interference microscopy (SLIM) is a highly sensitive QPI method due to its partially coherent illumination and common path interferometry geometry. However, SLIM’s acquisition rate is limited because of the four-frame phase-shifting scheme. On the other hand, off-axis methods such as diffraction phase microscopy (DPM) allow for single-shot QPI. However, the laser-based DPM system is plagued by spatial noise due to speckles and multiple reflections. In a parallel development, deep learning was proven valuable in the field of bioimaging, especially due to its ability to translate one form of contrast into another. Here, we propose using deep learning to produce synthetic, SLIM-quality, and high-sensitivity phase maps from DPM using single-shot images as the input. We used an inverted microscope with its two ports connected to the DPM and SLIM modules such that we have access to the two types of images on the same field of view. We constructed a deep learning model based on U-net and trained on over 1000 pairs of DPM and SLIM images. The model learned to remove the speckles in laser DPM and overcame the background phase noise in both the test set and new data. The average peak signal-to-noise ratio, Pearson correlation coefficient, and structural similarity index measure were 29.97, 0.79, and 0.82 for the test dataset. Furthermore, we implemented the neural network inference into the live acquisition software, which now allows a DPM user to observe in real-time an extremely low-noise phase image. We demonstrated this principle of computational interference microscopy imaging using blood smears, as they contain both erythrocytes and leukocytes, under static and dynamic conditions.

Significance The high incidence of human male factor infertility suggests a need for examining new ways of evaluating sperm cells. We present an approach that combines label-free imaging and artificial intelligence to obtain nondestructive markers for reproductive outcomes. Our phase-imaging system reveals nanoscale morphological details from unlabeled cells. Deep learning, on the other hand, provides a structural specificity map segmenting with high accuracy the head, midpiece, and tail. Using these binary masks applied to the quantitative phase images, we measure precisely the dry-mass content of each component. Remarkably, we found that the dry-mass ratios represent intrinsic markers with predictive power for zygote cleavage and blastocyst development., The ability to evaluate sperm at the microscopic level, at high-throughput, would be useful for assisted reproductive technologies (ARTs), as it can allow specific selection of sperm cells for in vitro fertilization (IVF). The tradeoff between intrinsic imaging and external contrast agents is particularly acute in reproductive medicine. The use of fluorescence labels has enabled new cell-sorting strategies and given new insights into developmental biology. Nevertheless, using extrinsic contrast agents is often too invasive for routine clinical operation. Raising questions about cell viability, especially for single-cell selection, clinicians prefer intrinsic contrast in the form of phase-contrast, differential-interference contrast, or Hoffman modulation contrast. While such instruments are nondestructive, the resulting image suffers from a lack of specificity. In this work, we provide a template to circumvent the tradeoff between cell viability and specificity by combining high-sensitivity phase imaging with deep learning. In order to introduce specificity to label-free images, we trained a deep-convolutional neural network to perform semantic segmentation on quantitative phase maps. This approach, a form of phase imaging with computational specificity (PICS), allowed us to efficiently analyze thousands of sperm cells and identify correlations between dry-mass content and artificial-reproduction outcomes. Specifically, we found that the dry-mass content ratios between the head, midpiece, and tail of the cells can predict the percentages of success for zygote cleavage and embryo blastocyst formation.

Traditional methods for cell cycle stage classification rely heavily on fluorescence microscopy to monitor nuclear dynamics. These methods inevitably face the typical phototoxicity and photobleaching limitations of fluorescence imaging. Here, we present a cell cycle detection workflow using the principle of phase imaging with computational specificity (PICS). The proposed method uses neural networks to extract cell cycle-dependent features from quantitative phase imaging (QPI) measurements directly. Our results indicate that this approach attains very good accuracy in classifying live cells into G1, S, and G2/M stages, respectively. We also demonstrate that the proposed method can be applied to study single-cell dynamics within the cell cycle as well as cell population distribution across different stages of the cell cycle. We envision that the proposed method can become a nondestructive tool to analyze cell cycle progression in fields ranging from cell biology to biopharma applications.TeaserWe present a non-destructive, high-throughput method for cell cycle detection combining label-free imaging and deep learning.

Efforts to mitigate the COVID-19 crisis revealed that fast, accurate, and scalable testing is crucial for curbing the current impact and that of future pandemics. We propose an optical method for directly imaging unlabeled viral particles and using deep learning for detection and classification. An ultrasensitive interferometric method was used to image four virus types with nanoscale optical path-length sensitivity. Pairing these data with fluorescence images for ground truth, we trained semantic segmentation models based on U-Net, a particular type of convolutional neural network. The trained network was applied to classify the viruses from the interferometric images only, containing simultaneously SARS-CoV-2, H1N1 (influenza-A virus), HAdV (adenovirus), and ZIKV (Zika virus). Remarkably, due to the nanoscale sensitivity in the input data, the neural network was able to identify SARS-CoV-2 vs. the other viruses with 96% accuracy. The inference time for each image is 60 ms, on a common graphic-processing unit. This approach of directly imaging unlabeled viral particles may provide an extremely fast test, of less than a minute per patient. As the imaging instrument operates on regular glass slides, we envision this method as potentially testing on patient breath condensates. The necessary high throughput can be achieved by translating concepts from digital pathology, where a microscope can scan hundreds of slides automatically., Rapid label-free detection of SARS-CoV-2 using phase imaging (spatial light-interference microscopy (SLIM)) with computational specificity. Different virus types captured in SLIM image are detected and classified by 2D U-Net.

We propose synthetic aperture gradient light interference microscopy (SA-GLIM) as a solution to avoid computational complexity in standard Fourier pytchographic microscopy. This new system combines direct phase measurements from GLIM with various illumination angles, and a synthetic aperture reconstruction method, to produce high resolution, large FOV quantitative phase maps. Using a 5× objective lens (NA = 0.15), SA-GLIM generates phase maps with a spatial resolution of 850 nm and FOV approximately 1.7×1.7 mm2. We tested the performance using a mixture of polystyrene beads (1 μm and 3 μm in diameter), and the smaller beads can be easily resolved in the final image. Compared with standard FPM, SA-GLIM records substantially fewer low-resolution images, which makes the data throughput highly efficient.

We demonstrate that live-dead cell assay can be conducted in a label-free manner using quantitative phase imaging and deep learning. We apply the concept of our newly-developed phase imaging with computational specificity (PICS) to digitally stain for the live/dead markers. HeLa cultured mixed with viability fluorescent reagents (ReadyProbes, ThermoFisher) were imaged for 24 hours by spatial light interference microscopy (SLIM) and fluorescent microscopy. Based on the ratio of the two fluorescence signals, semantic segmentation maps were generated to label the state of the cell as either live, injured, or dead. We trained an EfficientNet to infer cell viability from SLIM images with semantic maps as ground truth. Validated on the testing dataset, the trained network reported an F1 score of 73.4%, 97.0%, and 94.3% in identifying live, injured, and dead cells, respectively.

Quantitative Phase Imaging (QPI) has been widely applied in characterizing cells and tissues. Spatial light interference microscopy (SLIM) is a highly sensitive QPI method. However, as a phase-shifting technique, SLIM is limited in acquisition rate to at most 15 fps. On the other hand, Diffraction Phase Microscopy (DPM) is such a method, with the advantage of being common-path. However, laser-based DPM systems are plagued by spatial noise due to speckles and multiple reflections. Here, we propose using deep learning to produce SLIM-quality phase maps from DPM, single shot, images. We constructed a deep learning model based on U-Net and trained on over 1,000 pairs of DPM and SLIM images. From the test set, we observed that the model learned to remove the speckles in DPM and overcame the background phase noise. We implemented the neural network inference into the live acquisition software and allows us to acquire single-shot DPM images and infer from them SLIM images in real time.

In this study, we use phase imaging with computational specificity (PICS) to detect single Adenovirus and SARS-CoV2 particles. These viruses are sub-diffraction particles, with maximum diameter of approximately 120nm, which implies that we cannot fully visualize their internal structure. However, due to the very high spatial sensitivity of SLIM (0.3 nm pathlength), we can detect and localize individual viruses and, furthermore, using deep learning, classify them with high accuracy.

Quantitative phase imaging (QPI), with its capability to capture intrinsic contrast within transparent samples, has emerged as an important imaging method for biomedical research. However, due to its label-free nature, QPI lacks specificity and thus faces limitations in complex cellular systems. In our previous works, we have proposed phase imaging with computational specificity (PICS), a novel AI-enhanced imaging approach that advances QPI by utilizing deep learning for specificity. Here we present that PICS can be applied to study individual cell behavior and cellular dry mass change across different phases of the cell cycle. The cell cycle information is traditionally obtained by fluorescence microscopy with markers like Fluorescence Ubiquitin Cell Cycle Indicator (FUCCI). Our work showed that using deep learning, we can train a neural network to accurately predict the cell cycle phase (G1, S, or G2) for each individual cell.

Efforts to mitigate the COVID-19 crisis revealed that fast, accurate, and scalable testing is crucial for curbing the current impact and that of future pandemics. We propose an optical method for directly imaging unlabeled viral particles and using deep learning for detection and classification. An ultrasensitive interferometric method was used to image four virus types with nanoscale optical pathlength sensitivity. Pairing these data with fluorescence images for ground truth, we trained semantic segmentation models based on U-Net, a particular type of convolutional neural network. The trained network was applied to classify the viruses from the interferometric images only, containing simultaneously SARS-CoV-2, H1N1 (influenza-A), HAdV (adenovirus), and ZIKV (Zika). Remarkably, due to the nanoscale sensitivity in the input data, the neural network was able to identify SARS-CoV-2 vs. the other viruses with 96% accuracy. The inference time for each image is 60 ms, on a common graphic processing unit. This approach of directly imaging unlabeled viral particles may provide an extremely fast test, of less than a minute per patient. As the imaging instrument operates on regular glass slides, we envision this method as potentially testing on patient breath condensates.The necessary high throughput can be achieved by translating concepts from digital pathology, where a microscope can scan hundreds of slides automatically.One Sentence SummaryThis work proposes a rapid (

Microscopic imaging modalities can be classified into two categories: those that form contrast from external agents such as dyes, and label-free methods that generate contrast from the object’s unmodified structure. While label-free methods such as brightfield, phase contrast, or quantitative phase imaging (QPI) are substantially easier to use, as well as non-toxic, their lack of specificity leads many researchers to turn to labels for insights into biological processes, despite limitations due to photobleaching and phototoxicity. The label-free image may contain the structures of interest, but it is often difficult or time-consuming to distinguish these structures from their surroundings. Here we summarize our recent progress in shattering this tradeoff, by using machine learning to perform automated segmentation on label-free, intrinsic contrast, quantitative phase images.

Histological staining of tissue samples is one of the most helpful tools in diagnosing and prognosing various cancers. However, in order to prepare the slide for a histopathologist to examine, the tissue must first undergo a series of time-consuming processes, such as a staining technique to visually differentiate features in the sample. In this study, we use a label-free method to generate a virtually-stained microscopic image using a single spatial light interference microscopy (SLIM) image of an unlabeled tissue sample, therefore eliminating the need for standard histochemical administration. This novel approach will render histopathological practices faster and more cost-effective, while providing medically relevant dry mass information associated with SLIM images.

Fluorescence microscopy has been proven a valid method of classifying sperm with different characteristics such as gender. However, it has been observed that they introduced an increase in oxidative stress as well as undesired bias. We show that spatial light interference microscopy, a QPI method that can reveal the intrinsic contrast of cell structures, is ideal for the study of sperm. To enable high-throughput sperm quality assessment using QPI, we propose a new analysis method based on deep learning and the U-Net architecture. We show that our model can achieve satisfying precision and accuracy and that it can be integrated within our image acquisition software for near real-time analysis.