Search

Phase recovery (PR) refers to calculating the phase of the light field from its intensity measurements. As exemplified from quantitative phase imaging and coherent diffraction imaging to adaptive optics, PR is essential for reconstructing the refractive index distribution or topography of an object and correcting the aberration of an imaging system. In recent years, deep learning (DL), often implemented through deep neural networks, has provided unprecedented support for computational imaging, leading to more efficient solutions for various PR problems. In this review, we first briefly introduce conventional methods for PR. Then, we review how DL provides support for PR from the following three stages, namely, pre-processing, in-processing, and post-processing. We also review how DL is used in phase image processing. Finally, we summarize the work in DL for PR and outlook on how to better use DL to improve the reliability and efficiency in PR. Furthermore, we present a live-updating resource (https://github.com/kqwang/phase-recovery) for readers to learn more about PR., Comment: 82 pages, 32 figures

Optical computing systems provide high-speed and low-energy data processing but face deficiencies in computationally demanding training and simulation-to-reality gaps. We propose a gradient-based model-free optimization (G-MFO) method based on a Monte Carlo gradient estimation algorithm for computationally efficient in situ training of optical computing systems. This approach treats an optical computing system as a black box and back-propagates the loss directly to the optical computing weights' probability distributions, circumventing the need for a computationally heavy and biased system simulation. Our experiments on diffractive optical computing systems show that G-MFO outperforms hybrid training on the MNIST and FMNIST datasets. Furthermore, we demonstrate image-free and high-speed classification of cells from their marker-free phase maps. Our method's model-free and high-performance nature, combined with its low demand for computational resources, paves the way for accelerating the transition of optical computing from laboratory demonstrations to practical, real-world applications.

Single neural networks have achieved simultaneous phase retrieval with aberration compensation and phase unwrapping in off-axis Quantitative Phase Imaging (QPI). However, when designing the phase retrieval neural network architecture, the trade-off between computation latency and accuracy has been largely neglected. Here, we propose Neural Architecture Search (NAS) generated Phase Retrieval Net (NAS-PRNet), which is an encoder-decoder style neural network, automatically found from a large neural network architecture search space. The NAS scheme in NAS-PRNet is modified from SparseMask, in which the learning of skip connections between the encoder and the decoder is formulated as a differentiable NAS problem, and the gradient decent is applied to efficiently search the optimal skip connections. Using MobileNet-v2 as the encoder and a synthesized loss that incorporates phase reconstruction and network sparsity losses, NAS-PRNet has realized fast and accurate phase retrieval of biological cells. When tested on a cell dataset, NAS-PRNet has achieved a Peak Signal-to-Noise Ratio (PSNR) of 36.1 dB, outperforming the widely used U-Net and original SparseMask-generated neural network. Notably, the computation latency of NAS-PRNet is only 31 ms which is 12 times less than U-Net. Moreover, the connectivity scheme in NAS-PRNet, identified from one off-axis QPI system, can be well fitted to another with different fringe patterns.

Two-photon photopolymerization (TPP) has recently become a popular method for the fabrication of three-dimensional (3D) micro- and nanostructures. The reproduction fidelity of the designed micro- and nanostructures is influenced by experimental writing conditions, including laser power, exposure time, etc. To determine the appropriate writing parameters, characterization of morphological features and surface roughness during the experiment is needed. Traditional characterization methods for TPP, e.g., scanning electron microscopy and atomic force microscopy, have limited speed and cannot study internal structures without invasive approaches. Optical diffraction tomography (ODT) is an emerging label-free 3D imaging technique based on reconstructing the object's 3D refractive index (RI) distribution with diffraction-limited resolution. Here, we propose a non-invasive solution to fully characterize the TPP-fabricated structures using a high-speed ODT technique, which can eliminate the need for complex sample preparation, such as fluorescence labelling or metal-coating, and achieve a full 3D measurement time of 6 ms. By visualizing and studying different TPP-fabricated structures, including embedded spirals and cubes, via the ODT system, the fabrication quality, including 3D morphological features, exposure levels, and surface roughness, can be examined quantitatively. The results suggest our method can effectively improve the fabrication quality and reproducibility of TPP, generating impacts on the nanofabrication community.

The physical properties of two-dimensional (2D) materials may drastically vary with their thickness profiles. Current thickness profiling methods for 2D material (e.g., atomic force microscopy and ellipsometry) are limited in measurement throughput and accuracy. Here we present a novel high-speed and high-precision thickness profiling method, termed Transmission-Matrix Quantitative Phase Profilometry (TM-QPP). In TM-QPP, picometer-level optical pathlength sensitivity is enabled by extending the photon shot-noise limit of a high sensitivity common-path interferometric microscopy technique, while accurate thickness determination is realized by developing a transmission-matrix model that accounts for multiple refractions and reflections of light at sample interfaces. Using TM-QPP, the exact thickness profiles of monolayer and few-layered 2D materials (e.g., MoS2, MoSe2 and WSe2) are mapped over a wide field of view within seconds in a contact-free manner. Notably, TM-QPP is also capable of spatially resolving the number of layers of few-layered 2D materials.

Named Entity Recognition task is one of the core tasks of information extraction. Word ambiguity and word abbreviation are important reasons for the low recognition rate of named entities. In this paper, we propose a novel named entity recognition model WCL-BBCD (Word Contrastive Learning with BERT-BiLSTM-CRF-DBpedia), which incorporates the idea of contrastive learning. The model first trains the sentence pairs in the text, calculate similarity between sentence pairs, and fine-tunes BERT used for the named entity recognition task according to the similarity, so as to alleviate word ambiguity. Then, the fine-tuned BERT is combined with BiLSTM-CRF to perform the named entity recognition task. Finally, the recognition results are corrected in combination with prior knowledge such as knowledge graphs, so as to alleviate the low-recognition-rate problem caused by word abbreviations. The results of experimentals conducted on the CoNLL-2003 English dataset and OntoNotes V5 English dataset show that our model outperforms other similar models on.

Optical diffraction tomography (ODT) has emerged as a powerful label-free three-dimensional (3D) bioimaging techniques for observing living cells and thin tissue layers. We report a new reflection-mode ODT (rODT) method for imaging thick biological specimens with 500 nm lateral resolution and 1 {\mu}m axial resolution. In rODT, multiple scattering background is rejected through spatio-temporal gating provided by dynamic speckle-field interferometry, while depth-resolved refractive index maps are reconstructed by developing a comprehensive inverse scattering model that also considers specimen-induced aberration. Benefiting from the high-resolution and full-field quantitative imaging capabilities of rODT, we succeeded in imaging red blood cells and quantifying their membrane fluctuations behind a turbid sample with a thickness of 2.8 scattering mean-free-paths. We further realized volumetric imaging of cornea inside an ex vivo rat eye and quantified its optical properties, including mapping the topography of Dua's and Descemet's membrane surfaces on the nanometer scale.

Three-dimensional (3D) image cytometers may significantly improve the cell analysis accuracy to facilitate biological discoveries and clinical diagnosis, but their development is curbed by the low imaging throughput. Here we report SIngle-frame LAbel-free Cell Tomography (SILACT) with diffraction-limited resolution and unprecedented imaging speed of over 10,000 volumes/second. SILACT is built on a unique interferometric microscope with angle-multiplexing illumination and a pre-trained physics-incorporating Deep Neural Network for efficient 3D Refractive Index (RI) reconstruction, from which 3D morphological and biophysical parameters of cells are extracted. With microfluidics and a high-speed camera, SILACT is capable of imaging over 20,000 cells/second and distinguishing different cell species during rapid measurements of large cell quantities, as well as visualizing shear-induced 3D transient deformation of red blood cells on a sub-millisecond scale., Comment: Baoliang Ge, Yanping He, Mo Deng contributed equally to this work

Polarization light microscopes are powerful tools for probing molecular order and orientation in birefringent materials. While a multitude of polarization light microscopy techniques are often used to access steady-state properties of birefringent samples, quantitative measurements of the molecular orientation dynamics on the millisecond time scale have remained a challenge. We propose polarized shearing interference microscopy (PSIM), a single-shot quantitative polarization imaging method, for extracting the retardance and orientation angle of the laser beam transmitting through optically anisotropic specimens with complex structures. The measurement accuracy and imaging performances of PSIM are validated by imaging a rotating wave plate and a bovine tendon specimen. We demonstrate that PSIM can quantify the dynamics of a flowing lyotropic chromonic liquid crystal in a microfluidic channel at an imaging speed of 506 frames per second (only limited by the camera frame rate), with a field-of-view of up to $350\times350 \mu m^2$ and a diffraction-limit spatial resolution of $\sim 2\mu m$. We envision that PSIM will find a broad range of applications in quantitative material characterization under dynamical conditions.

Leukocyte differential test is a widely performed clinical procedure for screening infectious diseases. Existing hematology analyzers require labor-intensive work and a panel of expensive reagents. Here we report an artificial-intelligence enabled reagent-free imaging hematology analyzer (AIRFIHA) modality that can accurately classify subpopulations of leukocytes with minimal sample preparation. AIRFIHA is realized through training a two-step residual neural network using label-free images of separated leukocytes acquired from a custom-built quantitative phase microscope. We validated the performance of AIRFIHA in randomly selected test set and cross-validated it across all blood donors. AIRFIHA outperforms current methods in classification accuracy, especially in B and T lymphocytes, while preserving the natural state of cells. It also shows a promising potential in differentiating CD4 and CD8 cells. Owing to its easy operation, low cost, and strong discerning capability of complex leukocyte subpopulations, we envision AIRFIHA is clinically translatable and can also be deployed in resource-limited settings, e.g., during pandemic situations for the rapid screening of infectious diseases.

Phase sensitivity determines the lowest optical path length (OPL) value that can be detected from the noise floor in a quantitative phase microscopy (QPM) system. The temporal phase sensitivity is known to be limited by both photon shot-noise and a variety of noise sources from electronic devices and environment. To beat temporal phase sensitivity limit, we explore different ways to reduce different noise factors in off-axis interferometry-based QPM using laser-illumination. Using a high electron-well-capacity camera, we measured the temporal phase sensitivity values using non-common-path and common-path interferometry based QPM systems under different environmental conditions. A frame summing method and a spatiotemporal filtering method are further used to reduce the noise contributions, thus enabling us to push the overall temporal phase sensitivity to less than 2 picometers.

Close to half of the world population have smartphones, while a typical flagship smartphone today has been integrated with more than 20 smart components and sensors, making a smartphone a highly integrated platform that can potentially mimic the five senses of humans. Recent advancement in achieving high compactness, high performance computing, high flexibility, and multiplexed functionality in smartphones have enabled them for many cutting-edge healthcare applications, such as single-molecule imaging, medical diagnosis, and biosensing, which were conventionally done with bulky and sophisticated devices. Most of the current healthcare applications are developed based on using the photon-sensitive components, such as CMOS sensors, flash & fill lights, lens modules, and LED lights in the screen, leaving the rest of the smart and high-performance sensors rarely explored. In this Perspective, we review recent progresses in advanced sensors in modern smartphones and discuss how those sensors have great, as yet unmet, promise to offer widespread and easy-to-implement solutions to many emerging healthcare applications, including nanoscale sensing, point-of-care testing, pollution monitoring, etc., Comment: 23 pages, 4 figures

Here we report nonlinear focal modulation microscopy (NFOMM) to achieve super-resolution imaging. Abandoning the previous persistence on minimizing the size of Gaussian emission pattern by directly narrowing (e.g. Minimizing the detection pinhole in Airyscan, Zeiss) or by indirectly peeling its outer profiles (e.g., Depleting the outer emission region in STED, stimulated emission microscopy) in pointwise scanning scenarios, we stick to a more general basis------ maximizing the system frequency shifting ability. In NFOMM, we implement a nonlinear focal modulation by applying phase modulations with high-intensity illumination, thereby extending the effective spatial-frequency bandwidth of the imaging system for reconstructing super-resolved images. NFOMM employs a spatial light modulator (SLM) for assisting pattern-modulated pointwise scanning, making the system single beam path while achieving a transverse resolution of 60 nm on imaging fluorescent nanoparticles. While exploring a relatively simple and flexible system, the imagingperformance of NFOMM is comparable with STED as evidenced in imaging nuclear pore complexes, demonstrating NFOMM is a suitable observation tool for fundamental studies in biology. Since NFOMM is implemented as an add-on module to an existing laser scanning microscope and easy to be aligned, we anticipate it will be adopted rapidly by the biological community., Comment: 5 Figures, under review in PRL