Search

In this paper, an evolutionary single-pixel imaging (SPI) scheme is proposed to solve combinational optimization problems. SPI is a unique optical imaging technique by replacing the pixelated sensor array in a conventional camera with a single-pixel detector. SPI is conventionally employed for capturing object images or performing image processing tasks. It is a novel attempt to leverage SPI for processing other types of data in addition to images. An Ising machine model is implemented optically with SPI for solving two combinational optimization problems including number partition and graph maximum cut. The binary illumination patterns are encoded based on the spinning states of all the elements and the object image pixel values are encoded to be proportional to the pairwise weighting factors. As the mathematical inner product between object image and illumination pattern, the recorded single-pixel intensity values simulate the Hamiltonian function values. In each iteration, the feedback of single-pixel values is used to update the illumination patterns by selection, crossover and mutation. As the illumination patterns are evolving, an optimal solution of the Hamiltonian function can be finally obtained by our proposed optoelectronic Ising machine scheme.

Multi-angle structured illumination lensless (MASIL) microscopy enables high-resolution image recovery over a large field of view. Successful image recovery of MASIL microscopy, however, relies on an accurate knowledge of the multi-angle illumination. System misalignments and slight deviations from the true illumination angle may result in image artifacts in reconstruction. Here we report a MASIL microscopy system that is robust against illumination misalignment. To calibrate the illumination angles, we design and use a double-sided mask, which is a glass wafer fabricated with a ring-array pattern on the upper surface and a disk-array pattern on the lower surface. As such, the illumination angles can be decoded from the captured images by estimating the relative displacement of the two patterns. We experimentally demonstrate that this system can achieve successful image recovery without any prior knowledge of the illumination angles. The reported approach provides a simple yet robust resolution for wide-field lensless microscopy. It can solve the LED array misalignment problem and calibrate angle-varied illumination for a variety of applications.

Fourier single-pixel imaging (FSI) allows an image to be reconstructed by acquiring the Fourier spectrum of the image using a single-pixel detector. Fast FSI is typically achieved by acquiring a truncated Fourier spectrum, that is, only low-frequency Fourier coefficients are acquired, with the high-frequency coefficients discarded. However, the truncation of the Fourier spectrum leads to undesirable ringing artifacts in the resulting image. Ringing artifacts produce false edges in the image and reduce the image contrast, resulting in image quality degeneration. The artifact is particularly severe in dynamic FSI, where the sampling ratio is generally ultra-low. We propose an effective and fast deringing algorithm to achieve ringing-free fast FSI. The algorithm eliminates ringing artifacts through 2D sub-pixel shifting and preserves image details through image fusion. Both static and dynamic imaging results demonstrate that the proposed method can reconstruct ringing-free images from under-sampled data in FSI. The deringing algorithm not only provides FSI with the capability of fast high-quality single-pixel imaging but also might prove its applicability in other areas, such as Fourier-based data compression algorithms.

Multi-height phase retrieval introduces different object-to-detector distances for obtaining phase diversity measurements. In the acquisition process, the slow-varying phase information, however, cannot be converted to intensity variations for detection. Therefore, the low-frequency contents of the phase profile are lost during acquisition and cannot be properly restored via phase retrieval. Here, we demonstrate the use of a coded image sensor for addressing this challenge in multi-height phase retrieval. In our scheme, we add a coded layer on top of the image sensor for encoding the slow-varying complex wavefronts into intensity variations of the modulated patterns. Inspired by the concept of blind ptychography, we report a reconstruction scheme to jointly recover the complex object and the unknown coded layer using multi-height measurements. With both simulation and experimental results, we show that the recovered phase is quantitative and the slow-varying phase profiles can be properly restored using lensless multi-height measurements. We also show that the image quality using the coded sensor is better than that of a regular image sensor. For demonstrations, we validate the reported scheme with various biospecimens and compare the results to those of regular lensless multi-height phase retrieval. The use of a coded image sensor may enable true quantitative phase imaging for the lensless multi-height, multi-wavelength, and transport-of-intensity equation approaches.

We report a new, to the best of our knowledge, lensless microscopy configuration by integrating the concepts of transverse translational ptychography and defocus multi-height phase retrieval. In this approach, we place a tilted image sensor under the specimen for introducing linearly increasing phase modulation along one lateral direction. Similar to the operation of ptychography, we laterally translate the specimen and acquire the diffraction images for reconstruction. Since the axial distance between the specimen and the sensor varies at different lateral positions, laterally translating the specimen effectively introduces defocus multi-height measurements while eliminating axial scanning. Lateral translation further introduces sub-pixel shift for pixel super-resolution imaging and naturally expands the field of view for rapid whole slide imaging. We show that the equivalent height variation can be precisely estimated from the lateral shift of the specimen, thereby addressing the challenge of precise axial positioning in conventional multi-height phase retrieval. Using a sensor with 1.67 µm pixel size, our low-cost and field-portable prototype can resolve the 690 nm linewidth on the resolution target. We show that a whole slide image of a blood smear with a 120mm^2 field of view can be acquired in 18 s. We also demonstrate accurate automatic white blood cell counting from the recovered image. The reported approach may provide a turnkey solution for addressing point-of-care and telemedicine-related challenges.

ABSTRACTAs an important part of art and culture, ancient murals depict a variety of different artistic images, and these individual images have important research value. For research purposes, it is often important to first determine the type of objects represented in a painting. However, the mural painting environment makes datasets difficult to collect, and long-term exposure leads to underlying features that are not distinct, which makes this task challenging. This study proposes a convolutional neural network model based on the classic AlexNet network model and combines it with feature fusion to automatically classify ancient mural images. Due to the lack of large-scale mural datasets, the model first expands the dataset by applying image enhancement algorithms such as scaling, brightness conversion, noise addition, and flipping; then, it extracts the underlying features (such as fresco edges) shared by the first stage of a dual channel structure. Subsequently, a second-stage deep abstraction is conducted on the features extracted by the first stage using a two-channel network, each of which has a different structure. The obtained characteristics from both channels are merged, and a loss function is constructed to obtain the classification result. This approach improves the model's robustness and feature expression ability. The model achieves an accuracy of 84.24%, a recall rate of 84.15%, and an F1-measure of 84.13% when applied to a constructed mural image dataset. Compared with the AlexNet model and other improved convolutional neural network models, the proposed model improves each evaluation index by approximately 5%, verifying the rationality and effectiveness of the model for automatic mural image classification. The mural classification model proposed in this paper comprehensively considers the influences of network width and depth and can extract rich details from mural images from multiple local channels. An effective classification method could help researchers manage and protect mural images in an orderly fashion and quickly and effectively search for target images in a digital mural library based on a specified image category, aiding mural condition monitoring and restoration efforts as well as archaeological and art historical research.

Real-time 3-D tracking of a fast-moving object has found important applications in industry, traffic control, sports, biomedicine, defense, etc. However, it is difficult to adopt typical image-based object tracking systems in a fast-moving object tracking in real time and for a long duration, because reliable and robust image processing and analysis algorithms are often computationally exhausted, and limited storage and bandwidth can hardly fulfill the great demand of high-speed photography. Here we report an image-free 3-D tracking approach. The approach uses only two single-pixel detectors and a high-speed spatial light modulator for data acquisition. By illuminating the target moving object with six single-period Fourier basis patterns, the approach is able to analytically calculate the position of the object with the corresponding single-pixel measurements. The approach is low-cost, and data- and computation-efficient. We experimentally demonstrate that the proposed approach can detect and track a fast-moving object at a frame rate of 1666 frames per second by using a 10,000 Hz digital micromirror device. Benefiting from the wide working spectrum of single-pixel detectors, the reported approach might be applicable for hidden fast-moving object tracking.

Differential measurement has strong anti-interference ability. We report the first, to the best of our knowledge, experimental demonstration of differential self-mixing interference signals using a randomly polarized laser for differential self-mixing interferometry (SMI). In the differential SMI system, the detection light can be divided into interference signals of $p$p-polarized and $s$s-polarized light with identical intensity and pi-shift phase difference. By exploiting such a new experimental phenomenon, we propose a noise-robust and low-cost self-mixing interferometer. Experiments show that the proposed approach can effectively suppress both periodic and aperiodic noise. Thus, the reported phenomenon and approach has a good application prospect in self-mixing interferometers.

Conventional phase-based surface plasmon resonance (SPR) sensing can achieve 10^−8??RIU, but commonly requires two-beam interference. It therefore leads to complexity in terms of utilized devices, poor anti-noise ability, and demand for fine working conditions. With these requirements imposed, conventional SPR sensing has difficulties in commercial use. In this Letter, we report a simple, compact, and phase-sensitivity-doubled self-mixing interference (SMI)-based SPR sensing approach. The reported approach employs SMI and, therefore, needs only one optical path, enabling the advantages of compactness and simplicity in experimental setup, and strong anti-vibration property. With the proposed setup, the phase of light from the prism to the sample changes twice. Consequently, the sensitivity of phase is doubled. For experimental demonstration, we monitor the refractive index change of NaCl solution by using the proposed technique. The experimental results coincide with the theoretical analysis and simulation results.

Single-pixel imaging can capture images using a detector without spatial resolution, which enables imaging in various situations that are challenging or impossible with conventional pixelated detectors. Here we report a compressive single-pixel imaging approach that can simultaneously encode and recover spatial, spectral, and 3D information of the object. In this approach, we modulate and condense the object information in the Fourier space and detect the light signals using a single-pixel detector. The data-compressing operation is similar to conventional compression algorithms that selectively store the largest coefficients of a transform domain. In our implementation, we selectively sample the largest Fourier coefficients, and no iterative optimization process is needed in the recovery process. We demonstrate an 88% compression ratio for producing a high-quality full-color 3D image. The reported approach provides a solution for information multiplexing in single-pixel imaging settings. It may also generate new insights for developing multi-modality computational imaging systems.

Fluorescence imaging through a turbid layer holds great promise for various biophotonics applications. Conventional wavefront shaping techniques aim to create and scan a focus spot through the turbid layer. Finding the correct input wavefront without direct access to the target plane remains a critical challenge. In this paper, we explore a new strategy for imaging through turbid layer with a large field of view. In our setup, a fluorescence sample is sandwiched between two turbid layers. Instead of generating one focus spot via wavefront shaping, we use an unshaped beam to illuminate the turbid layer and generate an unknown speckle pattern at the target plane over a wide field of view. By tilting the input wavefront, we raster scan the unknown speckle pattern via the memory effect and capture the corresponding low-resolution fluorescence images through the turbid layer. Different from the wavefront-shaping-based single-spot scanning, the proposed approach employs many spots (i.e., speckles) in parallel for extending the field of view. Based on all captured images, we jointly recover the fluorescence object, the unknown optical transfer function of the turbid layer, the translated step size, and the unknown speckle pattern. Without direct access to the object plane or knowledge of the turbid layer, we demonstrate a 13-fold resolution gain through the turbid layer using the reported strategy. We also demonstrate the use of this technique to improve the resolution of a low numerical aperture objective lens allowing to obtain both large field of view and high resolution at the same time. The reported method provides insight for developing new fluorescence imaging platforms and may find applications in deep-tissue imaging.

Typical single-pixel imaging techniques inherently consume a large number of measurements to reconstruct a high-quality and high-resolution image. Three-dimensional (3-D) single-pixel imaging with both high sampling efficiency and high depth accuracy remains a challenge. We implement fringe projection virtually by exploiting Helmholtz reciprocity. Depth information is modulated into a deformed fringe pattern whose Fourier spectrum is sampled by using sinusoidal intensity pattern illumination and single-pixel detection. The fringe pattern has a highly focused first-order component in its Fourier spectrum, which allows us to efficiently acquire the depth information from measurements far fewer than illumination pattern pixels. The 3-D information is retrieved through Fourier analysis. We experimentally obtained a 3-D reconstruction of a complex object with 599×599 effective pixels, achieving a measurement-to-pixel ratio of 5.78%. The depth accuracy is evaluated at sub-millimetric level by using a test object.

How to effectively protect ancient murals has become an urgent and important problem. Digital image processing developments have made it possible to repair damaged murals to a certain extent. This study proposes a consistency-enhanced generative adversarial network (GAN) model to repair missing mural areas. First, the convolutional layer from a fully convolutional network (FCN) is used to extract deep image features; then, through deconvolution, the features are mapped to the size of the original image and the repaired image is output, thereby completing the regenerative network. Next, global and local discriminant networks are applied to determine whether the repaired mural image is “authentic” in terms of both the modified and unmodified areas. In adversarial learning, the generative and discriminant network models are optimized to better complete the mural repair. The network introduces a dilated convolution that increases the convolution kernel’s receptive field. Each network convolutional layer joins in the batch standardization (BN) process to accelerate network convergence and increase the number of network layers and adopts a residual module to avoid the vanishing gradient problem and further optimizing the network. Compared with existing mural restoration algorithms, the proposed algorithm increases the peak signal-to-noise ratio (PSNR) by an average of 6–8 dB and increases the structural similarity (SSIM) index by 0.08–0.12. From a visual perspective, this algorithm successfully complements mural images with complex textures and large missing areas; thus, it may contribute to digital restorations of ancient murals.

The rapid classification of ancient murals is a pressing issue confronting scholars due to the rich content and information contained in images. Convolutional neural networks (CNNs) have been extensively applied in the field of computer vision because of their excellent classification performance. However, the network architecture of CNNs tends to be complex, which can lead to overfitting. To address the overfitting problem for CNNs, a classification model for ancient murals was developed in this study on the basis of a pretrained VGGNet model that integrates a depth migration model and simple low-level vision. First, we utilized a data enhancement algorithm to augment the original mural dataset. Then, transfer learning was applied to adapt a pretrained VGGNet model to the dataset, and this model was subsequently used to extract high-level visual features after readjustment. These extracted features were fused with the low-level features of the murals, such as color and texture, to form feature descriptors. Last, these descriptors were input into classifiers to obtain the final classification outcomes. The precision rate, recall rate and F1-score of the proposed model were found to be 80.64%, 78.06% and 78.63%, respectively, over the constructed mural dataset. Comparisons with AlexNet and a traditional backpropagation (BP) network illustrated the effectiveness of the proposed method for mural image classification. The generalization ability of the proposed method was proven through its application to different datasets. The algorithm proposed in this study comprehensively considers both the high- and low-level visual characteristics of murals, consistent with human vision.