Search

Imaging through scattering media is a classical inverse issue in computational imaging. In recent years, deep learning(DL) methods have excelled in speckle reconstruction by extracting the correlation of speckle patterns. However, high-performance DL-based speckle reconstruction also costs huge hardware computation and energy consumption. Here, we develop an opto-electronic DL method with low computation complexity for imaging through scattering media. We design the "end-to-end" optronic structure for speckle reconstruction, namely optronic fully convolutional neural network (OP-FCNN). In OP-FCNN, we utilize lens groups and spatial light modulators to implement the convolution, down/up-sampling, and skip connection in optics, which significantly reduces the computational complexity by two orders of magnitude, compared with the digital CNN. Moreover, the reconfigurable and scalable structure supports the OP-FCNN to further improve imaging performance and accommodate object datasets of varying complexity. We utilize MNIST handwritten digits, EMNIST handwritten letters, fashion MNIST, and MIT-CBCL-face datasets to validate the OP-FCNN imaging performance through random diffusers. Our OP-FCNN reveals a good balance between computational complexity and imaging performance. The average imaging performance on four datasets achieves 0.84, 0.91, 0.79, and 16.3dB for JI, PCC, SSIM, and PSNR, respectively. The OP-FCNN paves the way for all-optical systems in imaging through scattering media.

The development of optical neural networks greatly slows the urgent demand of searching for fast computing approaches to solve big data processing. However, most optical neural networks following electronic training and optical inferencing do not really take full advantage of optical computing to reduce computational burden. Take the extensively used optronic convolutional neural networks (OPCNN) as an example, the convolutional operations still require vast computational operations in training stages on the computer. To address this issue, this study proposes the in-situ training algorithm to train the networks directly in optics. We derive the backpropagation algorithms of OPCNN hence the complicated gradient calculation in backward propagating processes can be obtained through optical computing. Both forward propagation and backward propagation are all executed on the same optical system. Furthermore, we successfully realize the introduction of optical nonlinearity in networks through utilizing photorefractive crystal SBN:60 and we also derive the corresponding backpropagation algorithm. The numerical simulation results of classification performance on several datasets validates the feasibility of the proposed algorithms. Through in-situ training, the reduction in performance resulting from the inconsistency of the plantform between training and inferencing stages can be eliminated completely. For example, we demonstrate that by using the optical training approach, OPCNN is capable of gaining a strong robustness under several misalignmed situations, which enhances the practicability of OPCNN and greatly expands its application range.

Although deeper convolutional neural networks (CNNs) generally obtain better performance on classification tasks, they incur higher computation costs. To address this problem, this study proposes the optronic convolutional neural network (OPCNN) in which all computation operations are executed in optics, and data transmission and control are executed in electronics. In OPCNN, we implement convolutional layers with multi input images by the lenslet 4f system, downsampling layers by optical-strided convolution and obtaining nonlinear activation by adjusting the camera's curve and fully connected layers by optical dot product. The OPCNN demonstrates good performance on the classification tasks in simulations and experiments and achieves better performance than other current optical convolutional neural networks by comparison due to the more complex architecture. The scalability of OPCNN contributes to building deeper networks when facing complicated datasets.

A clear image of an observed object may deteriorate into unrecognizable speckle when encountering heterogeneous scattering media, thus it is necessary to recover the object image from the speckle. A method combining least square and semidefinite programming is proposed, which can be used for imaging through scattering media. The proposed method consists of two main stages, that is, media scattering characteristics (SCs) estimation and image reconstruction. SCs estimation is accomplished through LS concept after establishing a database of known object-and-speckle pairs. Image reconstruction is realized by solving an SDP problem to obtain the product of the unknown object image and its Hermitian transposition. Finally, the unknown object image can be reconstructed by extracting the largest rank-1 component of the product. Structural similarity (SSIM) index is employed as a performance indicator in speckle prediction and image reconstruction. Numerical simulations and physical experiments are performed to verify the feasibility and practicality of the proposed method. Compared with the existing phase shift interferometry mean square optimization method and the single-shot phase retrieval algorithm, the proposed method is the most precise to obtain the best reconstruction results with highest SSIM index value. The work can be used for exploring the potential applications of scattering media, especially for imaging through turbid media in biomedical, scattering property measurement, and optical image encryption.

Imaging through scattering media is a common practice in many applications of biomedical imaging. Object image would deteriorate into unrecognizable speckle pattern when scattering media is presented. Many methods have been investigated to reconstruct the object image when only speckle pattern is available. In this paper, we demonstrate a method of single-shot imaging through scattering media. This method is based on classification and support vector regression of the measured speckle pattern. We prove the possibility of speckle pattern classification and related formulas are presented. The specified and limited imaging capability without speckle pattern classification is demonstrated. Our proposed approach, that is, speckle pattern classification based support vector regression method, makes up the deficiency. Experimental results show that, with our approach, speckle patterns could be utilized for classification when object images are unavailable, and object images can be reconstructed with high fidelity. The proposed approach for imaging through scattering media is expected to be applicable to various sensing schemes.