Search

This paper describes a pedestrian re-identification algorithm, which was developed by integrating semi-supervised learning and similarity-preserving generative adversarial networks (SPGAN). The pedestrian re-identification task aimed to rapidly capture the same target using different cameras. Importantly, this process can be applied in the field of security. Because real-life environments are complex, the number of detected identities is uncertain, and the cost of manual labeling is high; therefore, it is difficult to apply the re-identification model based on supervised learning in real-life scenarios. To use the existing labeled dataset and a large amount of unlabeled data in the application environment, this report proposes a semi-supervised pedestrian re-identification model, which combines a teacher–student model with SPGAN. SPGAN was used to reduce the difference between the target domain and the source domain by transferring the style of the labeled dataset from the source domain. Additionally, the dataset from the source domain was used after the style transfer to pre-train the model; this enabled the model to adapt more rapidly to the target domain. The teacher–student model and the transformer model were then employed to generate soft pseudo-labels and hard pseudo-labels (via iterative training) and to update the parameters through distillation learning. Thus, it retained the learned features while adapting to the target domain. Experimental results indicated that the maps of the applied method on the Market-to-Duke, Duke-to-Market, Market-to-MSMT, and Duke-to-MSMT domains were 70.2, 79.3, 30.2, and 33.4, respectively.

Objective quality measurement of a three-dimensional (3D) image is a challenging issue in various 3D visual applications since it is influenced by multiple aspects such as binocular fusion, binocular rivalry and visual comfort, etc. Existing studies show that classic 2D and some 3D image quality measurement (IQM) are only perform well for symmetric distorted 3D images, but not able to evaluate the quality of asymmetrical distorted 3D images accurately. In this paper, based on the statistical characteristics of natural images and perceptual properties of human visual system (HVS), we propose a novel reduced-reference (RR) 3D quality assessment evaluator (R3DQAE) to deal with the characteristics of 3D images. Two key technical steps are involved in R3DQAE: the statistical characteristics of 3D images and the perceptual properties of HVS. Specifically, in spatial domain, the generalized Gaussian density fits of luminance wavelet coefficients and correlations of luminance and disparity wavelet coefficients are used to represent the statistical characteristics of 3D image. Furthermore, in gradient domain, the enhanced gradient magnitudes are computed by using neighborhood phase congruency information to weight the gradient magnitudes in a locally adaptive manner. Afterward, the entropy differencing of discrete wavelet transform coefficients of enhanced gradient magnitudes are extracted as the perceptual features of HVS. Finally, the qualities index of both the statistical characteristics of 3D image and the perceptual properties of HVS are combined to yield 3D image quality index. Experiments are performed on published 3D image quality assessment database show that the proposed model achieves highly competitive performance as compared with the state-of-the art some typical full-reference and RR 3DIQM models.

Image super resolution (SR) is an important image processing technique in computer vision to improve the resolution of images and videos. In recent years, deep convolutional neural network (CNN) has made significant progress in the field of image SR; however, the existing CNN-based SR methods cannot fully search for background information in the measurement of feature extraction. In addition, in most cases, different scale factors of image SR are assumed to be different assignments and completed by training different models, which does not meet the actual application requirements. To solve these problems, we propose a multi-scale learning wavelet attention network (MLWAN) model for image SR. Specifically, the proposed model consists of three parts. In the first part, low-level features are extracted from the input image through two convolutional layers, and then a new channel-spatial attention mechanism (CSAM) block is concatenated. In the second part, CNN is used to predict the highest-level low-frequency wavelet coefficients, and the third part uses recursive neural networks (RNN) with different scales to predict the wavelet coefficients of the remaining subbands. In order to further achieve lightweight, an effective channel attention recurrent module (ECARM) is proposed to reduce network parameters. Finally, the inverse discrete wavelet transform (IDWT) is used to reconstruct HR image. Experimental results on public large-scale datasets demonstrate the superiority of the proposed model in terms of quantitative indicators and visual effects.

Reduced-reference stereoscopic image quality assessment (RRSIQA) models evaluate stereoscopic image quality degradation with partial information about the “ideal-quality” reference stereopair. On one hand, sparse representation in recent theoretical studies of visual cognition has been proved to resemble the strategy used to represent natural images in the primary visual cortex. On the other hand, the joint statistics of gradient magnitude (GM) and Laplacian of Gaussian (LOG) features are popularly utilized to form image semantic structures. Motivated by these findings, we present a new RRSIQA metric using gradient sparse representation and structural degradation in this paper. Concretely, the proposed metric is based on two main tasks: the first task extracts the distribution statistics of visual primitives by gradient sparse representation, while the second task measures structural degradation of stereoscopic image due to the presence of distortion by extracting the joint statistics of GM and LOG features. The former, so-called the binocular perceptual visual information (PVI), aims to effectively integrates the gradient map that is sparser than the image itself. Especially, the process of binocular fusion is simulated by using the mutual information of the gradient-based visual primitives between left and right view’s images as binocular cue. Furthermore, the perceptual loss vectors are taken as the differences of binocular perceptual visual information and structural degradation between reference and distorted stereopairs. Finally, the perceptual loss vectors are utilized to calculate the quality score by a prediction function which is trained using kernel ridge regressing (KRR). The experiments are performed on the popular LIVE 3D IQA databases and Waterloo IVC 3D databases, and experimental results show highly competitive performance with the state-of-the-art algorithms. Moreover, in some challenging cases with particular asymmetric distortion types, the proposed metric can achieves the best quality prediction accuracy in LIVE 3D phase II and Waterloo IVC 3D Phase II.

Objective quality measurement of a three-dimensional (3D) image is a challenging issue in various 3D visual applications. In this paper, we proposes a novel reduced-reference (RR) 3D quality assessment evaluator to deal with the characteristics of 3D images. Specifically, in spatial domain, the generalized Gaussian density (GGD) fits of luminance wavelet coefficients, and the correlations of luminance and disparity wavelet coefficients are used to represent the statistical characteristics of 3D image. Furthermore, in gradient domain, the enhanced gradient magnitudes are computed by using neighborhood phase congruency (PC) information to weight the gradient magnitudes (GM) in a locally adaptive manner. Afterward, the entropy differencing of discrete wavelet transform coefficients of enhanced gradient magnitudes are extracted as the perceptual features of HVS. Finally, we combined the quality indexes of both the statistical characteristics of 3D image and the perceptual properties of HVS to yield 3D image quality index. Experiments are performed on published 3D image quality assessment database show that our proposed R3DQAE model achieves highly competitive performance as compared with the state-of-the art some typical full-reference (FR) and RR models.

Speckles noise generated by the relative in-plane motion between the Laser Doppler Vibrometry (LDV) and the target damages the quality of the LDV-captured speech severely. As such, it is necessary to research on the effect of speckles noise on the LDV for remote voice detection and remove it. In this paper, we comprehensively discuss the statistical properties of dynamic speckles with the combination of theories and experiments, and the results show that the fluctuation of speckles intensity depends strongly on the relative velocity for a given value of the detection distance. Then, we investigate the effects of the speckles noise on the LDV for remote audio detection, and the results indicate that speckles noise can cause spikes and bursts appearing in the LDV-captured voice signals, which degrades the quality of the speech. To address this issue, an algorithm of the speckle noise reduction in the LDV-measured audio signals is proposed and the experimental results demonstrate that the proposed algorithm performs well.

To address the challenges of non-cooperative and remote human activity detection, a multimodal remote audio/video acquisition system is developed. The system mainly consists of a Pan-Tilt-Zoom (PTZ) camera and a Laser Doppler Virbometer (LDV). The traditional all-fiber structure has residual carriers, which degrades the system performance badly. To solve the problem, a partial-fiber LDV is developed to obtain remote audio by detecting the vibration of the object (caused by the acoustic pressure around the target). Besides, to improve the quality of LDV audio signals, a speech enhancement algorithm (OM-LSA) is applied to remove noises in the LDV audio signals. The PTZ camera can provide remote visual information. We also use the YOLO algorithm to discriminate human from the photos which are updated from the PTZ camera continuously. That is the primary application of the YOLO algorithm. Moreover, the YOLO algorithm is used to recognize the objects around the target person by processing the video signals acquired by PTZ camera, which can aid the LDV in finding a suitable vibration target. In experiments, we show that the remote (50 m) speech signals and visual signals can be obtained by this surveillance system. That means this system has the ability to detect remote human activities.

Speaker recognition technique is one of the popular biometric identification technology, which identifies the speaker’s identity based on the speaker’s voice. Whereas almost all speaker verification system shows poor performance when the system and speaker are far apart. To address the challenges of remote speaker recognition, a Laser Doppler Vibrometer (LDV) is used to recognize remote speaker. In this paper, three LDV speech corpuses, each consists of 50 speakers, are collected from the vibrations of a plastic bag, a mineral water bottle and a computer screen, using the LDV developed by us. The distance from the LDV sensor to the vibration targets is approximately 50 m. In order to improve the quality of the LDV-captured speech, the speech enhancement technology based on optimally modified log-spectral amplitude (OM-LSA) algorithm is used. According to the enhanced LDV-captured speech, a GMM-UBM model is built to recognize remote speaker. The experiment results show that the average EER using LDV-captured speech is 16.9590%. These results show great promise of using LDV for long range speaker recognition applications.

All-fiber laser Doppler vibrometer (LDV) systems have great potential in the application of remote acoustic detection. However, the system performance is impaired due to the serious intermediate frequency (IF) crosstalk signal, which is caused by the fiber circulator. In this paper, a system adopting a partial-fiber structure is proposed and analyzed. Simulative and experimental analysis indicate that the IF crosstalk signal is mainly caused by the fiber circulator, and it damages demodulation signal. In order to reduce the IF crosstalk signal, a polarization prism is employed to substitute the circulator. Experimental results reveal that the IF crosstalk signal can be eliminated by the partial-fiber LDV effectively. The comprehensible speech signals within the range of 50m can be obtained.

In this paper, we proposed an adaptive two-scale image fusion method using latent low-rank representation (LatLRR). First, both IR and VI images are decomposed into a two-scale representation using LatLRR to generate low-rank parts (the global structure) and saliency parts (the local structure). The algorithm denoises at the same time. Then, the guided filter is used in the saliency parts to make full use of the spatial consistency, which reduces artifacts effectively. With respect to the fusion rule of the low-rank parts, we construct adaptive weights by adopting fusion global-local-topology particle swarm optimization (FGLT-PSO) to obtain more useful information from the source images. Finally, the resulting image is reconstructed by adding the fused low-rank part and the fused saliency part. The experimental results validate that the proposed method outperforms several representative image fusion algorithms on publicly available datasets for infrared and visible image fusion in terms of subjective visual effect and objective assessment.

Ship detection in panchromatic optical remote sensing images is faced with two major challenges, locating candidate regions from complex backgrounds quickly and describing ships effectively to reduce false alarms. Here, a practical method was proposed to solve these issues. Firstly, we constructed a novel visual saliency detection method based on a hyper-complex Fourier transform of a quaternion to locate regions of interest (ROIs), which can improve the accuracy of the subsequent discrimination process for panchromatic images, compared with the phase spectrum quaternary Fourier transform (PQFT) method. In addition, the Gaussian filtering of different scales was performed on the transformed result to synthesize the best saliency map. An adaptive method based on GrabCut was then used for binary segmentation to extract candidate positions. With respect to the discrimination stage, a rotation-invariant modified local binary pattern (LBP) description was achieved by combining shape, texture, and moment invariant features to describe the ship targets more powerfully. Finally, the false alarms were eliminated through SVM training. The experimental results on panchromatic optical remote sensing images demonstrated that the presented saliency model under various indicators is superior, and the proposed ship detection method is accurate and fast with high robustness, based on detailed comparisons to existing efforts.

We describe an innovative form of polymer micro-tubes with diverse functions including biodegradation, magnetic manipulation, and photothermal effect that employs and activates photothermal therapy to target cancer cells. The micro-tube comprised soybean protein isolate, poly-l-glutamic acid, magnetite nanoparticles, plus gold nanoparticles. Through electrostatic force, these components, with opposite charges, formed pairs of layers in the pores of the template, various bilayers of soybean protein isolate and poly-l-glutamic acid served as the biodegradable building wall to each micro-tube. The layers of magnetite nanoparticle functionalized micro-tubes enabled the micro-tube manipulate to target the cancer cells by using an external magnetic field. The photo-thermal effect of the layer of gold nanoparticles on the outer surface of the micro-tubes, when under irradiation and when brought about by the near infrared radiation, elevated each sample’s temperature. In addition, and when under the exposure of the near infrared radiation, the elevated temperature of the suspension of the micro-tubes, likewise with a concentration of 0.2 mg/mL, and similarly with a power of 2 W and as well maintained for 10 min, elevated the temperature of the suspension beyond 42 °C. Such temperatures induced apoptosis of target cancer cells through the effect of photothermal therapy. The findings assert that structured micro-tubes have a promising application as a photothermal agent. From this assertion, the implications are that this multifunctional agent will significantly improve the methodology for cancer diagnosis and therapy.

This paper proposes a novel method for data hiding in encrypted image using image interpolation. Before the image encryption, the original image is sampled and an interpolation algorithm is used to calculate an estimation of the original image. Errors between the original image and the estimated image are compressed by Huffman encoding, which are further embedded into the estimated image to generate the redundant room. After image encryption using an encryption key, the secret bits are embedded into the reserved room. On the receiver side, the hidden bits can be extracted and the original content of the image can be perfectly recovered. Compared with the published results, the proposed method provides a larger embedding payload.