Search

Frame deletion detection is an arrestive field for video forensics in recent years. This paper presents a frame deletion detection algorithm based on optical flow orientation variation. Optical flow field has been proved to be effective for the detection of the frame deletion operation. However, current optical-flow based methods have relatively poor detection performance for real-world videos. Our research aims to develop a new forensic method to detect frame deletion for real-world videos. The proposed algorithm firstly develops an analytical model for flow orientation variation between adjacent optical flow fields, and exhibits a novel forensic clue for frame deletion detection, under an ideal but reasonable assumption. Accordingly, an effective descriptor, namely pseudo flow orientation variation (PFOV), is created to approximate the flow orientation variation, as a discriminative description feature to capture the frame deletion trace. The Frobenius norm and a smooth function are also introduced to quantify each descriptor to form an one-dimensional time series, and further statistics technique is adopted to detect the anomalous points in the time series caused by frame deletion. In addition, to improve the descriptor performance, the Robust Principal Component Analysis (RPCA) is applied to extract moving objects from video sequences and compute the proposed descriptor on them. The tests are made on 324 real-world videos, and experiment results demonstrate that the true positive rate can reach 90.12%, meanwhile the false alarm rate is 7.71%, which indicates the superiority of the proposed method in the forensic algorithms for real-world video.

The capability of a radar system to detect and identify the low-slow-small rotor unmanned aerial vehicle (UAV) is intensely important for management and control in low altitude, and it can be enhanced by the characteristics of UAV, which inherently carries micro-Doppler information. In this paper, we establish a micro-motion model of rotor UAV, and analyze the range walking and Doppler broadening caused by micro-motion. Hence, based on the characteristics of micro-Doppler of the rotors apart from the main body for the moving UAV, a micro-motion target detection and micro-Doppler parameters estimation method is proposed. Firstly, a method based on datashift in range dimension is used to compensate the translational component of the main body. Secondly, for the echo after translation compensation, a parameter estimation method of micro-Doppler based on the optimal demodulation operator is proposed. The operator contents the maximum likelihood criterion, which strengthens the difference between the micro-Doppler signal and the background clutters, so that more signal energy can be accumulated while the clutter is suppressed. Subsequently, the micro-Doppler parameters of the micro-motion component are obtained by the optimal demodulator, so that the detection and identification of the target are realized. Additionally, the performance of proposed method, including parameter estimation, signal-to-noise ratio (SNR) improvement and the calculation efficiency is analyzed. Finally, the simulation and experimental results reveal that the proposed method does possess the capability to improve the performance of detection and recognition of rotor UAV.

Infrared and visible image fusion aims to describe the same scene from different aspects by combining complementary information of multi-modality images. The existing Generative adversarial networks (GAN) based infrared and visible image fusion methods cannot perceive the most discriminative regions, and hence fail to highlight the typical parts existing in infrared and visible images. To this end, we integrate multi-scale attention mechanism into both generator and discriminator of GAN to fuse infrared and visible images (AttentionFGAN). The multi-scale attention mechanism aims to not only capture comprehensive spatial information to help generator focus on the foreground target information of infrared image and background detail information of visible image, but also constrain the discriminators focus more on the attention regions rather than the whole input image. The generator of AttentionFGAN consists of two multi-scale attention networks and an image fusion network. Two multi-scale attention networks capture the attention maps of infrared and visible images respectively, so that the fusion network can reconstruct the fused image by paying more attention to the typical regions of source images. Besides, two discriminators are adopted to force the fused result keep more intensity and texture information from infrared and visible image respectively. Moreover, to keep more information of attention region from source images, an attention loss function is designed. Finally, the ablation experiments illustrate the effectiveness of the key parts of our method, and extensive qualitative and quantitative experiments on three public datasets demonstrate the advantages and effectiveness of AttentionFGAN compared with the other state-of-the-art methods.

Methods based on generative adversarial network (GAN) have been widely used in infrared and visible images fusion. However, these methods cannot perceive the discriminative parts of an image. Therefore, we introduce a multigrained attention module into encoder–decoder network to fuse infrared and visible images (MgAN-Fuse). The infrared and visible images are encoded by two independent encoder networks due to their diverse modalities. Then, the results of the two encoders are concatenated to calculate the fused result by the decoder. To exploit the features of multiscale layers fully and force the model focus on the discriminative regions, we integrate attention modules into multiscale layers of the encoder to obtain multigrained attention maps, and then, the multigrained attention maps are concatenated with the corresponding multiscale features of the decoder network. Thus, the proposed method can preserve the foreground target information of the infrared image and capture the context information of the visible image. Furthermore, we design an additional feature loss in the training process to preserve the important features of the visible image, and a dual adversarial architecture is employed to help the model capture enough infrared intensity information and visible details simultaneously. The ablation studies illustrate the validity of the multigrained attention network and feature loss function. Extensive experiments on two infrared and visible image data sets demonstrate that the proposed MgAN-Fuse has a better performance than state-of-the-art methods.

Generative adversarial network (GAN) has shown great potential in infrared and visible image fusion. The existing GAN-based methods establish an adversarial game between generative image and source images to train the generator until the generative image contains enough meaningful information from source images. However, they only design one discriminator to force the fused result to complement gradient information from visible image, which may lose some detail information that existing in infrared image and omit some texture information that existing in visible image. To this end, we propose an end-to-end dual discriminators Wasserstein generative adversarial network, termed as D2WGAN, a framework that extends GAN to dual discriminators. In D2WGAN, the fused image can keep pixel intensity and details of infrared image by the first discriminator, and capture rich texture information of visible image by the second discriminator. In addition, to improve the performance of D2WGAN, we employ the GAN with Wasserstein distance. Moreover, in order to make the fused image keep more details from visible image in texture feature domain, we define a novel LBP (local binary pattern) loss. The extensive qualitative and quantitative experiments on public datasets demonstrate that D2WGAN can generate better results compared with the other state-of-the-art methods.

High-precision frequency estimation is an ubiquitous issue in fundamental physics and a critical task in spectroscopy. Here, we propose a quantum Ramsey interferometry to realize high-precision frequency estimation in spin-1 Bose-Einstein condensate via driving the system through quantum phase transitions(QPTs). In our scheme, we combine adiabatically driving the system through QPTs with {\pi}/2 pulse to realize the initialization and recombination. Through adjusting the laser frequency under fixed evolution time, one can extract the transition frequency via the lock-in point. The lock-in point can be determined from the pattern of the population measurement. In particular, we find the measurement precision of frequency can approach to the Heisenberg-limited scaling. Moreover, the scheme is robust against detection noise and non-adiabatic effect. Our proposed scheme does not require single-particle resolved detection and is within the reach of current experiment techniques. Our study may point out a new way for high-precision frequency estimation.

Saliency-based methods have been widely used in the fusion of infrared (IR) and visible (VIS) images, which can highlight the salient object region and preserve the detailed background information simultaneously. However, most existing methods ignore the salient information in the VIS image or they fail to highlight the boundaries of objects, which makes the final saliency map incomplete and the edges of the object blurred. To address the above-mentioned issues, we propose a novel IR and VIS images' fusion algorithm based on the Poisson reconstruction and saliency detection using the Dempster-Shafer (DS) theory. In detail, we mix the gradient using a mask map derived from the saliency map, which could avoid low contrast and halo effects in the results. Besides, both the intensity saliency of the IR image and the structural saliency of all source images are considered by DS to suppress some noise in the IR image. Thus, we could obtain smooth object contours and enhance the edge information of the salient region. Moreover, we also propose a novel probability mass function to calculate the probabilistic map in the process of applying DS to decrease the error from manually assigning the prior probability. Finally, the extensive qualitative and quantitative experiments have demonstrated the advantages and effectiveness of our method compared with other nine state-of-the-art IR and VIS image fusion methods.

Entanglement is one of the key ingredients for enhancing the measurement precision of quantum sensors. Generally, there is a trade-off between state preparation and sensing within a limited coherence time. To fully exploit temporal resources, concurrent entanglement generation and sensing with designed sequence of rotations are proposed. Based on twist-and-turn dynamics, modulated rotations along only one axis may be sufficient to drive the state to the optimal one for tiny estimated parameter. However, when the estimated parameter is not tiny, it may impact the evolved state and hence degrade the final measurement precision. Here, we introduce another modulated rotations along different axis and find out the optimal control sequences by means of machine optimization. The optimal measurement precision bounds become independent on the estimated parameter, which improves the dynamic range of the machine designed sensors. Particularly, by optimizing the interaction strength for different particle number and the time-modulated rotations along two different axes via machine optimization, the Heisenberg-limited precision scaling can be attained. Our work points out a way for designing optimized quantum-enhanced metrology protocols, which is promising for developing practical quantum sensors.

In view of the inaccuracy of existing methods for calculating the central anchor position and the lack of calculation method for the central anchor clamp position on long straight line section with a large slope, the current work introduces the slope of a line and uses the measured length of a span to calculate the tension difference of a wire caused by the deviation of a dropper and a positioning device in the line direction. According to the obtained tension difference and downhill component force caused by the gravity of the wire, the position of the central anchor is calculated. Then, according to the position of the central anchor knot, the tension difference of the dropper offset, the tension difference of the positioning device offset and downhill component force caused by the gravity of the wire, the position of the central anchor knot clamp is calculated. The maximum relative error of the position of the central anchor rope clamp obtained by this method is 1.401%, thus verifying the effectiveness of the proposed method.

Fringe projection profilometry technique has attracted more and more attention in the computer vision and found wide applications in the areas of high-precision measurement. This paper presents an accurate and robust system calibration method to formulate the phase-to-height conversion. This method uses the Levenberg-Marquardt nonlinear optimization algorithm to fit the phase value of a virtual reference plane, which is parallel to the line between the optical centers of the camera and the projector. The final expression just has three coefficients. With no need to manually adjust the position of components, several experiments were conducted to evaluate the performance of proposed method and demonstrated that the novel system calibration outperforms state-of-the-art methods in terms of stability and accuracy.

Recently, kernelised correlation filter (KCF)-based trackers aroused increasing interest and achieved extremely compelling results in different competitions and benchmarks in the field of visual object tracking. However, the training mechanism of the KCF that exploits simple linear combinations of filter from the previous frame easily cause error accumulation. To overcome this problem, the authors propose a novel training strategy that utilises all of the previous training samples, and a sparsity-related loss function regularised by the L1 norm to deal with the problem of the fixed template size in KCF trackers, a separate scale filter is learned for scale estimation during the tracking process. Moreover, powerful features that include histogram of oriented gradients (HOG) and colour features are integrated to further improve the robustness of the authors’ tracking. Extensive experiments in various challenging situations demonstrate that the proposed method performs favourably against several state-of-the-art tracking algorithms.

Generative adversarial network (GAN) has been widely applied to infrared and visible image fusion. However, the existing GAN-based image fusion methods only establish one discriminator in the network to make the fused image capture gradient information from the visible image, which may result in the loss of some infrared intensity information and texture information on the fused images. To solve this problem and improve the performance of GAN, we extend GAN to multiple discriminators and propose an end-to-end multi-discriminators Wasserstein generative adversarial network (MD-WGAN). In this framework, the fused image can preserve major infrared intensity and detail information from the first discriminator, and keep more texture information that existing in visible image from the second discriminator. We also design a texture loss function via local binary patterns to preserve more texture from visible image. The extensive qualitative and quantitative experiments show the advantages of our method compared with other state-of-the-art fusion methods.

This study presents a novel human action recognition method based on the sequences of depth maps, which provide additional body shape and motion information for action recognition. First, the authors divide each depth sequence into a number of sub-sequences. All these sub-sequences are of uniform length. By controlling vague boundary (VB), they construct a VB-sequence which consists of an original sub-sequence and its adjacent sequences. Then, each depth frame in a VB-sequence is projected onto three orthogonal Cartesian planes, and the absolute value of the difference between two consecutive projected maps is accumulated to form a depth motion map (DMM) to describe the dynamic feature of a VB-sequence. Finally, they concatenate the DMMs of all the VB-sequences in one video sequence to describe an action. Collectively, they call them VB division of depth model. For classification, they apply robust probabilistic collaborative representation classification. The recognition results applied to the MSR Action Pairs, MSR Gesture 3D, MSR Action3D, and UTD-MHAD datasets indicate superior performance of their method over most existing methods.

Sparse and low rank matrix decomposition is a method that has recently been developed for estimating different components of hyperspectral data. The rank component is capable of preserving global data structures of data, while a sparse component can select the discriminative information by preserving details. In order to take advantage of both, we present a novel decision fusion based on joint low rank and sparse component (DFJLRS) method for hyperspectral imagery in this paper. First, we analyzed the effects of different components on classification results. Then a novel method adopts a decision fusion strategy which combines a SVM classifier with the information provided by joint sparse and low rank components. With combination of the advantages, the proposed method is both representative and discriminative. The proposed algorithm is evaluated using several hyperspectral images when compared with traditional counterparts.

The accuracy of camera calibration depends significantly on the control point detection which customarily subjects to image noise, distortion and resolution. This paper uses concentric circle grids as the calibration pattern and accordingly proposes a robust and accurate control point detection method for camera calibration. Each set of concentric circles on the grids pattern consists of multiple circles whose radii decrease progressively. The geometric structure of multi-hierarchies can not only robustly extract control points from complicated scenarios but also build a voting scheme to avoid numerical instability resulting from the estimation of projected centers. According to the common self-polar triangle of concentric circles and the local sub-pixel edge detection algorithm, we efficiently estimated sub-pixel coordinates of control points under projective transformation. Simulations and real experiments were conducted to demonstrate that the performance of calibration using concentric circle grids is superior to the popular grid patterns with respect to the image noise, distortion and resolution.

We propose a parallel network with spatial–temporal attention for video-based person re-identification. Many previous video-based person re-identification methods use two-dimensional convolutional neural networks to extract spatial features, then, temporal features are extracted by temporal pooling or recurrent neural networks. Unfortunately, these series networks will cause the loss of spatial information when extracting temporal information. Different from previous methods, our parallel network can extract temporal and spatial features simultaneously, which can effectively reduce the loss of space information. In addition, we design a global temporal attention module, which obtains the attention weight through the correlation between the current frame and all the frames in the sequence. At the same time, the temporal module can act on the information extraction of spatial module. In this way, we can increase the temporal and spatial constraints. Experiments show that our method can effectively improve the re-id accuracy, better than the state-of-the-art methods.

The launching of Microsoft Kinect with skeleton tracking technique opens up new potentials for skeleton based human action recognition. However, the 3D human skeletons, generated via skeleton tracking from the depth map sequences, are generally very noisy and unreliable. In this paper, we introduce a robust informative joints based human action recognition method. Inspired by the instinct of the human vision system, we analyze the mean contributions of human joints for each action class via differential entropy of the joint locations. There is significant difference between most of the actions, and the contribution ratio is highly in accordance with common sense. We present a novel approach named skeleton context to measure similarity between postures and exploit it for action recognition. The similarity is calculated by extracting the multi-scale pairwise position distribution for each informative joint. Then feature sets are evaluated in a bag-of-words scheme using a linear CRFs. We report experimental results and validate the method on two public action dataset. Experiments results have shown that the proposed approach is discriminative for similar human action recognition and well adapted to the intra-class variation. Graphical abstractDisplay Omitted HighlightsOur informative joints based method eliminates noise by ignoring the joints of small contributions.We use binned pairwise space distribution of informative joints to build discriminative skeleton contexts.Our representation is strongly invariant to individual size and shape, and viewpoint.Improved affinity propagation was used to automatically find the exemplar features without worrying about bad initialization.The proposed approach is discriminative for similar human action recognition and well adapted to the intra-class variation.

The emergence of low-cost depth cameras creates potential for RGB-D based human action recognition. However, most of the existing RGB-D based approaches simply concatenate original heterogeneous features without discovering the latent relations among different modalities. We propose a discriminative common structure learning (DCSL) model for human action recognition from RGB-D sequences. Specifically, we extract deep learning-based features and hand-crafted features from multimodal data (skeleton, depth, and RGB). In particular, we propose a deep architecture based on 3-D convolutional neural network to automatically extract deep spatiotemporal features from raw sequences. The proposed DCSL model utilizes a generalized version of collective matrix factorization to learn shared features among different modalities. To perform supervised learning and preserve intermodal similarity, we formulate a graph regularization term by considering both label information and similar geometric structure of multimodal data, which intends to improve the discriminative power of shared features. Moreover, we solve the objective function using an iterative optimization algorithm. Then, an improved collaborative representation classifier is employed to perform computationally efficient action recognition. Experimental results on four action datasets demonstrate the superior performance of the proposed method.

Face recognition is a challenging task in computer vision. Numerous efforts have been made to design low-level hand-crafted features for face recognition. Low-level hand-crafted features highly depend on prior knowledge, which is difficult to obtain without learning new domain knowledge. Recently, ConvNets have generated great attention for their ability of feature learning and achieved state-of-the-art results on many computer vision tasks. However, typical ConvNets are trained by a gradient descent method in supervised mode, which results in high computational complexity. To solve this problem, an efficient unsupervised deep learning network is proposed for face recognition in this paper, which combines both 2-D Gabor filters and ( 2 D ) 2 PCA to learn the multistage convolutional filters. To speed up the calculation, the learned high-dimensional features are further encoded using short binary hashes. Finally, the obtained output features are trained using LinearSVM. Extensive experimental results on several facial benchmark databases show that the proposed network can obtain competitive performance and robust distortion-tolerance for face recognition.

Experience has been playing an important role in history of human and current social activities. Automating experience is also one of the most important parts for artificial intelligence. This paper will examine experience and experience based reasoning, experience management, experience engineering and their interrelationships. Then it will propose a unified architecture of experience engineering from a viewpoint of systems development methodologies. This architecture ties together philosophies, methodologies, techniques, tools and applications into a unified framework that includes both logical and intelligent embodiments of the aspects of experience engineering. The proposed approach will facilitate the development of experience management, experience engineering and knowledge based systems.