Search

Multibaseline (MB) phase unwrapping (PU) is a vital processing procedure for MB synthetic aperture radar interferometry (InSAR) signal processing and can improve the traditional InSAR by changing the ill-posed problem to the well-posed problem. The existing research has shown that the MB PU problem can be successfully converted into an unsupervised cluster analysis problem. Using the high feature descriptiveness of the deep learning technique, an unsupervised deep convolutional neural network, referred to as CANet, is proposed to cluster all the pixels into different groups according to the input's recognizable pattern of the ambiguity number of the MB interferometric phase. Subsequently, we extend our previous two-stage programming-based MB processing approach (TSPA) to processing MB PU on a sparse irregular network, which is established from the clustering result of CANet. Both theoretical analysis and experimental results show that the proposed method is an effective MB PU method, and its execution time is drastically lower than those of many classical MB PU methods.

Person reidentification, as one of the most important areas in the video surveillance field, is a crucial task in computer vision. It has attracted more and more attention in both academic research and industry due to its extremely high application value. However, the recognition accuracy of person reidentification is subjected to factors, such as illumination, pose, occlusion, and viewpoint. To alleviate the effect of such factors, the multiscale Retinex with color restoration (MSRSC) algorithm is adopted to preprocess the original images so that the color information can be restored and the illumination condition can be improved. To obtain pose-invariant features (PIFs), the convolution pose machine that can generate the body joint points of pedestrians is applied to divide the body into seven parts, the pose transform network is then used to align the body parts, and finally, the PIFs can be obtained from a designed pseudo-Siamese network by using the original and aligned images as the training samples. To further improve recognition accuracy, a reranking method based on bidirectional k-nearest neighbors (KNN) is presented to optimize the ranking list. Experimentally, the proposed method is implemented on three data sets: viewpoint invariant pedestrian recognition (VIPeR), CUHK03, and Market1501. The results demonstrate our method outperforms the other methods, as a result of both the representation of a more discriminative feature descriptor and the introduction of a reranking method.

Person re-identification aims at automatically retrieving a person of interest across multiple non-overlapping camera views. Because of increasing demand for real-world applications in intelligent video surveillance, person re-identification has become an important computer vision task and achieved high performance in recent years. However, the traditional person re-identification research mainly focus on matching cropped pedestrian images between queries and candidates on commonly used datasets and divided into two steps: pedestrian detection and person re-identification, there is still a big gap with practical applications. Under the premise of model optimization, based on the existing object detection and person re-identification, this paper achieves a one-step search of the specific pedestrians in the whole images or video sequences in the real scene. The experimental results show that our method is effective in commonly used datasets and has achieved good results in real-world applications, such as finding criminals, cross-camera person tracking, and activity analysis., 26th International Symposium on Artificial Life and Robotics, AROB 26th 2021, January 21–23, 2021, Beppu, Japan and Online

Behavioral decision systems may suffer from poor performance due to the failure in capturing the vibrations of environmental information. To better capture such vibrations and then make more accurate predictions, a parallel deep neural network based on dual fusions including feature and decision is proposed, called DFFD-Net. DFFD-NET is composed of two parts, the feature fusion network and the driving data network. The feature fusion model adopts two different operations, deconvolution and linear weighting, to fuse local features and global features, respectively. Deconvolution is applied between the convolutional layers, while linear weighting is operated among the outputs of SPP and LSTM. To further improve the accuracy of the prediction, the decisions generated from both networks are further weighed to get the final decision. Experimentally, DFFD-NET is implemented in the benchmarks BDDV and TORCS, and the results show that the final performance is benefited from both feature fusion and decision fusion. From the comparison, DFFD-NET can get state-of-the-art results on both perplexity and precision by only using the images captured from the front-facing camera as well as a few sensing data.

Synthetic aperture radar (SAR) images are widely used for aerial and spatial image applications. However, Most of SAR images are usually grayscale images with no color information. Hence, the study of SAR image colorization is meaningful. At present, deep learning has become the mainstream method of SAR coloring, and with the most advanced pix2pix method, it achieves satisfactory results. However, such an approach is limited to the corresponding paired data, which may be difficult to get. We then notice that the cycle-consistency loss can remove this constraint to some extent. In this letter, we present a novel method to colorize the SAR image using a multidomain cycle-consistency generative adversarial network (MC-GAN). The proposed method improves the performance of coloring SAR images from two aspects: first, we propose a mask vector for images of every particular terrain combined with cycle-consistency loss, which does not need the paired SAR-optical images to train the model. Second, we define the multidomain classification loss, which can together get the correct output image with the color we hope it to be. We examined the proposed method on the newly SEN1-2 data set compared with the pix2pix and CycleGAN methods, which demonstrates the effectiveness of our proposed method.

Person re-identification, as an important supplement to face recognition, it can solve the cross-camera and cross-scene pedestrian recognition and retrieval, determining whether there is a specific pedestrian to be detected in the image or video sequence. At present, many person re-identification researches are mainly carried out through experimental verification and evaluation on large person re-identification datasets such as Market-1501, DukeMTMC-reID, MSMT17, and CUHK03. In this paper, based on the existing deep learning person re-identification research and combining with the actual application scenarios on the premise of analyzing the technical feasibility, we propose a complete process for person re-identification with mobile devices, which aims to combine pedestrian detection with person re-identification to perform real-time pedestrian detection and query. In this process, not only the features extracted by pedestrians can be reused, but also the research on person re-identification can be applied effectively, such as tracking criminals and searching for missing children.

Based on the Z-Stack protocol Stack, with the CC2530 radio frequency transceiver chip and DHT22 digital temperature and humidity sensor as the core, this article puts forward a new implementation designed for the wireless temperature and humidity acquisition system on the basis of the Zigbee wireless communication technology. With the method of upper machine software by GUI, changes in temperature and humidity can be shown on the display terminal intuitively. The article mainly expounds the system structure design, temperature andhumiditydataacquisitionprocessandgraphicalinterfacemonitoringimplementation. Experimentsshowthat the wireless sensor system can make the real-time data acquisition of temperature and humidity, which has not only good efficiency and scalability, but also high reliability and excellent stability.

Event recognition in surveillance video has gained extensive attention from the computer vision community. This process still faces enormous challenges due to the tiny inter-class variations that are caused by various facets, such as severe occlusion, cluttered backgrounds, and so forth. To address these issues, we propose a spatio-temporal deep residual network with hierarchical attentions (STDRN-HA) for video event recognition. In the first attention layer, the ResNet fully connected feature guides the Faster R-CNN feature to generate object-based attention (O-attention) for target objects. In the second attention layer, the O-attention further guides the ResNet convolutional feature to yield the holistic attention (H-attention) in order to perceive more details of the occluded objects and the global background. In the third attention layer, the attention maps use the deep features to obtain the attention-enhanced features. Then, the attention-enhanced features are input into a deep residual recurrent network, which is used to mine more event clues from videos. Furthermore, an optimized loss function named softmax-RC is designed, which embeds the residual block regularization and center loss to solve the vanishing gradient in a deep network and enlarge the distance between inter-classes. We also build a temporal branch to exploit the long- and short-term motion information. The final results are obtained by fusing the outputs of the spatial and temporal streams. Experiments on the four realistic video datasets, CCV, VIRAT 1.0, VIRAT 2.0, and HMDB51, demonstrate that the proposed method has good performance and achieves state-of-the-art results.

Nowadays content-based video event detection faces great challenges due to complex scenes and blurred actions in surveillance videos. To alleviate these challenges, we propose a novel spatial-temporal architecture of deep convolutional neural networks for this task. By taking advantage of spatial-temporal information, we fine-tune two-stream networks, and then, fuse spatial and temporal features at convolution layers using a 2D pooling fusion method to enforce the consistence of spatial-temporal information. Based on the two-stream networks and spatial-temporal layer, a triple-channel model is obtained. Furthermore, we implement trajectory-constrained pooling to deep features and hand-crafted features to combine their merits. A fusion method on triple-channel yields the final detection result. The experiments on two benchmark surveillance video data sets including VIRAT 1.0 and VIRAT 2.0, which involve a suit of challenging events, such as person loading an object to a vehicle or person opening a vehicle trunk, manifest that the proposed method can achieve superior performance compared with the state-of-the-art methods on these event benchmarks.

Semi-supervised video object segmentation aims at segmenting the target of interest throughout a video sequence when only the annotated mask of the first frame is given. A feasible method for segmentation is to capture the spatial-temporal coherence between frames. However, it may suffer from mask drift when the spatial-temporal coherence is unreliable. To relieve this problem, we propose an encoder-decoder-recurrent model for semi-supervised video object segmentation. The model adopts a U-shape architecture that combines atrous convolution and ConvLSTM to establish the coherence in both the spatial and temporal domains. Furthermore, the weight ratio for each block is also reconstructed to make the model more suitable for the VOS task. We evaluate our method on two benchmarks, DAVIS-2017 and Youtube-VOS, where state-of-the-art segmentation accuracy with a real-time inference speed of 21.3 frames per second on a Tesla P100 is obtained.

Person Re-identification as an important supplement to face recognition, refers to a network of cameras with non-overlapping vision domains, through the use of computer vision technology to solve the cross-camera and cross-scene pedestrian recognition and retrieval. That is, determine whether there is a specific pedestrian to be detected in the different images or different video sequences. At present, many person re-identification research works mainly carried out through experimental verification and evaluation on large ReID datasets such as Market-1501, DukeMTMC-reID, MSMT17, and CUHK03. In this paper, based on the existing deep-learning person re-identification research, combines with the actual application scenarios, under the premise of analyzing the technical feasibility, we propose a complete process for Person Re-identification based on mobile devices, aims to combine pedestrian detection and person re-identification to perform real-time pedestrian detection and query. In this process, not only the features extracted by pedestrians can be reused, but also the research on person re-identification can be better and effectively applied, such as tracking criminals and searching for missing children.

Learning of the value function and the policy for continuous MDPs is non-trial due to the difficulty in collecting enough data. Model learning can use the collected data effectively, to learn a model and then use the learned model for planning, so as to accelerate the learning of the value function and the policy. Most of the existing works about model learning only concern the improvement of the single-step or multiple-step prediction, while the combination of them may be a better choice. Therefore, we propose an online algorithm where the samples for learning the model are both from the samples and from the trajectories, called Online-ML-ST. Other than the existing work, the trajectories collected in the interaction with the environment are not only used to learn the model offline, but also to learn the model, the value function and the policy online. The experiments are implemented in two typical continuous benchmarks such as the Pole Balancing and Inverted Pendulum, and the result shows that Online-ML-ST outperforms the other three typical methods in learning rate and convergence rate.

To solve the challenging person re-identification problem, great efforts have been devoted to feature representation and metric learning. However, existing feature extractors are either stripe-based or dense-block-based, the fine details and coarse appearance are not well integrated. What is more, the metrics are generally learned independently from distance view or bilinear similarity view. Few works have exploited the mutual complementary effects of their combination. To address these issues, we propose a new feature representation termed enhanced Local Maximal Occurrence (eLOMO) which fuses a new overlapping-stripe-based descriptor with the Local Maximal Occurrence (LOMO) extracted from dense blocks. Such integration makes eLOMO resemble the coarse-to-fine recognition mechanism of human vision system, thus it can provide a more discriminative descriptor for re-identification. Besides, we show the advantages of learning generalized similarity by combining the Mahalanobis distance and bilinear similarity together. Specifically, we derive a logistic metric learning method to jointly learn a distance metric and a bilinear similarity metric, which exploits both the distance and angle information from training data. Taking advantage of learning in the intra-class subspace, the proposed method can be solved efficiently by coordinate descent optimization. Experiments on four challenging datasets including VIPeR, PRID450S, QMUL GRID, and CUHK01, show that the proposed method outperforms the state-of-the-art approaches significantly.

Distance metric learning has achieved great success in person re-identification. Most existing methods that learn metrics from pairwise constraints suffer the problem of imbalanced data. In this study, the authors present a large margin relative distance learning (LMRDL) method which learns the metric from triplet constraints, so that the problem of imbalanced sample pairs can be bypassed. Different from existing triplet-based methods, LMRDL employs an improved triplet loss that enforces penalisation on the triplets with minimal inter-class distance, and this leads to a more stringent constraint to guide the learning. To suppress the large variations of pedestrian's appearance in different camera views, the authors propose to learn the metric over the intra-class subspace. The proposed method is formulated as a logistic metric learning problem with positive semi-definite constraint, and the authors derive an efficient optimisation scheme to solve it based on the accelerated proximal gradient approach. Experimental results show that the proposed method achieves state-of-the-art performance on three challenging datasets (VIPeR, PRID450S, and GRID).

Image-to-image translation is an important topic in the field of computer vision. It aims to learn the mapping between input image and output image by training the datasets, and finally translates the image style from one domain to another. In terms of the form of translation, it can be divided into the translation between two domains and multiple domains from different datasets. And it is also divided into pairs and unpaired by the training datasets. As a successful representation of the translation of an unpaired image between two domains, the CycleGAN model is of great significance to the research and application. Starting from the application background of the CycleGAN model, this paper compares it with the basic deep learning model GAN, the paired image translation model pix2pix and other unpaired image-to-image translation models separately, and gives their own model structure diagram. Then, the further application of the CycleGAN in other fields of computer vision is analyzed, especially in the field of Person re-recognition and face change. Finally, according to the actual running results of each model, the existing problems are analyzed and summarized. The further research work is given.

Learning discriminative local features is the key to improving the accuracy of fine-grained recognition. Methods based on local area labeling are very labor intensive and time costly. Existing weakly supervised methods decide the discriminative areas according to the response of the advanced feature maps. However, the details of the many small objects, which are vital for the region localization, were lost. We propose a region selection model with saliency constraint to capture the details, where the feature pyramid network is used to obtain higher resolution features with stronger semantics, and then the regions on different level feature maps that are consistently to be most informative are selected. This process enhances the model’s ability to represent details and capture small but discriminative local regions. Furthermore, a saliency extractor which shares convolutional layers with the backbone network is built to locate the object in an image, which will help to locate the discriminative regions and improve the training efficiency. We use a ranking loss function to optimize the multi-scale and multi-ratio regions which are selected by our model. Experimentally, the proposed method is implemented in datasets CUB-200-2011 and Stanford Cars and it is achieved the state-of-the-art results.

One of the challenges in fine-grained classification is that subcategories with significant similarity are hard to be distinguished due to the equal treatment of all subcategories in existing algorithms. In order to solve this problem, a fine-grained image classification method by combining a bilinear convolutional neural network (B-CNN) and the measurement of subcategory similarities is proposed. Firstly, an improved weakly supervised localization method is designed to obtain the bounding box of the main object, which allows the model to eliminate the influence of background noise and obtain more accurate features. Then, sample features in the training set are computed by B-CNN so that the fuzzing similarity matrix for measuring interclass similarities can be obtained. To further improve classification accuracy, the loss function is designed by weighting triplet loss and softmax loss. Extensive experiments implemented on two benchmarks datasets, Stanford Cars-196 and Caltech-UCSD Birds-200-2011 (CUB-200-2011), show that the newly proposed method outperforms in accuracy several state-of-the-art weakly supervised classification models.

Based on the Z-Stack protocol Stack, with the CC2530 radio frequency transceiver chip and DHT22 digital temperature and humidity sensor as the core, this article puts forward a new implementation designed for the wireless temperature and humidity acquisition system on the basis of the Zigbee wireless communication technology. With the method of upper machine software by GUI, changes in temperature and humidity can be shown on the display terminal intuitively. The article mainly expounds the system structure design, temperature and humidity data acquisition process and graphical interface monitoring implementation. Experiments show that the wireless sensor system can make the real-time data acquisition of temperature and humidity, which has not only good efficiency and scalability, but also high reliability and excellent stability.

Semantic image segmentation plays a crucial role in scene understanding tasks. In autonomous driving, the driving of the vehicle causes the scale changes of objects in the street scene. Although multi-scale features can be learned through concatenating multiple different atrous-convolved features, it is difficult to accurately segment pedestrians with only partial feature information due to factors such as occlusion. Therefore, we propose a Xiphoid Spatial Pyramid Pooling method integrated with detailed information. This method, while connecting the features of multiple atrous-convolved, retains the image-level features of target boundary information. Based on the above methods, we design an encoder-decoder architecture called DXNet. The encoder is composed of a deep convolution neural network and two XSPP modules, and the decoder decodes the advanced features through up-sampling operation and skips connection to gradually restore the target boundary. We evaluate the effectiveness of our approach on the Cityscapes dataset. Experimental results show that our method performs better in the case of occlusion, and the mean intersection-over-union score of our model outperforms some representative works.

This chapter is focused on some classical feature representations for image and video analysis. In particular, we will introduce the histogram-based features, texture features, and some local point features.

Object recognition, one of the important tasks of image recognition, mainly aimed at the recognition of visible images. It can accurately define and describe objects by attributes and features of geometric appearance, texture and material of images. In a broad sense, the recognition process can distinguish objects from backgrounds and other suspicious objects, such as cars and roads, that is the object detection.

In this chapter, we talk about one of the typical image understanding problems—cross-camera person re-identification. Some classical visual descriptors and metric learning algorithms for person re-identification are detailed.

In this chapter we firstly introduce the development and the main reasons of the success of deep learning, then the structure and principle of the deep CNN are explored, and several classical convolution network models are analyzed, finally two instances based on CNN architecture are given.

This chapter briefly introduces the background reading of scene classification and two topic models: LDA model and Topic Model using Belief Propagation (TMBP).

This chapter is devoted to some segmentation method of image and video. For image segmentation, five types of methods are detailed, including threshold segmentation, region-based segmentation, partial differential equation based segmentation, clustering based segmentation, and the graph theory based segmentation. For video segmentation, we shall introduce the motion region extraction method based on cumulative difference.

3D human action recognition is a challenging task due to its complexity. In this paper, we propose a new framework to tackle the matter. Firstly, we propose to extract the orientation vectors from several groups of skeleton joints instead of the original data to strengthen the spatial features. Then the skeleton features are fed to a stacked residual Bi-directional Long-Short Term Memory (stacked residual Bi-LSTM) network, which is set up to capture the robust bi-directional spatio-temporal features. Furthermore, the usage of residual connection can solve the vanishing gradient problem and further improve the results. We evaluate our method on the NTU RGB+D dataset. The experimental results manifest that our method can achieve excellent performance.

Image co-segmentation is the problem of extracting the common objects from multiple images. Foreground segmentation is always effected by the diverse objects and complex background. However, the existing methods didn't pay much attention to images' background as object, especially the similar background. To address the similar scene co-segmentation problems, a method which considers the foreground consistency and background consistency in Markov random field(MRF) model is introduced in this paper. In addition, we use co-saliency map to measure the consistency instead of histogram matching. We obtain the better results with the state-of-the-art similar scene co-segmentation method.

In this paper, we are tackling the problem of distinguishing scenes, including static and dynamic scenes. We propose a framework of scene recognition, based on bag of visual words and topic model. We achieve the task using the topic model by belief propagation (TMBP), which belongs to the family of the latent Dirichlet allocation model. We also extend the TMBP model, called as the knowledge TMBP model, by introducing the prior information of visual words and scenes. Experimental results on the static and dynamic scenes demonstrated that our proposed framework is effective and efficient. The scene semantics can be obtained from two levels of visual words and topics in our framework. Our result significantly outperforms the others using low-level visual features, such as spatial, temporal and spatiotemporal features.

 Abstract—Pedestrian recognition in aerial video is a challenge problem for the problem of low resolution, camera movement and target's blurred detail in aerial video. This paper proposes weighted region matching algorithm with Kalman filter, Multi-features fusion model and saliency segmentation (KMFS-WRM) to detect and recognize pedestrian. The KMFS-WRM algorithm first uses Kalman filter algorithm to mark candidate's region, which can avoid the problem of selecting candidates under supervision. Then we proposed the fusion algorithm of multi-feature, including HOG, LBP and SIFT features, namely HLS model to detect the pedestrian in aerial video. Our proposed detection method is robust for whether the camera is moving. And instructing human percept and concept, we segment the pedestrians in marked region using Context-Aware saliency detection algorithm that proposed by Goferman et al. and revised the segmentation results by HST model (Head Shoulder and Torso) and AAM model (Active Appearance Model) to obtain the candidates set. Last the matching of voter and candidates set using weighted region matching algorithm. Experimental results in complex aerial video demonstrated that our KMFS-WRM algorithm not only cuts down calculated complexity, but also improves adaptive and real-time ability. Moreover proposed method outperforms recent state-of-the-art methods.

We utilize a tomography-based persistent scatterer interferometric synthetic aperture radar (Tomo-PSInSAR) approach for monitoring the deformation performance and analysis of the thermal expansion of 33 high-rise buildings in Shanghai Lujiazui. For the purpose of this study, we use 31 stripmap acquisitions from TerraSAR-X missions spanning from December 2009 to February 2013. Considering that thermal expansion, concrete creep, and shrinkage are three long-term movements that occur in high-rise buildings with concrete structures, we use an extended 4-D SAR phase model, and three parameters (height, deformation velocity, and thermal amplitude) are estimated simultaneously. Additionally, we apply a two-tier network strategy to detect single and double PSs with no need for preliminary removal of the atmospheric phase screen in the study area. The thermal amplitude map demonstrates that the derived thermal amplitude of the buildings is highly related to the building height due to the upward accumulative effect of thermal expansion. The deformation velocity map reveals that the Shanghai World Financial Center is subject to a great deal of deformation during the newly built period due to concrete creep and shrinkage effects, which are height-dependent downward movements. In addition, the deformation rates caused by concrete creep and shrinkage are largest at the beginning and gradually decrease, finally achieving a steady state as time goes to infinity. In contrast, the deformation velocity map shows that the Jin Mao Tower is almost stable during this period. We observe a negative correlation between deformation characteristics and the time of completion of high-rise buildings, inferring that the newly built concrete buildings are more sensitive to concrete creep and shrinkage effects than the older concrete buildings. Our study emphasizes the potential of the Tomo-PSInSAR solution for monitoring the deformation performance and analysis of the thermal expansion of high-rise buildings.

Crowd counting, a high accuracy and high-speed technology, has been applied in new retail, shopping mall, underground, rail station and vehicle surveillance systems. However, due to the inconsistent sizes of human heads, there are a lot of counting errors and instability of the crowd density estimation in extremely dense crowd images. Therefore, a scale-adaptive Convolutional Neural Network (CNN) architecture is proposed by introducing residual network on the basis of multi-column CNN. In the process of model training, joint learning is proposed in this paper. Through alternating training for residual network and multi-column CNN, network parameters with the best accuracy are selected after iteration. Joint learning helps to enhance the modelling ability for massive scale transformation and the scale self-adaptability of the network. The proposed method is experimented on public dense crowd data sets. Experimental results prove that scale-adaptive CNN shows higher counting capability than the current state-of-the-art method.

This paper proposes a novel invariant local descriptor, a combination of gradient histograms with contrast intensity (CGCI), for image matching and object recognition. Considering the different contributions of sub-regions inside a local interest region to an interest point, we divide the local interest region around the interest point into two main sub-regions: an inner region and a peripheral region. Then we describe the divided regions with gradient histogram information for the inner region and contrast intensity information for the peripheral region respectively. The contrast intensity information is defined as intensity difference between an interest point and other pixels in the local region. Our experimental results demonstrate that the proposed descriptor performs better than SIFT and its variants PCA-SIFT and SURF with various optical and geometric transformations. It also has better matching efficiency than SIFT and its variants PCA-SIFT and SURF, and has the potential to be used in a variety of realtime applications.

SIFT is an image local feature description algorithm based on scale-space. Due to its strong matching ability, SIFT has many applications in different fields, such as image retrieval, image stitching, and machine vision. After SIFT was proposed, researchers have never stopped tuning it. The improved algorithms that have drawn a lot of attention are PCA-SIFT, GSIFT, CSIFT, SURF and ASIFT. In this paper, we first systematically analyze SIFT and its variants. Then, we evaluate their performance in different situations: scale change, rotation change, blur change, illumination change, and affine change. The experimental results show that each has its own advantages. SIFT and CSIFT perform the best under scale and rotation change. CSIFT improves SIFT under blur change and affine change, but not illumination change. GSIFT performs the best under blur change and illumination change. ASIFT performs the best under affine change. PCA-SIFT is always the second in different situations. SURF performs the worst in different situations, but runs the fastest.

In this paper, we present a method that combines a sparse appearance model into the Bayesian inference framework for tracking pedestrians in video sequences captured by a fixed camera. We formulate sparse appearance model as a linear combination of a set of 4D smoothed colour histograms for each pedestrian. These colour histograms are computed for all detection windows with different confidence values from human detector proposed by Dalal and Triggs. Object tracking is carried out using the Bayesian inference method. For occlusion handling, we integrate the Kalman filter to get the potential region containing target's observation and then use maximum a posteriori estimation to get the most likely observation. We test our tracking method on the benchmark video datasets. Our experimental results show that our tracking method outperforms one without using occlusion handling technique and can handle partial occlusion and false negative errors from human detector.

Hand gesture and sign language are important in facilitating non-verbal communication between human beings. This paper proposes a new approach to code 3d hand gestures for presentation and recognition. The coding is based on 3D hand and finger structure extracted from Leap Motion Controller (LMC). From the 3D coordinates tracked by device, shapes, positions or distances between finger tips are coded and normalized according to a set of rules. This coding process can be completed manually or semi-automatically. The recognition stage is automatic. In experiments, our system applied automatic coding and recognition on a subset of Chinese sign language, which is a special set of meaningful hand gestures. Integrated in a Unity application, the method can recognize learned gestures quickly and accurately. For unknown gestures, the system also provided interactive learning method to further extend recognizable alphabet signs.

Block matching has been used for motion estimation and motion compensation in the Audio and Video Standard (AVS) for years. While having an acceptable performance in describing motion between frames, it requires quite a few bits to represent the motion vectors. In certain circumstances, the use of global motion estimation and compensation would perform equally well or even better than the block matching in terms of motion accuracy, while it results in the coding of global motion model parameters. In this letter, we modify an AVS coder by adding (1) six global motion model parameters to the frame header, and (2) mode selection among INTRA, SKIP, INTER-16×16, INTER-16×8, INTER-8×16, INTER-8×8, and Global Motion Compensation (GMC) modes by Lagrange optimal rate-distortion criteria. Simulation results demonstrate that over 0.1dB improvement in PSNR is obtained compared to the AVS coder for an average coded P-frame with the same bitrate.

Person re-identification is the task of associating people across cameras with non-overlapping view field. Two key aspects of Person re-identification are the feature representation and metric learning. The feature representation employed should be both discriminative and invariant, which is also our considering in this paper. To enhance person re-identification performance, we propose to combine improved Local Maximal Occurrence (LOMO) descriptor with semantic color names (SCN). Especially, we introduce symmetry information of human body to suppress the impact of background in LOMO. When fused with mid-level attribute-based description — sematic color names, our more discriminative signature is obtained. Based on the KISS metric, evaluation on the challenging VIPeR dataset shows that the proposed method improves the re-identification significantly.

Domain adaptation methods show better ability to learn when the training data is not identically and independently distributed. The key task of domain adaptation is to find a suitable measure to scale the distributed difference between source domain and target domain. So a projected maximum divergence discrepancy distance measure is proposed. Based on the structural risk minimization theory and the projected maximum divergence discrepancy distance measure, the support vector machine based on difference of divergence is also proposed. Experimental results on artificial and real world problems show the proposed approach could learn across the domains effectively.

The task of matching persons across non-overlapping camera views, known as person re-identification, is rather challenging due to strong visual similarity and large appearance changes caused by illumination, pose and occlusion. Most approaches rely on low-level features that are both discriminative and invariant. In this work, we propose a novel method to address this problem by fusing mid-level semantic attributes with kernelized ranking. First, a kernelized ranking model is learned, and it gives the initial ranking scores. Next, an adaptive similarity model based on spatially constrained attributes is used to refine the ranking list. Fusion of the two models leads to much better performance than each individual alone. Experiments demonstrate complements of the two models and the results achieve new state-of-the-art performance on two benchmark datasets.

This paper proposes a novel method to learn a set of high-level feature representations for face verification across aging. Conventional hand-crafted features are not capable to overcome aging effects. In order to obtain an accurate face representation, we apply the combination of a nine-layer deep convolutional neural network and Local Binary Pattern(LBP) histograms, both of which are essential to face recognition. On account of the need of large quantity data in deep learning methods, we train the model on the publicly available cross-age face dataset CACD (Cross-Age Celebrity Dataset), which contains more than 160000 face images of 2000 different celebrities. Experiments on the CACD and LFW (Labeled Faces in the Wild) dataset demonstrate that the proposed approach outperforms the state-of-the-art methods. In addition, hairstyle, facial expression, changes of background and occlusion provide discriminative cues to the system of face verification.

In this paper, we introduce a more principled pooling strategy for the Convolutional Restricted Boltzmann Machine. In order to solve the information loss problem of pooling operation and inspired by the idea of spatial pyramid, we replace the probabilistic max-pooling with our sparse pyramid pooling, which produces outputs of different sizes for different pyramid levels. And then we use sparse coding method to aggregate the multi-level feature maps. The experimental results on KTH action dataset and Maryland dynamic scenes dataset show that the sparse pyramid pooling achieves superior performance to the conventional probabilistic max-pooling. In addition, our pooling strategy can effectively improve the performance of deep neural network on video classification.

In this paper, we propose an improved fuzzy C-means clustering algorithm based on cooperative quantum-behaved particle swarm optimization with learning behavior. Though FCM is a widely used clustering method, it has the inherent limitation of being sensitive to initial value and prone to fall in local optimum. To address this problem, we utilize the widely used global searching algorithm--QPSO, and employ new strategies to enhance its performance. First, we use the cooperative evolution strategy to improve the global searching capacity. Second, for each particle, the behavior of learning from others is granted, which effectively boosts the local searching capability. Furthermore, a gene pool is constructed to share information among all subgroups periodically. Since the iteration process is replaced by the improved version of QPSO, FCM no longer depends on the initialization values. Our experiments show that the proposed algorithm outperforms FCM and its improved versions significantly. The convergence and clustering accuracy are both improved effectively.

For many tracking and surveillance applications, Gaussian mixture model GMM provides an effective mean to segment the foreground from background. Though, because of insufficient and noisy data in complex dynamic scenes, the estimated parameters of the GMM, which are based on the assumption that the pixel process meets multi-modal Gaussian distribution, may not accurately reflect the underlying distribution of the observations. And the existing block-based GMM BGMM method may be able to segment only rough foreground objects with time-consuming calculations. To solve these difficulties, this paper proposes to use type-2 fuzzy sets T2FSs to handle GMM's uncertain parameters T2GMM. Furthermore, this paper also introduces a novel representation of contextual spatial information including the color, edge and texture features for each block which is faster and almost lossless T2BGMM. Experimental results demonstrate the efficiency of the proposed methods.

Slow feature analysis (SFA) can extract slowly varying signals from quickly varying input data. Inspired by the temporal slowness principle, we propose a novel spatiotemporal saliency algorithm for dynamic scenes analysis. In the training phase, slow feature functions are learned from different video patches using SFA. At the stage of saliency computation, we first exploit two-layer slow feature functions to extract pixel-level high-level motion features, which represent temporal slowness of every local space-time cuboid. Temporal saliency of each location is measured by the average of the corresponding feature vector. Finally, a saliency map is generated by combining the proposed temporal saliency and existing spatial saliency. The algorithm is qualitatively and quantitatively evaluated on challenging video sequences, and achieves competitive performance in contrast to the state-of-the-art algorithm.

In this paper, contrary to most existing hand-crafted descriptor, we propose an automatic feature learning method to solve the problem of dynamic natural scenes classification. Our model use convolutional Restricted Boltzmann machine as building block, called Temporal-Spatial Deep Belief Network (TS-DBN). We train the model over both fine-scale and coarse-scale, which automatically selected from the scale space according to the related information theory knowledge, to learn multi-scale features from each video sequence. The results on Maryland dataset show that feature representation based on automatic deep feature learning methods can achieve comparable accuracy with hand-crafted descriptors. Simultaneously, both coarse and fine-scale features can get better accuracy as compared to single scale features.

This paper proposed a new object detection method based on codebook with super pixels (CBSP-OD), which overcomes large memory requirements, calculated quantity problems and code words created are not exact in dynamic scenes. The CBSP-OD algorithm converts pixels from RGB space to HSL space and cluster pixels into super pixels. In this way, we just use L component as light value to reduce computational complexity and enforce robust to light. CBSP-OD use super pixel instead of single pixel to build background model, in which consider spatial consistency well and create code words more exactly. And then, build code words on every super pixel. In this way, large memory requirements and calculated quantity problems can be avoided. Experiments in the dynamic scenes demonstrate the proposed method outperform recent state-of the-art methods.

This paper proposes a novel framework for tracking multiple targets with occlusion and illumination changing in scenes. Under our framework, the extension of Mean Shift algorithm via five global and local features: central location, width, height, area and Harris corner information, and Mean Shift (CFMS) are used to track multiple targets. To effectively track multiple objects with occluded and don't split up the object to different part, Harris corner information as local feature is extracted by proposed double threshold Harris corner detection algorithm when there exist occluding, moreover classify the corners in the occluding region by K-NN classifier. The proposed CFMS algorithm achieved tracking of multiple targets based on feature fusion and mean shift algorithm. The experimental results have showed that the proposed method of CFMS can track multiple objects more robustly.

In this paper we propose an automatic salient object extraction method for nature scene. The proposed method first utilizes an algorithm based on visual attention model to obtain a prior knowledge for Graph Cut, and then constructs the weighted graph of Graph Cut based on super-pixels pre-segmented by the improved watershed algorithm in order to accelerate the speed of proposed method. In this framework, Visual saliency map is obtained using chrominance and intensity features in HSV color space, which provides the approximate region that contains salient object to be segmented. Then the salient object region after extension is cropped as input image, and pre-segmented by the improved watershed algorithm into several regions to construct weighted graph. Finally the salient object is obtained by Graph Cut algorithm. Experiment results show that our algorithm can automatically get salient object without human interactions, and speed up the segmentation without decreasing segmentation accuracy.

This paper presents an improved warping background subtraction model for moving object detection, in order to effectively detect moving targets when background undergoes complex motions with self-occlusions. Unlike many approaches modelling each pixel independently, in this paper, each pixel is believed to correlate with the surrounding ones,which enable us to compare the image pixel with the surrounding pixels on their intensity and distribution to distinguish foreground from background. The underlying warping of pixel location is molded which arose from background motion. The background is modeled as a set of warping layers, and different layers may be visible due to the motion of an occluding layer. Foreground regions are thus defined as those that can't be modeled by the set. Through the training set, we could get each pixel warping range, and generate a reference background associated with neighborhood table. Then through this table, the canonical image can be divided into dynamic and static block, for further computing with different strategies. From the research result, it is proved that this approach compares favorably with the state of the art, while showing more precision and less time.