Search

Minimum-cost flow algorithms have recently achieved state-of-the-art results in multi-object tracking. However, they rely on the whole image sequence as input. When deployed in real-time applications or in distributed settings, these algorithms first operate on short batches of frames and then stitch the results into full trajectories. This decoupled strategy is prone to errors because the batch-based tracking errors may propagate to the final trajectories and cannot be corrected by other batches. In this paper, we propose a greedy batch-based minimum-cost flow approach for tracking multiple objects. Unlike existing approaches that conduct batch-based tracking and stitching sequentially, we optimize consecutive batches jointly so that the tracking results on one batch may benefit the results on the other. Specifically, we apply a generalized minimum-cost flows (MCF) algorithm on each batch and generate a set of conflicting trajectories. These trajectories comprise the ones with high probabilities, but also those with low probabilities potentially missed by detectors and trackers. We then apply the generalized MCF again to obtain the optimal matching between trajectories from consecutive batches. Our proposed approach is simple, effective, and does not require training. We demonstrate the power of our approach on data sets of different scenarios. [ABSTRACT FROM PUBLISHER]

Images, as high-dimensional data, usually embody large variabilities. To classify images for versatile applications, an effective algorithm is necessarily designed by systematically considering the data structure, similarity metric, discriminant subspace, and classifier. In this paper, we provide evidence that, besides the Fisher criterion, graph embedding, and tensorization used in many existing methods, the correlation-based similarity metric embodied in supervised multilinear discriminant subspace learning can additionally improve the classification performance. In particular, a novel discriminant subspace learning algorithm, called correlation tensor analysis (CTA), is designed to incorporate both graph-embedded correlational mapping and discriminant analysis in a Fisher type of learning manner. The correlation metric can estimate intrinsic angles and distances for the locally isometric embedding, which can deal with the case when Euclidean metric is incapable of capturing the intrinsic similarities between data points. CTA learns multiple interrelated subspaces to obtain a low-dimensional data representation reflecting both class label information and intrinsic geometric structure of the data distribution. Extensive comparisons with most popular subspace learning methods on face recognition evaluation demonstrate the effectiveness and superiority of CTA. Parameter analysis also reveals its robustness. [ABSTRACT FROM AUTHOR]

Visual tracking can be treated as a parameter estimation problem that infers target states based on image observations from video sequences. A richer target representation may incur better chances of successful tracking in cluttered and dynamic environments, and thus enhance the robustness. Richer representations can be constructed by either specifying a detailed model of a single cue or combining a set of rough models of multiple cues. Both approaches increase the dimensionality of the state space, which results in a dramatic increase of computation. To investigate the integration of rough models from multiple cues and to explore computationally efficient algorithms, this paper formulates the problem of multiple cue integration and tracking in a probabilistic framework based on a factorized graphical model. Structured variational analysis of such a graphical model factorizes different modalities and suggests a co-inference process among these modalities. Based on the importance sampling technique, a sequential Monte Carlo algorithm is proposed to provide an efficient simulation and approximation of the co-inferencing of multiple cues. This algorithm runs in real-time at around 30 Hz. Our extensive experiments show that the proposed algorithm performs robustly in a large variety of tracking scenarios. The approach presented in this paper has the potential to solve other problems including sensor fusion problems. [ABSTRACT FROM AUTHOR]

This paper presents the methodology and theory for automatic spatial pattern discovery from multiple attributed relational graph samples. The spatial pattern is modelled as a mixture of probabilistic parametric attributed relational graphs. A statistic learning procedure is designed to learn the parameters of the spatial pattern model from the attributed relational graph samples. The learning procedure is formulated as a combinatorial non-deterministic process, which uses the expectation–maximization (EM) algorithm to find the maximum-likelihood estimates for the parameters of the spatial pattern model. The learned model summarizes the samples and captures the statistic characteristics of the appearance and structure of the spatial pattern, which is observed under various conditions. It can be used to detect the spatial pattern in new samples. The proposed approach is applied to unsupervised visual pattern extraction from multiple images in the experiments. [Copyright &y& Elsevier]

Digital watermarking is an emerging technique to protect data security and intellectual property right. Identification or verification of watermarking patterns can be achieved by detecting watermarks in received signals. However, one of the biggest challenges in watermarking detection is that the strengths of the watermark signals will change after being distorted by an attacker in a watermarking channel. Meanwhile, the embedding strengths may be adapted to original signals, which are unknown at the receiver end. Further, the original signals are often highly non-Gaussian. Although some work has been done on optimum detection of watermarks, the uncertainty of watermark signal strengths and real statistical behavior of multimedia contents have not been taken into account simultaneously. Much more study is needed to enhance the performance of watermarking systems. Since multiplicative watermarks are robust and well suited for copyright protection, this paper presents our investigation on robust optimum detection of multiplicative watermarks. For sub-band transformed domains such as the discrete cosine transform (DCT), discrete wavelet transform (DWT), and pyramid transform, a class of generalized correlators is constructed based on the generalized Gaussian distributions. Thresholding methods to achieve a given false alarm rate, and the performance analyses are provided. The square-root detector is designed and demonstrated to have near optimal performance for a large set of natural images and can be employed as a "universally optimal" detector or decoder for images and video. The locally most powerful detection method is then extended to DFT domain multiplicative watermarking, with the magnitudes of coefficients modeled by the Weibull distributions. Another class of detectors is built based on this statistical modeling. The robust optimum detection of multiplicative watermarks can be applied to copyright notification, enforcement, and broadcast monitoring. We... [ABSTRACT FROM AUTHOR]

Discusses how reliance on powerful pattern recognition and machine learning techniques is increasing in the field of multimedia retrieval. State of the art in multimedia database management; Association of multimedia content with semantic concepts; Knowledge representations for multimedia content.

Investigates color-based image segmentation and nonstationary color-based target tracking for vision-based human-computer interaction. Representations of color distributions; Description of a structure adaptive self-organizing map neural network; Formulation of the color tracking problem.

Many visual learning tasks are usually confronted by some common difficulties. One of them is the lack of supervised information, due to the fact that labeling could be tedious, expensive or even impossible. Another difficulty is the high dimensionality of the visual data. Fortunately, these difficulties could be alleviated by using a hybrid of labeled and unlabeled training data for learning. Since the unlabeled data characterize the joint probability across different features, they could be used to boost weak classifiers by exploring discriminating features in a self-supervised fashion. This paper proposes a novel method, the Discriminant-EM (D-EM) algorithm, which attacks these difficulties by integrating discriminant analysis with the EM framework in this hybrid formulation. Both linear and nonlinear methods are investigated in this paper. Based on kernel multiple discriminant analysis, the nonlinear D-EM provides better ability to simplify the probabilistic structures of data distributions in a discrimination space. We also propose a novel data-sampling scheme for efficient learning of kernel discriminants. Our experimental results show that D-EM outperforms a variety of supervised and semi-supervised learning algorithms for many visual learning tasks, such as content-based image retrieval, invariant object recognition, and nonstationary color tracking. The proposed approach could be easily applied for many other learning tasks. [Copyright &y& Elsevier]

In this article, a new video coding system is proposed which takes advantage of both intrascale and interscale similarities present in video sequences. Each block in every frame is approximated with a linear combination of members of an adaptive library of blocks constructed for that block. This library is made up of some fixed blocks (e.g., DCT basis blocks) and some adaptive blocks. The adaptive library blocks are blocks of the same size or larger blocks that are shrunk, taken from the previous frame. For each block in every frame, a rate-distortion optimized fully orthogonal matching pursuit algorithm is used to determine how many and which ones of the members of the library constructed for that block should be selected and linearly combined to approximate the block most efficiently in terms of bit rate and rms error. Simulation results on sample gray-scale video sequence “Miss America” suggests that this method has promising coding performance in terms of PSNR and bit rate,especially compared to other published fractal-based video compression methods. © 1998 John Wiley & Sons, Inc. Int J Imaging Syst Technol, 9, 305–319, 1998 [ABSTRACT FROM AUTHOR]

We propose a motion estimation system that uses stereo image pairs as the input data. To perform experimental work, we also obtain a sequence of outdoor stereo images taken by two metric cameras. The system consists of four main stages, which are (1) determination of point correspondences on the stereo images, (2) correction of distortions in image coordinates, (3) derivation of 3D point coordinates from 2D correspondences, and (4) estimation of motion parameters based on 3D point correspondences. For the first stage of the system, we use a four-way matching algorithm to obtain matched points on two stereo image pairs at two consecutive brae instants (t, and t,.,). Since the input data are stereo images taken by cameras, it has two types of distortions, which are (i) film distortion and (ii) lens distortion. These two distortions must be corrected before any process can be applied on the matched points. To accomplish this goal, we use (i) bilinear transform for film distortion correction and (ii) lens formulas for lens distortion correction. After correcting the distortions, the results are 2D coordinates of each matched point that can be used to derive 3D coordinates. However, due to data noise, the calculated 3D coordinates do not usually represent a consistent rigid structure that is suitable for motion estimation; therefore, we suggest a procedure to select good 3D point sets as the input for motion estimation. The procedure exploits two constraints, rigidity between different time instants and uniform point distribution across the object on the image. For the last stage, we use an algorithm to estimate the motion parameters. We also wish to know what is the effect of quantization error on the estimated results; therefore an error analysis based on quantization error is performed on the estimated motion parameters. In order to test our system, eight sets of stereo image pairs are extracted from an outdoor stereo image sequence and used as the input data. The experimental results indicate that the proposed system does provide reasonable estimated motion parameters. [ABSTRACT FROM AUTHOR]

This article presents a model-based approach for nonrigid object motion and deformation analysis from 3D data. The modeling primitives used in this research are the superquadrics, which have already been proven useful in describing a variety of natural and man-made objects. This model-based approach is not only in accordance with the human visual perception process but also able to decouple the large and unstructured nonrigid motion estimation system into simple and well structured subsystems. We develop a recursive algorithm for estimating global motion and object shape, which effectively incorporates a priori knowledge of the object into the estimation procedure and obtains a good estimate of the global motion and object shape even if the given 3D points are distributed bias. After compensating for the global motion of the object, a tensor model of local deformation is introduced and a spherical harmonic surface-fitting algorithm is described such that the localized deformations of the object surface can be characterized. The local deformations of the object are then estimated using tensor-description-based analysis and parametrized by the directions and magnitudes of the extreme deformations in a localized surface element. To illustrate the potential of this model-based approach for nonrigid motion analysis, a real data example is presented using the proposed approach. This example involves estimating the left ventricle motion and deformations from a time sequence of 3D coordinates of coronary artery bifurcation points. The estimation results show the success of the model-based approach even when the given bifurcation points are distributed only on half of the left ventricle surface. [ABSTRACT FROM AUTHOR]

This paper describes a new algorithm which uses Gaussian and mean curvature values of the terrain surface to extract feature points. These points are then used for recognition of particular subregions of the terrain and in estimating relative positions of these subregions in the terrain. The Gaussian and mean curvatures are chosen because they are invariant under rotation and translation. In the Gaussian and mean curvature images, the points of maximum and minimum curvatures are extracted and used for matching. The stability of the position of these points in the presence of noise and with resampling is investigated. The input for this algorithm is 3D digital terrain data. Curvature values are calculated from the data by fitting a quadratic surface over a square window and calculating directional derivatives of this surface. A method of surface fitting that is invariant to sensor-centered coordinate system transformation is suggested and implemented. Real terrain data are used in our experiments. The algorithm is tested with and without the presence of noise, and its performance is described. [ABSTRACT FROM AUTHOR]

Looks at a computational framework, which is used in the extraction of salient patterns, called vortex structures, from 3D turbulent fluid flows, relating to pattern recognition. Detailed information on the vortex structure and its detection; Information on the results of experiments conducted; Other detailed information.

Presents relevant feedback from two characteristics of content-based image retrieval (CBIR). Details on the experimental results; Description of the multimedia object model; Integrating relevance feedback in CBIR.

Presents a new edge detection scheme that detects two-dimensional (2-D) edges by a curve-segment-based detection functional guided by the zero-crossing contours of the Laplacian-of-Gaussian (LOG) to approach the true edge locations. Optimal 2-D edge detection and edge regularization; Detecting salient edges at different scales.

Self-similarity refers to the image prior widely used in image restoration algorithms that small but similar patterns tend to occur at different locations and scales. However, recent advanced deep convolutional neural network-based methods for image restoration do not take full advantage of self-similarities by relying on self-attention neural modules that only process information at the same scale. To solve this problem, we present a novel Pyramid Attention module for image restoration, which captures long-range feature correspondences from a multi-scale feature pyramid. Inspired by the fact that corruptions, such as noise or compression artifacts, drop drastically at coarser image scales, our attention module is designed to be able to borrow clean signals from their "clean" correspondences at the coarser levels. The proposed pyramid attention module is a generic building block that can be flexibly integrated into various neural architectures. Its effectiveness is validated through extensive experiments on multiple image restoration tasks: image denoising, demosaicing, compression artifact reduction, and super resolution. Without any bells and whistles, our PANet (pyramid attention module with simple network backbones) can produce state-of-the-art results with superior accuracy and visual quality. Our code is available at https://github.com/SHI-Labs/Pyramid-Attention-Networks [ABSTRACT FROM AUTHOR]

Subspace clustering methods partition the data that lie in or close to a union of subspaces in accordance with the subspace structure. Such methods with sparsity prior, such as sparse subspace clustering (SSC) (Elhamifar and Vidal in IEEE Trans Pattern Anal Mach Intell 35(11):2765-2781, 2013) with the sparsity induced by the ℓ1-norm, are demonstrated to be effective in subspace clustering. Most of those methods require certain assumptions, e.g. independence or disjointness, on the subspaces. However, these assumptions are not guaranteed to hold in practice and they limit the application of existing sparse subspace clustering methods. In this paper, we propose ℓ0-induced sparse subspace clustering (ℓ0-SSC). In contrast to the required assumptions, such as independence or disjointness, on subspaces for most existing sparse subspace clustering methods, we prove that ℓ0-SSC guarantees the subspace-sparse representation, a key element in subspace clustering, for arbitrary distinct underlying subspaces almost surely under the mild i.i.d. assumption on the data generation. We also present the “no free lunch” theorem which shows that obtaining the subspace representation under our general assumptions can not be much computationally cheaper than solving the corresponding ℓ0 sparse representation problem of ℓ0-SSC. A novel approximate algorithm named Approximate ℓ0-SSC (Aℓ0-SSC) is developed which employs proximal gradient descent to obtain a sub-optimal solution to the optimization problem of ℓ0-SSC with theoretical guarantee. The sub-optimal solution is used to build a sparse similarity matrix upon which spectral clustering is performed for the final clustering results. Extensive experimental results on various data sets demonstrate the superiority of Aℓ0-SSC compared to other competing clustering methods. Furthermore, we extend ℓ0-SSC to semi-supervised learning by performing label propagation on the sparse similarity matrix learnt by Aℓ0-SSC and demonstrate the effectiveness of the resultant semi-supervised learning method termed ℓ0-sparse subspace label propagation (ℓ0-SSLP). [ABSTRACT FROM AUTHOR]

Abstract: The World-Wide Web provides tremendous resources in Multimedia Data, Ubiquitous Interconnection, and Storage/Computing Power. In this short note, we raise but not answer the question: Can the WWW help bridge the Semantic Gap in Computer Vision? Even if the answer turns out to be NO, we hope that by exploring these resources, we may gain a deeper understanding of the Semantic Gap challenge. [Copyright &y& Elsevier]

Focuses on the impact of the development in hardware, storage, connectivity and sensor technology to the introduction of large image repositories. Efficacy of man and machine interaction; Challenges of the computer vision in content-based retrieval; Citation of the essential components of image retrieval systems.

Aggregating features in terms of different convolutional blocks or contextual embeddings has been proven to be an effective way to strengthen feature representations for semantic segmentation. However, most of the current popular network architectures tend to ignore the misalignment issues during the feature aggregation process caused by step-by-step downsampling operations and indiscriminate contextual information fusion. In this paper, we explore the principles in addressing such feature misalignment issues and inventively propose Feature-Aligned Segmentation Networks (AlignSeg). AlignSeg consists of two primary modules, i.e., the Aligned Feature Aggregation (AlignFA) module and the Aligned Context Modeling (AlignCM) module. First, AlignFA adopts a simple learnable interpolation strategy to learn transformation offsets of pixels, which can effectively relieve the feature misalignment issue caused by multi-resolution feature aggregation. Second, with the contextual embeddings in hand, AlignCM enables each pixel to choose private custom contextual information adaptively, making the contextual embeddings be better aligned. We validate the effectiveness of our AlignSeg network with extensive experiments on Cityscapes and ADE20K, achieving new state-of-the-art mIoU scores of 82.6 and 45.95 percent, respectively. Our source code is available at https://github.com/speedinghzl/AlignSeg. [ABSTRACT FROM AUTHOR]

Link prediction is one of the most fundamental problems in graph modeling and mining. It has been studied in a wide range of scenarios, from uncovering missing links between different entities in databases, to recommending relations between people in social networks. In this problem, we wish to predict unseen links in a growing target network by exploiting existing structures in source networks. Most of the existing methods often assume that abundant links are available in the target network to build a model for link prediction. However, in many scenarios, the target network may be too sparse to enable robust inference process, which makes link prediction challenging with the paucity of link data. On the other hand, in many cases, other (more densely linked) auxiliary networks can be available that contains similar link structure relevant to that in the target network. The linkage information in the existing networks can be used in conjunction with the node attribute information in both networks in order to make more accurate link recommendations. Thus, this paper proposes the use of learning methods to perform link inference by transferring the link information from the source network to the target network. We also note that the source network may contain the link information irrelevant to the target network. This leads to cross-network bias between the networks, which makes the link model built upon the source network misaligned with the link structure of the target network. Therefore, we re-sample the source network to rectify such cross-network bias by maximizing the cross-network relevance measured by the node attributes, as well as preserving as rich link information as possible to avoid the loss of source link structure caused by the re-sampling algorithm. The link model based on the re-sampled source network can make more accurate link predictions on the target network with aligned link structures across the networks. We present experimental results illustrating the effectiveness of the approach. [ABSTRACT FROM PUBLISHER]

We present a semisupervised method for single-pixel classification of hyperspectral images. The proposed method is designed to address the special problematic characteristics of hyperspectral images, namely, high dimensionality of hyperspectral pixels, lack of labeled samples, and spatial variability of spectral signatures. To alleviate these problems, the proposed method features the following components. First, being a semisupervised approach, it exploits the wealth of unlabeled samples in the image by evaluating the confidence probability of the predicted labels, for each unlabeled sample. Second, we propose to jointly optimize the classifier parameters and the dictionary atoms by a task-driven formulation, to ensure that the learned features (sparse codes) are optimal for the trained classifier. Finally, it incorporates spatial information through adding a Laplacian smoothness regularization to the output of the classifier, rather than the sparse codes, making the spatial constraint more flexible. The proposed method is compared with a few comparable methods for classification of several popular data sets, and it produces significantly better classification results. [ABSTRACT FROM PUBLISHER]

Expression recognition from faces with varying pose and illumination conditions is a challenging research area with growing interest. In this paper, we develop a novel supervised super-vector encoding framework to learn discriminative image feature representations. The framework is then validated on the Multi-PIE and BU3D-FE databases for multi-view facial expression recognition. Extensive experiments show that our supervised framework gives significant improvement over the unsupervised counterpart and outperforms the state-of-the-arts. [ABSTRACT FROM AUTHOR]

In this paper, a novel discriminative dictionary learning method is proposed for sparse-representation-based classification (SRC) to label highly dimensional hyperspectral imagery (HSI). In SRC, a dictionary is conventionally constructed using all of the training pixels, which is not only inefficient due to the large size of typical HSI images but also ineffective in capturing class-discriminative information crucial for classification. We address the dictionary design problem with the inspiration from the learning vector quantization technique and propose a hinge loss function that is directly related to the classification task as the objective function for dictionary learning. The resulting online learning procedure systematically “pulls” and “pushes” dictionary atoms so that they become better adapted to distinguish between different classes. In addition, the spatial context for a test pixel within its local neighborhood is modeled using a Bayesian graph model and is incorporated with the sparse representation of a single test pixel in a unified probabilistic framework, which enables further refinement of our dictionary to capture the spatial class dependence that complements the spectral information. Experiments on different HSI images demonstrate that the dictionaries optimized using our method can achieve higher classification accuracy with substantially reduced dictionary size than using the whole training set. The proposed method also outperforms existing dictionary learning methods and attains the state-of-the-art results in both the spectral-only and spatial-spectral settings. [ABSTRACT FROM PUBLISHER]

Abstract: We propose a pose-robust face recognition method to handle the challenging task of face recognition in the presence of large pose difference between gallery and probe faces. The proposed method exploits the sparse property of the representation coefficients of a face image over its corresponding view-dictionary. By assuming the representation coefficients are invariant to pose, we can synthesize for the probe image a novel face image which has smaller pose difference with the gallery faces. Furthermore, face recognition in the presence of pose variations is achieved based on the synthesized face image again via sparse representation. Extensive experiments on CMU Multi-PIE face database are conducted to verify the efficacy of the proposed method. [Copyright &y& Elsevier]

Abstract: Sparsity driven classification method has been popular recently due to its effectiveness in various classification tasks. It is based on the assumption that samples of the same class live in the same subspace, thus a test sample can be well represented by the training samples of the same class. Previous methods model the subspace for each class with either the training samples directly or dictionaries trained for each class separately. Although enabling strong reconstructive ability, these methods may not have desirable discriminative ability, especially when there are high correlations among the samples of different classes. In this paper, we propose to learn simultaneously a discriminative projection and a dictionary that are optimized for the sparse representation based classifier, to extract discriminative information from the raw data while respecting the sparse representation assumption. By formulating the task of projection and dictionary learning into an optimization framework, we can learn the discriminative projection and dictionary effectively. Extensive experiments are carried out on various datasets and the experimental results verify the efficacy of the proposed method. [Copyright &y& Elsevier]

This paper presents a new approach to single-image superresolution, based upon sparse signal representation. Research on image statistics suggests that image patches can be well-represented as a sparse linear combination of elements from an appropriately chosen over-complete dictionary. Inspired by this observation, we seek a sparse representation for each patch of the low-resolution input, and then use the coefficients of this representation to generate the high-resolution output. Theoretical results from compressed sensing suggest that under mild conditions, the sparse representation can be correctly recovered from the downsampled signals. By jointly training two dictionaries for the low- and high-resolution image patches, we can enforce the similarity of sparse representations between the low-resolution and high-resolution image patch pair with respect to their own dictionaries. Therefore, the sparse representation of a low-resolution image patch can be applied with the high-resolution image patch dictionary to generate a high-resolution image patch. The learned dictionary pair is a more compact representation of the patch pairs, compared to previous approaches, which simply sample a large amount of image patch pairs refid="ref1"/, reducing the computational cost substantially. The effectiveness of such a sparsity prior is demonstrated for both general image super-resolution (SR) and the special case of face hallucination. In both cases, our algorithm generates high-resolution images that are competitive or even superior in quality to images produced by other similar SR methods. In addition, the local sparse modeling of our approach is naturally robust to noise, and therefore the proposed algorithm can handle SR with noisy inputs in a more unified framework. [ABSTRACT FROM PUBLISHER]

In this paper, we present a fusion approach to solve the nonrigid shape recovery problem, which takes advantage of both the appearance information and the local features. We have two major contributions. First, we propose a novel progressive finite Newton optimization scheme for the feature-based nonrigid surface detection problem, which is reduced to only solving a set of linear equations. The key is to formulate the nonrigid surface detection as an unconstrained quadratic optimization problem that has a closed-form solution for a given set of observations. Second, we propose a deformable Lucas-Kanade algorithm that triangulates the template image into small patches and constrains the deformation through the second-order derivatives of the mesh vertices. We formulate it into a sparse regularized least squares problem, which is able to reduce the computational cost and the memory requirement. The inverse compositional algorithm is applied to efficiently solve the optimization problem. We have conducted extensive experiments for performance evaluation on various environments, whose promising results show that the proposed algorithm is both efficient and effective. [ABSTRACT FROM AUTHOR]

In this paper, our contributions to the subspace learning problem are two-fold. We first justify that most popular subspace learning algorithms, unsupervised or supervised, can be unitedly explained as instances of a ubiquitously supervised prototype. They all essentially minimize the intraclass compactness and at the same time maximize the interclass separability, yet with specialized labeling approaches, such as ground truth, self-labeling, neighborhood propagation, and local subspace approximation. Then, enlightened by this ubiquitously supervised philosophy, we present two categories of novel algorithms for subspace learning, namely, misalignment-robust and semi-supervised subspace learning. The first category is tailored to computer vision applications for improving algorithmic robustness to image misalignments, including image translation, rotation and scaling. The second category naturally integrates the label information from both ground truth and other approaches for unsupervised algorithms. Extensive face recognition experiments on the CMU PIE and FRGC ver1.0 databases demonstrate that the misalignment-robust version algorithms consistently bring encouraging accuracy improvements over the counterparts without considering image misalignments, and also show the advantages of semi-supervised subspace learning over only supervised or unsupervised scheme. [ABSTRACT FROM AUTHOR]

This paper introduces a head pose estimation system that automatically localizes the nose-tips of the faces and estimates head poses in images simultaneously. In the training stage, the nose-tips of the faces are first manually labeled. The appearance variations caused by head pose changes are then characterized by a Tensorposes model. Given an image with unknown head pose and nose-tip location, the nose-tip of the face is automatically localized in a coarse-to-fine fashion after the skin color segmentation. The head pose is also estimated simultaneously. The performance of our system is evaluated on the Pointing'04 head pose image data set. We first evaluate the classification performance of the Tensorposes models with image patches of the faces cropped according to the manually labeled nose-tip locations of the faces in the Pointing '04 data set. By leaving-one-person-out evaluation strategy, we obtain the optimal parameters of the Tensorposes model, and evaluate the discriminative power of the Tensorposes model built based on high order singular value decomposition (HOSVD) and multilinear independent component analysis (MICA), and naive principal component analysis (PCA) subspace models. It is shown Tensorposes model by HOSVD and MICA decomposition performs similarly good but much better than naive PCA subspace models. The Tensorposes model is then utilized to automatically localize nose-tip location in the testing image and to simultaneously estimate the head pose. The nose-tip localization and pose estimation accuracy of the proposed system are evaluated against the ground truth. Finally, cross-database evaluation of the performance of our system is carried out on Pointing'04 database, a selected subset of CMU PIE database, and some pictures from CLEAR'07 head pose evaluation database. The experiments show that our system generalizes reasonably well to the real-world scenarios. [ABSTRACT FROM AUTHOR]

Beyond conventional linear and kernel-based feature extraction, we present a more generalized formulation for feature extraction in this paper. Two representative algorithms using the correlation metric are proposed based on this formulation. Correlation Embedding Analysis (CEA), which incorporates both correlational mapping and discriminant analysis, boosts the discriminating power by mapping the data from a high-dimensional hypersphere onto another low-dimensional hypersphere and preserving the neighboring relations with local-sensitive graph modeling. Correlational Principal Component Analysis (CPCA) generalizes the Principal Component Analysis (PCA) algorithm to the case with data distributed on a high-dimensional hypersphere. Their advantages stem from two facts: 1) directly working on normalized data, which are often the outputs from data preprocessing, and 2) directly designed with the correlation metric, which is shown to be generally better than euclidean distance for classification purpose in many real-world applications. Extensive visual recognition experiments compared with existing feature extraction algorithms demonstrate the effectiveness of the proposed algorithms. [ABSTRACT FROM AUTHOR]

This paper presents a novel real-time multimodal human-avatar interaction (RTM-HAI) framework with vision-based remote animation control (RAC). The framework is designed for both mobile and desktop avatar-based human-machine or human-human visual communications in real-world scenarios. Using 3-D components stored in the Java mobile 3-D (M3G) file format, the avatar models can be flexibly constructed and customized on the fly on any mobile devices or systems that support the M3G standard. For the RAC head tracker, we propose a 2-D real-time face detection/tracking strategy through an interactive loop, in which the detection and tracking complement each other for efficient and reliable face localization, tolerating extreme user movement. With the face location robustly tracked, the RAC head tracker selects a main user and estimates the user's head rolling, tilting, yawing, scaling, horizontal, and vertical motion in order to generate avatar animation parameters. The animation parameters can be used either locally or remotely and can be transmitted through socket over the network. In addition, it integrates audio-visual analysis and synthesis modules to realize multichannel and runtime animations, visual TTS and real-time viseme detection and rendering. The framework is recognized as an effective design for future realistic industrial products of humanoid kiosk and human-to-human mobile communication. [ABSTRACT FROM AUTHOR]

In this paper we present a novel class of human model described by convolution surface attached to articulated kinematics skeletons. The human pose can be estimated from silhouette in monocular images. The contribution of this paper consists of three points: First, human model of convolution surface is presented and its shape is deformable when changing polynomial parameters and radius parameters. Second, convolution surface and curve correspondence theorem is presented to give a map between 3D pose and 2D contour. Third, we model the human silhouette with convolution curve in order to estimate joint parameters from monocular images and we also give an effective constraint function. Evaluation of this approach is performed on some video frames about a walking man. The experiment result shows that our method works well without self-occlusion. [Copyright &y& Elsevier]

We propose an HMM model for contour detection based on multiple visual cues in spatial domain and improve it by joint probabilistic matching to reduce background clutter. It is further integrated with unscented Kalman filter to exploit object dynamics in nonlinear systems for robust contour tracking. [ABSTRACT FROM AUTHOR]

This paper presents a new method of motion analysis of articulated objects from feature point correspondences over monocular perspective images without imposing any constraints on motion. An articulated object is modeled as a kinematic chain consisting of joints and links, and its 3D joint positions are estimated within a scale factor using the connection relationship of two links over two or three images. Then, twists and exponential maps are employed to represent the motion of each link, including the general motion of the base link and the rotation of other links around their joints. Finally, constraints from image point correspondences, which are similar to that of the essential matrix in rigid motion, are developed to estimate the motion. In the algorithm, the characteristic of articulated motion, i.e., motion correlation among links, is applied to decrease the complexity of the problem and improve the robustness. A point pattern matching algorithm for articulated objects is also discussed in this paper. Simulations and experiments on real images show the correctness and efficiency of the algorithms. [ABSTRACT FROM AUTHOR]

Capturing the human hand motion from video involves the estimation of the rigid global hand pose as well as the nonrigid finger articulation. The complexity induced by the high degrees of freedom of the articulated hand challenges many visual tracking techniques. For example, the particle filtering technique is plagued by the demanding requirement of a huge number of particles and the phenomenon of particle degeneracy. This paper presents a novel approach to tracking the articulated hand in video by learning and integrating natural hand motion priors. To cope with the finger articulation, this paper proposes a powerful sequential Monte Carlo tracking algorithm based on importance sampling techniques, where the importance function is based on an initial manifold model of the articulation configuration space learned from motion-captured data. In addition, this paper presents a divide-and-conquer strategy that decouples the hand poses and finger articulations and integrates them in an iterative framework to reduce the complexity of the problem. Our experiments show that this approach is effective and efficient for tracking the articulated hand. This approach can be extended to track other articulated targets. [ABSTRACT FROM AUTHOR]

Digital watermarking is an important technique to protect intellectual property right and to transmit useful secondary data. This paper investigates the performance analysis and error exponents of asymmetric watermarking systems. Asymmetric watermarking provides potentially better levels of security. Its detection is much different from commonly used watermarking detectors, particularly when both gain factors and exact values of the watermark are not known to the detector. Optimum detectors are constructed in this paper. To handle unwieldy computations of large matrices, we develop equivalent yet more practical detectors in the frequency domain using asymptotic analysis techniques. Due to the nonlinearity nature of these detectors, their performance analysis is challenging, especially when the size of the watermark becomes large. Gaussian approximations using the central limit theorem is limited in modeling the performance. To obtain fundamental performance limits, we go a step further to make use of asymptotic statistical analysis techniques to derive the exact error exponents. The analytical method gives insights into the performance of asymmetric watermarking, and paves the way toward seeking efficient watermarks for more general asymmetric watermarking systems. [Copyright &y& Elsevier]

Motion can provide useful information about the structure of an articulated object. In this paper, we develop a method to estimate the revolute joints from two monocular images of the moving articulated object. Firstly, according to the characteristic of the articulated structure and motion, constraint equations about motion and joint parameters from image point correspondences are deduced, which provide crucial information for joint estimation. Then, the position ambiguity caused by the one DOF attribute, and the scale ambiguity resulted from monocular images are discussed. Also, a skillful function is employed to avoid the degeneration cases of estimation. Finally, the joint axis are computed from the estimated rotation matrices, and all estimated joint points are scaled to have the same scale factor to their ground truth. Simulations and experiments on real images show the correctness and efficiency of the algorithm. [Copyright &y& Elsevier]

This paper presents a novel fused hidden Markov model (fused HMM) for integrating tightly coupled time series, such as audio and visual features of speech. In this model, the time series are first modeled by two conventional HMMs separately. The resulting HMMs are then fused together using a probabilistic fusion model, which is optimal according to the maximum entropy principle and a maximum mutual information criterion. Simulations and bimodal speaker verification experiments show that the proposed model can significantly reduce the recognition errors in noiseless or noisy environments. [ABSTRACT FROM AUTHOR]

In content-based image retrieval (CBIR), experimental (trial-and-error) query with relevance feedback is essential for successful retrieval. Unfortunately, the traditional user interfaces are not suitable for trying different combinations of query examples. This is because first, these systems assume query examples are always added incrementally. Second, the query and the result display are done on the same workspace. Once the user removes an image from the query examples, the image may disappear from the user interface. In addition, it is difficult to combine the result of different queries.In this paper, we propose a new interface for Content-based image retrieval named ImageGrouper. ImageGrouper is a Group-Oriented user interface in that all operations are done by creating groups of images. This approach has several advantages. First, the users can interactively compare different combinations of the query examples by dragging and grouping the images on the workspace (Query-by-Group). Because the query results are displayed on another pane, the user can quickly review the results and modify the query. Combining different queries is also easy. Furthermore, the concept of “Image Groups” is also applied for annotating and organizing many images. The Annotation-by-Groups method relieves the user of tedious task of annotating textual information on a large number of images. This method realizes a hierarchical annotation of the images as well as Bulk Annotation. The Organize-by-Group method lets the users manipulate the image groups as “Photo Albums” to organize the images. Finally, the usability of the system is compared with the traditional user interfaces. By incorporating the lessons from the experiments, the usability of ImageGrouper is further improved. [Copyright &y& Elsevier]

This paper deals with motion estimation from image corner correspondences in two cases: the orthogonal corner and the general corner with known space angles. The contribution of the paper is in three folds: First, the three-dimensional structure of a corner is recovered easily from its image by introducing a new coordinate system; second, it is shown that the one corner and two points correspondences over two views are sufficient to uniquely determine the motion, i.e., the rotation and translation; third, experiments using both simulated data and real images are conducted, which present good results. [Copyright &y& Elsevier]

Describes a framework that systematically addresses facial deformation modeling, automatic facial motion analysis and real-time speech-driven face animation with expression using neural networks. Core of speech-driven face animation; Discussion of motion units and facial action coding system.

Presents a factor graph framework for semantic video indexing. Challenges in the research issues in video data management; Gap between the low-level representation and high level semantics; Improvement in the detection performance using the multinet.

We present the iFACE system, a visual speech synthesizer that provides a form of virtual face-to-face communication. The system provides an interactive tool for the user to customize a graphic head model for the virtual agent of a person based on his/her range data. The texture is mapped onto the customized model to achieve a realistic appearance. Face animations are produced by using text stream or speech stream to drive the model. A set of basic facial shapes and head action is manually built and used to synthesize expressive visual speech based on rules. [ABSTRACT FROM AUTHOR]

Cross-domain visual problems, such as image-to-image translation and domain adaptive object detection, have attracted increasing attentions in the last few years, and also become new rising and challenging directions for the computer vision community. Recently, despite enormous efforts of the field in data collection, there are still few datasets covering the instance-level image-to-image translation and domain adaptive object detection tasks simultaneously. In this work, we introduce a large-scale cross-domain benchmark CDTD (contains 155,529 high-resolution natural images across four different modalities with object bounding box annotations. A summary of the entire dataset is provided in the following sections. Dataset is available at: http://zhiqiangshen.com/projects/INIT/index.html.) for the new instance-level translation and object detection tasks. We provide comprehensive baseline results of the benchmark on both of these two tasks. Moreover, we proposed a novel instance-level image-to-image translation approach called INIT and a gradient detach method for the domain adaptive object detection to harvest and exert dataset's function of the instance level annotations across different domains. [ABSTRACT FROM AUTHOR]

In this paper, we propose a deep CNN to tackle the image restoration problem by learning formatted information. Previous deep learning based methods directly learn the mapping from corrupted images to clean images, and may suffer from the gradient exploding/vanishing problems of deep neural networks. We propose to address the image restoration problem by learning the structured details and recovering the latent clean image together, from the shared information between the corrupted image and the latent image. In addition, instead of learning the pure difference (corruption), we propose to add a residual formatting layer and an adversarial block to format the information to structured one, which allows the network to converge faster and boosts the performance. Furthermore, we propose a cross-level loss net to ensure both pixel-level accuracy and semantic-level visual quality. Evaluations on public datasets show that the proposed method performs favorably against existing approaches quantitatively and qualitatively. [ABSTRACT FROM AUTHOR]

Image denoising and high-level vision tasks are usually handled independently in the conventional practice of computer vision, and their connection is fragile. In this paper, we cope with the two jointly and explore the mutual influence between them with the focus on two questions, namely (1) how image denoising can help improving high-level vision tasks, and (2) how the semantic information from high-level vision tasks can be used to guide image denoising. First for image denoising we propose a convolutional neural network in which convolutions are conducted in various spatial resolutions via downsampling and upsampling operations in order to fuse and exploit contextual information on different scales. Second we propose a deep neural network solution that cascades two modules for image denoising and various high-level tasks, respectively, and use the joint loss for updating only the denoising network via back-propagation. We experimentally show that on one hand, the proposed denoiser has the generality to overcome the performance degradation of different high-level vision tasks. On the other hand, with the guidance of high-level vision information, the denoising network produces more visually appealing results. Extensive experiments demonstrate the benefit of exploiting image semantics simultaneously for image denoising and high-level vision tasks via deep learning. The code is available online: https://github.com/Ding-Liu/DeepDenoising [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel semi-online approach to tracking multiple people. In contrast to conventional offline approaches that take the whole image sequence as input, our semi-online approach tracks people in a frame-by-frame manner by exploring the time, space and multi-camera relationship of detection hypotheses in the near future frames. We cast the multi-people tracking task as a re-identification problem, and explicitly account for objects' appearance changes and longer-term associations. We model our approach using a Multi-Label Markov Random Field, and introduce a fast α -expansion algorithm to solve it efficiently. To our best knowledge, this is the first semi-online approach achieved by re-identification. It yields very promising tracking results especially in challenging cases, such as scenarios of the crowded streets where pedestrians frequently occlude each other, scenes captured with moving cameras where objects may disappear and reappear randomly, and videos under changing illuminations wherein the appearances of objects are influenced. [ABSTRACT FROM AUTHOR]