Search

Despite the growing number of unmanned aerial vehicles (UAVs) and aerial videos, there is a paucity of studies focusing on the aesthetics of aerial videos that can provide valuable information for improving the aesthetic quality of aerial photography. In this article, we present a method of deep multimodality learning for UAV video aesthetic quality assessment. More specifically, a multistream framework is designed to exploit aesthetic attributes from multiple modalities, including spatial appearance, drone camera motion, and scene structure. A novel specially designed motion stream network is proposed for this new multistream framework. We construct a dataset with 6,000 UAV video shots captured by drone cameras. Our model can judge whether a UAV video was shot by professional photographers or amateurs together with the scene type classification. The experimental results reveal that our method outperforms the video classification methods and traditional SVM-based methods for video aesthetics. In addition, we present three application examples of UAV video grading, professional segment detection and aesthetic-based UAV path planning using the proposed method., Comment: IEEE Trans. on Multimedia, 2020

Due to the latest progress in image sensor manufacturing technology, the emergence of a sensor equipped with an RGGB Bayer filter and a directional polarizing filter has brought significant advantages to computer vision tasks where RGB and polarization information is required. In this regard, joint chromatic and polarimetric image demosaicing is indispensable. However, as a new type of array pattern, there is no dedicated method for this challenging task. In this Letter, we collect, to the best of our knowledge, the first chromatic-polarization dataset and propose a chromatic-polarization demosaicing network (CPDNet) to address this joint chromatic and polarimetric image demosaicing issue. The proposed CPDNet is composed of the residual block and the multi-task structure with the costumed loss function. The experimental results show that our proposed methods are capable of faithfully recovering full 12-channel chromatic and polarimetric information for each pixel from a single mosaic image in terms of quantitative measures and visual quality.

Unsupervised co-segmentation for a set of 3D shapes is a challenging problem as no prior information is provided. The accuracy of the current approaches is necessarily restricted by the accuracy of the unsupervised face classification, which is used to provide an initialization for the following optimization to improve the consistency between adjacent faces. However, it is exceedingly difficult to obtain a satisfactory initialization pre-segmentation owing to variation in topology and geometry of 3D shapes. In this study, we consider the unsupervised 3D shape co-segmentation as an exemplar-based clustering problem, aimed at simultaneously discovering optimal exemplars and obtaining co-segmentation results. Therefore, we introduce a novel exemplar-based clustering method based on affinity propagation for 3D shape co-segmentation, which can automatically identify representative exemplars and patterns in 3D shapes considering the high-order statistics, yielding consistent and accurate co-segmentation results. Experiments using various datasets, especially large sets with 200 or more shapes that would be challenging to manually segment, demonstrate that our method exhibits a better performance compared to state-of-the-art methods.

Online 3D reconstruction of real-world scenes has been attracting increasing interests from both the academia and industry, especially with the consumer-level depth cameras becoming widely available. Recent most online reconstruction systems take live depth data from a moving Kinect camera and incrementally fuse them to a single high-quality 3D model in real time. Although most real-world scenes have static environment, the daily objects in a scene often move dynamically, which are non-trivial to reconstruct especially when the camera is also not still. To solve this problem, we propose a single depth camera-based real-time approach for simultaneous reconstruction of dynamic object and static environment, and provide solutions for its key issues. In particular, we first introduce a robust optimization scheme which takes advantage of raycasted maps to segment moving object and background from the live depth map. The corresponding depth data are then fused to the volumes, respectively. These volumes are raycasted to extract views of the implicit surface which can be used as a consistent reference frame for the next iteration of segmentation and tracking. Particularly, in order to handle fast motion of dynamic object and handheld camera in the fusion stage, we propose a sequential 6D pose prediction method which largely increases the registration robustness and avoids registration failures occurred in conventional methods. Experimental results show that our approach can reconstruct moving object as well as static environment with rich details, and outperform conventional methods in multiple aspects.

This paper presents a new method for 3D model labeling guided by weakly tagged 2D color images. Many previous methods on 3D model labeling achieve impressive performances using large training data sets. However, it is difficult and time-consuming to build such a carefully annotated data set. In order to solve this problem, we take advantage of the large number of weakly tagged color images to label the 3D models. In our approach, we first collect and tag the web color images with semantic annotations. Then we project the input 3D model into multiview projections. Through the multiview alignment, we transfer the semantic labels onto the model projections via a color-weighting process. Combining pre-segment information, we back-project the labels and get final labeling results. Experimental results between two benchmarks show that our approach could get comparable labeling accuracy compared to other two state-of-art methods without expensive training cost.

For the task of mobility analysis of 3D shapes, we propose joint analysis for simultaneous motion part segmentation and motion attribute estimation, taking a single 3D model as input. The problem is significantly different from those tackled in the existing works which assume the availability of either a pre-existing shape segmentation or multiple 3D models in different motion states. To that end, we develop Shape2Motion which takes a single 3D point cloud as input, and jointly computes a mobility-oriented segmentation and the associated motion attributes. Shape2Motion is comprised of two deep neural networks designed for mobility proposal generation and mobility optimization, respectively. The key contribution of these networks is the novel motion-driven features and losses used in both motion part segmentation and motion attribute estimation. This is based on the observation that the movement of a functional part preserves the shape structure. We evaluate Shape2Motion with a newly proposed benchmark for mobility analysis of 3D shapes. Results demonstrate that our method achieves the state-of-the-art performance both in terms of motion part segmentation and motion attribute estimation., Comment: CVPR 2019 (oral presentation); Corresponding author: Kai Xu (kevin.kai.xu@gmail.com); Project page: www.kevinkaixu.net/projects/shape2motion.html

With the increasing availability of high-resolution images, videos, and 3D models, the demand for scalable large data processing techniques increases. We introduce a method of sparse dictionary learning for edit propagation of large input data. Previous approaches for edit propagation typically employ a global optimization over the whole set of pixels (or vertexes), incurring a prohibitively high memory and time-consumption for large input data. Rather than propagating an edit pixel by pixel, we follow the principle of sparse representation to obtain a representative and compact dictionary and perform edit propagation on the dictionary instead. The sparse dictionary provides an intrinsic basis for input data, and the coding coefficients capture the linear relationship between all pixels and the dictionary atoms. The learned dictionary is then optimized by a novel scheme, which maximizes the Kullback-Leibler divergence between each atom pair to remove redundant atoms. To enable local edit propagation for images or videos with similar appearance, a dictionary learning strategy is proposed by considering range constraint to better account for the global distribution of pixels in their feature space. We show several applications of the sparsity-based edit propagation, including video recoloring, theme editing, and seamless cloning, operating on both color and texture features. Our approach can also be applied to computer graphics tasks, such as 3D surface deformation. We demonstrate that with an atom-to-pixel ratio in the order of 0.01% signifying a significant reduction on memory consumption, our method still maintains a high degree of visual fidelity.

Clouds are important components of the fascinating natural images. However, extracting cloud shapes from images remains a challenging task. This paper presents a calculation method for estimating the shape of a cumulus cloud from a single image suitable for flight simulations and games. The shape of the cloud is assumed to be symmetric. Based on this assumption, the intensities of pixels are correlated with the geometry of a cloud's surface via a simplified single scattering model. A propagation scheme is designed to derive the surface progressively, and mesh editing techniques are used to improve the surface. Finally, the cloud is represented by a particle system. The results show that the proposed method can generate realistic cumulus clouds that are similar to those found in the images in terms of the shape distribution.

Skin synthesis is important for the actual appearance of digital human models. However, it is difficult to design a general algorithm to efficiently produce high quality results. This paper proposes a parallel texture synthesis method for large scale skin of digital human models. Two major procedures are included in this method, a parallel matching procedure and a multi-pass optimizing procedure. Compared with other methods, this algorithm is easy to use, requires only a small size of skin image as input, and generates an arbitrary size of skin texture with high quality. As demonstrated by experiments, the effectiveness of this skin texture synthesis method is confirmed.

This paper systematically advocates an interactive volumetric image manipulation framework, which can enable the rapid deployment and instant utility of patient-specific medical images in virtual surgery simulation while requiring little user involvement. We seamlessly integrate multiple technical elements to synchronously accommodate physics-plausible simulation and high-fidelity anatomical structures visualization. Given a volumetric image, in a user-transparent way, we build a proxy to represent the geometrical structure and encode its physical state without the need of explicit 3-D reconstruction. On the basis of the dynamic update of the proxy, we simulate large-scale deformation, arbitrary cutting, and accompanying collision response driven by a non-linear finite element method. By resorting to the upsampling of the sparse displacement field resulted from non-linear finite element simulation, the cut/deformed volumetric image can evolve naturally and serves as a time-varying 3-D texture to expedite direct volume rendering. Moreover, our entire framework is built upon CUDA Beihang University, Beijing, China and thus can achieve interactive performance even on a commodity laptop. The implementation details, timing statistics, and physical behavior measurements have shown its practicality, efficiency, and robustness. Copyright © 2013 John Wiley & Sons, Ltd.

Automatic image completion can provide convenient editing of consumer images. Most image completion methods find the best patch locally and then copy it to the hole region for texture inpainting. Since the best patch is fixed in size, it is hard to adapt these methods efficiently either to various patterns or to content synthesis. Meanwhile, salient image structures can be estimated and propagated to guide the texture inpainting process for more plausible results. This paper presents a novel image completion method using structure guided texture inpainting. We do not require any interactions to achieve automatic two-stage image completion. In the structure completion stage, the salient structures around the hole region are detected, and then the missing structure curves are completed with Euler spiral. The textures along the structure curves are completed through belief propagation. In the texture inpainting stage, we propose multi-scale patches and global optimization to inpaint the remaining texture in the hole regions guided by the completed structures. First, with defined patch sizes, the hole region is divided into lattice patches, making it possible for multiple patch sizes to render multiscale descriptions of this image. A multi-scale graph is then built for the hole region and formulated as a posterior probability model. Second, using a simulated annealing based Markov chain Monte Carlo method, an inference algorithm is designed to find a global optimization solution for the posterior probability model. The experiments show that our method can automatically complete the hole region and preserve plausible structure shapes of existing ones in various scenarios. The texture inpainting results are more convincing with guidance from the completed structures, and our method can guarantee and accelerate convergence of the global optimization.

Estimation of human shape from images has numerous applications ranging from graphics to surveillance. A single image provides insufficient constraints (e.g. clothing), making human shape estimation more challenging. We propose a method to simultaneously estimate a person's clothed and naked shapes from a single image of that person wearing clothing. The key component of our method is a deformable model of clothed human shape. We learn our deformable model, which spans variations in pose, body, and clothes, from a training dataset. These variations are derived by the non-rigid surface deformation, and encoded in various low-dimension parameters. Our deformable model can be used to produce clothed 3D meshes for different people in different poses, which neither appears in the training dataset. Afterward, given an input image, our deformable model is initialized with a few user-specified 2D joints and contours of the person. We optimize the parameters of the deformable model by pose fitting and body fitting in an iterative way. Then the clothed and naked 3D shapes of the person can be obtained simultaneously. We illustrate our method for texture mapping and animation. The experimental results on real images demonstrate the effectiveness of our method.

To bring computer vision closer to human vision, we attempt to enable computer to understand the occlusion relationship in an image. In this paper, we propose five low dimensional region-based occlusion cues inspired by the human perception of occlusion. These cues are semantic cue, position cue, compactness cue, shared boundary cue and junction cue. We apply these cues to predict the region-wise occlusion relationship in an image and infer the layer sequence of the image scene. A preference function, trained with samples consisting of these cues, is defined to predict the occlusion relationship in an image. Then we put all the occlusion predictions into the layering algorithm to infer the layer sequence of the image scene. The experiments on rural, artificial and outdoor scene datasets show the effectiveness of our method for occlusion relationship prediction and image scene layering.

We propose a novel edit propagation algorithm for interactive image and video manipulations. Our approach uses the locally linear embedding (LLE) to represent each pixel as a linear combination of its neighbors in a feature space. While previous methods require similar pixels to have similar results, we seek to maintain the manifold structure formed by all pixels in the feature space. Specifically, we require each pixel to be the same linear combination of its neighbors in the result. Compared with previous methods, our proposed algorithm is more robust to color blending in the input data. Furthermore, since every pixel is only related to a few nearest neighbors, our algorithm easily achieves good runtime efficiency. We demonstrate our manifold preserving edit propagation on various applications.

Relighting a portrait in a single image is still a challenging problem, particularly when only a single artistic reference photograph or painting is provided. In this paper, we propose an artistic illumination transfer system for portraits based on a database of portrait images (photographs and paintings) associated with hand-drawn illumination templates (276) by artists. Users can select a reference portrait image in the database, and the corresponding illumination template is transferred to an input portrait using image warping. Users can also provide reference portrait images those are not in the database. Based on the Face Illumination Descriptor (FID), the system selects from the database the reference image with the closest illumination to that of the user-provided reference image and adjusts the corresponding illumination template to match the contrast of the user-provided reference image. Experiments on not only paintings but also photographs, paper-cuts and sketches demonstrate that convincing illumination transferred results can be rendered by our system. © 2012 Wiley Periodicals, Inc.

Stitching motions in multiple videos into a single video scene is a challenging task in current video fusion and mosaicing research and film production. In this paper, we present a novel method of video motion stitching based on the similarities of trajectory and position of foreground objects. First, multiple video sequences are registered in a common reference frame, whereby we estimate the static and dynamic backgrounds, with the former responsible for distinguishing the foreground from the background and the static region from the dynamic region, and the latter functioning in mosaicing the warped input video sequences into a panoramic video. Accordingly, the motion similarity is calculated by reference to trajectory and position similarity, whereby the corresponding motion parts are extracted from multiple video sequences. Finally, using the corresponding motion parts, the foregrounds of different videos and dynamic backgrounds are fused into a single video scene through Poisson editing, with the motions involved being stitched together. Our major contributions are a framework of multiple video mosaicing based on motion similarity and a method of calculating motion similarity from the trajectory similarity and the position similarity. Experiments on everyday videos show that the agreement of trajectory and position similarities with the real motion similarity plays a decisive role in determining whether two motions can be stitched. We acquire satisfactory results for motion stitching and video mosaicing.

D scans are usually stored as range images or point clouds, which are inconvenient to be utilized in common graphics pipelines. Therefore non-rigid registration is required to convert the scans into triangulated 3D models. Traditional non-rigid registration methods deform and register a topology- stable template to approximate the point cloud without considering the complexity of the scanned object, this will lead to bad registration or redundancies in the results, especially for 3D dynamic, densely-sampled facial scans with a large number of degrees of freedom. To address this problem, in this paper we propose a new topology-adaptive non-rigid registration method for 3D facial scans. Our method avoids the drawback of traditional methods by combining the advantages of non- rigid registration and mesh subdivision/simplification. It reduces the redundancy of registered results effectively and guarantees the accuracy of registration by adding or removing triangles of template models. We introduce of the working principle of our method, then evaluate our technique with a variety of models, and finally give a quantitative analysis of the registration results.

Rendering global illumination for objects with mesostructure surfaces is a time-consuming task, and cannot presently be applied to interactive graphics. This paper presents a real-time rendering method based on a mesostructure height gradient map (MHGM) to exhibit lighting effects on meso-scale details in dynamic environments. We approximate global illumination using a lighting model including three components: incident ambient light, direct light and single bounce indirect light. MHGM is introduced to create local apex sets, which would help us to compute the three components adaptively. Our approach runs entirely on the graphics hardware, and uses deferred shading and the graphics pipeline to accelerate computation. We achieve high quality results which can render meso-scale details with approximate global illumination even for low-resolution geometric models. Moreover, our approach fully supports dynamic scenes and deformable objects.

In this paper, we present a novel automatic image completion solution in a greedy manner inspired by a primal sketch representation model. Firstly, an image is divided into structure (sketchable) components and texture (non-sketchable) components, and the missing structures, such as curves and corners, are predicted by tensor voting. Secondly, the textures along structural sketches are synthesized with the sampled patches of some known structure components. Then, using the texture completion priorities decided by the confidence term, data term and distance term, the similar image patches of some known texture components are found by selecting a point with the maximum priority on the boundary of hole region. Finally, these image patches inpaint the missing textures of hole region seamlessly through graph cuts. The characteristics of this solution include: (1) introducing the primal sketch representation model to guide completion for visual consistency; (2) achieving fully automatic completion. The experiments on natural images illustrate satisfying image completion results.

We propose a novel approach to generate 4D light field in the physical world for lighting reproduction. The light field is generated by projecting lighting images on a lens array. The lens array turns the projected images into a controlled anisotropic point light source array which can simulate the light field of a real scene. In terms of acquisition, we capture an array of light probe images from a real scene, based on which an incident light field is generated. The lens array and the projectors are geometric and photometrically calibrated, and an efficient resampling algorithm is developed to turn the incident light field into the images projected onto the lens array. The reproduced illumination, which allows per-ray lighting control, can produce realistic lighting result on real objects, avoiding the complex process of geometric and material modeling. We demonstrate the effectiveness of our approach with a prototype setup.

This paper presents a semantic labeling framework with geodesic propagation (GP). Under the same framework, three algorithms are proposed, including GP, supervised GP (SGP) for image, and hybrid GP (HGP) for video. In these algorithms, we resort to the recognition proposal map and select confident pixels with maximum probability as the initial propagation seeds. From these seeds, the GP algorithm iteratively updates the weights of geodesic distances until the semantic labels are propagated to all pixels. On the contrary, the SGP algorithm further exploits the contextual information to guide the direction of propagation, leading to better performance but higher computational complexity than the GP. For video labeling, we further propose the HGP algorithm, in which the geodesic metric is used in both spatial and temporal spaces. Experiments on four public data sets show that our algorithms outperform several state-of-the-art methods. With the GP framework, convincing results for both image and video semantic labeling can be obtained.

We introduce a method of sparse dictionary learning for edit propagation of high-resolution images or video. Previous approaches for edit propagation typically employ a global optimization over the whole set of image pixels, incurring a prohibitively high memory and time consumption for high-resolution images. Rather than propagating an edit pixel by pixel, we follow the principle of sparse representation to obtain a compact set of representative samples (or features) and perform edit propagation on the samples instead. The sparse set of samples provides an intrinsic basis for an input image, and the coding coefficients capture the linear relationship between all pixels and the samples. The representative set of samples is then optimized by a novel scheme which maximizes the KL-divergence between each sample pair to remove redundant samples. We show several applications of sparsity-based edit propagation including video recoloring, theme editing, and seamless cloning, operating on both color and texture features. We demonstrate that with a sample-to-pixel ratio in the order of 0.01%, signifying a significant reduction on memory consumption, our method still maintains a high-degree of visual fidelity.

It is hard to extract skeletons using geometric models with limited information such as point models without topology and models with low resolution. This paper proposes a novel skeleton-extracting and animation approach for point models, which explores the differential properties of point models without triangulating the discrete points. First, this paper gives definitions of the attributes of surface and tangency, and then uses these attributes to iteratively contract the model to get the skeleton. At last the point models animation driven by the resulting skeleton is presented. Results show that our approach can get good skeletons even if the model is lacking connection information and in low resolution. Also, it can obtain good animation results and make the animation more convenient since the binding of skeleton points and skin is obtained.

This paper presents a novel method to simultaneously estimate the clothed and naked 3D shapes of a person. The method needs only a single photograph of a person wearing clothing. Firstly, we learn a deformable model of human clothed body shapes from a database. Then, given an input image, the deformable model is initialized with a few user-specified 2D joints and contours of the person. And the correspondence between 3D shape and 2D contours is established automatically. Finally, we optimize the parameters of the deformable model in an iterative way, and then obtain the clothed and naked 3D shapes of the person simultaneously. The experimental results on real images demonstrate the effectiveness of our method.

In this paper we propose a novel semantic label transfer method using supervised geodesic propagation (SGP). We use supervised learning to guide the seed selection and the label propagation. Given an input image, we first retrieve its similar image set from annotated databases. A Joint Boost model is learned on the similar image set of the input image. Then the recognition proposal map of the input image is inferred by this learned model. The initial distance map is defined by the proposal map: the higher probability, the smaller distance. In each iteration step of the geodesic propagation, the seed is selected as the one with the smallest distance from the undetermined superpixels. We learn a classifier as an indicator to indicate whether to propagate labels between two neighboring superpixels. The training samples of the indicator are annotated neighboring pairs from the similar image set. The geodesic distances of its neighbors are updated according to the combination of the texture and boundary features and the indication value. Experiments on three datasets show that our method outperforms the traditional learning based methods and the previous label transfer method for the semantic segmentation work.

This paper advocates a novel material-aware feature descriptor for volumetric image registration. We rigorously formulate a novel probability density function (PDF) based distance metric to devise a compact local feature descriptor supporting invariance of full 3D orientation and isometric deformation. The central idea is to employ anisotropic heat diffusion to characterize the detected local volumetric features. It is achieved by the elegant unification of diffusion tensor (DT) space construction based on local Hessian eigen-system, multi-scale feature extraction based on DT-weighted dyadic wavelet transform, and local distance definition based on PDF formulated in DT space. The diffusion, intrinsic structure-aware nature makes our volumetric feature descriptor more robust to noise. With volumetric images registration as verifiable application, various experiments on different volumetric images demonstrate the superiority of our descriptor.

We propose a method for interactive deformation of large detail meshes. Our method allows the users to manipulate the mesh directly using freely-selected handles on the mesh. To best preserve surface details, we introduce a new surface representation, the skin-detached surface. It represents a detail surface model as a peeled "skin" added over a simplified surface model. The "skin" contains the details of the surface while the simplified mesh maintains the rough shape. The deformation process consists of three steps: first, the "detached skin" is precomputed according to the detail mesh at the loading stage. Then we deform the simplified mesh following the nonlinear gradient domain mesh editing approach to satisfy the handle position constraints. Finally the "detached skin" is remapped onto the simplified mesh, resulting in a deformed detail mesh. We investigate the advantages as well as the limitations of our method by implementing a prototype system and applying it to several examples.

This paper proposes a Key-Frame-Free (KFF) prediction method for the medium-grain scalable (MGS) coding of H.264/SVC, in which the key pictures can be completely avoided to reduce the memory complexity and improve the coding efficiency. In our proposed method, the encoder employs a joint rate-distortion model to decide which quality layer is reconstructed and used for prediction of base quality macroblocks in the coarsest temporal layer, while the remaining macroblocks always predict and reconstruct from the highest quality layer. The proposed method requires no change to the H.264/SVC standard and experimental results show that compared with the MGS key-picture control method the proposed scheme significantly improves the scalable coding efficiency from 0.8 to 1.5dB at higher bit rates while maintains similar drift error as MGS with key pictures at lower bit rates. Our proposed method can also coexist with SVC-to-AVC rewrite function which is conflict with the key picture of MGS.

This paper presents an automatic video highlighting approach in relation to a soccer match video lasting over ninety minutes, in which only the match video and the target length of the highlight video are required for the composition. Firstly, we propose a core-around event model, which represents the highlight as a complex event and consists of three components: semantic relations, temporal relationship among the activities and local motion appearance of each activity. Secondly, we detect activities involved in a highlight in the soccer match video by the local motion appearance. Thirdly, a candidate highlight video segment is aligned with the model by a specific kernel activity extracted on the basis of the semantic relations, and then assigned a matching score. Finally, we splice together the highlight segments of high matching score. Experimental results demonstrate that the composed highlight videos are attractive episodes of soccer matches, and similar to those edited by professional editors.

This article proposes a novel image-based method to transfer illumination from a reference face image to a target face image through edge-preserving filters. According to our method, only a single reference image, without any knowledge of the 3D geometry or material information of the target face, is needed. We first decompose the lightness layers of the reference and the target images into large-scale and detail layers through weighted least square (WLS) filter after face alignment. The large-scale layer of the reference image is filtered with the guidance of the target image. Adaptive parameter selection schemes for the edge-preserving filters is proposed in the above two filtering steps. The final relit result is obtained by replacing the large-scale layer of the target image with that of the reference image. We acquire convincing relit result on numerous target and reference face images with different lighting effects and genders. Comparisons with previous work show that our method is less affected by geometry differences and can preserve better the identification structure and skin color of the target face.

In this paper we propose a novel nonparametric image parsing method for the image parsing problem in certain environment. A novel and efficient nearest neighbor matching scheme, the ANN bilateral matching scheme, is proposed. Based on the proposed matching scheme, we first retrieve some partially similar images for each given test image from the training image database. The test image can be well explained by these retrieved images, with similar regions existing in the retrieved images for each region in the test image. Then, we match the test image to the retrieved training images with the ANN bilateral matching scheme, and parse the test image by integrating multiple cues in a markov random field. Experiment on three datasets shows our method achieved promising parsing accuracy and outperformed two state-of-the-art nonparametric image parsing methods.

This paper presents a method for learning artistic portrait lighting template from a dataset of artistic and daily portrait photographs. The learned template can be used for (1) classification of artistic and daily portrait photographs, and (2) numerical aesthetic quality assessment of these photographs in lighting usage. For learning the template, we adopt Haar-like local lighting contrast features, which are then extracted from pre-defined areas on frontal faces, and selected to form a log-linear model using a stepwise feature pursuit algorithm. Our learned template corresponds well to some typical studio styles of portrait photography. With the template, the classification and assessment tasks are achieved under probability ratio test formulations. On our dataset composed of 350 artistic and 500 daily photographs, we achieve a 89.5% classification accuracy in cross-validated tests, and the assessment model assigns reasonable numerical scores based on portraits' aesthetic quality in lighting.

We propose a method for interactive deformation of large detailed meshes. Our method allows the users to manipulate the mesh directly using freely-selected handles on the mesh. To best preserve surface details, we introduce a new surface representation, the skin-detached surface. It represents a detailed surface model as a peeled “skin” added over a simplified surface model. The “skin” contains the details of the surface while the simplified mesh maintains the basic shape. The deformation process consists of three steps: At the mesh loading stage, the “skin” is precomputed according to the detailed mesh and detached from the simplified mesh. Then we deform the simplified mesh following the nonlinear gradient domain mesh editing approach to satisfy the handle position constraints. Finally the detailed “skin” is remapped onto the simplified mesh, resulting in a deformed detailed mesh. We investigate the advantages as well as the limitations of our method by implementing a prototype system and applying it to several examples.

An efficient polarization restraint based fast motion estimation algorithm for H.264 stereoscopic video coding is proposed in this paper. For parallel binocular-camera systems, there exists a certain principle called polarization restraint, according to which, for the object in a 3D scene, there is no vertical disparity between its projection point on the left plane and that on the right plane. The Motion Estimation (ME) complexity in the right view channel can be greatly reduced by adopting the algorithm presented in this paper. Experimental results show that, compared with the Full Search Block Match Algorithm (FSBMA), the number of search candidates of the proposed algorithm are only 1/136 of that of the FSBMA with similar decoded video quality. Besides, the computation complexity of the proposed algorithm is lower than that of Diamond Search (DS) and of Three Step Search (TSS) with much better decoded pictures’ quality.

This paper presents a rapid and robust method to align large sets of range scans captured by a 3D scanner automatically. The method incorporates the color information from the range data into the pairwise registration. Firstly, it detects the features using SIFT (Scale-Invariant Feature Transform) on grayscale images generated from two range scans to align. Then a quasi-dense matching algorithm, based on the match propagation principle, is applied to specify the matching pixel pairs between two images. All matches obtained are mapped to 3D space but in different world coordinates, and fitered by the 3D geometry constraint discovered from the range data. The remaining set of point correspondences is used to estimate the rigid transformation. Finally, a modified ICP (Iterative Closest Point) algorithm is applied to refine the result. The paper also describes a framework to use this alignment method for object reconstruction. The reconstruction proceeds by acquiring several range scans with color information from different directions, following which pair-wise of range data are aligned with the above method selectively and iteratively. Then a model graph containing the correct pair-wise matches is created and a span tree specifying a complete model is constructed. Finally a global optimization is performed to refine the result. This reconstruction technique achieves a robust and high performance in the application of rebuilding the 3D models of culture heritages for virtual museum automatically.

Texture atlas is widely used in many applications such as texture mapping, 3D Paint and illumination map texture synthesis etc. Several methods have been proposed for generating texture atlas, but most of them are not compact enough for some applications or require too much manual intervention. This paper presents a fully automatic BFD approach to generate compact texture atlas for triangular mesh models. Our main contribution is making some modifications to a piecewise mesh parameterization method to control the shape of charts by introducing a new local criterion, and developing a novel rectangle-packing algorithm.

Registering range images captured by 3D scanner and constructing a complete model of an object is frequently used in reverse engineering, computer game and animation, digital museum and digital heritage, etc. Many algorithms about registering the range images have been developed up to now. In this paper, a robust automated algorithm is presented based on surface characteristics extracting and matching techniques. The algorithm restores a set of point correspondences between every two adjacent range images without image position information or user interaction. Differential quantities are used as geometric descriptor and some other constraints are used to accelerate searching and matching processes and maintain result accuracy. After calculating the estimate of the rigid motion transformation between range images with selected point correspondences, a modified ICP algorithm is used to refine the result. The whole algorithm can be applied in many areas, especially in building the digital models of cultural artifacts and works of art.

Virtual Five Animals Exercise is an interactive game which is an important part of Virtual Olympic Museum for the Games of the XXIX Olympiad. Users can learn and imitate Five Animals Exercise without any interactive device. To do these, we make use of motion capture data to drive the virtual human motion, and use template matching method to recognize users' motion. We improve the motion editing and motion retargetting methods in our system. Many people enjoy our game.

In this paper, we describe a novel method to analyze captured human motion data. As described later, we analyze the sequences of footprint of the moving figure to extract different motion stages, as well as the switch frame between two adjacent stages. The displacement between left and right footprints is a time varying function. The change of motion (in direction, posture, speed etc.) can be reflected by the displacement between left and right footprints. Each motion stage is a sequence of motion data. Two adjacent motion stages can be different just in movement direction despite they are in the same movement style. In the second half of this paper, we apply footprint analysis on a short human motion clip of walking forward, and synthesis new motions of walking (running) along a path, of which the direction can be changed interactively in running time.

Texture Mapping plays a very important role in Computer Graphics. Texture Synthesis is one of the main methods to obtain textures, it makes use of sample textures to generate new textures. Texture Transfer is based on Texture Synthesis, it renders objects with textures taken from different objects. Currently, most of Texture Synthesis and Transfer methods use a single sample texture. A method for Texture Synthesis adn Transfer from multi samples was presented. For texture synthesis, the L-shaped neighborhood seaching approach was used. Users specify the proportion of each sample, the number of seed points, and these seed points are scattered randomly according to their samples in horizontal and vertical direction synchronously to synthesize textures. The synthesized textures are very good. For texture transfer, the luminance of the target image and the sample textures are analyzed. This procedure is from coarse to fine, and can produce a visually pleasing result.

Image-based rendering technique is a very interesting research area in Virtual Reality. This paper presents a novel approach to create spherical panoramas. With these panoramas, we make use of the idea of hyperlink organizing viewpoints and the virtual objects to construct image-based spherical virtual spaces. In such spaces, users can look in all directions in one viewpoint, wander widely by jumping from one viewpoint to another, and manipulate virtual objects freely.

Funded by National High Technology 863 Project, the distributed virtual environment network DVENET has been focused at incorporating realistic 3D visual simulation into a joint exercise. Visual tracking allows visual simulation to measure visual field and visual angle relied on by multiresolution rendering techniques including LOD (level of detail), morphing and LOL (level of light), which economize the computational cost. Based on vision perception of user, this paper describes our recent research in real-time multiresolution rendering of 3D models in virtual environment. This method calculates the critical parameters, such as view-distance, imaging parameter, brightness parameter, and so on, which control the switches between neighbor multiple resolution models and are correlative to user's visual field and visual angle. It might also be utilized for kinds of simulators responding to the presence of the multi-resolution target but based on the special visions of infrared equipment or radar.