Search

Textureless 3D object tracking of the object's position and orientation is a considerably challenging problem, for which a 3D model is commonly used. The 3D-2D correspondence between a known 3D object model and 2D scene edges in an image is standardly used to locate the 3D object, one of the most important problems in model-based 3D object tracking. State-of-the-art methods solve this problem by searching correspondences independently. However, this often fails in highly cluttered backgrounds, owing to the presence of numerous local minima. To overcome this problem, we propose a new method based on global optimization for searching these correspondences. With our search mechanism, a graph model based on an energy function is used to establish the relationship of the candidate correspondences. Then, the optimal correspondences can be efficiently searched with dynamic programming. Qualitative and quantitative experimental results demonstrate that the proposed method performs favorably compared to the state-of-the-art methods in highly cluttered backgrounds. [ABSTRACT FROM AUTHOR]

Current low-cost depth sensing techniques, such as Microsoft Kinect, still can achieve only limited precision. The resultant depth maps are often found to be noisy, misaligned with the color images, and even contain many large holes. These limitations make it difficult to be adopted by many graphics applications. In this paper, we propose a computational approach to address the problem. By fusing raw depth values with image color, edges and smooth priors in a Markov random field optimization framework, both misalignment and large holes can be eliminated effectively, our method thus can produce high-quality depth maps that are consistent with the color image. To achieve this, a confidence map is estimated for adaptive weighting of different cues, an image inpainting technique is introduced to handle large holes, and contrasts in the color image are also considered for an accurate alignment. Experimental results demonstrate the effectiveness of our method. [ABSTRACT FROM AUTHOR]

Action recognition solely based on video data has known to be very sensitive to background activity, and also lacks the ability to discriminate complex 3D motion. With the development of commercial depth cameras, skeleton-based action recognition is becoming more and more popular. However, the skeleton-based approach is still very challenging because of the large variation in human actions and temporal dynamics. In this paper, we propose a hierarchical model for action recognition. To handle confusing motions, a motion-based grouping method is proposed, which can efficiently assign each video a group label, and then for each group, a pre-trained classifier is used for frame-labeling. Unlike previous methods, we adopt a bottom-up approach that first performs action recognition for each frame. The final action label is obtained by fusing the classification to its frames, with the effect of each frame being adaptively adjusted based on its local properties. To achieve online real-time performance and suppressing noise, bag-of-words is used to represent the classification features. The proposed method is evaluated using two challenge datasets captured by a Kinect. Experiments show that our method can robustly recognize actions in real-time. [ABSTRACT FROM AUTHOR]

All depth data captured by Kinect devices are noisy, and sometimes even lost or shifted, especially around the edges of the depth. In this paper, we propose an approach to generate a per-pixel confidence measurement for each depth map captured by Kinect devices in indoor environments through supervised learning. Several distinguishing features from both the color images and depth maps are selected to train depth map estimators using Random Forest regressor. Using this estimator, we can predict a confidence map of any depth map captured by Kinect devices. Usage of other devices, such as an industrial laser scanner, is unnecessary, making the implementation more convenient. The experiments demonstrate precise confidence prediction of the depth. [ABSTRACT FROM AUTHOR]

In augmented reality, it is essential that the rendered virtual objects are embedded harmonically into the view of the background scenes and their appearance should reflect the changing lighting condition of the real scene to ensure illumination consistency. In this paper, we propose a novel method to solve for the sunlight and skylight basis images of static outdoor scenes from a time-lapse image sequence. It is proved that the resulted basis images encapsulate the geometry and material reflectivity of the scene, correspond to the global illumination effects of the outdoor scene under a unit intensity of the sunlight and skylight. Our method is fully automatic. Unlike previous methods, it gets rid of the constraints that the reflectance of all objects in scenes should be ideal diffuse, or the weather condition should be overcast or sunshine. During decomposition, we first detect shadowed pixels by analyzing the time-lapse curve of each pixel through k-means clustering, the basis images of sunlight and skylight are then solved by an iterative procedure with the decomposition equation. The basis images are further optimized by exploiting their constraints and priors. Experimental results demonstrate the effectiveness and flexibility of the proposed method. Our method can also be applied in image understanding and compressing. [ABSTRACT FROM AUTHOR]

Stratum structure detection is a fundamental problem in geological engineering. One of the most commonly employed detection technologies is to shoot videos of a borehole using a forward moving camera. Using this technology, the problem of stratum structure detection is transformed into the problem of constructing a panoramic image from a low quality video. In this paper, we propose a novel method for creating a panoramic image of a borehole from a video sequence without the need of camera calibration and tracking. To stitch together pixels of neighboring image frames, our camera model is designed with a focal length changing feature, along with a small rotational freedom in the two-dimensional image space. Our camera model assumes that target objects lie on a cylindrical wall and that the camera moves forward along the central axis of the cylindrical wall. Based on these two assumptions, our method robustly resolves these two degrees-of-freedoms in our camera model through KLT feature tracking. Since the quality of the result video is affected by possible illumination overflow, camera lens blurring, and low video resolution, we introduce a cost function for eliminating seams between stitching strips. Our cost function is designed based on Markov Random Field and optimized using a belief propagation algorithm. Using our method, we can automatically construct a panorama image with good resolution, smoothness, and continuousness both in the texture and illumination space. Experiment results show that our method could efficiently generate panoramas for long video sequences with satisfying visual quality. [ABSTRACT FROM AUTHOR]

We propose a novel method, based on the two-pass Z-buffer algorithm, to calculate shadows with sufficient precision and efficiency for rendering a daytime landscape with solar penumbrae. The special feature of the proposed method is that the shadows can be preserved with a precision superior to that of any visible surface. We use the optimal number of plural shadow buffers to do this; it gives a fairly satisfying trade–off between computation time and quality of shadows. [ABSTRACT FROM AUTHOR]