Search

In recent years, robotic arm technology is in dire need of reform because of the remarkable advances in artificial intelligence and computer vision. The traditional robotic arm techniques, e.g., template matching algorithm and iterative closest point algorithm, suffer from the low precision issue, especially when the target objects overlap with each other, resulting in inaccurate estimation of overlapping objects. This paper proposes a precise grabbing of overlapping objects system based on an end-to-end deep neural network. The successful grabbing is realized in the case of overlapping objects. First, the datasets needed for network training were established, utilizing structured light to obtain the point cloud information of the arbitrarily placed target objects. Furthermore, we collect the corresponding postures as data labels via the teaching device of the robotic arm, and train the network models using the datasets and labels. Finally, we can predict the postures of the target objects in real time and transmit the results to a robotic arm to complete the grabbing work. The experiment results indicate that the proposed grabbing system can grab small irregular objects accurately, only using the point cloud information, estimating the posture of multiple target objects in the scene simultaneously, and estimating the posture of overlapping small objects in the scene.

Image-guided therapies have been on the rise in recent years as they can achieve higher accuracy and are less invasive than traditional methods. By combining augmented reality technology with image-guided therapy, more organs, and tissues can be observed by surgeons to improve surgical accuracy. In this review, 233 publications (dated from 2015 to 2020) on the design and application of augmented reality-based systems for image-guided therapy, including both research prototypes and commercial products, were considered for review. Based on their functions and applications. Sixteen studies were selected. The engineering specifications and applications were analyzed and summarized for each study. Finally, future directions and existing challenges in the field were summarized and discussed.

Structured light is a widely employed approach for profile measurement in active stereo vision. Recent methods have developed learning-based single pattern structured light systems and unstructured light stereo vision systems. However, their methods require additional cameras or a diffractive optical element. In addition, the calibration methods of the learning-based structured light systems require a large number of images, while the stereo vision method cannot detect the blocked region. Classically, a digital projector is treated as a pseudocamera, and the stripe pattern is employed to measure the depth. However, an additional pattern is employed to calibrate the camera-projector system in the stripe-based method. In this paper, 24 reliable features and a novel calibration algorithm are proposed. With the proposed calibration algorithm, the stripe patterns are calibrated without additional patterns, and with the proposed RGB features, the number of encoded columns is enlarged significantly. Experiments demonstrate that the proposed method is accurate and robust.

The recent advancement of motion tracking technology offers better treatment tools for conditions, such as movement disorders, as the outcome of the rehabilitation could be quantitatively defined. The accurate and fast angular information output of the inertial measurement unit tracking systems enables the collection of accurate kinematic data for clinical assessment. This article presents a study of a low-cost microelectromechanical system inertial measurement unit-based tracking system in comparison with the conventional optical tracking system. The system consists of seven microelectromechanical system inertial measurement units, which could be mounted on the lower limbs of the subjects. For the feasibility test, 10 human participants were instructed to perform three different motions: walking, running, and fencing lunges when wearing specially designed sleeves. The subjects’ lower body movements were tracked using our inertial measurement unit-based system and compared with the gold standard—the NDI Polaris Vega optical tracking system. The results of the angular comparison between the inertial measurement unit and the NDI Polaris Vega optical tracking system were as follows: the average cross-correlation value was 0.85, the mean difference of joint angles was 2.00°, and the standard deviation of joint angles was ± 2.65°. The developed microelectromechanical system–based tracking system provides an alternative low-cost solution to track joint movement. Moreover, it is able to operate on an Android platform and could potentially be used to assist outdoor or home-based rehabilitation.

We tackle the problem of dense reconstruction with a practical system, in which near point lighting (NPL) is employed. Different from the conventional formulation of photometric stereo (PS) that assumes parallel lighting, PS under the NPL condition is a nonlinear problem as the local surface normals are coupled with its distance to the camera as well as the light sources. After obtaining the locations of point lights by a calibration process, we develop a new framework to solve this nonlinear reconstruction problem via mesh deformation, in which each facet is corresponding to a pixel in the image captured by the camera. In our framework, mesh deformation is decoupled into an iteration of interlaced steps of local projection and global blending. Experimental results verify that our method can generate accurate estimation of surface shape under NPL in a few iterations. Besides, this approach is robust to errors on the positions of light sources and is easy to be implemented.

As the credibility and accuracy of binocular endoscope reconstruction are significantly affected by narrow baseline and large distortion of taken pictures, we develop a binocular endoscopic speckle system to address these limitations. The system consists of a projector and a binocular endoscope. First, the binocular endoscope captures the object surface, which illuminated by a projector with pseudo-random speckles. Then, we proposed a high-precision calibration method based on three-dimensional to improve the calibration precision of the system. Finally, in order to enhance the exactness and reliability of the stereo matching, Semi-Global Matching (SGM) algorithm is studied and improved. This paper aims to reduce the deviation and improve the 3D measurement accuracy of the system. In comparison with traditional stereo reconstruction, the reconstruction accuracy of cylinder and plane was improved 31.2% and 34.0% respectively.

Structured light sensing system can be used to obtain a high-precision 3D model of an object. In this paper, we present a mirror-based structured light sensing system for acquiring complete 3D surface models. Our system can obtain three point clouds by using planar mirror and register them together. First, we describe the mathematical model of our system, and propose a method to register the point clouds gained by our system. Finally, we evaluate the feasibility and accuracy of our mirror-based structured light sensing system by extensive experiments. The result shows that our system can reconstruct a full-body 3D model of the object successfully.

Phase measuring deflectometry (PMD) is an effective technique for three-dimensional measurement of specular surfaces. However, the ambiguity of monoscopic PMD and the time-consuming searching process of stereoscopic PMD are challenges for specular surface reconstruction. To solve it, we propose an iterative reconstruction algorithm for the stereoscopic phase measuring deflectometry system free of the time-consuming searching process for each pixel. An arbitrary seed point on the specular surface is accurately obtained via a coarse-to-fine optimization means without any other expensive and complicate auxiliaries. Then, a plane with the height of seed point is set as the initial surface form for the iteration, in which the pinhole model is used to find the linear relation to update the surface form. The converging height is the output as the final result. Simulations and experiments verify the feasibility and efficiency of our proposed method based on the stereoscopic phase measuring deflectometry system. The accuracy and robustness are comprehensively evaluated as well.

Both intensity and phase information of images have been the most important similarity measures in solving the general stereo matching problem. Intensity contains most of the imaging information of the scene/object, yet the phase information could reflect the local structure of images, which is more robust than the grayscale value. Plenty of work has been done in intensity-based or phase-based stereo matching methods. However, neither of them could work well enough when process images were taken under varied illuminations. A robust depth recovery method by making use of both intensity and phase information of stereo images properly is proposed. Firstly, 2D signal analysis is conducted by using the multiscale monogenic wavelet transform, from which local phase and intensity amplitude information are extracted into different scales. Secondly, disparity maps are estimated in different scales based on the intensity information. Thirdly, the optimal disparity is obtained by weighted-combining the disparity maps in different scales. The weighted coefficients are computed by making use of the phase information. Extensive experimental evaluation demonstrates the benefits of the proposed method.

The acquisition of the geometry of general scenes is related to the interplay of surface geometry, material properties and illumination characteristics. Surface texture and non-Lambertian reflectance properties degrade the reconstruction results by structured light technique. Existing structured light techniques focus on different coding strategy and light sources to improve reconstruction accuracy. The hybrid system consisting of a structured light technique and photometric stereo combines the depth value with normal information to refine the reconstruction results. In this paper, we propose a novel hybrid system consisting of stripe-based structured light and photometric stereo. The effect of surface texture and non-Lambertian reflection on stripe detection is first concluded. Contrary to existing fusion strategy, we propose an improved method for stripe detection to reduce the above factor&rsquo, s effects on accuracy. The reconstruction problem for general scene comes down to using reflectance properties to improve the accuracy of stripe detection. Several objects, including checkerboard, metal-flat plane and free-form objects with complex reflectance properties, were reconstructed to validate our proposed method, which illustrates the effectiveness on improving the reconstruction accuracy of complex objects. The three-step phase-shifting algorithm was implemented and the reconstruction results were given and also compared with ours. In addition, our proposed framework provides a new feasible scheme for solving the ongoing problem of the reconstruction of complex objects with variant reflectance. The problem can be solved by subtracting the non-Lambertian components from the original grey values of stripe to improve the accuracy of stripe detection. In the future, based on stripe structured light technique, more general reflection models can be used to model different types of reflection properties of complex objects.

Compared with existing depth cameras, such as RGB-D, RealSense and Kinect, stripe-based structured light (SL) has the potential for micrometer-level 3D measurement; this can be attributed to its higher coding capacity. While surface texture, high-reflective region, and occlusion remain some of the main sources leading to degraded reconstruction quality in complex objects, methods that are only based on SL cannot completely solve such problems in complex object reconstruction. In this paper, we developed an advanced fusion strategy for the reconstruction of complex objects in micrometer-level 3D measurement. This includes solving the above-mentioned inherent problems of a stripe-based SL system with the aid of photometric stereo (PS). Firstly, to improve the robustness of decoding and eliminate the effects of noise and occlusion on stripe detection, a novel scene-adaptive decoding algorithm based on a binary tree was proposed. Further, a robust and practical calibration method for area light sources in the PS system, which utilizes the absolute depth information from SL system, was introduced. A piecewise integration algorithm, which is based on a subregion divided by Gray code, was proposed by combining the depth values from SL with the normal information from PS. Remarkably, this method eliminates the effects of surface texture and high-reflective region on the reconstruction quality and improves the resolution to camera-level resolution. In experimental parts, a regular cylinder was reconstructed to demonstrate micrometer-level measurement accuracy and resolution enhancement by the proposed method. Then, improvement of the reconstruction accuracy for objects with surface texture was validated with a regular pyramid that had textures on it and a white paper with characters printed on it. Lastly, a complex object containing multiple phenomena was reconstructed with the newly proposed method to show its effectiveness for micrometer-level 3D measurement in complex objects. Evaluation of our proposed method shows the improvement of the proposed method on the existing methods being used for micrometer-level 3D measurement in complex objects.

To improve the calibration efficiency of the structured light stereo vision system, a closed-form single-pose calibration method is proposed. Previous works calibrate the structured light stereo vision system via the planar-based method since the world coordinates of the projected pixels are easy to find. But the disadvantage of these methods is that multiple poses are required, so that the mechanical movement is necessary in these calibration methods. In this paper, the inherent geometry between the spheres and their image is exploited deeply. Thus, a sphere-based calibration method is proposed. The proposed method first calibrates the camera from the image of the absolute conic through an improved algorithm. Then, the three-dimensional coordinates of the projected pixels are found via the inherent geometrical relation. Next, the projector is calibrated using the direct linear method. Finally, the distortion of the projector is found through the nonlinear optimization method with an improved updating direction. Experiments demonstrate that not only the calibration efficiency is improved, but also the calibration accuracy is improved.

Decoding is a challenging and complex problem in a coded structured light system. In this paper, a robust pattern decoding method is proposed for the shape-coded structured light in which the pattern is designed as grid shape with embedded geometrical shapes. In our decoding method, advancements are made at three steps. First, a multi-template feature detection algorithm is introduced to detect the feature point which is the intersection of each two orthogonal grid-lines. Second, pattern element identification is modelled as a supervised classification problem and the deep neural network technique is applied for the accurate classification of pattern elements. Before that, a training dataset is established, which contains a mass of pattern elements with various blurring and distortions. Third, an error correction mechanism based on epipolar constraint, coplanarity constraint and topological constraint is presented to reduce the false matches. In the experiments, several complex objects including human hand are chosen to test the accuracy and robustness of the proposed method. The experimental results show that our decoding method not only has high decoding accuracy, but also owns strong robustness to surface color and complex textures.

Structured light-based 3D sensing techniques has been an important means for 3D digitization of real objects. However, to obtain a complete 3D model of target, multiple scans are demanded and that makes the operation time-consuming. In this paper, an efficient 3D scanning system is introduced, which is composed of two structured light sensors. The two sensors are placed in opposite directions, and the target is placed between them. With a single scan operation, two point clouds can be obtained and aligned by the proposed system calibration parameters. For some objects with relative simple geometry like human hands, a complete 3D model can be obtained via single scan. But regarding to complex targets, by changing the target orientation several times and using the proposed point cloud registration method, complete 3D model can be obtained more efficiently. Experiments with different objects are implemented to verify feasibility and accuracy of the proposed method.

This paper developed a miniature 3D vision system for dynamic depth recovery. Our system is constituted by a DOE projector and two cameras. A single pattern with pseudo-random speckles was projected onto the surface of the target object and the coupling pattern was imaged by the stereo cameras. An optimal camera calibration algorithm is adopted to guarantee the measurement accuracy. The speckle pattern projected through a diffractive optical element (DOE) of size 8mm×8mm. The baseline between the stereo cameras is as short as 50mm. The system has the advantages of simple structure, small size, and low cost. It can be easily integrated into mobile devices such as tablet computers and mobile phones. In comparison with several other mature products, the proposed 3D depth camera can achieve 3D reconstruction with higher accuracy.

Vehicle positioning and ranging are the current research hotspots. To attain the competition goal of the MMSP Witcomm Challenge 2019 and promote the development of the autonomous driving technology, a novel framework is proposed in this paper, which combines the 2D object tracking and 3D reprojection methodologies. Firstly, a correlation filter using the deep convolutional features is designed to detect the bounding box of the moving objects which is achieved via finding the maximum response of the initial object in the convolutional feature maps. Next the homography matrix is calculated based on the image coordinate points and the corresponding world coordinate points to project the vehicle 2D boundary into the real world. The validity of the proposed framework is verified using the dataset provided by the MMSP Witcomm Challenge 2019 competition, where our method was awarded the second-place prize.

Accurate 3D modelling of real clothing is of great significance for many applications. However, most existing methods of acquiring these 3D models are too expensive and non-automatic. In this paper, we propose a practical framework for the automatic 3D reconstruction of clothing by scanning real clothes using a motorized rotation stage and three RGB-D cameras (Intel RealSense Depth Camera D415). The framework consists of three parts, including data acquisition module, point cloud process module and model reconstruction module. Users can easily use it to reconstruct clothes because it works automatically and quickly. Experimental results show that the proposed system and method can obtain 3D models of clothing with satisfied 3D mesh and texture. This research provides manufacturers a fast and economical way to create 3D clothing models for applications like virtual reality display and online shopping etc.

In a structured light-based 3D scanning system, the overall 3D information of to-be-measured objects cannot be retrieved at one time automatically. Currently the 3D registration algorithms can be divided into the auxiliary objects-based method and the feature points-based method. The former requires extra calibration objects or positioning platforms, which limits its application in free-form 3D scanning task. The latter can be conducted automatically, however, most of them tried to recover the motion matrix from extracted 2D features, which has been proved to be inaccurate. This paper proposed an automatic and accurate full-view registration method for 3D scanning system. Instead of using the 3D information of detected feature points to estimate the coarse motion matrix, 3D points reconstructed by the 3D scanning system were utilized. Firstly, robust SIFT features were extracted from each image and corresponding matching point pairs are achieved from two adjacent left images. Secondly, re-project all of the 3D point clouds onto the image plane of each left camera and corresponding 2D image points can be obtained. Filter out correct matching points from all 2D reprojection points under the guidance of the extracted SIFT matching points. Then, the covariance method was adopted to estimate the coarse registration matrix of adjacent positions. This procedure was repeated among every adjacent viewing position of the 3D scanning system. Lastly, fast ICP algorithm was performed to conduct fine registration of multi-view point clouds. Experiments conducted on real data have verified the effectiveness and accuracy of the proposed method.

Color encoded structured light is an important means for single-shot 3D reconstruction. In this article, we present a pseudorandom array-based structured light method. The proposed pattern is composed with four colors, and one geometric shape is inlaid to the color elements to extend the coding element from four to eight. As a result, a relatively small coding window with the size of 2×2 can be obtained. Grid-point between each two adjacent pattern elements is defined as the pattern feature. And an improved feature detector is proposed for the robust and accurate grid-point localization. The grid-points are divided into two types and represented by a topological structure. The decoding procedure is performed in two steps, the shape decoding and color decoding. Finally, the epipolar constraint and neighboring constraint are introduced to improve the decoding reliability. Accuracy and robustness of the proposed method are evaluated in the experiments. Some real objects including human face are reconstructed to demonstrate the 3D reconstruction quality of the proposed structured light method. Compared with conventional color SL pattern, a novel 'color+geometrical' pattern was proposed.The pattern consists of four colors and one embedded geometrical feature.With a coding capacity of 65×63, a relative small unique window size of 2×2 was obtained.The decoding procedure was implemented as color decoding and geometrical decoding.A robust and accurate pattern feature detection algorithm was also presented.

Speckle dots have the advantage of easy projection, which makes them good candidate features of structured light (SL) cameras, such as Kinect v1. However, they generally yield poor accuracy due to block matching. To improve their accuracy, this paper proposes a dot-coded SL, the coding information of which is added into dot distribution. Some of the dots are arranged regularly to provide easy-to-locate corner features, while others are specially designed to form different shapes of unique identification. A Gaussian-cross module and a simplified ResNet have been proposed to conduct robust decoding. Various experiments are performed to verify the accuracy and robustness of our framework.

3D real-time acquisition plays a vital role in computer graphics and computer vision. In this paper, we present a dynamic IR structured light sensing system with high resolution and accuracy for real-time 3D scanning. We adopt the Gray code combined with stripe shifting as our 3D acquisition’s coding strategy and parallelize the algorithm via the GPU in our IR 3D scanning system. Our built-up system can capture dense and high-precision 3D model sequences with a speed of 29 Hz. Furthermore, we propose a practical calibration method to obtain accurate calibration parameters for our system. Finally, various experiments are performed to verify the feasibility and accuracy of our proposed IR structured light sensing system.

Segmentation of targets from a set of disordered objects is always plays a significant role in the field of computer vision. In this paper, a novel method of object segmentation of scattered parts, of which dense and accurate 3D point cloud can be obtained by visual measurement technology of the structured light, is proposed and confirmed to be valid without training large datasets. The randomly placed parts are almost separated completely after two dimensional image processing and point cloud segmentation using local convex convexity connections. The segmentation results can guide the grabbing work of robot arms in the bin-picking system.

Time of Flight (ToF) and stereo vision are the two major technologies for rapid depth information acquisition. The ToF sensor is able to obtain depth map directly at a high rate of speed, but the resolution and accuracy is relatively low. The binocular stereo vision means can obtain high-accuracy disparity map based on triangulation principle, but its performance depends on the textures of the target surface. Therefore it is an interesting research topic to combine above two techniques. In order to carry out the fusion of the two 3D techniques, an efficient calibration method is investigated in this work. Based on a simple calibration rig, point correspondences between the depth image and color image are extracted and then used to calculate both intrinsic and extrinsic parameters of two devices. The experimental result shows that the proposed method could achieve better accuracy in comparison with classical calibration methods.

Registration of 3D point clouds is an important issue in the field of 3D reconstruction. In this work, we proposed a non-iterative registration method based on the motion screw theory and trackable features. The screw theory is derived from the theory of rigid body mechanics, which holds the idea that the motion of rigid body can be regarded as a kind of spiral motion and can be effectively represented by an angular velocity vector and a linear velocity vector. It has not been utilized in 3D data registration before as far as we know. In this paper, 3D data registration based on the motion screw theory is specifically introduced, and a searching strategy based on the trackable features in image sequences is presented to improve the accuracy of 3D registration. The proposed method has been successfully tested on real multi-view data. Experimental results showed that it could simplify the computational process, accelerate the speed of registration, and achieve higher precision than other methods.

This paper introduces a robust feature detection method based on an infrared structured light system (SLS). Subject to the absorption of infrared light by human face, the projected grid-pattern is difficult to be extracted directly. In the proposed image preprocessing stage, two nonlinear filters are firstly applied and their subtraction is calculated. Then, a local adaptive threshold filtering procedure is implemented to boost the contrast around grid-lines and to suppress the image background. Based on the enhanced structured light image, a cross filter is applied to extract candidate grid-points. Finally, a symmetry-based feature detector is introduced to localize the grid-points precisely. Real human face is used in the experiments. And results show that, most of the grid-points can be accurately detected and 3D model of human face can be successfully retrieved via single projection of an infrared grid-line structured light pattern.

Calibration is a critical issue in the development of projector-camera-based structured light systems. In this paper, underlying calibration principle and research progress are briefly reviewed firstly. Then, three representative calibration methods, i.e. the ProcamCalib, SLSCalib and 3DCalib methods are introduced in details. To compare the performance of various calibration approaches, a prototype of projector-camera system is constructed and calibrated by three calibration methods respectively. To evaluate the calibration accuracy, three metric error terms are used based on a reference plane with printed markers. By the analysis of experimental results, some conductive conclusions are obtained which can provide guidance for the researchers in structured light research domain.

Conventional Photometric Stereo (PS) techniques are usually based on the assumption that the light sources are assumed sufficiently far from the object that all incoming light can be modeled using parallel rays. Meanwhile, for near-field lighting conditions the light sources are close to the object so the parallel ray model cannot be used. To determine the surface normal for each point on the object more accurately, the incoming light direction should be calculated individually for each point. In this work, based on a simple PS setup consisting of LED lamps and one camera, we present a practical method for calibrating lighting directions. First, an optical model of an LED was introduced in the calibration procedure to represent the surface irradiance and image irradiance more accurately. A reference sphere was used for the calibration so that the LED optical axis could be estimated by extracting the specular points from the reference sphere. By introducing the LED emitting model, distance between the LED and the specular point along the optical axis can be calculated. Thus, the incident lighting directions for various image points can be estimated individually. To improve the estimation robustness, a non-linear fitting approach was also applied. Experiments were conducted using objects and the results are compared with traditional methods to demonstrate its feasibility and improvement.

This paper presents a novel 3D panoramic system which is composed with two panoramic cameras and a laser line projector. By rotating the laser line, a serial of panoramic images with laser strip projections can be obtained. By calibrating the panoramic cameras, epipolar constrain can be used to precisely solve the corresponding points on the laser strips in the panoramic images. As a result, a dense 3D reconstruction result can be obtained based on the binocular panoramic cameras. In the experiment, a complex indoor scene is used. The experimental results show that, the scenario can be well reconstructed by our method in comparison with classical stereo matching approaches. With proposed method, not only panoramic images can be generated, but also dense 3D information can be reconstructed, which has great potentials in various applications.

Structured light (SL) has a trade-off between acquisition time and spatial resolution. Temporally coded SL can produce a 3D reconstruction with high density, yet it is not applicable to dynamic reconstruction. On the contrary, spatially coded SL works with a single shot, but it can only achieve sparse reconstruction. This paper aims to achieve accurate 3D dense and dynamic reconstruction at the same time. A speckle-based SL sensor is presented, which consists of two cameras and a diffractive optical element (DOE) projector. The two cameras record images synchronously. First, a speckle pattern was elaborately designed and projected. Second, a high-accuracy calibration method was proposed to calibrate the system, meanwhile, the stereo images were accurately aligned by developing an optimized epipolar rectification algorithm. Then, an improved semi-global matching (SGM) algorithm was proposed to improve the correctness of the stereo matching, through which a high-quality depth map was achieved. Finally, dense point clouds could be recovered from the depth map. The DOE projector was designed with a size of 8 mm &times, 8 mm. The baseline between stereo cameras was controlled to be below 50 mm. Experimental results validated the effectiveness of the proposed algorithm. Compared with some other single-shot 3D systems, our system displayed a better performance. At close range, such as 0.4 m, our system could achieve submillimeter accuracy.

Current image matting approaches are often implemented based upon color samples under various local assumptions. In this letter, a novel image matting algorithm is investigated by treating the alpha matting as a regression problem. Specifically, we learn spatially-varying relations between pixel features and alpha values using support vector regression. Via the learning-based approach, limitations caused by local image assumptions can be greatly relieved. In addition, the computed confidence terms in learning phase can be conveniently integrated with other matting approaches for the matting accuracy improvement. Qualitative and quantitative evaluations are implemented with a public matting benchmark. And the results are compared with some recent matting algorithms to show its advantages in both efficiency and accuracy.

Three-dimensional measurement of shiny or reflective surface is a challenging issue for optical-based instrumentations. In this paper, we present a novel structured light approach for direct measurement of shiny target so as to skip the coating preprocedure. In comparison with traditional image-intensity-based structured light coding strategies like sinusoidal and line patterns, strip edges not raw image intensities are encoded in the illuminated patterns. With strip edges generally better preserved than individual image intensity in the image data in the presence of surface reflections, such a coding strategy is more robust. To remove the periodic ambiguity within strip patterns, traditional Gray code patterns are adopted. To localize the strip edges more precisely, both positive and negative strip patterns are used. An improved zero-crossing feature detector that has subpixel accuracy is proposed for strip-edge localization. The experimental setup is configured with merely an off-the-shelf pico-projector and a camera. Extensive experiments including accuracy evaluation, comparison with previous structured light algorithms, and the reconstruction of some real shiny objects are shown to demonstrate the system's accuracy and endurance against reflective nature of surfaces.

3D reconstruction of indoor scenes is important for lots of applications like SLAM (Simultaneous Localization and Mapping), VR (virtual reality) and AR (augmented reality) etc. This paper presents a precise and low-cost 3D reconstruction system, which consists of a laser emitter, a monocular camera and a turntable. The system is firstly calibrated to determine the intrinsic and extrinsic parameters of the camera and laser plane. By rotating the scanning device, a serial of laser strip images can be obtained. With the proposed point cloud registration algorithm, multiple 3D profiles of the target scene can be computed precisely and aligned automatically. As a result, a complete 3D model of the indoor scene with color information and high accuracy can be obtained. In the experiments, some real scenes are reconstructed and evaluated from the aspects of both visual quality and measurement accuracy.

In this paper, a calibration method for structured light system is proposed, which is based on pseudo-random coding theory to generate binary shape-coded pattern. In this method, the checkerboard and binary shape-coded patterns are captured by the camera of the structured light system method. Based on the geometric feature of binary shape-coded pattern, a feature point detector is designed. Then, the feature points in the binary geometric image are extracted, and the topological structure is constructed. After that, the pattern elements are extracted with the affine transformation theory and bilinear interpolation algorithm. The identification of pattern elements is modeled as a supervised classification problem, and the convolutional neural network technique is adopted to recognize the pattern elements by collecting a large number of training samples. Thus, the code-words of the feature points are confirmed. According to the projective transformation principle, the correspondence between the camera image plane and projector image plane is determined. Then, the corner points in the camera image plane are transformed into the projector image plane with the correspondence. Thereby the camera and projector are calibrated with Zhang’s calibration method, and the system calibration is achieved. The experimental results show that the calibration accuracy can reach about 0.2 pixels with the proposed method and the quality of reconstructed surface is great.

This paper presents a novel textured model building method using stripe structured light system. It is implemented by automatic registration of multiple point clouds obtained by the structured light system. Firstly, point clouds captured from different viewpoints are pairwise coarsely registered by feature matching of their corresponding RGB images. Secondly, we use an appropriate function to evaluate the quality of every pairwise coarse registration, and construct a pairwise coarse registration graph which uses point clouds as nodes and the evaluation function between them to weight their corresponding edges. Thirdly, an optimal registration tree will be generated by finding the maximum weight spanning tree of the graph and selecting a node as the root to minimize the depth of the tree. Finally, global fine registration is performed by applying ICP algorithm along the optimal registration tree. Median filtering in luminance space is also applied in the color of the integrated point model to adjust the RGB values. Experiment shows that this approach can automatically build full models which are well-registered and compatible in color for textured objects even when those objects are not rich in geometrical information.

This paper presents a photometric stereo method with nonisotropic point light sources. Subject to the non-uniform lighting conditions produced by the nonisotropic point sources, each incident light ray should be precisely determined so as to realize an accurate calculation of surface normal. In the proposed method, radiance model of the light source is firstly introduced to the classical photometric stereo framework. By considering the distance and angular attenuations of incident light rays, a precise description for the lighting field can be established. Based on the initial 3D reconstruction result, an iterative process is introduced to optimize the primary light model parameters with respect to the unknown distance factor. The experimental setup is quite simple, which only consists of some LEDs and one camera. And the experimental results show that, with the proposed method, accuracy of the reconstructed surface normal can be greatly improved in comparison with some conventional light models.

This paper presents a new calibration method of photometric stereo system which works under near-field lighting condition and with non-isotropic light sources. To estimate the position of light source, a multiple-sphere-based approach is introduced. According to the axial symmetry property of the lighting field, we show that there exists a unique brightest point which locates on the symmetry axis. By the using of a reference plane and detecting the brightest image point, principle optical axis of light source can be precisely calculated. By considering the radiance model of light source, lighting condition for each surface point can be accurately represented. Both planar and free-form surfaces are used for the experiments. And the results show that, high-quality 3D reconstruction results can be obtained by the proposed calibration approach in comparison with conventional methods.

The goal of this paper is to review the state-of-the-art progress on visual tracking methods, classify them into different categories, as well as identify future trends. Visual tracking is a fundamental task in many computer vision applications and has been well studied in the last decades. Although numerous approaches have been proposed, robust visual tracking remains a huge challenge. Difficulties in visual tracking can arise due to abrupt object motion, appearance pattern change, non-rigid object structures, occlusion and camera motion. In this paper, we first analyze the state-of-the-art feature descriptors which are used to represent the appearance of tracked objects. Then, we categorize the tracking progresses into three groups, provide detailed descriptions of representative methods in each group, and examine their positive and negative aspects. At last, we outline the future trends for visual tracking research.

Structured light-based sensing (SLS) requires the illumination to be coded either spatially or temporally in the illuminated pattern. However, while the former demands the use of uniquely coded spatial windows whose size grows with the reconstruction resolution and thereby demanding increasing smoothness on the imaged scene, the latter demands the use of multiple image captures. This article presents how the illumination of a very simple pattern plus a single image capture can also achieve 3D reconstruction. The illumination and imaging setting has the configuration of a typical SLS system, comprising a projector and a camera. The difference is, the illumination is not much more than a checkerboard-like pattern - a non-structured pattern in the language of SLS - that does not provide direct correspondence between the camera's image plane and the projector's display panel. The system works from the image progressively, first constructing the orientation map of the target object from the observed grid-lines, then inferring the depth map by the use of a few tricks related to interpolation. The system trades off little accuracy of the traditional SLSs with simplicity of its operation. Compared to temporally coded SLSs, the system has the essence that it requires only one image capture to operate; compared with spatially coded SLSs, it requires no use of spatial windows, and in turn a less degree of smoothness on the object surface; compared with methods like shape from shading and photometric stereo, owing to the use of artificial illumination it is less affected by the surface reflectance property of the target surface and the ambient lighting condition.

Calibration is a crucial step in structured-light-based range sensing device. The step involves the determination of the intrinsic parameters of both the camera and the projector that constitute the device and the extrinsic parameters between the two instruments. The traditional solution requires the use of an external calibration object with an accurately measured pattern printed on it. This paper presents a calibration design that makes use of a liquid-crystal display (LCD) panel as the calibration object. The LCD panel's planarity is of industrial grade and is thus dependable. The pattern shown on the LCD panel is programmable and is thus convenient to produce in high precision. We show that, with the design, the projector-and-camera system parameters can be calibrated with far fewer images with much higher accuracy. Extensive experiments are shown to illustrate the dramatic improvement in performance.

The recent years have witnessed rapid developments in the field of image encryption algorithms for secure color image processing. Image encryption algorithms have been classified in different ways in the past. This paper reviews the different image encryption algorithms developed during the period 2007-2015, highlighting their contrasting features. At the same time a broad classification of the said algorithms into: (1) full encryption algorithm and (2) partial encryption algorithm, each further sub-classified with respect to their domain orientations (spatial, frequency and hybrid domains) have been attempted. Efforts have also been made to cover different algorithms useful for color images of various color spaces like Red-Green-Blue (RGB), Hue-Saturation-Intensity (HSI), Cyan-Magenta-Yellow (CMY) etc. Chaotic cryptosystems, various transforms like wavelets, affine transforms etc. and visual cryptography systems are being discussed in detail.

3D measurement of specular surface is a difficult issue for optical-based structured light techniques. Due to strong reflectivity of target surface, the projected patterns are usually invisible to the camera. In this work, we present a novel structured light sensing approach by exploiting the high dynamic range imaging techniques. The target surface is scanned by the structured light device multiple times with various exposure levels. And then, images of the same pattern index are fused together to get a high dynamic range radiance map. By compressing the radiance map to the image space, a new set of images with more homogenous image intensity distribution and well exposure quality can be obtained. And finally, the 3D reconstruction is applied to these reconstructed images. A stainless steel sheet with very strong reflectance is experimented. And the results show that, the specular surface can be well reconstructed with satisfied accuracy.

Registration of 3D point clouds is an important issue in 3D scanning domain. In this work, we developed a turntable-base structured light system for the automatically 3ED scanning purpose. To realize the fully automatically 3D point clouds registration, a planar surface is placed on the turntable and scanned with different rotation angles. The turntable axis direction vector is calculated by averaging the intersection lines of the reconstructed planes firstly. And the linear minimization function is constructed for the robust and accurate estimation of the turntable origin. A variety of objects are used in the experiment. And the results show that, point clouds under different scanning angles can be precisely registered. In addition, there are no iterative procedures in the proposed algorithm, and that makes the registration procedure more efficiently.?

Existing tomography instrumentation and imaging systems use a centralized approach for data collection and image reconstruction and lack the capability of obtaining real-time information. A new distributed approach is required that can obtain a high-resolution tomography image in real time. In this chapter we introduce you to a new paradigm called large-scale in-network tomographic inversion and provide some candidate algorithms applicable for large-scale networks. This chapter focuses on seismic tomography as an example and builds a system and algorithm around it; however, the application of distributed imaging is not limited to seismic tomography, and spans various domains such as oil–natural gas fields and Martian surface exploration.

Calibration is a critical step for the projector-camera-based structured light system (SLS). Conventional SLS calibration means usually use the calibrated camera to calibrate the projector device, and the optimization of calibration parameters is applied to minimize the two-dimensional (2-D) reprojection errors. A three-dimensional (3-D)-based method is proposed for the optimization of SLS calibration parameters. The system is first calibrated with traditional calibration methods to obtain the primary calibration parameters. Then, a reference plane with some precisely printed markers is used for the optimization of primary calibration results. Three metric error criteria are introduced to evaluate the 3-D reconstruction accuracy of the reference plane. By treating all the system parameters as a global optimization problem and using the primary calibration parameters as initial values, a nonlinear multiobjective optimization problem can be established and solved. Compared with conventional calibration methods that adopt the 2-D reprojection errors for the camera and projector separately, a global optimal calibration result can be obtained by the proposed calibration procedure. Experimental results showed that, with the optimized calibration parameters, measurement accuracy and 3-D reconstruction quality of the system can be greatly improved.

This paper represents a hand posture recognition method that combines both skin and shape cues. In the algorithm, implicit skin image is firstly computed to suppress the background disturbances as well as to enhance the hand region; and vanishing component analysis (VCA) algorithm is applied to each posture category to learn a group of Approximate Vanishing Ideal Generators (AVIGs) which are used for the follow-up feature extraction. Each generator is essentially an effective characterization for geometrical structure of corresponding hand posture. In recognition phase, features acquired from implicit skin image and AVIGs are inputted to a softmax model for classification. A dataset comprising 5 hand postures are constructed for its evaluation. The proposed algorithm is demonstrated to be robust to complex environment and challenging illuminations.