Search

Six degrees of freedom (6D) pose estimation or measurement of target objects are important robotic vision techniques that are key aspects for achieving autonomously executed robot tasks. The existing techniques solve this problem well for ordinary textured or rough-surfaced objects; however, 6D pose estimation remains a challenge for textureless, shiny metal parts in industry. This article proposes both a new framework for performing 6D pose estimation and a practical method based on the proposed framework that outperforms the state-of-the-art methods. All the existing methods follow an image-to-feature framework: they first extract features from images and then use the features to infer object poses. In contrast, our idea is a novel generative feature-to-image framework based on generative models, whose pipeline is a reverse mapping from feature to image. In other words, given a feature representing a pose, our method generates an image of the object in the exact same pose. Based on this framework, we propose a specific algorithm to indirectly infer the pose, which enables an initially random answer to gradually be regressed to the correct result. The experimental results demonstrate that the proposed deep learning method possesses greater robustness and achieves higher accuracy than the traditional methods

Pose estimation is an essential technology for product grasping and assembly in intelligent manufacturing. Finding local correspondences between the 2D image and 3D model is the key step to estimate the 6D pose of an object. However, when the objects are textureless, it is difficult to identify distinguishable point features. In this paper, we propose a novel deep learning framework called the pseudo-Siamese graph matching network to tackle the problem of feature matching of textureless objects, and estimate accurate object poses with a single RGB-only image. We utilize a pseudo-Siamese network structure to learn the similarity between the 2D image features and the 3D mesh model of the object. A fully convolutional network and a graph convolutional network are used to extract high-dimensional deep features of 2D image and 3D model, respectively. Dense 2D-3D correspondences are inferred using the pseudo-Siamese matching network. Then, the pose of the object is calculated by the PnP and RANSAC methods. Experiments on the LINEMOD dataset and a grasping task for metal part show the accuracy and robustness of our proposed method. Code is available at https://github.com/Ray0089/PSGMN.git.

Conventional moment-based visual servoing methods suffer from several problems in industrial applications due to the utilization of high-order image moments. In this paper, we analyze the shortcomings of the moment-based visual servoing from the viewpoint of practical industrial applications, and propose a novel moment-based two-and-a-half-dimensional visual servoing method for grasping textureless planar parts. We use hybrid visual features that combine image moments with three-dimensional (3-D) rotation in the Cartesian space to control the robot motion. Instead of applying high-order image moments, we use rotation features, which provide a decoupled interaction matrix that is full rank and with no local minimum in the control scheme. Furthermore, to estimate the relative rotation of the textureless part in real time, a new estimation method based on a cross-correlation analysis is proposed. The proposed visual servoing method provides a better motion control and 3-D trajectory of the robot arm and remains stable in the workspace. Experimental results demonstrate the effectiveness of the method.

We propose a new lossy image compression algorithm for images from consumer photo storage on the generative adversarial networks (GANs). It uses a pre-trained generator model as decoder and an algorithm called back-propagation-to-the-input as encoder. In compression process, this method could compress images with extremely small compression storage. Experiments demonstrate that this method surpasses the traditional ways in compression ratio and reconstruct representation.

The performances of traditional visual odometry algorithms may degenerate when a scene contains dynamic objects. In this paper, we propose a novel spatiotemporal visual odometry for dynamic indoor environments by using RGB-D cameras. First, to improve the data association, the complete ground plane is detected and used in the optimization function of coarse pose estimation. Then, the spatial information and temporal information are fused by an undirected network to improve the segmentation of the moving objects. Last, the pose is computed accurately via a coarse-to-fine strategy. Experimental results that demonstrate the performance of the proposed method are presented, and the factors that affect the measurement accuracy are analyzed.

Small blurred blowholes are difficult to detect, especially those in castings having a complex structure. In this paper, we propose a blowhole detection method based on bidirectional enhancement and omnidirectional analysis for X-ray inspection of castings. After obtaining the X-ray images of a casting from the X-ray inspection equipment, there are two main processes to detect the blowholes. First, in the extraction process for the potential defect region, we propose a method based on bidirectional enhancement that highlights the blurred defects and models the nondefective structure. Second, in the precise defect identification process, we propose a method to distinguish real defects by 3D model fitting-based omnidirectional analysis. The experimental results indicate that the proposed method has a higher performance in blowhole detection than comparable methods.

This paper presents a new method for improving video coding efficiency based on a sparse contractive mapping approach. The proposed method introduces a new sparse contractive mapping approach to replace the traditional intra prediction in the video coding standards such as H.264/AVC. Specifically, the intra- and its following inter-frame are respectively approximated by the sparse representation, satisfying contractive mapping. Then these two frames can be reconstructed from an arbitrary initial image by utilizing a few representation coefficients. With this advantage, the proposed method reduces the total amount of bits by removing MBs in the target I frame, whose approximation performance is higher than the others in the encoder. Furthermore, by transmitting the representation coefficients of the removed MBs, these MBs can be accurately reconstructed in the decoder. Since the reconstruction performance is better than that of the conventional approach, the proposed method can remove more MBs from the target video sequences, and reduction of total amount of bits can be feasible. Therefore, the proposed method realizes the improvement of the video coding efficiency. Some experimental results are shown to verify the superior performance of the proposed method to that of H.264/AVC. The results also demonstrate that the bit-saving performance of the proposed method is comparable to that of H.265/HEVC.

In this paper, a robust approach using a shape distribution histogram (SDH) feature and modified sparse representation classification (MSRC) for pedestrian detection in thermal infrared imagery is proposed. In this framework, the candidate regions that are more likely to contain the pedestrians are first detected based on the Contour Saliency Map. Then distances between random points on the thinned contour map of objects in the candidate regions are applied to acquire the SDH feature. SDH is a robust and discriminative feature which can precisely describe the pedestrian characteristics. Finally, a robust MSRC classifier which has high accuracy is used to recognize the true pedestrians. Experiments are conducted over the OSU thermal pedestrian database by comparing with other algorithms. The proposed method shows an excellent performance in detecting pedestrians. HighlightsRobust pedestrian detection using the SDH feature and MSRC classifier in infrared imagery.Discriminative SDH feature can precisely describe the pedestrian characteristic.Robust MSRC classifier is used to classify the pedestrians and non-pedestrians.

In this paper, a robust sparse-representation-based inspection system for the detection and classification of casting hidden defects in radiographs is presented. Four common types of casting defects including cracks, blow holes, shrinkage porosities and shrinkage cavities are considered in our system. In the framework, a Gray Arranging Pairs (GAP) based segmentation method is implemented firstly. This method can deal with the case of casting that has complex structure well and is robust against non-uniform illumination variations and noise. Second, a Randomly Distributed Triangle (RDT) feature is extracted to represent the geometric characteristic of each defect. This feature uses random triangle samplings which are formed from the defect shape to produce a continuous probability distribution. It is simple and can discriminate defects correctly despite of rotation, scale and noise. Third, a Sparse Representation-based Classification (SRC) is trained to classify each of the input defect into one of the classes. The performance of the proposed method is shown in the experiment by comparing with the SVM classifier.

Additive manufacturing allows for the production of custom parts with complex internal features. However, a lack of online detection of the printing process is one of the challenges that has prevented further improvements in additive manufacturing. This paper proposes a novel online monitoring technique based on a fringe projector to keep the stability of the process under control on a layer-by-layer basis. To detect microdefects with a magnitude smaller than the noise, we propose a region-based defect detection method that takes each subregion, instead of each point, as a detection unit. Compared with a single point, a subregion contains hundreds of points, which represent a domain with greater stability. We combine Voxel Cloud Connectivity Segmentation (VCCS) and Fast Point Feature Histogram (FPFH) to divide the printing area into hundreds of subregions, smooth the surface to resist the noise based on Moving Least Square (MLS) and evaluate each subregion using local point features. The projection model and normal deviation are developed to extract the sizes and locations of the flaws. The performances of the techniques are demonstrated with some experiments using different 3D printing objects, which indicates that the proposed method is able to detect various flaws and its detection accuracy is higher than the noise magnitude.

In this paper, we propose a novel approach based on Grayscale Arranging Pairs (GAP) for face recognition. A facial model is built by using the stable point pairs of the GAP feature. Then the similarity between the facial model and the input facial image is calculated by checking whether the intensity relationship of these point pairs is the same. Different from current face recognition algorithms, GAP is a robust holistic feature without losing its local property. By using a stable intensity relationship of multiple point pairs, the GAP feature shows a great invariance property of facial features, and exhibits high robustness to resist illumination variations. Meanwhile, it describes the holistic information in the entire facial image, which is more similar to the human recognition mechanism. In addition, a novel weighting model that exploits the local characteristics of faces is applied in the framework, leading to a higher accuracy in face recognition. We compare the proposed method with four other famous methods on the Extended Yale B face and PIE face databases. The experimental results showed that the proposed method provides outstanding results in recognizing faces.

In the current keypoint-based image matching methods, not all the keypoints can be reliably matched because of the influence of noise, illumination, and image distortion. In the paper, we propose a novel method based on the Delaunay triangulation to detect and remove possible mismatches. Given previously matched keypoints by a detector in two images, the proposed method utilizes four steps to remove the mismatched point pairs: first, triangulating keypoints in the reference image, and producing a graph consisting of edges that connects the keypoints; second, drawing a graph in the test image by connecting the corresponding points as same as in the reference image; third, detecting abnormal edges in the test image using special constraints; fourth, detecting and removing mismatched point pairs based on the abnormal edges. The experimental results show that the method can detect the mismatches accurately and improve the robustness of current matching algorithms.

The grasping of circular metal parts such as bearings and flanges is a common task in industry. Limited by low texture and repeated features, the point-feature-based method is not applicable in pose measurement of these parts. In this paper, we propose a novel pose measurement method for grasping circular metal parts. This method is based on cone degradation and involves a monocular camera. To achieve higher measurement accuracy, a position-based visual servoing method is presented to continuously control an eye-in-hand, six-degrees-of-freedom robot arm to grasp the part. The uncertainty of the part's coordinate frame during the control process is solved by defining a fixed virtual coordinate frame. Experimental results are provided to illustrate the effectiveness of the proposed method and the factors that affect measurement accuracy are analyzed.

Post-consumer PET bottle recycling is attracting increasing attention due to its value as an energy conservation and environmental protection measure. Sorting by color is a common method in bottle recycling; however, manual operations are unstable and time consuming. In this paper, we design a vision-based facility to perform high-speed bottle sorting. The proposed facility consists mainly of electric and mechanical hardware and image processing software. To solve the recognition problem of isolated and overlapped bottles, we propose a new shape descriptor and utilize the support vector data description classifier. We use color names to represent the colors in the real world in order to avoid problems introduced by colors that are similar. The facility is evaluated by the target error, outlier error and total error. The experimental results demonstrate that the facility we developed is capable of recycling various PET bottles.