Search

Virtual assembly technology has developed rapidly in recent years. As one of its key technologies, object registration has been widely studied, whose main task is to register the target object. However, the existing methods cannot effectively register reflective texture-less objects such as metal parts. In this paper, an object registration method based on multiple edge features is proposed, which includes a sparse uniform template sampling method and a multiple edge feature matching method. At the same time, we also used two oil pumps to carry out virtual assembly experiments and verify the accuracy of object registration. The experimental results show that this method can achieve high accuracy which is in line with the requirements of the virtual assembly algorithm in the industry.

Six degrees of freedom (6D) pose estimation or measurement of target objects are important robotic vision techniques that are key aspects for achieving autonomously executed robot tasks. The existing techniques solve this problem well for ordinary textured or rough-surfaced objects; however, 6D pose estimation remains a challenge for textureless, shiny metal parts in industry. This article proposes both a new framework for performing 6D pose estimation and a practical method based on the proposed framework that outperforms the state-of-the-art methods. All the existing methods follow an image-to-feature framework: they first extract features from images and then use the features to infer object poses. In contrast, our idea is a novel generative feature-to-image framework based on generative models, whose pipeline is a reverse mapping from feature to image. In other words, given a feature representing a pose, our method generates an image of the object in the exact same pose. Based on this framework, we propose a specific algorithm to indirectly infer the pose, which enables an initially random answer to gradually be regressed to the correct result. The experimental results demonstrate that the proposed deep learning method possesses greater robustness and achieves higher accuracy than the traditional methods

Pose estimation is an essential technology for product grasping and assembly in intelligent manufacturing. Finding local correspondences between the 2D image and 3D model is the key step to estimate the 6D pose of an object. However, when the objects are textureless, it is difficult to identify distinguishable point features. In this paper, we propose a novel deep learning framework called the pseudo-Siamese graph matching network to tackle the problem of feature matching of textureless objects, and estimate accurate object poses with a single RGB-only image. We utilize a pseudo-Siamese network structure to learn the similarity between the 2D image features and the 3D mesh model of the object. A fully convolutional network and a graph convolutional network are used to extract high-dimensional deep features of 2D image and 3D model, respectively. Dense 2D-3D correspondences are inferred using the pseudo-Siamese matching network. Then, the pose of the object is calculated by the PnP and RANSAC methods. Experiments on the LINEMOD dataset and a grasping task for metal part show the accuracy and robustness of our proposed method. Code is available at https://github.com/Ray0089/PSGMN.git.

A major obstacle in digital projector technology is that images projected onto nonideal surfaces with large depth variances can easily become blurred. In this article, present a method to overcome projection defocus for projection surfaces that inevitably have complex shapes and large depth variances. The proposed method has two main advantages over traditional methods. First, an edge-intensification-based defocus compensation algorithm is proposed to manipulate the input image to compensate for its projection defocus before blurring occurs. Unlike previous time-consuming compensation algorithms, the proposed algorithm has very high efficiency, as it is noniterative and open loop. Second, a sinusoidal-projection-based estimation method is proposed to reduce kernel estimation errors on complex surface types. Unlike previous methods limited to specific surface types, the proposed method can provide consistently good kernel estimation results even for discontinuous and textured (nonpure white) projection surfaces. Hence, the proposed method can be applied to a wider range of applicable surfaces. These two contributions are demonstrated through extensive experiments and compared with the state-of-the-art methods. 1 1 The MATALB code and experimental data for this article are available at https://github.com/lpl-code/FastProjDefocusComp .

Conventional moment-based visual servoing methods suffer from several problems in industrial applications due to the utilization of high-order image moments. In this paper, we analyze the shortcomings of the moment-based visual servoing from the viewpoint of practical industrial applications, and propose a novel moment-based two-and-a-half-dimensional visual servoing method for grasping textureless planar parts. We use hybrid visual features that combine image moments with three-dimensional (3-D) rotation in the Cartesian space to control the robot motion. Instead of applying high-order image moments, we use rotation features, which provide a decoupled interaction matrix that is full rank and with no local minimum in the control scheme. Furthermore, to estimate the relative rotation of the textureless part in real time, a new estimation method based on a cross-correlation analysis is proposed. The proposed visual servoing method provides a better motion control and 3-D trajectory of the robot arm and remains stable in the workspace. Experimental results demonstrate the effectiveness of the method.

For mechanical elements running under severe working condition, there inevitably exist some small defects caused during working. The weak area, which is sensitive to defects, of the structure is more vulnerable, resulting in early damage during service. This paper describes a novel anti-defect design method for optimizing structure to enhance the reliability of vulnerable areas. First, a half-real defect model derived from the real defect is developed to model the geometrical characteristics of defects. Then, the damage degrees of model parameters are identified by Sobol’s sensitivity method and the vulnerable area can be labeled according to the damage degree. Finally, we take the vulnerable area as the object of anti-defect optimization for structure and the optimization variables are selected tendentiously based on parameters with larger damage degrees. The multiobjective particle swarm algorithm is then employed to find the optimal distribution of the variables in order to improve the safety of the sensitive area of structure. We take the impeller blade as the research subject to verify the validity of the proposed method. Analysis results showed that the proposed method can increase the structure strength and delay the damage of the mechanical elements.

Closely adjacent objects are a kind of challenging scene in three-dimensional point cloud segmentation. To address this issue, we propose a novel periphery-restrictive region-growing-based segmentation method. First, the clouds are smoothed by the moving least squares method and the periphery of the target clouds is extracted. Then an initial seed generation based on minimum curvature first of non-periphery points, namely, non-periphery seed generation algorithm is proposed. Finally, a nearest periphery restrictive growing algorithm is proposed for accurate segmentation. In addition, we establish a point cloud dataset of adjacent objects which contains three typically adjacent objects on the pipeline. We evaluate the effectiveness and accuracy of the proposed method on this dataset, and the extensive experiments show that the proposed method performs well on closely adjacent objects.

Spoken language understanding (SLU) is a fundamental to service robot handling of natural language task requests. There are two main basic problems in SLU, namely, intent determination (ID) and slot filling (SF). The slot-gated recurrent neural network joint model for the two tasks has been proven to be superior to the single model, and has achieved the most advanced performance. However, in the context of task requests for home service robots, there exists a phenomenon that the information about a current word is strongly dependent on key verbs in the sentence, and it is difficult to capture this relation well with current methods. In this paper, we extract the key instructional verb containing greater core task information based on dependency parsing, and construct a feature that combines the key verb with its contextual information to solve this problem. To further improve the performance of the slot-gated model, we consider the strong relations between intent and slot. By introducing intent attention vectors into the slot attention vectors through the global-level gate and element-level gate, a novel dual slot-gated mechanism is proposed to explicitly model the complex relations between the results of the ID tasks and SF prediction tasks and optimize the global prediction results. Our experimental results on the ATIS dataset and an extended home service task (SRTR) dataset based on FrameNet show that the proposed method outperforms the most advanced methods in both tasks. Especially, for SRTR, the results of SF, ID, and sentence-level semantic frame-filling are improved by 1.7%, 1.1%, and 1.7%, respectively.

We propose a new lossy image compression algorithm for images from consumer photo storage on the generative adversarial networks (GANs). It uses a pre-trained generator model as decoder and an algorithm called back-propagation-to-the-input as encoder. In compression process, this method could compress images with extremely small compression storage. Experiments demonstrate that this method surpasses the traditional ways in compression ratio and reconstruct representation.