Search

As an important field of computer vision, object detection has been studied extensively in recent years. However, existing object detection methods merely utilize the visual information of the image and fail to mine the high-level semantic information of the object, which leads to great limitations. To take full advantage of multi-source information, a knowledge update-based multimodal object recognition model is proposed in this paper. Specifically, our method initially uses Faster R-CNN to regionalize the image, then applies a transformer-based multimodal encoder to encode visual region features (region-based image features) and textual features (semantic relationships between words) corresponding to pictures. After that, a graph convolutional network (GCN) inference module is introduced to establish a relational network in which the points denote visual and textual region features, and the edges represent their relationships. In addition, based on an external knowledge base, our method further enhances the region-based relationship expression capability through a knowledge update module. In summary, the proposed algorithm not only learns the accurate relationship between objects in different regions of the image, but also benefits from the knowledge update through an external relational database. Experimental results verify the effectiveness of the proposed knowledge update module and the independent reasoning ability of our model.

Owing to insufficient illumination of the space station, the image information collected by the intelligent robot will be degraded, and it will not be able to accurately identify the tools required for the robot’s on-orbit maintenance. This situation increases the difficulty of the robot’s maintenance in a low-illumination environment. We proposes a novel enhancement method for images under low-illumination, namely, a deep learning algorithm based on the combination of deep convolutional and Wasserstein generative adversarial networks (DC-WGAN) in CIELAB color space. The original low-illuminance image is converted from the RGB space to the CIELAB color space which is relatively close to human vision, to accurately estimate the illumination image, and effectively reduce the effect of uneven illumination. DC-WGAN is applied to enhance the brightness component by increasing the width of the generation network to obtain more image features. Subsequently, the LAB is converted into RGB space to obtain the final enhanced image. The feasibility of the algorithm is verified by experiments on low-illuminance image under general, special, and actual conditions and comparing the experimental results with four commonly used algorithms. This study lays a technical foundation for robot target recognition and on-orbit maintenance in a space environment.