A pseudo-color image-based cylindrical object surface text detection method.

With the development of deep learning theory and computer vision technology, text detection has been widely used in automatic navigation, product recognition and language translation. However, the results of the existing text detection methods on the surface of cylindrical-shape objects are not ideal; the main reason for their poor performance is that these methods only use the color, texture and deep learning feature and do not make full use of the spatial differences of text instances. In order to solve the problem of poor text detection on cylindrical-shape object such as bottles and jars, a novel method for text detection on the curved surface of cylindrical-shape objects based on 3D point cloud is proposed in this paper. Firstly, the captured multi-view images of cylinder-shaped objects such as bottles and jars are used to reconstruct the 3D point cloud. Secondly, the 2D grid of the point cloud is established by using the graph rendering technology, which is followed by the 2D grid pixels are filled with XYZ coordinates, RGB and SWT features to generate a pseudo-color image. Finally, the U-Net network is used to segment the generated pseudo-color image, and the detection result of the text instance is obtained after the post-processing. For verifying the effectiveness of this proposed method, extensive experiments are carried out on the 3D text point cloud dataset established by us. The Precision (P), Recall (R) and F-score average of our method are 87.1%, 79.4% and 83.1%, respectively. Compared with the classical algorithms such as PSENet and CRAFT, the accuracy is increased by 11.2% and 7.0%, respectively, and the average of F-score is increased by 2.7% and 2.2%, respectively. And compared to color images, the F-score was improved by 4.4% using 3D information, which further demonstrates the effectiveness of 3D position detection for text instances on bottles and jars. The ablation experiment also demonstrated the applicability of the generated pseudo-color images based on 3D information on different models. Thus, the experimental results show that the proposed method can accurately detect the text on the surface of cylindrical objects such as bottles and jars and can be well applied to the text recognition of pharmacies, supermarkets, and daily bottle and jar products. [ABSTRACT FROM AUTHOR]