Vision-based seam tracking for GMAW fillet welding based on keypoint detection deep learning model

Pre-programmed welding robots significantly improved the efficiency and quality of the welds in large batch production. In small and medium batch production, the robots need appropriate sensors to perform well and adapt to the changes and uncertainties in a noisy welding environment. Vision-based sensors enabled by machine learning are making it possible to sense in process previously not measurable. One challenge is developing artificial intelligent models capable of real-time seam tracking, particularly in fillet joints where visual analysis is hindered by non-perpendicular camera angles and arc reflections. In this paper, we propose a vision system that enables automated seam tracking with a collaborative robot. The vision-based deep learning classification model detects the tacks, where the seam is not visible. It is based on a keypoint detection deep learning model that addresses the challenges in distorted and noisy images of fillet joints between the pipes and flanges during the real-time Gas Metal Arc Welding to track the location of the seam in non tack images. The system is optimized for real time seam tracking by proposing the appropriate input image size. Multiple images and multiple points are also considered to provide a controllable signal of the location of the seam with less errors and outliers. Our proposed model can track the seam with more than 80 percent accuracy for errors less than 0.3 mm in fillet joints. The high accuracy of the proposed method would result in fewer flaws and defects and reduced rework, resulting in significant cost saving in manufacturing. The real-time monitoring also enables the adaptability to slight variations in gap.