Efficient virtual-to-real dataset synthesis for amodal instance segmentation of occlusion-aware rockfill material gradation detection.

• An efficient dataset synthesis method for occluded rockfill particle image segmentation. • An occlusion-aware gradation detection method based on amodal instance segmentation. • A comparison between occlusion-aware and ordinary image-based gradation detection. Image-based gradation detection methods of rockfill materials mostly ignore occluded regions of particles, and for the mainstream methods with deep learning, dataset annotation is time-consuming and labour-intensive. This study proposes an efficient virtual-to-real dataset synthesis method for rapid dataset synthesis and occlusion-aware gradation detection. Instead of photographing or scanning real particles with large time cost and labour input, Diffusion-GAN, which is trained with 600 virtual images generated by 3D modeling, efficiently generates 50,000 various individual particle images to synthesize initial stacked particle images and amodal annotations. Post-processing CycleGAN is subsequently proposed to preserve the background with image processing based on CycleGAN, which superiorly converts the style of synthetic images from virtual to real. The proposed dataset synthesis method is 50 times faster than manual labelling. Occlusion-aware gradation detection employs Bilayer Convolutional Network (BCNet) to predict both visible and occluded areas of particles, whose maximum absolute error is 4.72%, less than the error of 7.73% produced by ignoring occluded regions. The AP 50 of BCNet trained with the synthetic dataset is 0.941, extraordinarily close to the result trained with the real dataset. [ABSTRACT FROM AUTHOR]