An efficient semantic segmentation method based on transfer learning from object detection

Abstract Nowadays, numerous semantic segmentation techniques were used to complex scenes such as urban streets. However, speed issues are not considered in most of these methods, and real‐time methods do not mainly include enough accuracy. In this paper, an efficient semantic segmentation method is proposed, using the feature extractor of a real‐time object detection model, Darknet53, as the backbone of DeepLabv3+. By the high accuracy of DeepLabv3+ structure and great efficiency of Darknet53, a mean intersection was obtained over union of 76.3% in Cityscapes test set, and fast inference speed simultaneously (0.178 s per frame on one GTX 1080Ti GPU). A huge imbalance of objects was noticed on Cityscapes dataset. To solve this problem, a Focal Loss like loss function was proposed to concentrate more on the hard difficult pixels. Moreover, an atrous convolution block was proposed to extract more high‐level features. Based on the experimental results, it is proved that these changes contribute to a better result on the Cityscapes test set (77.8% mean Intersection over Union) and faster inference speed (0.171 s per frame). Authors' model achieves state‐of‐art results on Cityscapes test set (79.1% mean Intersection over Union) after fine‐tuning on Cityscapes coarsely annotated dataset.