Search

An intelligent cleaning vehicle improves the efficiency of road cleaning to a great extent. In this case, road garbage recognition is fundamental and crucial. Usually, the pebbles are too small, and the sand has an inconspicuous boundary and unfixed shape on the roads. These issues make the stones and sand too hard to be detected by a garbage recognition system. Hence, we designed a novel semantic segmentation network that acquires the areas and categories of road garbage. We designed a Deep Supervision and High Fusion (DSHF) block to improve the road garbage segmentation accuracy. The designed block with a backbone network of HFCN and UNet++ is comparable with a model that only has either a deep supervision block or a high fusion block. According to the collected road garbage segmentation dataset comprising four categories (stones, leaves, sand and bottles), the model we designed has improved the metric value of MPA by 3% over the state-of-art methods and achieved the highest MIoU (Mean Intersection over Union) with the value of 77.92%. The results of our experiments show that the semantic segmentation of road garbage is feasible and that the proposed DSHF block is practical for improving the segmentation effect of road garbage.

Road garbage recognition and cleanliness assessment are very important for intelligent cleaning of vehicles. However, widely used garbage detection methods cannot provide accurate information to assess cleanliness. Also, only a few contributions are available in the road cleanliness assessment, and they too mainly depend on the category and quantity of garbage. In this paper, deep supervision UNet++ (DUNet++) is proposed to solve the problem of road garbage classification and segmentation, which can directly impact the output of the category of garbage and the occupied ground area. A new, simple and accurate method is established to assess road cleanliness by combining the results of garbage segmentation. Compared with the cleanliness index (CI), the road cleanliness index based on semantic segmentation (CISS) not only considers the road area occupied by different types or categories of the garbage but also considers the difficulty of cleaning them. A road garbage segmentation dataset, which consists of four categories (stones, leaves, sand, and bottles), is collected to train the designed garbage segmentation model, and it has achieved considerable garbage segmentation improvement with an MIoU (Mean Intersection over Union) of $76.73\pm 0.11$ . Especially when compared with the state-of-the-art methods, the model we designed has greatly improved the accuracy of garbage segmentation. Experimental results have shown that our road garbage segmentation and cleanliness assessment methods are relatively simple, and can provide accurate road cleanliness information for cleaning vehicles.

Due to distortion, limitations of vision, and occlusion, most of the existing vacant parking slot detection methods with a standalone around view monitor (AVM) are prone to miss some parking slots and incorrectly identify whether the parking slot is vacant. To overcome this problem, we propose a complete method for vacant parking slot detection and tracking during driving and parking. Considering the different conditions of driving and parking, two different deep convolutional neural networks (DCNNs) are used to detect parking slots, of which the vacant parking slot detection network (VPS-Net) is used to detect vacant parking slots during driving, and the directional marking point detection network (DMPR-PS) is used to detect the directional marking points of the target parking slot during parking. Furthermore, in the driving process, we design a new matching rule and tracking management rule based on the Kernelized Correlation Filter (KCF) to track the parking slots, and fuse classification results of multiple frames to determine the occupancy status. In the parking process, since the parking slot is easily blocked by the vehicle, we design another new tracker to track the directional marking points and infer the complete parking slot using tracking results and prior geometric information. To evaluate the proposed method, a labeled video sequence dataset is established. Experiments show that the proposed method has improved the accuracy and continuity of vacant parking slots detection and positioning whether in the driving process or parking process.

Left-behind humans inside the car or bus have caused a lot of accidents, so it is essential to detect the humans in vehicle. Current human detection methods rely on wearable devices, oxygen sensors, and special seat designs in vehicles, but those sensors cannot adapt to ever-changing environments. To solve those problems and especially to improve passengers’ safety on the bus, we propose a method to accomplishing human detection by fusion vision and microwave radar information in various environments in vehicle. For vision information, we use different networks to extract human and human face features, and fusion of the detection results in different models to improve human detection accuracy. The human detection model is MobileNet-V2, and the human face detection model is MTCNN. A new matching schedule and tracking objects management rule based on the Kernelized Correlation Filter tracker are designed to track the human and human face detection boxes. The microwave radar information is used to detect moving objects. Finally, the fusion vision and microwave radar detection results are implemented. Experiments show that our method has improved the human detection accuracy in vehicle, and this method can be used for detection of left-behind children on the school bus.

Linear feature extraction is crucial for special objects in semantic segmentation networks, such as slot marking and lanes. The objects with linear characteristics have global contextual information dependency. It is very difficult to capture the complete information of these objects in semantic segmentation tasks. To improve the linear feature extraction ability of the semantic segmentation network, we propose introducing the dilated convolution with vertical and horizontal kernels (DVH) into the task of feature extraction in semantic segmentation networks. Meanwhile, we figure out the outcome if we put the different vertical and horizontal kernels on different places in the semantic segmentation networks. Our networks are trained on the basis of the SS dataset, the TuSimple lane dataset and the Massachusetts Roads dataset. These datasets consist of slot marking, lanes, and road images. The research results show that our method improves the accuracy of the slot marking segmentation of the SS dataset by 2%. Compared with other state-of-the-art methods, our UnetDVH-Linear (v1) obtains better accuracy on the TuSimple Benchmark Lane Detection Challenge with a value of 97.53%. To prove the generalization of our models, road segmentation experiments were performed on aerial images. Without data argumentation, the segmentation accuracy of our model on the Massachusetts roads dataset is 95.3%. Moreover, our models perform better than other models when training with the same loss function and experimental settings. The experiment result shows that the dilated convolution with vertical and horizontal kernels will enhance the neural network on linear feature extraction.