Search

In recent years, image segmentation based on deep learning has been widely used in medical imaging, automatic driving, monitoring and security. In the fields of monitoring and security, the specific location of a person is detected by image segmentation, and it is segmented from the background to analyze the specific actions of the person. However, in low-illumination conditions, it is a great challenge to the traditional image-segmentation algorithms. Unfortunately, a scene with low light or even no light at night is often encountered in monitoring and security. Given this background, this paper proposes a multi-modal fusion network based on the encoder and decoder structure. The encoder, which contains a two-branch swin-transformer backbone instead of the traditional convolutional neural network, fuses the RGB and depth features with a multiscale fusion attention block. The decoder is also made up of the swin-transformer backbone and is finally connected via the encoder with several residual connections, which are proven to be beneficial in improving the accuracy of the network. Furthermore, this paper first proposes the low light–human segmentation (LLHS) dataset of portrait segmentation, with aligned depth and RGB images with fine annotation under low illuminance, by combining the traditional monocular camera and a depth camera with active structured light. The network is also tested in different levels of illumination. Experimental results show that the proposed network has good robustness in the scene of human segmentation in a low-light environment with varying illumination. The mean Intersection over Union (mIoU), which is often used to evaluate the performance of image segmentation model, of the Swin-MFA in the LLHS dataset is 81.0, is better than those of ACNet, 3DGNN, ESANet, RedNet and RFNet at the same level of depth in a mixed multi-modal network and is far ahead of the segmentation algorithm that only uses RGB features, so it has important practical significance. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the problem of unmanned aerial vehicles (UAVs) autonomous tracking moving target with only an airborne camera sensor. We proposed a novel integrated controller framework for this problem based on multi-neural-network modules (MNNMs). In this framework, two neural networks are designed for target perception and guidance control, respectively. The deep learning method and reinforcement learning method are applied to train the integrated controller. The training result demonstrates that the integrated controller can be trained more quickly and efficiently than the end-to-end controller trained by the deep reinforcement learning method. The flight tests with the integrated controller are implemented in simulated and realistic environments, the results show that the integrated controller trained in simulation can easily be transferred to the realistic environment and achieve the UAV tracking randomly moving target, which has a faster motion velocity. The integrated controller based on the MNNMs structure has a better performance on an autonomous tracking target than the control mode that combines with a perception network and a proportional integral derivative controller. [ABSTRACT FROM AUTHOR]