Search

Recently, with most mobile phones coming with dual cameras, stereo image super-resolution is becoming increasingly popular in phones and other modern acquisition devices, leading stereo super-resolution images spread widely on the Internet. However, current image forensics methods are carried out in monocular images, and high false positive rate appears when detecting stereo super-resolution images by these methods. Therefore, it is important to develop stereo super-resolution image detection method. In this paper, a convolutional neural network with multi-scale feature extraction and hierarchical feature fusion is proposed to detect the stereo super-resolution images. Multi-atrous convolutions are employed to extract multi-scale features and be adapt for varying stereo super-resolution images, and hierarchical feature fusion further improve the performance and robustness of the model. Experimental results demonstrate that the proposed network can detect stereo super-resolution images effectively and achieve strong generalization and robustness. To the best of our knowledge, it is the first attempt to investigate the performance of current forensics methods when tested under stereo super-resolution images, and represent the first study of stereo super-resolution images detection. We believe that it can raise the awareness about the security of stereo super-resolution images. [ABSTRACT FROM AUTHOR]

Multi-scale geometric processing has been a popular and powerful tool in graphics, which typically employs isotropic diffusion across scales. This paper proposes a novel method of multi-scale anisotropic heat diffusion on manifold, based on the new normal-driven shape representation and Edge-weighted Heat Kernels (EHK). The new shape representation, named as Normal-Controlled Coordinates (NCC), can encode local geometric details of a vertex along its normal direction and rapidly reconstruct surface geometry. Moreover, the inner product of NCC and its corresponding vertex normal, called Normal Signature (NS), defines a scalar/heat field over curved surface. The anisotropic heat diffusion is conducted using the weighted heat kernel convolution governed by local geometry. The convolution is computed iteratively based on the semigroup property of heat kernels toward accelerated performance. This diffusion is an efficient multi-scale procedure that rigorously conserves the total heat. We apply our new method to multi-scale feature detection, scalar field smoothing and mesh denoising, and hierarchical shape decomposition. We conduct various experiments to demonstrate the effectiveness of our method. Our method can be generalized to handle any scalar field defined over manifold. [ABSTRACT FROM AUTHOR]

Loop closure detection (LCD) is an indispensable part of simultaneous localization and mapping (SLAM) systems, making it significant for eliminating accumulative trajectory error and constructing a consistent map. Focusing on visual SLAM, we propose a LCD method using local-global similarity measurement strategies. As with most deep learning-based methods, we treat the LCD problem as a classification problem. We use a pretrained VGG-16 network to obtain advanced features and a multiscale feature extraction module to integrate the features of different receptive fields. Finally, we overcome the problems of dynamic object occlusion and large-area repetitive texture using a local-global similarity measurement module to measure the similarity between current and historical frames in all dimensions. This approach decouples feature extraction from classification determination, which enables us to use the advantages of both supervised and unsupervised learning to cope with the imbalance in the number of nonloop and loop pairs. Experimental results on three different datasets showed that the proposed method is significantly better than the state-of-the-art appearance-based LCD methods in dynamic urban, suburban, and highway scenarios. [ABSTRACT FROM AUTHOR]

The point clouds scanned by lidar are generally sparse, which can result in fewer sampling points of objects. To perform precise and effective 3D object detection, it is necessary to improve the feature representation ability to extract more feature information of the object points. Therefore, we propose an adaptive feature enhanced 3D object detection network based on point clouds (AFE-RCNN). AFE-RCNN is a point-voxel integrated network. We first voxelize the raw point clouds and obtain the voxel features through the 3D voxel convolutional neural network. Then, the 3D feature vectors are projected to the 2D bird's eye view (BEV), and the relationship between the features in both spatial dimension and channel dimension is learned by the proposed residual of dual attention proposal generation module. The high-quality 3D box proposals are generated based on the BEV features and anchor-based approach. Next, we sample key points from raw point clouds to summarize the information of the voxel features, and obtain the key point features by the multi-scale feature extraction module based on adaptive feature adjustment. The neighboring contextual information is integrated into each key point through this module, and the robustness of feature processing is also guaranteed. Lastly, we aggregate the features of the BEV, voxels, and point clouds as the key point features that are used for proposal refinement. In addition, to ensure the correlation among the vertices of the bounding box, we propose a refinement loss function module with vertex associativity. Our AFE-RCNN exhibits comparable performance on the KITTI dataset and Waymo open dataset to state-of-the-art methods. On the KITTI 3D detection benchmark, for the moderate difficulty level of the car and the cyclist classes, the 3D detection mean average precisions of AFE-RCNN can reach 81.53% and 67.50%, respectively. [ABSTRACT FROM AUTHOR]

Early screening based on computed tomography (CT) pulmonary nodule detection is an important means to reduce lung cancer mortality, and in recent years three dimensional convolutional neural network (3D CNN) has achieved success and continuous development in the field of lung nodule detection. We proposed a pulmonary nodule detection algorithm by using 3D CNN based on a multi-scale attention mechanism. Aiming at the characteristics of different sizes and shapes of lung nodules, we designed a multi-scale feature extraction module to extract the corresponding features of different scales. Through the attention module, the correlation information between the features was mined from both spatial and channel perspectives to strengthen the features. The extracted features entered into a pyramid-similar fusion mechanism, so that the features would contain both deep semantic information and shallow location information, which is more conducive to target positioning and bounding box regression. On representative LUNA16 datasets, compared with other advanced methods, this method significantly improved the detection sensitivity, which can provide theoretical reference for clinical medicine.

Educational theory claims that integrating learning style into learning-related activities can improve academic performance. Traditional methods to recognize learning styles are mostly based on questionnaires and online behavior analyses. These methods are highly subjective and inaccurate in terms of recognition. Electroencephalography (EEG) signals have significant potential for use in the measurement of learning style. This study uses EEG signals to design a deep-learning-based model of recognition to recognize people's learning styles with EEG features by using a non-overlapping sliding window, one-dimensional spatio-temporal convolutions, multi-scale feature extraction, global average pooling, and the group voting mechanism; this model is named the TSMG model (Temporal-Spatial-Multiscale-Global model). It solves the problem of processing EEG data of variable length, and improves the accuracy of recognition of the learning style by nearly 5% compared with prevalent methods, while reducing the cost of calculation by 41.93%. The proposed TSMG model can also recognize variable-length data in other fields. The authors also formulated a dataset of EEG signals (called the LSEEG dataset) containing features of the learning style processing dimension that can be used to test and compare models of recognition. This dataset is also conducive to the application and further development of EEG technology to recognize people's learning styles. [ABSTRACT FROM AUTHOR]

How to accurately detect small targets from the complex maritime environment has been a bottleneck problem. The strong wind-wave backlight conditions (SWWBC) is the most common situation in the process of distress target detection. In order to solve this problem, the main contribution of this paper is to propose a small target detection method suitable for SWWBC. First of all, for the purpose of suppressing the gray value of the background, it is analyzed that some minimum points with the lowest gray value tend to gather in the interior of the small target. As the distance from the extreme point increases, the gray value of the pixel in all directions also increases by the same extent. Therefore, an inverse Gaussian difference (IGD) preprocessing method similar to the distribution of the target pixel value is proposed to suppress the uniform sea wave and intensity of the sky background. So as to achieve the purpose of background suppression. Secondly, according to the feature that the small target tends to "ellipse shape" in both horizontal and vertical directions, a multi-scale and multi-directional Gabor filter is applied to filter out interference without "ellipse shape". Combined with the inter-scale difference (IsD) operation and iterative normalization operator to process the results of the same direction under different scales, it can further suppress the noise interference, highlight the significance of the target, and fuse the processing results to enrich the target information. Then, according to different texture feature distributions of the target and noise in the multi-scale feature fusion results, a cross-correlation (CC) algorithm is proposed to eliminate noise. Finally, according to the dispersion of the number of extreme points and the significance of the intensity of the small target compared with the sea wave and sky noise, a new peak significance remeasurement method is proposed to highlight the intensity of the target and combined with a binary method to achieve accurate target segmentation. In order to better evaluate the performance index of the proposed method, compared with current state-of-art maritime target detection technologies. The experimental results of multiple image sequence sets confirm that the proposed method has higher accuracy, lower false alarm rate, lower complexity, and higher stability. [ABSTRACT FROM AUTHOR]

• The novelty of the work presented in this paper is that a information fusion algorithm of the maximum multi-modal weight decision is proposed. The emotion recognition scheme is developed by using the multi-modal information fusion algorithm. • Another major contribution of this work is the innovative conversion of traditional EEG data into corresponding EEG topographic maps for the emotion recognition. • Besides, this paper builds a multi-scale feature extraction network model for the facial expression recognition. In view of the low accuracy of the traditional emotion recognition methods based on facial expressions, an emotion recognition method based on maximum weight multi-modal information fusion of electroencephalographs (EEGs) and facial expression information is proposed in this paper. First, the induced emotional EEG data is converted into the corresponding EEG topographic map data and sent to the convolutional network for training and outputting decision information. Second, the illumination compensation method is utilized to filter the noise of the face image data. Then, the face image data is trained in the multi-scale feature extraction network, and the decision information is output. Finally, aiming at the decision-level information fusion, a weighted fusion method is proposed in this paper for emotion recognition. Experimental tests show that the recognition accuracy of the multi-scale feature extraction network on the CK+ data set and Fer2013 data reached 94.4% and 72%, respectively. Simultaneously, the multi-modal information fusion method achieves 92.6% accuracy in emotion recognition. [Display omitted] The novelty of the work presented in this paper is that a fusion method of maximum weight decision information is proposed. Emotion recognition is realized by the weighted fusion of EEG information and facial expression information. Another major contribution of this work is the innovative conversion of traditional EEG data into corresponding state EEG topographic maps for processing and analysis. Besides, this paper builds a multi-scale feature extraction network for emotion recognition. The graphical abstract of this paper is as below. [ABSTRACT FROM AUTHOR]

Wide field-of-view (FOV) and high-resolution (HR) imaging are essential to many applications where high-content image acquisition is necessary. However, due to the insufficient spatial sampling of the image detector and the trade-off between pixel size and photosensitivity, the ability of current imaging sensors to obtain high spatial resolution is limited, especially under low-light-level (LLL) imaging conditions. To solve these problems, we propose a multi-scale feature extraction (MSFE) network to realize pixel-super-resolved LLL imaging. In order to perform data fusion and information extraction for low resolution (LR) images, the network extracts high-frequency detail information from different dimensions by combining the channel attention mechanism module and skip connection module. In this way, the calculation of the high-frequency components can receive greater attention. Compared with other networks, the peak signal-to-noise ratio of the reconstructed image was increased by 1.67 dB. Extensions of the MSFE network are investigated for scene-based color mapping of the gray image. Most of the color information could be recovered, and the similarity with the real image reached 0.728. The qualitative and quantitative experimental results show that the proposed method achieved superior performance in image fidelity and detail enhancement over the state-of-the-art. [ABSTRACT FROM AUTHOR]

The main challenges of semantic segmentation in vehicle-mounted scenes are object scale variation and trading off model accuracy and efficiency. Lightweight backbone networks for semantic segmentation usually extract single-scale features layer-by-layer only by using a fixed receptive field. Most modern real-time semantic segmentation networks heavily compromise spatial details when encoding semantics, and sacrifice accuracy for speed. Many improving strategies adopt dilated convolution and add a sub-network, in which either intensive computation or redundant parameters are brought. We propose a multi-level and multi-scale feature aggregation network (MMFANet). A spatial pyramid module is designed by cascading dilated convolutions with different receptive fields to extract multi-scale features layer-by-layer. Subseqently, a lightweight backbone network is built by reducing the feature channel capacity of the module. To improve the accuracy of our network, we design two additional modules to separately capture spatial details and high-level semantics from the backbone network without significantly increasing the computation cost. Comprehensive experimental results show that our model achieves 79.3% MIoU on the Cityscapes test dataset at a speed of 58.5 FPS, and it is more accurate than SwiftNet (75.5% MIoU). Furthermore, the number of parameters of our model is at least 53.38% less than that of other state-of-the-art models. [ABSTRACT FROM AUTHOR]

Due to the rapid development of deep learning and artificial intelligence techniques, denoising via neural networks has drawn great attention due to their flexibility and excellent performances. However, for most convolutional network denoising methods, the convolution kernel is only one layer deep, and features of distinct scales are neglected. Moreover, in the convolution operation, all channels are treated equally; the relationships of channels are not considered. In this paper, we propose a multi-scale feature extraction-based normalized attention neural network (MFENANN) for image denoising. In MFENANN, we define a multi-scale feature extraction block to extract and combine features at distinct scales of the noisy image. In addition, we propose a normalized attention network (NAN) to learn the relationships between channels, which smooths the optimization landscape and speeds up the convergence process for training an attention model. Moreover, we introduce the NAN to convolutional network denoising, in which each channel gets gain; channels can play different roles in the subsequent convolution. To testify the effectiveness of the proposed MFENANN, we used both grayscale and color image sets whose noise levels ranged from 0 to 75 to do the experiments. The experimental results show that compared with some state-of-the-art denoising methods, the restored images of MFENANN have larger peak signal-to-noise ratios (PSNR) and structural similarity index measure (SSIM) values and get better overall appearance. [ABSTRACT FROM AUTHOR]

Cloud detection plays a vital role in remote sensing data preprocessing. Traditional cloud detection algorithms have difficulties in feature extraction and thus produce a poor detection result when processing remote sensing images with uneven cloud distribution and complex surface background. To achieve better detection results, a cloud detection method with multi-scale feature extraction and content-aware reassembly network (MCNet) is proposed. Using pyramid convolution and channel attention mechanisms to enhance the model's feature extraction capability, MCNet can fully extract the spatial information and channel information of clouds in an image. The content-aware reassembly is used to ensure that sampling on the network can recover enough in-depth semantic information and improve the model cloud detection effect. The experimental results show that the proposed MCNet model has achieved good detection results in cloud detection tasks. [ABSTRACT FROM AUTHOR]

• We proposed a model based on SE-Inception to correctly predict solanaceae diseases. • SE-Inception has a good performance on our constructed dataset and public dataset. • SE-Inception can meet the demand of disease identification on mobile devices. Aiming at the diseases of tomato and eggplant, we present a solanaceae disease recognition model based on SE-Inception. Our model uses batch normalization layer (BN) to accelerate network convergence. Besides, SE-Inception structure and multi-scale feature extraction module is adopted to improve accuracy of this model. Our sample data set consists of 4 disease categories including whitefly, powdery mildew, yellow smut, cotton blight. We also add healthy leaves into it. In order to reduce overfitting, the data set is expanded by the data enhancement method of translation, rotation and flip. Experiments show that the average recognition accuracy of this model is 98.29% and the model size is 14.68 MB on our constructed dataset. In addition, in order to verify the robustness of this model, it was also verified on the public data set of PlantVillage, and the top-1, top-5 accuracy and the size of our proposed model is 99.27%, 99.99% and 14.8 MB respectively. Moreover, we implemented a solanaceae disease image recognition system using this model based on the Android. The accuracy of average recognition and the recognition time of a single photo are 95.09% and 227 ms, respectively. Our constructed model has a small number of parameters with maintaining high accuracy, which can meet the needs of automatic recognition of disease images on mobile devices. Data and code are available at https://github.com/Jujube-sun/diseaseRecognition. [ABSTRACT FROM AUTHOR]

• A novel ReSidual U-block (RSU) is designed to capture multi-scale deep features. • A nested U-structure, called U2-Net, that uses RSU is developed for salient object detection. • Both large (176.3 MB) and small (4.7 MB) instances of U2-Net get competitive results. In this paper, we design a simple yet powerful deep network architecture, U2-Net, for salient object detection (SOD). The architecture of our U2-Net is a two-level nested U-structure. The design has the following advantages: (1) it is able to capture more contextual information from different scales thanks to the mixture of receptive fields of different sizes in our proposed ReSidual U-blocks (RSU), (2) it increases the depth of the whole architecture without significantly increasing the computational cost because of the pooling operations used in these RSU blocks. This architecture enables us to train a deep network from scratch without using backbones from image classification tasks. We instantiate two models of the proposed architecture, U2-Net (176.3 MB, 30 FPS on GTX 1080Ti GPU) and U2-Net† (4.7 MB, 40 FPS), to facilitate the usage in different environments. Both models achieve competitive performance on six SOD datasets. The code is available: https://github.com/NathanUA/U-2-Net. [ABSTRACT FROM AUTHOR]

The quality of remotely sensed images is usually determined by their spatial resolution, spectral resolution, and coverage. However, due to limitations in the sensor hardware, the spectral resolution, spatial resolution, and swath width of the coverage are mutually constrained. Remote sensing image fusion aims at overcoming the different constraints of remote sensing images, to achieve the purpose of combining the useful information in the different images. However, the traditional spatial–spectral fusion approach is to use data in the same swath width that covers the same area and only considers the mutually constrained conditions between the spectral resolution and spatial resolution. To simultaneously solve the image fusion problems of the swath width, spatial resolution, and spectral resolution, this paper introduces a method with multi-scale feature extraction and residual learning with recurrent expanding. To discuss the sensitivity of convolution operation to different variables of images in different swath widths, we set the sensitivity experiments in the coverage ratio and offset position. We also performed the simulation and real experiments to verify the effectiveness of the proposed framework with the Sentinel-2 data, which simulated the different widths. [ABSTRACT FROM AUTHOR]

Spectral features cannot effectively reflect the differences among the ground objects and distinguish their boundaries in hyperspectral image (HSI) classification. Multi-scale feature extraction can solve this problem and improve the accuracy of HSI classification. The Gaussian pyramid can effectively decompose HSI into multi-scale structures, and efficiently extract features of different scales by stepwise filtering and downsampling. Therefore, this paper proposed a Gaussian pyramid based multi-scale feature extraction (MSFE) classification method for HSI. First, the HSI is decomposed into several Gaussian pyramids to extract multi-scale features. Second, we construct probability maps in each layer of the Gaussian pyramid and employ edge-preserving filtering (EPF) algorithms to further optimize the details. Finally, the final classification map is acquired by a majority voting method. Compared with other spectral-spatial classification methods, the proposed method can not only extract the characteristics of different scales, but also can better preserve detailed structures and the edge regions of the image. Experiments performed on three real hyperspectral datasets show that the proposed method can achieve competitive classification accuracy. [ABSTRACT FROM AUTHOR]