Search

This article discusses the importance of multi-sensor imaging and fusion technology in various fields such as medical imaging, engineering construction, and object detection. It presents different research studies related to multi-sensor imaging and fusion, including methods for imaging detection, feature extraction, and fusion, as well as objective evaluation methods. The research results demonstrate the effectiveness and potential applications of multi-sensor imaging and fusion technology in different areas. The document also provides a summary of various research topics related to the specific applications of multi-sensor technology in engineering problems, such as metal corrosion depth estimation, tunnel management, sentiment analysis, highway vehicle speed detection, road surface detection, and brainstem hematoma puncture and drainage surgery. The authors hope that this research will inspire further advancements in multi-sensor imaging and fusion. [Extracted from the article]

Learning discriminative features is of vital importance for automatic facial expression recognition (FER) in the wild. In this article, we propose a novel Slide-Patch and Whole-Face Attention model with SE blocks (SPWFA-SE), which jointly perceives the discriminative locality characteristics and informative global features of the face for effective FER. Specifically, the well-designed slide patches are proposed to extract local features. Different from the existing methods, our slide patches not only can maintain the information at the edge area of patches, but also do not need to detect facial landmarks. Moreover, to make the model adaptively focus on the distinguishable regions, an attention module is proposed in the patch level to learn the weight of each patch. Furthermore, squeeze-and-excitation blocks are explored in the channel level to learn the weight of each channel. As such, the proposed multi-level feature extraction and attention mechanisms can enhance the representative ability of the learned features. Extensive experiments on five challenging datasets demonstrate that our method can achieve state-of-the-art performance. Cross database experiments on another three databases show the superior generalization performance of our model. Furthermore, complexity analysis results show that our model contains fewer parameters with fast training advantages than other competing models. [ABSTRACT FROM AUTHOR]

Single image super-resolution (SISR) using deep convolutional neural networks (CNNs) achieves the state-of-the-art performance. Most existing SISR models mainly focus on pursuing high peak signal-to-noise ratio (PSNR) and neglect textures and details. As a result, the recovered images are often perceptually unpleasant. To address this issue, in this paper, we propose a texture and detail-preserving network (TDPN), which focuses not only on local region feature recovery but also on preserving textures and details. Specifically, the high-resolution image is recovered from its corresponding low-resolution input in two branches. First, a multi-reception field based branch is designed to let the network fully learn local region features by adaptively selecting local region features in different reception fields. Then, a texture and detail-learning branch supervised by the textures and details decomposed from the ground-truth high resolution image is proposed to provide additional textures and details for the super-resolution process to improve the perceptual quality. Finally, we introduce a gradient loss into the SISR field and define a novel hybrid loss to strengthen boundary information recovery and to avoid overly smooth boundary in the final recovered high-resolution image caused by using only the MAE loss. More importantly, the proposed method is model-agnostic, which can be applied to most off-the-shelf SISR networks. The experimental results on public datasets demonstrate the superiority of our TDPN on most state-of-the-art SISR methods in PSNR, SSIM and perceptual quality. We will share our code on https://github.com/tocaiqing/TDPN. [ABSTRACT FROM AUTHOR]

The informativeness and discriminativeness of features collaboratively ensure high-accuracy Facial Expression Recognition (FER) in the wild. Most of existing methods use the single-path deep convolutional neural network with softmax loss for basic FER, while they cannot deal with the challenging situations of the compound FER in the wild, because they fail to learn informative and discriminative features in a targeted manner. To this end, we present an Informative and Discriminative Feature Learning (IDFL) framework that consists of two key components: the Multi-Path Attention Convolutional Neural Network (MPACNN) and Balanced Separate loss (BS loss), for both basic and compound high-accuracy FER in the wild. Specifically, MPACNN leverages different paths to learn diverse features. These features are then adaptively fused into informative ones via an attention module, such that the model can adequately capture detailed information for both basic and compound FER. The BS loss maximizes the inter-class distance of features and minimizes the intra-class one. In this way, the features are discriminative enough for high-accuracy FER in the wild. Particularly, the BS loss is invoked as the objective function of MPACNN, so the model can learn informative and discriminative features at the same time, yielding better performance. Seven databases are utilized to evaluate the proposed method, and the results demonstrate that our method achieves state-of-the-art performance on both basic and compound expressions with good generalization ability. Moreover, our model contains fewer parameters and can be trained faster than other related models. [ABSTRACT FROM AUTHOR]

With the increased model size of convolutional neural networks (CNNs), overfitting has become the main bottleneck to further improve the performance of networks. Currently, the weighting regularization methods have been proposed to address the overfitting problem and they perform satisfactorily. Since these regularization methods cannot be used in all the networks and they are usually not flexible enough in different phases of the training and test processes, this article proposes a multiscale conditional (MSC) regularization method. MSC divides the intermediate features into different scales and then generates new data for each scale features, respectively. In addition, the new data are generated by employing the information from two conditions: 1) each sample feature and 2) each layer pattern. Finally, a self-identity structure is proposed to supplement the features with the generated data. Therefore, MSC can adaptively and efficiently generate much finer and individualized data to make the entire regularization more flexible. Furthermore, MSC is more general and can be applied to all kinds of networks through the proposed self-identity structure. The experimental results on all the benchmark datasets showed that the proposed MSC regularization method achieves the best performances in all the networks. [ABSTRACT FROM AUTHOR]

Recently, the group convolutions are widely used in mobile convolutional neural networks (CNNs) to improve the model's efficiency. However, the training process of these popular group-conv mobile models is usually time-consuming compared to the regular models. Since some practical applications usually need retraining when the new data are added, the training process should be also efficient. Therefore, this paper firstly explores the inefficiency of current popular group-conv mobile CNNs to propose the guidelines of constructing mobile CNNs. According to these guidelines, the fully shared convolutional neural networks (FSC-Nets) are proposed in this paper. The FSC-Nets can share the same filter in both the channel and layer extents, and it is the first method to share the filters from all the extents. Experimental results show that, compared with current popular regular networks and the latest state-of-the-art group-conv mobile networks, the FSC-Nets can perform better while effectively decreasing the model size and runtime in both the training and test processes. [ABSTRACT FROM AUTHOR]

Currently, in order to deploy the convolutional neural networks (CNNs) on the mobile devices and address the over-fitting problem caused by the less abundant datasets, reducing the redundancy of parameters is the main target to construct the mobile CNNs. Based on this target, this paper proposes two novel convolutional kernels, multiple group reused convolutions (MGRCs) and decomposed point-wise convolutions (DPCs), to improve the efficiency of parameters by removing the parameter's redundancy. The summation of MGRC and DPC is called high-parameter-efficiency convolutions (HPEC) in this paper, and the relevant CNNs can be called HPE-CNNs. Experimental results showed that, compared with the traditional convolutional kernels, HPEC can greatly decrease the model size without affecting the performance. Additionally, since the HPE-CNNs reduce the redundancy of parameters more thoroughly than the other mobile CNNs, they can address the over-fitting problem more effectively on the challenging datasets with less abundant training information. [ABSTRACT FROM AUTHOR]

In this paper, a novel deep network is proposed for multi-focus image fusion, named Deep Regression Pair Learning (DRPL). In contrast to existing deep fusion methods which divide the input image into small patches and apply a classifier to judge whether the patch is in focus or not, DRPL directly converts the whole image into a binary mask without any patch operation, subsequently tackling the difficulty of the blur level estimation around the focused/defocused boundary. Simultaneously, a pair learning strategy, which takes a pair of complementary source images as inputs and generates two corresponding binary masks, is introduced into the model, greatly imposing the complementary constraint on each pair and making a large contribution to the performance improvement. Furthermore, as the edge or gradient does exist in the focus part while there is no similar property for the defocus part, we also embed a gradient loss to ensure the generated image to be all-in-focus. Then the structural similarity index (SSIM) is utilized to make a trade-off between the reference and fused images. Experimental results conducted on the synthetic and real-world datasets substantiate the effectiveness and superiority of DRPL compared with other state-of-the-art approaches. The source code can be found in https://github.com/sasky1/DPRL. [ABSTRACT FROM AUTHOR]

• We explore the factors of CNNs' inefficiency and the parameters' properties. • We propose mobile Highly Shared Convolutional Neural Networks (HSC-Nets). • HSC-Nets share the parameters thoroughly by employing two novel shared methods. • HSC-Nets can effectively improve efficiency both in the training and test. In order to deploy deep Convolutional Neural Networks (CNNs) on the mobile devices, many mobile CNNs are introduced. Currently, some online applications are usually re-trained because of the constantly-increasing data. However, compared with the regular models, it is not very efficient to train the present mobile models. Therefore, the purpose of this paper is to propose efficient mobile models both in the training and test processes through exploring the main causes of the current mobile CNNs' inefficiency and the parameters' properties. Finally, this paper introduces Highly Shared Convolutional Neural Networks (HSC-Nets). The HSC-Nets employ two shared mechanisms to reuse the filters comprehensively. Experimental results showed that, compared with the regular networks and the latest state-of-the-art group-conv mobile networks, the HSC-Nets can achieve promising performances and effectively decrease the model size. Furthermore, it is also more efficient in both the training and test processes. [ABSTRACT FROM AUTHOR]