Search

This paper presents a pansharpening technique based on the non-subsampled contourlet transform (NSCT) and convolutional autoencoder (CAE). NSCT is exceptionally proficient at presenting orientation information and capturing the internal geometry of objects. First, it’s used to decompose the multispectral (MS) and panchromatic (PAN) images into high-frequency and low-frequency components using the same number of decomposition levels. Second, a CAE network is trained to generate original low-frequency PAN images from their spatially degraded versions. Low-resolution multispectral images are then fed into the trained convolutional autoencoder network to generate estimated high-resolution multispectral images. Third, another CAE network is trained to generate original high-frequency PAN images from their spatially degraded versions. The result of low-pass CAE is fed to the trained high-pass CAE to generate estimated high-resolution multispectral images. The final pan-sharpened image is accomplished by injecting the detailed map of the spectral bands into the corresponding estimated high-resolution multispectral bands. The proposed method is tested on QuickBird datasets and compared with some existing pan-sharpening techniques. Objective and subjective results demonstrate the efficiency of the proposed method.

The recent years has witnessed a rapid development of Deep Learning (DL) based Automation Modulation Classification (AMC) methods, which has proved to outperform traditional classification approaches. In order to disturb the deep neural networks for AMC, in this paper, we propose an adversarial attack method to generate fake signals for fooling DL-based classifiers. Firstly, some constraints on visual difference and recoverability of fake signals are defined. Next, a Few Data Point Attacker (FDPA) is proposed to generate fake signals with few perturbed data points via differential evolution algorithm. Some experiments are taken on a public dataset, RML 2016.10a, and the results show that fake signals generated by the FDPA can remarkably reduce the accuracies of three types of DL-based AMC classifiers, a Convolutional Neural Network (CNN) based classifier, a Long Short-Term Memory Recurrent Neural Network (LSTM-RNN) based classifier, and a classifier combined with CNN and LSTM-RNN. The code will be available.

In this paper a new Sparse Tensor Auto-Encoder (STAE) model is proposed to learn a latent and discriminative feature representations for saliency detection. By formulating the background patches as holistic high-dimensional tensors and learning multi-dimensional dictionary to code image patches, the coding error can precisely reveal the difference between salient object and background. Then a saliency map can be derived by a subsequent refinement of the representation errors of patches via image segmentation. Several benchmark datasets are used to verify the effectiveness of the proposed method. The results show that the proposed STAE can accurately locate saliency region and outperform its counterparts.

Handcraft features are commonly used for signal classification, which is a time-consuming feature engineering. In order to develop a general and robust feature learning method for radio signals, a novel One-dimensional Deep Attention Convolution Network (ODACN) is proposed to automatically extract discriminative features and classify various kinds of signals. First, one-dimensional (1-D) sparse filters are designed to learn hierarchical features of raw signals. Second, an attention layer is constructed to weight and assemble feature maps, to derive more context-relevant representation. By using simple 1-D filtering, ODACN is characteristic of less parameters and lower computation complexity than traditional Convolutional Neural Networks (CNNs). Moreover, feature attention can mimic a succession of partial glimpses of humans and focus on context parts of signals, thus helps in recognizing signals even at low Signal-to-Noise Ratio (SNR). Some experiments are taken to classify 31 kinds of signals with different modulation and channel coding types, and the results show that ODACN can achieve accurate classification of very similar signals, without any prior knowledge and manual operation.

Compared with natural image super-resolution, hyperspectral image super-resolution (HSR) is more complex because the redundancy in spectral bands and spatial information. To overcome the difficulties exist in HSR, in this paper, we propose a tensor spatial-spectral joint correlation based HSR method. Start with the tensor representation, we construct a series of fourth-order tensors to preserve the intrinsic structure of hyperspectral images, and then explore the spatial-spectral joint correlation based on meaningful interpretations of tensor canonical matrices. To further constrain the spectral characteristics, we analyze the sparsity of the spectral gradients and model it with Laplacian prior. Then, the two regularizations are combined with the reconstruction model to develop a new HSR method. Finally, an iterative optimization algorithm based on alternating direction method of multiplier (ADMM) and augmented Lagrangian multiplier method is proposed to reconstruct the high-resolution hyperspectral images. Experimental results on several data sets illustrate the effectiveness of our proposed method both in visual and numerical comparisons.

The past decade has witnessed a prosperity of sparsity-inspired face hallucination methods that use sparse prior and instances to generate High-Resolution (HR) faces. However, they need numerous Low-Resolution (LR) and HR instance pairs and adopt approximate sparse coding, which will bring bias to the recovery and suffer from high computational burden. In this paper we advance a Single Face Image Hallucination (SFIH) method from a new perspective of Non-linear Learning Compressive Sensing (NLCS), which can recover HR faces from a surprisingly small number of HR faces. The nonlinear sparse coding of facial images is explored, and a Deep AutoEncoder (DAE) network is constructed for learning a kernel function from a single HR instance set. SFIH is then reduced to an analytic compressive recovery problem by reformulating linear sparse coding as a nonlinear DAE model. By exploring the nonlinear sparsity in the feature space, NLCS can accurately and rapidly recover HR facial images with large magnification factor and exhibit robustness to LR-HR instance pairs mapping. Some experiments are taken on realizing 3X, 6X, 9X amplification of face images, and the results prove its efficiency and superiority to its counterparts.

Deep Neural Networks (DNNs) have achieved remarkable accuracy improvements for automatic modulation classification. However, the employed networks often have millions of parameters and need very high computation, which makes it difficult to deploy these models on portable devices with limited resources. We propose a cross model deep learning scheme to build a lightweight deep network for accurate modulation classification. Firstly, a large Hybrid DNN (HDNN) that is composed of convolutional and recurrent layers is constructed and trained for automatic and accurate classification of signals. Then we build a smaller Layered Resnet Network (LRN) with shallow layers and few nodes. The HDNN and LRN are taken as a Teacher Model (TM) and a Student Model (SM) respectively. Finally, a knowledge distillation method is proposed to guide the learning of the SM, by formulating a teaching loss from the prediction of the TM to train the SM. The performances of the proposed HDNN and LRN are investigated on the public RadioML2016.10a and RadioML2016.10b data sets. The experimental results show that the trained HDNN presents state-of-the-art classification results and the LRN trained in this scheme takes only about a sixth of the HDNN's inference time and consumes only 472.3KB for storage, with a slight accuracy decrease compared with the large HDNN.

Desired by sparse subset learning, in this paper, a hyperspectral band selection method via pairwise band agreement with spatial-spectral graph regularier, referred as Regularized Band Selection via Learned Pairwise Agreement (RBS-LPA), was proposed. The process was formulated as a graph-regularized row-sparse constrained optimization problem, which select a few representative bands to code the all bands based on the learned pairwise band agreement. In RBS-LPA, a spatial-spectral informative graph, constructed by spatial-spectral neighbor relationship, is incorporated to encode both the spatial and spectral geometrical structure. By combining the learning procedure with graph regularizer jointly, the graph regularizer can adaptively change with the sparse representative bands, which ensures that the selected bands can well preserve the local manifold structure. Experimental results on three real urban hyperspectral data demonstrate the efficiency of the proposed RBS-LPA and achieve convincing performance.

Remote Sensing Images (RSIs) often have extremely wide width and abundant terrain. In order to achieve rapid object detection in large RSIs, in this paper, a Deep Hash Assisted Network (DHAN) is constructed by introducing a hashing encoding of images in a two-stage deep neural network. Different with the available detection networks, DHAN first locates candidate object regions and then transfers the learned features to another Region Proposal Network (RPN) for detection. On the one hand, it can avoid the calculations on the background irrelevant to objects. On the other hand, the built hash encoding layer in DHAN can accelerate the detection via binary hash features. Moreover, a self attention layer is designed and combined with the convolution layer, to distinguish relatively small objects regions from a very large scene. The proposed method is tested on several public data sets, and the comparison results show that DHAN can remarkably improve the detection efficiency on large RSIs and simultaneously achieve high detection accuracy.

Tensor representation is the most natural and effective way to preserve the structural information of hyperspectral image (HSI), and thus is very beneficial to HSI processing. This paper represents the spectral features of each testing pixel and its spatial neighbors as a Spatial Neighbor Tensor (SNT), whose spectral vectors can be simultaneously sparse coded by the spectral vectors of a few common training pixels. The obtained sparse coding coefficients could be regarded as a Slice Sparse Coding Tensor (SSCT), which can be adaptively learnt and utilized to predict the labels of HSI pixels. Furthermore, to improve the efficiency, the Compressive Dimensionality Reduction (CDR) is introduced into tensor slice sparse coding for optimizing SSCT and thus an SSCT based Classifier with CDR (SSCTC-CDR) is proposed for hyperspectral image classification (HIC). The performance of SSCTC-CDR is evaluated on three real HSI data, and the results show that it can obtain high-accurate classification result with relatively low computation lost.

In this paper, a new Separable Attention Capsule Network (SACN) is proposed for signal classification. SACN is a light-weight network composed of multi-channel separable convolution layer, attention module and classification layer. First, depth-wise convolution is employed to extract features of signals in a low-complexity manner, and the multi-channel network structure is designed to increase the network width to improve the diversity of features of signals. Then a channel attention module is followed by a capsule network whose element contains a group of neurons. This attention module can explore the interdependence among channels to use global information to selectively strengthen some important channels, thus achieving the improvement of generalization ability of SACN. Some experiments are taken on several datasets with communication and radar signals, and the comparison results prove the efficiency of SACN and the superiority to its counterparts.

Nowadays, with the development of remote sensing technology, very-high-resolution (VHR) remote sensing image object detection technology attracts more and more attention. However, various challenges still exist in remote sensing image object detection field, such as the complex and varied appearances, the expensive manual annotation, and difficult in fast detecting the large scene image. In this paper, we propose a multi-scale image block-level fully convolutional neural network (MIF-CNN) for remote sensing image object detection, which can solve the above problems to a certain degree. First, the training data sets which only require class labels and do not need the bounding box label can reduce the spend of manual annotation during training. Second, the MIF-CNN is designed to extract the multi-scale-based high-level feature which can better represent the various types of objects. The image block-level fully convolutional network contributes to improving computing efficiency and can directly detect any size of the input image, including a large scene remote sensing image. In the testing phase, the large scene image is directly input to MIF-CNN model, and the detection results are generated from the MIF-CNN output maps which are improved by the proposed bounding boxes modification strategy and local re-recognized strategy. The experiments on NWPU VHR-10 [1] and two Airports data sets demonstrate the effectiveness of the proposed method.

Synthetic aperture radar (SAR) image classification aims at labeling pixels with different categories and this is both, a fundamental step for automatic target recognition (ATR) and a prerequisite for further interpretation. In the past decades, various methods have been proposed for the classification of SAR targets and among them are discriminative dictionary learning (DDL) methods. These DDL methods have recently gained attention from researchers' community due to the fact that they are very powerful on both, representation and discrimination during the classification process of SAR images. However, most of the existing DDL methods adopt l0-norm or l1-norm to ensure the sparsity, but in general, these DDL methods suffer from a high computational burden. Furthermore, it is important to minimize the execution time in the phase of online testing for the scenario of onboard real-time or near real-time SAR automatic target recognition such as modern unmanned aerial vehicle SAR platforms. That said, on reducing execution time, we are confronted with the problem of enhancing recognition efficiency while maintaining its accuracy. In order to solve this problem, our paper proposes a fast DDL method (named as FaDDL) based on a nonlinear analysis co-sparse model by adopting an l1,∞-norm ball as a constraint to replace l0-norm or l1-norm on the coding coefficient matrix. The experimental results show that our proposed method significantly reduces execution time, without losing the classification accuracy.

The past few years have witnessed the prosperity of establishing deep architectures for modeling complex structured data. In this paper, motivated by the hierarchical, multi-scale and sparse characteristics of Human Visual System (HVS), we advance a new Deep and Sparse Representative (DSR) saliency detection approach for natural images. Under the assumption that salient regions are considered as sparse representatives of an image, we gradually select the salient regions via a deep and hierarchical sparse coding model. Moreover, in order to well preserve image structures and achieve more efficient coding, we formulate image patches as tensors and propose a Sparse Tensor Coding based DSR (STC-DSR) approach to find representatives. A generalized Laplacian regularizer is further cast on the coding to refine the coefficients. Finally, the saliency map is formulated by a spatial aggregation and multi-scale fusion of multilayer representatives. The proposed STC-DSR is investigated on the synthetic and public datasets and the experimental results prove its efficiency and superiority to its counterparts.

With the rapid development of earth observation satellites, on-orbit data processing is becoming more and more desirable. In this paper, a new on-orbit change detection method for Synthetic Aperture Radar (SAR) images, is proposed via an Extreme Self-paced Learning Machine (ESLM). First, a reflectivity-spatial affinity is defined to measure the similarity between two segmented super-pixels, to identify the initial three groups of pixels: strictly changed, strictly unchanged and fuzzy pixels. Then a new extreme self-paced learning machine is developed, by gradually selecting the most confident changed pixels and predicting the changed pixels in an incremental pattern. Moreover, both the labeled and unlabeled samples are explored to realize semi-supervised classification. Different with the available methods, ESLM works in a self-paced learning pattern and achieves accurate detection, for it can automatically choose the training samples and explore unlabeled samples to enhance the online prediction of changes. Therefore, ESLM has the characteristics of accurate and robust detection, parameter free, low-complexity and rapid implementation, which is very suitable for on-orbit processing. Some experiments are taken on five real benchmark datasets, and the results verify the effectiveness of ESLM.

Extracting matched details of the PANchromatic (PAN) image and injecting them into the MultiSpectral (MS) images, is very crucial in pansharpening. In this paper, a new pansharpening method based on Joint Local Low Rank Decomposition (JLLRD) and Hierarchical Geometric Filtering (HGF) is proposed. First, a cascaded geometric filtering is performed on the PAN and MS images, to extract their multi-scale directional details. Then a joint local low rank decomposition is developed to deduce low-rank and sparse components for injection. Finally, an adaptive injection rule based on spectral correlation coefficient, is designed to further reduce spectral distortion of the fused images. Several experiments are taken to investigate the performance of the proposed JLLRD-HGF method, and the results show that it can extract more accurate injection details and produce less spectral and spatial distortions than its counterparts.

Inspired by human’s learning characteristic that knowledge is gradually learned little by little, a Spatial-Polarimetric Reinforcement Learning (SPRL) approach is proposed for Polarimetric Synthetic Aperture Radar (PolSAR) data classification, from a new perspective of reinforcement learning. In our method, each pixel has its own “state” and “action”, and can modify its “action” based on interactions with the “environment”. A spatial-polarimetric “reward” function, is designed from a local neighborhood region to explore both the spatial and polarimetric information for more accurate classification. Thus a self-evolution and model-free classifier can be obtained, which has simple principle and robustness to speckle noises existed in the data. By an interaction with the environment, SPRL can obtain high classification accuracy when only very few labeled pixels are available. Several real PolSAR datasets are used to investigate the effectiveness of the proposed method, and the results show that SPRL is superior to its counterparts.

Object detection is one of the most important and challenging branches of computer vision, which has been widely applied in people’s life, such as monitoring security, autonomous driving and so on, with the purpose of locating instances of semantic objects of a certain class. With the rapid development of deep learning algorithms for detection tasks, the performance of object detectors has been greatly improved. In order to understand the main development status of object detection pipeline thoroughly and deeply, in this survey, we analyze the methods of existing typical detection models and describe the benchmark datasets at first. Afterwards and primarily, we provide a comprehensive overview of a variety of object detection methods in a systematic manner, covering the one-stage and two-stage detectors. Moreover, we list the traditional and new applications. Some representative branches of object detection are analyzed as well. Finally, we discuss the architecture of exploiting these object detection methods to build an effective and efficient system and point out a set of development trends to better follow the state-of-the-art algorithms and further research.