Search

Low-resolution characteristics of multispectral images restrict their usability. Various approaches (e.g., single-image super-resolution reconstruction (SISR) and pansharpening method) have been proposed to enrich the spatial details of low-resolution multispectral images (LRMSs) to obtain high-resolution ones. While the pansharpening method inevitably depends on panchromatic (PAN) images, which limits its application scenarios, SISR does not need auxiliary images, yet the blurred edge details within the reconstructed super-resolution multispectral (SRMS) images still remain a big challenge. In this work, we propose a novel high-frequency feature transfer (HFFT-PAN) method for multispectral image super-resolution to tackle the above drawbacks. Specifically, we first exploit the inherent low-frequency features among LRMS images to facilitate the extraction of high-frequency features from PAN images. After that, the high-frequency features from PAN images are transferred to the reconstruction procedure of multispectral images so that the SRMS images can not only benefit from the edge detail information from PAN images but also avoid the utilization of any PAN images during inference. Moreover, we employ an additional contrastive loss during training to ensure the fidelity of the generated SRMS images. Qualitative and quantitative evaluations exhibit that the proposed method performs favorably against state-of-the-art methods. The model and code are available at https://github.com/wx0110wx/HFFT.

With the rapid development and widespread deployment of radar technology, the interpretation of the vast amount of synthetic aperture radar (SAR) imagery obtained daily has emerged as a hot topic. The complexity of the electromagnetic scattering mechanisms contained within radar images makes SAR image interpretation a challenging task. Current methodologies for SAR image interpretation primarily involve feature extraction-based techniques, categorized into image-domain and frequency-domain algorithms. However, these methods are faced with issues, such as rough segmentation in images, high-computational complexity, and poor robustness, presenting significant challenges in the field. In this article, a novel framework for SAR image interpretation is proposed, leveraging reinforcement learning (RL) for the extraction of electromagnetic scattering features and the inversion of parameters. Within this framework, a nonsparse reward function, combined with curriculum learning, is introduced as the supervisory information. It enables more efficient policy updates through a structured two-stage training approach. In addition, an algorithm that integrates a four-neighbor breadth-first search (BFS) with the watershed segmentation process is proposed, aiming to enhance the accuracy of scattering center analysis in SAR imagery. The attribute scattering center model (ASCM) is utilized as a prototype for conducting algorithmic research and experimentation. Experimental results on both simulation data and measured data have indicated that the proposed method significantly improves efficiency while ensuring accuracy, demonstrating its capability to extract parameters from measured data in most scenarios.

Amidst the landscape of contemporary remote-sensing (RS) technology, the endeavor to detect and recognize aircraft within RS images (RSIs) assumes pivotal strategic and practical significance. The complex nature of fine-grained aircraft recognition is a result of the intricate interplay between aircraft and their background environments, alongside category imbalance, which collectively leads to the emergence of a long-tail distribution within the dataset. However, experts proficient in RSI interpretation can effectively address these challenges through the application of prior knowledge. This article introduces the aircraft reasoning network (ARNet), a framework tailored for aircraft detection and fine-grained recognition in RSIs, building upon prior knowledge employed in expert interpretation. Specifically, the knowledge reasoning module (KRM) introduces a knowledge graph (KG) that incorporates both common and expert knowledge into the end-to-end network. Additionally, the network encompasses a spatial context module (SCM) and an airport facility relationship module (AFRM). These components facilitate highly accurate detection and recognition of fine-grained aircraft in diverse environmental contexts by employing adaptive prior knowledge reasoning and optimizing target spatial location. Furthermore, an independent aircraft component discrimination module (ACDM) distinguishes aircraft based on their predominant component features, contributing to improved classification performance in both the few-shot and easily confused categories. Moreover, this article introduces the AR-RSI dataset, a compilation of RSIs capturing fine-grained aircraft targets from diverse locations. The effectiveness and superiority of ARNet are exemplified by AR-RSI, achieving a minimum of 3.7% higher mean average precision (mAP) than the mainstream aircraft detection framework.

Superpixel analysis is showing great potential for high-resolution synthetic aperture radar (SAR) image change detection (CD), as it uses a larger detection granularity and enhances computational efficiency. However, some deficiencies still exist. It is difficult for previous methods to extract the complete change regions from cluttered backgrounds. Meanwhile, the local spatial correlation and structure consistency are not well represented. To address the above problems and achieve better separation of changed and unchanged superpixels in complicated scenarios, we design a novel unsupervised CD framework from the perspective of low-rank matrix decomposition (LRMD) theory. The entire framework is carried out in two stages. First, the $\ell _{1}$ -norm sparsity constraint LRMD model is constructed to decompose change features into a low-rank component associated with background and a sparse component representing changed regions. Then, the local spatial correlation and structure consistency constraint are explicitly modeled by introducing a Laplacian regularization term. The unified model smooths the local similarity superpixels and enlarges the distance between changed regions and the background in the feature subspace. In this stage, the saliency difference image (DI) is generated to indicate the change probabilities of each superpixel. Furthermore, a classification refining (CR) module is designed to learn the projection from the change feature matrix to the saliency DI, which can further fine-tune such obscure regions and boost the binary classification. Extensive experiments on five challenging datasets from the TerraSAR-X sensor demonstrate the effectiveness and superiority of the proposed method.

Multimodal image fusion has recently garnered increasing interest in the field of remote sensing. By leveraging the complementary information in different modalities, the fused results may be more favorable in characterizing objects of interest, thereby increasing the chance of a more comprehensive and accurate perception of the scene. Unfortunately, most existing fusion methods tend to extract modality-specific features independently without considering intermodal alignment and complementarity, leading to a suboptimal fusion process. To address this issue, we propose a novel interactive generative adversarial network (IG-GAN), for the task of multimodal image fusion. IG-GAN comprises guided dual streams tailored for enhanced learning of details and content, as well as cross-modal consistency. Specifically, a details-guided interactive running-in module (GIR1) and a content-guided interactive running-in module (GIR2) are developed, with the stronger modality serving as guidance for detail richness or content integrity, and the weaker one assisting. To fully integrate multigranularity features from dual-modality, a hierarchical fusion and reconstruction branch is established. Specifically, a shallow interactive fusion (SIF) module followed by a multilevel interactive fusion (MIF) module is designed to aggregate multilevel local and long-range features. Concerning feature decoding and fused image generation, a high-level interactive fusion and reconstruction module (HRM) is further developed. In addition, to empower the fusion network to generate fused images with complete content, sharp edges, and high fidelity without supervision, a loss function facilitating the mutual game between the generator and two discriminators is also formulated. Comparative experiments with 14 state-of-the-art methods are conducted on three datasets. Qualitative and quantitative results indicate that IG-GAN exhibits obvious superiority in terms of both visual effect and quantitative metrics. Moreover, experiments on two RGB-IR object detection datasets are also conducted, which demonstrate that IG-GAN can enhance the accuracy of object detection by integrating complementary information from different modalities. The code will be available at https://github.com/flower6top.

This study introduces an innovative approach to hyperspectral imaging (HSI) classification by integrating convolution, recurrence, and self-attention mechanisms in a 3-D configuration. We address several challenges such as the 1) disruption of spectral continuity by traditional dimensionality reduction methods like PCA, 2) the overlooking of band-to-band continuous features in existing spatial-only 2-D multiscanning strategy, and 3) the limitations in model design by simply cascading recurrent neural networks (RNNs) with transformers for HSI analysis. Our solution involves three core components: 1) subband grouping with group-wise convolution for refined dimension reduction; 2) a novel cubed 3-D-multiscanning technique enabling thorough multidirectional analysis in both spectral and spatial domains; and 3) the development of a Cubic-Net framework with a specially designed segmented recurrent transformer (SRT). This SRT is tailored to effectively utilize spectral continuity along with spatial contextual features, overcoming common sequential data analysis challenges seen in RNNs and transformers. Furthermore, our feature fusion strategy successively integrates “short-term” and “long-term” SRT features, thereby enhancing the model’s ability to process both spectral and spatial features effectively. Experimental results from three public HSI datasets indicate our method’s improved performance over existing baselines and state-of-the-art methods. This research offers a new perspective on 3-D sequential HSI classification.

Remote sensing datasets pose long-tailed data distribution, and such unbalanced datasets will reduce the performance of existing remote sensing object classification models. Existing methods mainly rely on resampling datasets, modifying loss functions, data augmentation, and transfer learning to cope with such challenges. Unlike these, our study takes a novel perspective and focuses on mitigating the long-tailed distribution problem by generating a large number of tail-class images with consistency and diversity. Specifically, this article introduces a novel text-guided tail-class generation network (TGN). TGN comprises two main components: knowledge mutual distillation network (KMDN) and class-consistent diverse tail-class generation network (CDTG). KMDN resolves the isolation issue of the head and tail knowledge by facilitating mutual learning of feature representations between the head and tail data, thereby improving the feature extraction capability of the tail model. CDTG focuses on generating class-consistency diverse tail-class images that uses tail-class features extracted by KMDN. Especially, the class consistency is guaranteed by contrastive language-image pre-trainings (CLIP’s) powerful text-image alignment capability. These generated images are then added back into the original dataset to alleviate the long-tailed distribution, thereby improving the tail-class accuracy. Extensive experiments on the widely used DIOR, FGSC-23 and DOTA datasets demonstrate that the proposed method outperforms state-of-the-art methods. Dataset and code are publicly available at https://github.com/XinR-Tang/TGN.