Search

The differences in sensor imaging mechanisms, observation angles, and scattering characteristics of terrestrial objects significantly limit the registration performance of synthetic aperture radar (SAR) and optical heterologous images. Traditional methods particularly struggle in weak feature regions, such as harbors and islands with substantial water coverage, as well as in desolate areas like deserts. This paper introduces a robust heterologous image registration technique based on region-adaptive keypoint selection that integrates image texture features, targeting two pivotal aspects: feature point extraction and matching point screening. Initially, a dual threshold criterion based on block region information entropy and variance products effectively identifies weak feature regions. Subsequently, it constructs feature descriptors to generate similarity maps, combining histogram parameter skewness with non-maximum suppression (NMS) to enhance matching point accuracy. Extensive experiments have been conducted on conventional SAR-optical datasets and typical SAR-optical images with different weak feature regions to assess the method's performance. The findings indicate that this method successfully removes outliers in weak feature regions and completes the registration task of SAR and optical images with weak feature regions. [ABSTRACT FROM AUTHOR]

Faced with the problem of incompatibility between traditional information acquisition mode and spaceborne earth observation tasks, starting from the general mathematical model of compressed sensing, a theoretical model of block compressed sensing was established, and a full-process adaptive coding and decoding compressed sensing framework for remote sensing images was proposed, which includes five parts: mode selection, feature factor extraction, adaptive shape segmentation, adaptive sampling rate allocation and image reconstruction. Unlike previous semi-adaptive or local adaptive methods, the advantages of the adaptive encoding and decoding method proposed in this paper are mainly reflected in four aspects: (1) Ability to select encoding modes based on image content, and maximizing the use of the richness of the image to select appropriate sampling methods; (2) Capable of utilizing image texture details for adaptive segmentation, effectively separating complex and smooth regions; (3) Being able to detect the sparsity of encoding blocks and adaptively allocate sampling rates to fully explore the compressibility of images; (4) The reconstruction matrix can be adaptively selected based on the size of the encoding block to alleviate block artifacts caused by non-stationary characteristics of the image. Experimental results show that the method proposed in this article has good stability for remote sensing images with complex edge textures, with the peak signal-to-noise ratio and structural similarity remaining above 35 dB and 0.8. Moreover, especially for ocean images with relatively simple image content, when the sampling rate is 0.26, the peak signal-to-noise ratio reaches 50.8 dB, and the structural similarity is 0.99. In addition, the recovered images have the smallest BRISQUE value, with better clarity and less distortion. In the subjective aspect, the reconstructed image has clear edge details and good reconstruction effect, while the block effect is effectively suppressed. The framework designed in this paper is superior to similar algorithms in both subjective visual and objective evaluation indexes, which is of great significance for alleviating the incompatibility between traditional information acquisition methods and satellite-borne earth observation missions. [ABSTRACT FROM AUTHOR]

Monitoring aircraft using synthetic aperture radar (SAR) images is a very important task. Given its coherent imaging characteristics, there is a large amount of speckle interference in the image. This phenomenon leads to the scattering information of aircraft targets being masked in SAR images, which is easily confused with background scattering points. Therefore, automatic detection of aircraft targets in SAR images remains a challenging task. For this task, this paper proposes a framework for speckle reduction preprocessing of SAR images, followed by the use of an improved deep learning method to detect aircraft in SAR images. Firstly, to improve the problem of introducing artifacts or excessive smoothing in speckle reduction using total variation (TV) methods, this paper proposes a new nonconvex total variation (NTV) method. This method aims to ensure the effectiveness of speckle reduction while preserving the original scattering information as much as possible. Next, we present a framework for aircraft detection based on You Only Look Once v8 (YOLOv8) for SAR images. Therefore, the complete framework is called SAR-NTV-YOLOv8. Meanwhile, a high-resolution small target feature head is proposed to mitigate the impact of scale changes and loss of depth feature details on detection accuracy. Then, an efficient multi-scale attention module was proposed, aimed at effectively establishing short-term and long-term dependencies between feature grouping and multi-scale structures. In addition, the progressive feature pyramid network was chosen to avoid information loss or degradation in multi-level transmission during the bottom-up feature extraction process in Backbone. Sufficient comparative experiments, speckle reduction experiments, and ablation experiments are conducted on the SAR-Aircraft-1.0 and SADD datasets. The results have demonstrated the effectiveness of SAR-NTV-YOLOv8, which has the most advanced performance compared to other mainstream algorithms. [ABSTRACT FROM AUTHOR]

Visible and infrared image fusion is a strategy that effectively extracts and fuses information from different sources. However, most existing methods largely neglect the issue of lighting imbalance, which makes the same fusion models inapplicable to different scenes. Several methods obtain low-level features from visible and infrared images at an early stage of input or shallow feature extraction. However, these methods do not explore how low-level features provide a foundation for recognizing and utilizing the complementarity and common information between the two types of images. As a result, the complementarity and common information between the images is not fully analyzed and discussed. To address these issues, we propose a Self-Attention Progressive Network for the fusion of infrared and visible images in this paper. Firstly, we construct a Lighting-Aware Sub-Network to analyze lighting distribution, and introduce intensity loss to measure the probability of scene illumination. This approach enhances the model's adaptability to lighting conditions. Secondly, we introduce self-attention learning to design a multi-state joint feature extraction module (MSJFEM) that fully utilizes the contextual information among input keys. It guides the learning of a dynamic attention matrix to strengthen the capacity for visual representation. Finally, we design a Difference-Aware Propagation Module (DAPM) to extract and integrate edge details from the source images while supplementing differential information. The experiments across three benchmark datasets reveal that the proposed approach exhibits satisfactory performance compared to existing methods. [ABSTRACT FROM AUTHOR]

Entomological radar is one of the most effective tools for monitoring insect migration, capable of detecting migratory insects concentrated in layers and facilitating the analysis of insect migration behavior. However, traditional entomological radar, with its low resolution, can only provide a rough observation of layer concentrations. The advent of High-Resolution Phased Array Radar (HPAR) has transformed this situation. With its high range resolution and high data update rate, HPAR can generate detailed concentration spatiotemporal distribution heatmaps. This technology facilitates the detection of changes in insect concentrations across different time periods and altitudes, thereby enabling the observation of large-scale take-off, landing, and layering phenomena. However, the lack of effective techniques for extracting insect concentration data of different phenomena from these heatmaps significantly limits detailed analyses of insect migration patterns. This paper is the first to apply instance segmentation technology to the extraction of insect data, proposing a method for segmenting and extracting insect concentration data from spatiotemporal distribution heatmaps at different phenomena. To address the characteristics of concentrations in spatiotemporal distributions, we developed the Heatmap Feature Fusion Network (HFF-Net). In HFF-Net, we incorporate the Global Context (GC) module to enhance feature extraction of concentration distributions, utilize the Atrous Spatial Pyramid Pooling with Depthwise Separable Convolution (SASPP) module to extend the receptive field for understanding various spatiotemporal distributions of concentrations, and refine segmentation masks with the Deformable Convolution Mask Fusion (DCMF) module to enhance segmentation detail. Experimental results show that our proposed network can effectively segment concentrations of different phenomena from heatmaps, providing technical support for detailed and systematic studies of insect migration behavior. [ABSTRACT FROM AUTHOR]

The accurate extraction of agricultural parcels from remote sensing images is crucial for advanced agricultural management and monitoring systems. Existing methods primarily emphasize regional accuracy over boundary quality, often resulting in fragmented outputs due to uniform crop types, diverse agricultural practices, and environmental variations. To address these issues, this paper proposes DSTBA-Net, an end-to-end encoder–decoder architecture. Initially, we introduce a Dual-Stream Feature Extraction (DSFE) mechanism within the encoder, which consists of Residual Blocks and Boundary Feature Guidance (BFG) to separately process image and boundary data. The extracted features are then fused in the Global Feature Fusion Module (GFFM), utilizing Transformer technology to further integrate global and detailed information. In the decoder, we employ Feature Compensation Recovery (FCR) to restore critical information lost during the encoding process. Additionally, the network is optimized using a boundary-aware weighted loss strategy. DSTBA-Net aims to achieve high precision in agricultural parcel segmentation and accurate boundary extraction. To evaluate the model's effectiveness, we conducted experiments on agricultural parcel extraction in Denmark (Europe) and Shandong (Asia). Both quantitative and qualitative analyses show that DSTBA-Net outperforms comparative methods, offering significant advantages in agricultural parcel extraction. [ABSTRACT FROM AUTHOR]

Image super-resolution (SR) technology can improve the resolution of images and provide clearer and more reliable remote sensing images of high quality to better serve the subsequent applications. However, when reconstructing high-frequency feature areas of remote sensing images, existing SR reconstruction methods are prone to artifacts that affect visual effects and make it difficult to generate real texture details. In order to address this issue, a texture-detail-enhanced dense generative adversarial network (TDEGAN) for remote sensing image SR is presented. The generator uses multi-level dense connections, residual connections, and Shuffle attention (SA) to improve the feature extraction ability. A PatchGAN-style discrimination network is designed to effectively perform local discrimination and helps the network generate rich, detailed features. To reduce the impact of artifacts, we introduce an artifact loss function, which is combined with the exponential moving average (EMA) technique to distinguish the artifacts generated from the actual texture details through local statistics, which can help the network reduce artifacts and generate more realistic texture details. Experiments show that TDEGAN can better restore the texture details of remote sensing images and achieves certain advantages in terms of evaluation indicators and visualization. [ABSTRACT FROM AUTHOR]

Remote sensing images usually contain abundant targets and complex information distributions. Consequently, networks are required to model both global and local information in the super-resolution (SR) reconstruction of remote sensing images. The existing SR reconstruction algorithms generally focus on only local or global features, neglecting effective feedback for reconstruction errors. Therefore, a Global Residual Multi-attention Fusion Back-projection Network (SRBPSwin) is introduced by combining the back-projection mechanism with the Swin Transformer. We incorporate a concatenated Channel and Spatial Attention Block (CSAB) into the Swin Transformer Block (STB) to design a Multi-attention Hybrid Swin Transformer Block (MAHSTB). SRBPSwin develops dense back-projection units to provide bidirectional feedback for reconstruction errors, enhancing the network's feature extraction capabilities and improving reconstruction performance. SRBPSwin consists of the following four main stages: shallow feature extraction, shallow feature refinement, dense back projection, and image reconstruction. Firstly, for the input low-resolution (LR) image, shallow features are extracted and refined through the shallow feature extraction and shallow feature refinement stages. Secondly, multiple up-projection and down-projection units are designed to alternately process features between high-resolution (HR) and LR spaces, obtaining more accurate and detailed feature representations. Finally, global residual connections are utilized to transfer shallow features during the image reconstruction stage. We propose a perceptual loss function based on the Swin Transformer to enhance the detail of the reconstructed image. Extensive experiments demonstrate the significant reconstruction advantages of SRBPSwin in quantitative evaluation and visual quality. [ABSTRACT FROM AUTHOR]

Medium- to high-resolution imagery is indispensable for various applications. Combining images from Landsat 8 and Sentinel-2 can improve the accuracy of observing dynamic changes on the Earth's surface. Many researchers use Sentinel-2 10 m resolution data in conjunction with Landsat 8 30 m resolution data to generate 10 m resolution data series. However, current fusion techniques have some algorithmic weaknesses, such as simple processing of coarse or fine images, which fail to extract image features to the fullest extent, especially in rapidly changing land cover areas. Facing the aforementioned limitations, we proposed a multiscale and attention mechanism-based residual spatiotemporal fusion network (MARSTFN) that utilizes Sentinel-2 10 m resolution data and Landsat 8 15 m resolution data as auxiliary data to upgrade Landsat 8 30 m resolution data to 10 m resolution. In this network, we utilized multiscale and attention mechanisms to extract features from coarse and fine images separately. Subsequently, the features outputted from all input branches are combined and further feature information is extracted through residual networks and skip connections. Finally, the features obtained from the residual network are merged with the feature information of the coarsely processed images from the multiscale mechanism to generate accurate prediction images. To assess the efficacy of our model, we compared it with existing models on two datasets. Results demonstrated that our fusion model outperformed baseline methods across various evaluation indicators, highlighting its ability to integrate Sentinel-2 and Landsat 8 data to produce 10 m resolution data. [ABSTRACT FROM AUTHOR]

The unique imaging modality of synthetic aperture radar (SAR) has posed significant challenges for object detection, making it more complex to acquire and interpret than optical images. Recently, numerous studies have proposed cross-domain adaptive methods based on convolutional neural networks (CNNs) to promote SAR object detection using optical data. However, existing cross-domain methods focus on image features, lack improvement on input data, and ignore the valuable supervision provided by few labeled SAR images. Therefore, we propose a semi-supervised cross-domain object detection framework that uses optical data and few SAR data to achieve knowledge transfer for SAR object detection. Our method focuses on the data processing aspects to gradually reduce the domain shift at the image, instance, and feature levels. First, we propose a data augmentation method of image mixing and instance swapping to generate a mixed domain that is more similar to the SAR domain. This method fully utilizes few SAR annotation information to reduce domain shift at image and instance levels. Second, at the feature level, we propose an adaptive optimization strategy to filter out mixed domain samples that significantly deviate from the SAR feature distribution to train feature extractor. In addition, we employ Vision Transformer (ViT) as feature extractor to handle the global feature extraction of mixed images. We propose a detection head based on normalized Wasserstein distance (NWD) to enhance objects with smaller effective regions in SAR images. The effectiveness of our proposed method is evaluated on public SAR ship and oil tank datasets. [ABSTRACT FROM AUTHOR]

Deep learning is an important research topic in the field of image super-resolution. Problematically, the performance of existing hyperspectral image super-resolution networks is limited by feature learning for hyperspectral images. Nevertheless, the current algorithms exhibit some limitations in extracting diverse features. In this paper, we address limitations to existing hyperspectral image super-resolution networks, focusing on feature learning challenges. We introduce the Channel-Attention-Based Spatial–Spectral Feature Extraction network (CSSFENet) to enhance hyperspectral image feature diversity and optimize network loss functions. Our contributions include: (a) a convolutional neural network super-resolution algorithm incorporating diverse feature extraction to enhance the network's diversity feature learning by elevating the matrix rank, (b) a three-dimensional (3D) feature extraction convolution module, the Channel-Attention-Based Spatial–Spectral Feature Extraction Module (CSSFEM), to boost the network's performance in both the spatial and spectral domains, (c) a feature diversity loss function designed based on the image matrix's singular value to maximize element independence, and (d) a spatial–spectral gradient loss function introduced based on space and spectrum gradient values to enhance the reconstructed image's spatial–spectral smoothness. In contrast to existing hyperspectral super-resolution algorithms, we used four evaluation indexes, PSNR, mPSNR, SSIM, and SAM, and our method showed superiority during testing with three common hyperspectral datasets. [ABSTRACT FROM AUTHOR]

Multi-source image registration has often suffered from great radiation and geometric differences. Specifically, grayscale and texture from similar landforms in different source images often show significantly different visual features, and these differences disturb the corresponding point extraction in the following image registration process. Considering that edges between heterogeneous images can provide homogeneous information and more consistent features can be extracted based on image edges, an edge consistency radiation-change insensitive feature transform (EC-RIFT) method is proposed in this paper. Firstly, the noise and texture interference are reduced by preprocessing according to the image characteristics. Secondly, image edges are extracted based on phase congruency, and an orthogonal Log-Gabor filter is performed to replace the global algorithm. Finally, the descriptors are built with logarithmic partition of the feature point neighborhood, which improves the robustness of the descriptors. Comparative experiments on datasets containing multi-source remote sensing image pairs show that the proposed EC-RIFT method outperforms other registration methods in terms of precision and effectiveness. [ABSTRACT FROM AUTHOR]

The outstanding performance of deep neural networks (DNNs) in multiple computer vision in recent years has promoted its widespread use in aerial image semantic segmentation. Nonetheless, prior research has demonstrated the high susceptibility of DNNs to adversarial attacks. This poses significant security risks when applying DNNs to safety-critical earth observation missions. As an essential means of attacking DNNs, data poisoning attacks destroy model performance by contaminating model training data, allowing attackers to control prediction results by carefully crafting poisoning samples. Toward building a more robust DNNs-based aerial image semantic segmentation model, in this study, we proposed a robust invariant feature enhancement network (RIFENet) that can resist data poisoning attacks and has superior semantic segmentation performance. The constructed RIFENet improves the resistance to poisoning attacks by extracting and enhancing robust invariant features. Specifically, RIFENet uses a texture feature enhancement module (T-FEM), structural feature enhancement module (S-FEM), global feature enhancement module (G-FEM), and multi-resolution feature fusion module (MR-FFM) to enhance the representation of different robust features in the feature extraction process to suppress the interference of poisoning samples. Experiments on several benchmark aerial image datasets demonstrate that the proposed method is more robust and exhibits better generalization than other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Mapping arable field areas is crucial for assessing agricultural productivity but poses challenges in sub-Saharan agroecosystems because of diverse crop calendars, small and irregularly shaped fields, persistent cloud cover, and lack of high-quality model training data. This study proposes several methodological improvements to overcome these challenges. Specifically, it utilizes long-term MODIS data to stratify finer Sentinel-2 reflectance and Sentinel-1 backscatter image features on a per-pixel basis. It also incorporates texture features and employs a machine learning approach with over 300,000 samples. The eastern region of Ghana was stratified into seven seasonal strata exhibiting distinct vegetation seasonality, capturing diversity in crop calendars, using long-term MODIS (2001–2009) normalized difference vegetation index phenology. Three years (2017–2019) of Sentinel-1 and Sentinel-2 original bands at 20 m were composited into dry and wet seasonal features according to the strata, from which spectral, polarimetric, and texture features were extracted. The field boundaries were digitized using PlanetScope images (2018–2019). Random Forest classifier with 10-fold cross-validation and recursive feature elimination was used for feature selection and model building. Including topographic variables, out of 137 image features, only 11 features were found important. Sentinel-2 SWIR-based spectral features were most important, followed by Sentinel-1 polarimetric (VV) and elevation features. Half of the 11 features were variance texture features, followed by spectral features. The Random Forest classifier produced a 0.78 AUC score with overall precision, recall, and F1-score of 0.96, 0.78, and 0.85, respectively. While the precision for both classes was >0.90, the recall rate for arable areas was half that of non-arable areas. Future studies could improve the technical workflow with reliable balanced sampling, narrowband hyperspectral images, and fully polarized SAR images. [ABSTRACT FROM AUTHOR]

In order to improve the accuracy of the segmentation of buildings with small sample sizes, this paper proposes a building-segmentation network, ResFAUnet, with transfer learning and multi-scale feature fusion. The network is based on AttentionUnet. The backbone of the encoder is replaced by the ResNeXt101 network for feature extraction, and the attention mechanism of the skip connection is preserved to fuse the shallow features of the encoding part and the deep features of the decoding part. In the decoder, the feature-pyramid structure is used to fuse the feature maps of different scales. More features can be extracted from limited image samples. The proposed network is compared with current classical semantic segmentation networks, Unet, SuUnet, FCN, and SegNet. The experimental results show that in the dataset selected in this paper, the precision indicators of ResFAUnet are improved by 4.77%, 2.3%, 2.11%, and 1.57%, respectively, compared with the four comparison networks. [ABSTRACT FROM AUTHOR]

The annual flood cycle of the Mekong Basin in Vietnam plays an important role in the hydrological balance of its delta. In this study, we explore the potential of the C-band of Sentinel-1 SAR time series dual-polarization (VV/VH) data for mapping, detecting and monitoring the flooded and flood-prone areas in the An Giang province in the Mekong Delta, especially its rice fields. Time series floodable area maps were generated from five images per month taken during the wet season (6–7 months) over two years (2019 and 2020). The methodology was based on automatic image classification through the application of Machine Learning (ML) algorithms, including convolutional neural networks (CNNs), multi-layer perceptrons (MLPs) and random forests (RFs). Based on the segmentation technique, a three-level classification algorithm was developed to generate maps of the development of floods and floodable areas during the wet season. A modification of the backscatter intensity was noted for both polarizations, in accordance with the evolution of the phenology of the rice fields. The results show that the CNN-based methods can produce more reliable maps (99%) compared to the MLP and RF (97%). Indeed, in the classification process, feature extraction based on segmentation and CNNs has demonstrated an effective improvement in prediction performance of land use land cover (LULC) classes, deriving complex decision boundaries between flooded and non-flooded areas. The results show that between 53% and 58% of rice paddies areas and 9% and 14% of built-up areas are affected by the flooding in 2019 and 2020 respectively. Our methodology and results could support the development of the flood monitoring database and hazard management in the Mekong Delta. [ABSTRACT FROM AUTHOR]

High-quality remote sensing images play important roles in the development of ecological indicators' mapping, urban-rural management, urban planning, and other fields. Compared with natural images, remote sensing images have more abundant land cover along with lower spatial resolutions. Given the embedded longitude and latitude information of remote sensing images, reference (Ref) images with similar scenes could be more accessible. However, existing traditional super-resolution (SR) approaches always depend on increases in network depth to improve performance, which limits the acquisition and application of high-quality remote sensing images. In this paper, we proposed a novel, reference-image-based, super-resolution method with feature compression module (FCSR) for remote sensing images to alleviate the above issue while effectively utilizing high-resolution (HR) information from Ref images. Specifically, we exploited a feature compression branch (FCB) to extract relevant features in feature detail matching with large measurements. This branch employed a feature compression module (FCM) to extract features from low-resolution (LR) and Ref images, which enabled texture transfer from different perspectives. To decrease the impact of environmental factors such as resolution, brightness and ambiguity disparities between the LR and Ref images, we designed a feature extraction encoder (FEE) to ensure accuracy in feature extraction in the feature acquisition branch. The experimental results demonstrate that the proposed FCSR achieves significant performance and visual quality compared to state-of-the-art SR methods. Explicitly, when compared with the best method, the average peak signal-to-noise ratio (PSNR) index on the three test sets is improved by 1.0877%, 0.8161%, 1.0296%, respectively, and the structural similarity (SSIM) index on four test sets is improved by 1.4764%, 1.4467%, 0.0882%, and 1.8371%, respectively. Simultaneously, FCSR obtains satisfactory visual details following qualitative evaluation. [ABSTRACT FROM AUTHOR]

Smart vehicles with embedded Autonomous Vehicle (AV) technologies are currently equipped with different types of mounted sensors, aiming to ensure safe movement for both passengers and other road users. The sensors' ability to capture and gather data to be synchronically interpreted by neural networks for a clear understanding of the surroundings is influenced by lighting conditions, such as natural lighting levels, artificial lighting effects, time of day, and various weather conditions, such as rain, fog, haze, and extreme temperatures. Such changing environmental conditions are also known as complex environments. In addition, the appearance of other road users is varied and relative to the vehicle's perspective; thus, the identification of features in a complex background is still a challenge. This paper presents a pre-processing method using multi-sensorial RGB and thermal camera data. The aim is to handle issues arising from the combined inputs of multiple sensors, such as data registration and value unification. Foreground refinement, followed by a novel statistical anomaly-based feature extraction prior to image fusion, is presented. The results met the AV challenges in CNN's classification. The reduction of the collected data and its variation level was achieved. The unified physical value contributed to the robustness of input data, providing a better perception of the surroundings under varied environmental conditions in mixed datasets for day and night images. The method presented uses fused images, robustly enriched with texture and feature depth and reduced dependency on lighting or environmental conditions, as an input for a CNN. The CNN was capable of extracting and classifying dynamic objects as vehicles and pedestrians from the complex background in both daylight and nightlight images. [ABSTRACT FROM AUTHOR]

Registration between remote sensing images has been a research focus in the field of remote sensing image processing. Most of the existing image registration algorithms applied to feature point matching are derived from image feature extraction methods, such as scale-invariant feature transform (SIFT), speed-up robust features (SURF) and Siamese neural network. Such methods encounter difficulties in achieving accurate image registration where there is a large bias in the image features or no significant feature points. Aiming to solve this problem, this paper proposes an algorithm for multi-source image registration based on geographical location information (GLI). By calculating the geographic location information that corresponds to the pixel in the image, the ideal projected pixel position of the corresponding image is obtained using spatial coordinate transformation. Additionally, the corresponding relationship between the two images is calculated by combining multiple sets of registration points. The simulation experiment illustrates that, under selected common simulation parameters, the average value of the relative registration-point error between the two images is 12.64 pixels, and the registration accuracy of the corresponding ground registration point is higher than 6.5 m. In the registration experiment involving remote sensing images from different sources, the average registration pixel error of this algorithm is 20.92 pixels, and the registration error of the image center is 21.24 pixels. In comparison, the image center registration error given by the convolutional neural network (CNN) is 142.35 pixels after the registration error is manually eliminated. For the registration of homologous and featureless remote sensing images, the SIFT algorithm can only offer one set of registration points for the correct region, and the neural network cannot achieve accurate registration results. The registration accuracy of the presented algorithm is 7.2 pixels, corresponding to a ground registration accuracy of 4.32 m and achieving more accurate registration between featureless images. [ABSTRACT FROM AUTHOR]

Spectral-spatial classification of hyperspectral images (HSIs) has recently attracted great attention in the research domain of remote sensing. It is well-known that, in remote sensing applications, spectral features are the fundamental information and spatial patterns provide the complementary information. With both spectral features and spatial patterns, hyperspectral image (HSI) applications can be fully explored and the classification performance can be greatly improved. In reality, spatial patterns can be extracted to represent a line, a clustering of points or image texture, which denote the local or global spatial characteristic of HSIs. In this paper, we propose a spectral-spatial HSI classification model based on superpixel pattern (SP) and kernel based extreme learning machine (KELM), called SP-KELM, to identify the land covers of pixels in HSIs. In the proposed SP-KELM model, superpixel pattern features are extracted by an advanced principal component analysis (PCA), which is based on superpixel segmentation in HSIs and used to denote spatial information. The KELM method is then employed to be a classifier in the proposed spectral-spatial model with both the original spectral features and the extracted spatial pattern features. Experimental results on three publicly available HSI datasets verify the effectiveness of the proposed SP-KELM model, with the performance improvement of 10% over the spectral approaches. [ABSTRACT FROM AUTHOR]

The existing nonuniformity correction methods generally have the defects of image blur, artifacts, image over-smoothing, and nonuniform residuals. It is difficult for these methods to meet the requirements of image enhancement in various complex application scenarios. In particular, when these methods are applied to dim small target images, they may remove dim small targets as noise points due to the image over-smoothing. This paper draws on the idea of a residual network and proposes a two-stage learning network based on the imaging mechanism of an infrared line-scan system. We adopt a multi-scale feature extraction unit and design a gain correction sub-network and an offset correction sub-network, respectively. Then, we pre-train the two sub-networks independently. Finally, we cascade the two sub-networks into a two-stage network and train it. The experimental results show that the PSNR gain of our method can reach more than 15 dB, and it can achieve excellent performance in different backgrounds and different intensities of nonuniform noise. Moreover, our method can avoid losing texture details or dim small targets after effectively removing nonuniform noise. [ABSTRACT FROM AUTHOR]

Urban forests maintain the ecological balance of cities and are significant in promoting the sustainable development of cities. Therefore, using advanced remote sensing technology to accurately extract forest green space in the city and monitor its change in real-time is very important. Taking Nanjing as the study area, this research extracted 55 vegetation phenological features from Sentinel-2A time series images and formed a feature set containing 81 parameters together with 26 features, including polarimetric- and texture-related information extracted from dual-polarization Sentinel-1A data. On the basis of the improved ABC (ABC-LIBSVM) feature selection method, the optimal feature subset was selected, and the forest coverage areas in the study area were accurately described. To verify the feasibility of the improved feature selection method and explore the potential for the development of multi-source time series remote sensing for urban forest feature extraction, this paper also used the random forest classification model to classify four different feature sets. The results revealed that the classification accuracy based on the feature set obtained by the ABC-LIBSVM algorithm was the highest, with an overall accuracy of 86.80% and a kappa coefficient of 0.8145. The producer accuracy and user accuracy of the urban forest were 93.21% and 82.45%, respectively. Furthermore, by combining the multi-source time series Sentinel-2A optical images with Sentinel-1A dual-polarization SAR images, urban forests can be distinguished from the perspective of phenology, and polarimetric- and texture-related features can contribute to the accurate identification of forests. [ABSTRACT FROM AUTHOR]

Popular deep-learning-based spatiotemporal fusion methods for creating high-temporal–high-spatial-resolution images have certain limitations. The reconstructed images suffer from insufficient retention of high-frequency information and the model suffers from poor robustness, owing to the lack of training datasets. We propose a dual-branch remote sensing spatiotemporal fusion network based on a selection kernel mechanism. The network model comprises a super-resolution network module, a high-frequency feature extraction module, and a difference reconstruction module. Convolution kernel adaptive mechanisms are added to the high-frequency feature extraction module and difference reconstruction module to improve robustness. The super-resolution module upgrades the coarse image to a transition image matching the fine image; the high-frequency feature extraction module extracts the high-frequency features of the fine image to supplement the high-frequency features for the difference reconstruction module; the difference reconstruction module uses the structural similarity for fine-difference image reconstruction. The fusion result is obtained by combining the reconstructed fine-difference image with the known fine image. The compound loss function is used to help network training. Experiments are carried out on three datasets and five representative spatiotemporal fusion algorithms are used for comparison. Subjective and objective evaluations validate the superiority of our proposed method. [ABSTRACT FROM AUTHOR]

This paper presents two categories of features extraction and mapping suite, a very high-resolution suite and an ultra-resolution suite at 2 m and 0.5 m resolutions, respectively, for the differentiation and mapping of land cover and community-level vegetation types. The features extraction flow of the ultra-resolution suite involves pan-sharpening of the multispectral image, color-transformation of the pan-sharpened image, and the generation of panchromatic textural features. The performance of the ultra-resolution features extraction suite was compared with the very high-resolution features extraction suite that involves the calculation of radiometric indices and color-transformation of the multi-spectral image. This research was implemented in three mountainous ecosystems located in a cool temperate region. Three machine learning classifiers, Random Forests, XGBoost, and SoftVoting, were employed with a 10-fold cross-validation method for quantitatively evaluating the performance of the two suites. The ultra-resolution suite provided 5.3% more accuracy than the very high-resolution suite using single-date autumn images. Addition of summer images gained 12.8% accuracy for the ultra-resolution suite and 13.2% accuracy for the very high-resolution suite across all sites, while the ultra-resolution suite showed 4.9% more accuracy than the very high-resolution suite. The features extraction and mapping suites presented in this research are expected to meet the growing need for differentiating land cover and community-level vegetation types at a large scale. [ABSTRACT FROM AUTHOR]

For skywave over-the-horizon radar, beamforming techniques are often used to suppress airspace radio frequency interference because the high-frequency band is shared by many devices. To address the problems that the traditional beamforming method is not capable of recognizing the electromagnetic environment and that its performance is greatly affected by the accuracy of signal feature estimation, a cognitive beamforming method using range-Doppler (RD) map features for skywave radar is proposed. First, the RD map is weighted by a local attention model, and then, texture features are extracted as the inputs to a support vector machine. Finally, the support vector machine is used to predict the optimal diagonal loading factor. Simulation results show that the output signal-to-interference-plus-noise ratio is improved compared with previous methods. The proposed method is suitable for many kinds of common unsatisfactory scenarios, making it beneficial for engineering implementation. [ABSTRACT FROM AUTHOR]

Efficient detection of earthquake−triggered landslides is crucial for emergency response and risk assessment. With the development of multi−source remote sensing images, artificial intelligence has gradually become a powerful landslide detection method for similar tasks, aiming to mitigate time−consuming problems and meet emergency requirements. In this study, a relatively new deep learning (DL) network, called U−Net++, was designed to detect landslides for regions affected by the Iburi, Japan Mw = 6.6 earthquake, with only small training samples. For feature extraction, ResNet50 was selected as the feature extraction layer, and transfer learning was adopted to introduce the pre−trained weights for accelerating the model convergence. To prove the feasibility and validity of the proposed model, the random forest algorithm (RF) was selected as the benchmark, and the F1−score, Kappa coefficient, and IoU (Intersection of Union) were chosen to quantitatively evaluate the model's performance. In addition, the proposed model was trained with different sample sizes (256,512) and network depths (3,4,5), respectively, to analyze their impacts on performance. The results showed that both models detected the majority of landslides, while the proposed model obtained the highest metric value (F1−score = 0.7580, Kappa = 0.7441, and IoU = 0.6104) and was capable of resisting the noise. In addition, the proposed model trained with sample size 256 possessed optimal performance, proving that the size is a non−negligible parameter in U−Net++, and it was found that the U−Net++ trained with shallower layer 3 yielded better results than that with the standard layer 5. Finally, the outstanding performance of the proposed model on a public landslide dataset demonstrated the generalization of U−Net++. [ABSTRACT FROM AUTHOR]

Mapping smallholder fruit plantations using optical data is challenging due to morphological landscape heterogeneity and crop types having overlapping spectral signatures. Furthermore, cloud covers limit the use of optical sensing, especially in subtropical climates where they are persistent. This research assessed the effectiveness of Sentinel-1 (S1) and Sentinel-2 (S2) data for mapping fruit trees and co-existing land-use types by using support vector machine (SVM) and random forest (RF) classifiers independently. These classifiers were also applied to fused data from the two sensors. Feature ranks were extracted using the RF mean decrease accuracy (MDA) and forward variable selection (FVS) to identify optimal spectral windows to classify fruit trees. Based on RF MDA and FVS, the SVM classifier resulted in relatively high classification accuracy with overall accuracy (OA) = 0.91.6% and kappa coefficient = 0.91% when applied to the fused satellite data. Application of SVM to S1, S2, S2 selected variables and S1S2 fusion independently produced OA = 27.64, Kappa coefficient = 0.13%; OA= 87%, Kappa coefficient = 86.89%; OA = 69.33, Kappa coefficient = 69. %; OA = 87.01%, Kappa coefficient = 87%, respectively. Results also indicated that the optimal spectral bands for fruit trees mapping are green (B3) and SWIR_2 (B10) for S2, whereas for S1, the vertical-horizontal (VH) polarization band. Including the textural metrics from the VV channel improved crop discrimination and co-existing land use cover types. The fusion approach proved robust and well suited for accurate smallholder fruit plantation mapping. [ABSTRACT FROM AUTHOR]

Forest above-ground biomass (AGB) is an important index to evaluate forest carbon sequestration capacity, which is very important to maintain the stability of forest ecosystems. At present, the wide use of remote sensing technology makes it possible to estimate the large-scale forest AGB accurately and efficiently. Airborne hyperspectral remote sensing data can obtain rich spectral information and spatial structure information on the forest canopy with the characteristics of high spatial and hyperspectral resolution. Airborne LiDAR data can describe the three-dimensional structure characteristics of a forest and provide vertical structure information related to biomass. Based on the characteristics of the two data sources, this study takes Gaofeng forest farm in Nanning, Guangxi, as the study area, Chinese fir, pine tree, eucalyptus and other broadleaved trees as the research object, and constructs the AGB estimation models of different tree species by fusing airborne LiDAR and hyperspectral features. Firstly, spectral features, texture features, vegetation index, wavelet transform features and edge features are extracted from hyperspectral data. Canopy structure features, point cloud structure features, point cloud density features and terrain features are extracted from airborne LiDAR data. Secondly, the random forest (RF) method is used to screen the features of the two sets of data, and the features with the highest importance are selected. Finally, based on the optimal features of the two data sources, the forest AGB model is constructed using the multiple stepwise regression method. The results show that the texture features extracted by wavelet transform can be used for AGB modeling. The AGB of eucalyptus has high correlation with height features derived from airborne LiDAR, the AGB of other broadleaved trees mostly depends on the wavelet transform texture features from airborne hyperspectral data, while the AGB of Chinese fir and pine tree has high correlation with both height features and spectral features. Feature-fusion-based LiDAR and hyperspectral data can greatly improve the accuracy of the AGB models. The accuracy of the optimal AGB models of Chinese fir, pine tree, eucalyptus and other broadleaved trees is 0.78, 0.95, 0.72 and 0.89, respectively. In conclusion, more accurate estimation results can be obtained by combining active and passive remote sensing data for forest AGB estimation, which provides a solution for carbon storage assessment and forest ecosystem assessment. [ABSTRACT FROM AUTHOR]

Heterogeneous synthetic aperture radar (SAR) images contain more complementary information compared with homologous SAR images; thus, the comprehensive utilization of heterogeneous SAR images could potentially improve performance for the monitoring of sea surface objects, such as sea ice and enteromorpha. Image registration is key to the application of monitoring sea surface objects. Heterogeneous SAR images have intensity differences and resolution differences, and after the uniform resolution, intensity differences are one of the most important factors affecting the image registration accuracy. In addition, sea surface objects have numerous repetitive and confusing features for feature extraction, which also limits the image registration accuracy. In this paper, we propose an improved L2Net network for image registration with intensity differences and repetitive texture features, using sea ice as the research object. The deep learning network can capture feature correlations between image patch pairs, and can obtain the correct matching from a large number of features with repetitive texture. In the SAR image pair, four patches of different sizes centered on the corner points are proposed as inputs. Thus, local features and more global features are fused to obtain excellent structural features, to distinguish between different repetitive textural features, add contextual information, further improve the feature correlation, and improve the accuracy of image registration. An outlier removal strategy is proposed to remove false matches due to repetitive textures. Finally, the effectiveness of our method was verified by comparative experiments. [ABSTRACT FROM AUTHOR]

Resolution is a comprehensive reflection and evaluation index for the visual quality of remote sensing images. Super-resolution processing has been widely applied for extracting information from remote sensing images. Recently, deep learning methods have found increasing application in the super-resolution processing of remote sensing images. However, issues such as blurry object edges and existing artifacts persist. To overcome these issues, this study proposes an improved generative adversarial network with self-attention and texture enhancement (TE-SAGAN) for remote sensing super-resolution images. We first designed an improved generator based on the residual dense block with a self-attention mechanism and weight normalization. The generator gains the feature extraction capability and enhances the training model stability to improve edge contour and texture. Subsequently, a joint loss, which is a combination of L1-norm, perceptual, and texture losses, is designed to optimize the training process and remove artifacts. The L1-norm loss is designed to ensure the consistency of low-frequency pixels; perceptual loss is used to entrench medium- and high-frequency details; and texture loss provides the local features for the super-resolution process. The results of experiments using a publicly available dataset (UC Merced Land Use dataset) and our dataset show that the proposed TE-SAGAN yields clear edges and textures in the super-resolution reconstruction of remote sensing images. [ABSTRACT FROM AUTHOR]

The topographical morphology of the loess landform on the Loess Plateau exhibits remarkable textural features at different spatial scales. However, existing topographic texture analysis studies on the Loess Plateau are usually dominated by statistical characteristics and are missing structural characteristics. At the same time, there is a lack of regularity calculation methods for DEM digital terrain analysis. Taking the Loess Plateau as the study area, a regularity calculation method based on the improved normalized distance matching function (INDMF) is proposed and applied to the classification of a loess landform. The regularity calculation method used in this study (INDMF regularity) mainly includes two key steps. Step 1 calculates the INDMF sequence value and the peak and valley values for the terrain data. Step 2 calculates the significant peak and valley, constructs the significant peak and valley sequences, and then obtains the regularity using the normalised ratio value. The experimental results show that the proposed method has good anti-interference ability and can effectively extract the regularity of the main landform unit. Compared with previous methods, adding structural features (i.e., INDMF regularity) can effectively distinguish loess hill and loess ridge in the hilly and gully region. For the loess hill and loess ridge, the recognition rates of the proposed method are 84.62% and 92.86%, respectively. Combined with the existing topographic characteristics, the proposed INDMF regularity is a topographic structure feature extraction method that can effectively discriminate between loess hill and loess ridge areas on the Loess Plateau. [ABSTRACT FROM AUTHOR]

The wide adoption of dicamba-tolerant (DT) soybean has led to numerous cases of off-target dicamba damage to non-DT soybean and dicot crops. This study aimed to develop a method to differentiate soybean response to dicamba using unmanned-aerial-vehicle-based imagery and machine learning models. Soybean lines were visually classified into three classes of injury, i.e., tolerant, moderate, and susceptible to off-target dicamba. A quadcopter with a built-in RGB camera was used to collect images of field plots at a height of 20 m above ground level. Seven image features were extracted for each plot, including canopy coverage, contrast, entropy, green leaf index, hue, saturation, and triangular greenness index. Classification models based on artificial neural network (ANN) and random forest (RF) algorithms were developed to differentiate the three classes of response to dicamba. Significant differences for each feature were observed among classes and no significant differences across fields were observed. The ANN and RF models were able to precisely distinguish tolerant and susceptible lines with an overall accuracy of 0.74 and 0.75, respectively. The imagery-based classification model can be implemented in a breeding program to effectively differentiate phenotypic dicamba response and identify soybean lines with tolerance to off-target dicamba damage. [ABSTRACT FROM AUTHOR]

In this article, we develop a novel method for the detection of vineyard parcels in agricultural landscapes based on very high resolution (VHR) optical remote sensing images. Our objective is to perform texture-based image retrieval and supervised classification algorithms. To do that, the local textural and structural features inside each image are taken into account to measure its similarity to other images. In fact, VHR images usually involve a variety of local textures and structures that may verify a weak stationarity hypothesis. Hence, an approach only based on characteristic points, not on all pixels of the image, is supposed to be relevant. This work proposes to construct the local extrema-based descriptor (LED) by using the local maximum and local minimum pixels extracted from the image. The LED descriptor is formed based on the radiometric, geometric and gradient features from these local extrema. We first exploit the proposed LED descriptor for the retrieval task to evaluate its performance on texture discrimination. Then, it is embedded into a supervised classification framework to detect vine parcels using VHR satellite images. Experiments performed on VHR panchromatic PLEIADES image data prove the effectiveness of the proposed strategy. Compared to state-of-the-art methods, an enhancement of about 7% in retrieval rate is achieved. For the detection task, about 90% of vineyards are correctly detected. [ABSTRACT FROM AUTHOR]

Optical remote sensing images are widely used in the fields of feature recognition, scene semantic segmentation, and others. However, the quality of remote sensing images is degraded due to the influence of various noises, which seriously affects the practical use of remote sensing images. As remote sensing images have more complex texture features than ordinary images, this will lead to the previous denoising algorithm failing to achieve the desired result. Therefore, we propose a novel remote sensing image denoising network (RSIDNet) based on a deep learning approach, which mainly consists of a multi-scale feature extraction module (MFE), multiple local skip-connected enhanced attention blocks (ECA), a global feature fusion block (GFF), and a noisy image reconstruction block (NR). The combination of these modules greatly improves the model's use of the extracted features and increases the model's denoising capability. Extensive experiments on synthetic Gaussian noise datasets and real noise datasets have shown that RSIDNet achieves satisfactory results. RSIDNet can improve the loss of detail information in denoised images in traditional denoising methods, retaining more of the higher-frequency components, which can have performance improvements for subsequent image processing. [ABSTRACT FROM AUTHOR]

Large-scale optical sensing and precise, rapid assessment of seismic building damage in urban communities are increasingly demanded in disaster prevention and reduction. The common method is to train a convolutional neural network (CNN) in a pixel-level semantic segmentation approach and does not fully consider the characteristics of the assessment objectives. This study developed a machine-learning-derived two-stage method for post-earthquake building location and damage assessment considering the data characteristics of satellite remote sensing (SRS) optical images with dense distribution, small size, and imbalanced numbers. It included a modified You Only Look Once (YOLOv4) object detection module and a support vector machine (SVM) based classification module. In the primary step, the multiscale features were successfully extracted and fused from SRS images of densely distributed buildings by optimizing the YOLOv4 model toward the network structures, training hyperparameters, and anchor boxes. The fusion improved multi-channel features, optimization of network structure and hyperparameters have significantly enhanced the average location accuracy of post-earthquake buildings. Thereafter, three statistics (i.e., the angular second moment, dissimilarity, and inverse difference moment) were further discovered to effectively extract the characteristic value for earthquake damage from located buildings in SRS optical images based on the gray level co-occurrence matrix. They were used as the texture features to distinguish damage intensities of buildings, using the SVM model. The investigated dataset included 386 pre- and post-earthquake SRS optical images of the 2017 Mexico City earthquake, with a resolution of 1024 × 1024 pixels. Results show that the average location accuracy of post-earthquake buildings exceeds 95.7% and that the binary classification accuracy for damage assessment reaches 97.1%. The proposed two-stage method was validated by its extremely high precision in respect of densely distributed small buildings, indicating the promising potential of computer vision in large-scale disaster prevention and reduction using SRS datasets. [ABSTRACT FROM AUTHOR]

Most 3D CityGML building models in street-view maps (e.g., Google, Baidu) lack texture information, which is generally used to reconstruct real-scene 3D models by photogrammetric techniques, such as unmanned aerial vehicle (UAV) mapping. However, due to its simplified building model and inaccurate location information, the commonly used photogrammetric method using a single data source cannot satisfy the requirement of texture mapping for the CityGML building model. Furthermore, a single data source usually suffers from several problems, such as object occlusion. We proposed a novel approach to achieve CityGML building model texture mapping by multiview coplanar extraction from UAV remotely sensed or terrestrial images to alleviate these problems. We utilized a deep convolutional neural network to filter out object occlusion (e.g., pedestrians, vehicles, and trees) and obtain building-texture distribution. Point-line-based features are extracted to characterize multiview coplanar textures in 2D space under the constraint of a homography matrix, and geometric topology is subsequently conducted to optimize the boundary of textures by using a strategy combining Hough-transform and iterative least-squares methods. Experimental results show that the proposed approach enables texture mapping for building façades to use 2D terrestrial images without the requirement of exterior orientation information; that is, different from the photogrammetric method, a collinear equation is not an essential part to capture texture information. In addition, the proposed approach can significantly eliminate blurred and distorted textures of building models, so it is suitable for automatic and rapid texture updates. [ABSTRACT FROM AUTHOR]

The survival rate of seedlings is a decisive factor of afforestation assessment. Generally, ground checking is more accurate than any other methods. However, the survival rate of seedlings can be higher in the growing season, and this can be estimated in a larger area at a relatively lower cost by extracting the tree crown from the unmanned aerial vehicle (UAV) images, which provides an opportunity for monitoring afforestation in an extensive area. At present, studies on extracting individual tree crowns under the complex ground vegetation conditions are limited. Based on the afforestation images obtained by airborne consumer-grade cameras in central China, this study proposes a method of extracting and fusing multiple radii morphological features to obtain the potential crown. A random forest (RF) was used to identify the regions extracted from the images, and then the recognized crown regions were fused selectively according to the distance. A low-cost individual crown recognition framework was constructed for rapid checking of planted trees. The method was tested in two afforestation areas of 5950 m2 and 5840 m2, with a population of 2418 trees (Koelreuteria) in total. Due to the complex terrain of the sample plot, high weed coverage, the crown width of trees, and spacing of saplings vary greatly, which increases both the difficulty and complexity of crown extraction. Nevertheless, recall and F-score of the proposed method reached 93.29%, 91.22%, and 92.24% precisions, respectively, and 2212 trees were correctly recognized and located. The results show that the proposed method is robust to the change of brightness and to splitting up of a multi-directional tree crown, and is an automatic solution for afforestation verification. [ABSTRACT FROM AUTHOR]

The control of thermostats of a heating, ventilation, and air-conditioning (HVAC) system installed in commercial and residential buildings remains a pertinent problem in building energy efficiency and thermal comfort research. The ability to determine the number of people at a particular time in an area is imperative for energy efficiency in order to condition only occupied regions and thermally deficient regions. In this study of the best features comparison for detecting the number of people in an area, feature extraction techniques including wavelet scattering, wavelet decomposition, grey-level co-occurrence matrix (GLCM) and feature maps convolution neural network (CNN) layers were explored using thermal camera imagery. Specifically, the pretrained CNN networks explored are the deep residual (Resnet-50) and visual geometry group (VGG-16) networks. The discriminating potential of Haar, Daubechies and Symlets wavelet statistics on different distributions of data were investigated. The performance of VGG-16 and ResNet-50 in an end-to-end manner utilizing transfer learning approach was investigated. Experimental results showed the classification and regression trees (CART) model trained on only GLCM and Haar wavelet statistic features, individually achieved accuracies of approximately 80% and 84%, respectively, in the detection problem. Moreover, k-nearest neighbors (KNN) trained on the combined features of GLCM and Haar wavelet statistics achieved an accuracy of approximately 86%. In addition, the performance accuracy of the multi classification support vector machine (SVM) trained on deep features obtained from layers of pretrained ResNet-50 and VGG-16 was between 96% and 97%. Furthermore, ResNet-50 transfer learning outperformed the VGG-16 transfer learning model for occupancy detection using thermal imagery. Overall, the SVM model trained on features extracted from wavelet scattering emerged as the best performing classifier with an accuracy of 100%. A principal component analysis (PCA) on the wavelet scattering features proved that the first twenty (20) principal components achieved a similar accuracy level instead of training on the whole feature set to reduce the execution time. The occupancy detection models can be integrated into HVAC control systems for energy efficiency and security systems, and aid in the distribution of resources to people in an area. [ABSTRACT FROM AUTHOR]

The monitoring of human activity and vital signs plays a significant role in remote health-care. Radar provides a non-contact monitoring approach without privacy and illumination concerns. However, multiple people in a narrow indoor environment bring dense multipaths for activity monitoring, and the received vital sign signals are heavily distorted with body movements. This paper proposes a framework based on Frequency Modulated Continuous Wave (FMCW) and Impulse Radio Ultra-Wideband (IR-UWB) radars to address these challenges, designing intelligent spatial-temporal information fusion for activity and vital sign monitoring. First, a local binary pattern (LBP) and energy features are extracted from FMCW radar, combined with the wavelet packet transform (WPT) features on IR-UWB radar for activity monitoring. Then the additional information guided fusing network (A-FuseNet) is proposed with a modified generative and adversarial structure for vital sign monitoring. A Cascaded Convolutional Neural Network (CCNN) module and a Long Short Term Memory (LSTM) module are designed as the fusion sub-network for vital sign information extraction and multisensory data fusion, while a discrimination sub-network is constructed to optimize the fused heartbeat signal. In addition, the activity and movement characteristics are introduced as additional information to guide the fusion and optimization. A multi-radar dataset with an FMCW and two IR-UWB radars in a cotton tent, a small room and a wide lobby is constructed, and the accuracies of activity and vital sign monitoring achieve 99.9% and 92.3% respectively. Experimental results demonstrate the superiority and robustness of the proposed framework. [ABSTRACT FROM AUTHOR]

The use of multibeam echosounder systems (MBES) for detailed seafloor mapping is increasing at a fast pace. Due to their design, enabling continuous high-density measurements and the coregistration of seafloor's depth and reflectivity, MBES has become a fundamental instrument in the advancing field of acoustic seafloor classification (ASC). With these data becoming available, recent seafloor mapping research focuses on the interpretation of the hydroacoustic data and automated predictive modeling of seafloor composition. While a methodological consensus on which seafloor sediment classification algorithm and routine does not exist in the scientific community, it is expected that progress will occur through the refinement of each stage of the ASC pipeline: ranging from the data acquisition to the modeling phase. This research focuses on the stage of the feature extraction; the stage wherein the spatial variables used for the classification are, in this case, derived from the MBES backscatter data. This contribution explored the sediment classification potential of a textural feature based on the recently introduced Weyl transform of 300 kHz MBES backscatter imagery acquired over a nearshore study site in Belgian Waters. The goodness of the Weyl transform textural feature for seafloor sediment classification was assessed in terms of cluster separation of Folk's sedimentological categories (4-class scheme). Class separation potential was quantified at multiple spatial scales by cluster silhouette coefficients. Weyl features derived from MBES backscatter data were found to exhibit superior thematic class separation compared to other well-established textural features, namely: (1) First-order Statistics, (2) Gray Level Co-occurrence Matrices (GLCM), (3) Wavelet Transform and (4) Local Binary Pattern (LBP). Finally, by employing a Random Forest (RF) categorical classifier, the value of the proposed textural feature for seafloor sediment mapping was confirmed in terms of global and by-class classification accuracies, highest for models based on the backscatter Weyl features. Further tests on different backscatter datasets and sediment classification schemes are required to further elucidate the use of the Weyl transform of MBES backscatter imagery in the context of seafloor mapping. [ABSTRACT FROM AUTHOR]

Image-based three-dimensional (3D) reconstruction is a process of extracting 3D information from an object or entire scene while using low-cost vision sensors. A structure-from-motion coupled with multi-view stereo (SFM-MVS) pipeline is a widely used technique that allows 3D reconstruction from a collection of unordered images. The SFM-MVS pipeline typically comprises different processing steps, including feature extraction and feature matching, which provide the basis for automatic 3D reconstruction. However, surfaces with poor visual texture (repetitive, monotone, etc.) challenge the feature extraction and matching stage and affect the quality of reconstruction. The projection of image patterns while using a video projector during the image acquisition process is a well-known technique that has been shown to be successful for such surfaces. In this study, we evaluate the performance of different feature extraction methods on texture-less surfaces with the application of synthetically generated noise patterns (images). Seven state-of-the-art feature extraction methods (HARRIS, Shi-Tomasi, MSER, SIFT, SURF, KAZE, and BRISK) are evaluated on problematic surfaces in two experimental phases. In the first phase, the 3D reconstruction of real and virtual planar surfaces evaluates image patterns while using all feature extraction methods, where the patterns with uniform histograms have the most suitable morphological features. The best performing pattern from Phase One is used in Phase Two experiments in order to recreate a polygonal model of a 3D printed object using all of the feature extraction methods. The KAZE algorithm achieved the lowest standard deviation and mean distance values of 0.0635 mm and −0.00921 mm, respectively. [ABSTRACT FROM AUTHOR]

Illegal excavations in archaeological heritage sites (namely "looting") are a global phenomenon. Satellite images are nowadays massively used by archaeologists to systematically document sites affected by looting. In parallel, remote sensing scientists are increasingly developing processing methods with a certain degree of automation to quantify looting using satellite imagery. To capture the state-of-the-art of this growing field of remote sensing, in this work 47 peer-reviewed research publications and grey literature are reviewed, accounting for: (i) the type of satellite data used, i.e., optical and synthetic aperture radar (SAR); (ii) properties of looting features utilized as proxies for damage assessment (e.g., shape, morphology, spectral signature); (iii) image processing workflows; and (iv) rationale for validation. Several scholars studied looting even prior to the conflicts recently affecting the Middle East and North Africa (MENA) region. Regardless of the method used for looting feature identification (either visual/manual, or with the aid of image processing), they preferred very high resolution (VHR) optical imagery, mainly black-and-white panchromatic, or pansharpened multispectral, whereas SAR is being used more recently by specialist image analysts only. Yet the full potential of VHR and high resolution (HR) multispectral information in optical imagery is to be exploited, with limited research studies testing spectral indices. To fill this gap, a range of looted sites across the MENA region are presented in this work, i.e., Lisht, Dashur, and Abusir el Malik (Egypt), and Tell Qarqur, Tell Jifar, Sergiopolis, Apamea, Dura Europos, and Tell Hizareen (Syria). The aim is to highlight: (i) the complementarity of HR multispectral data and VHR SAR with VHR optical imagery, (ii) usefulness of spectral profiles in the visible and near-infrared bands, and (iii) applicability of methods for multi-temporal change detection. Satellite data used for the demonstration include: HR multispectral imagery from the Copernicus Sentinel-2 constellation, VHR X-band SAR data from the COSMO-SkyMed mission, VHR panchromatic and multispectral WorldView-2 imagery, and further VHR optical data acquired by GeoEye-1, IKONOS-2, QuickBird-2, and WorldView-3, available through Google Earth. Commonalities between the different image processing methods are examined, alongside a critical discussion about automation in looting assessment, current lack of common practices in image processing, achievements in managing the uncertainty in looting feature interpretation, and current needs for more dissemination and user uptake. Directions toward sharing and harmonization of methodologies are outlined, and some proposals are made with regard to the aspects that the community working with satellite images should consider, in order to define best practices of satellite-based looting assessment. [ABSTRACT FROM AUTHOR]

Sea ice is one of the causes of marine disasters. The classification of sea ice images is an important part of sea ice detection. The labeled samples in hyperspectral sea ice image classification are difficult to acquire, which causes minor sample problems. In addition, most of the current sea ice classification methods mainly use spectral features for shallow learning, which also limits further improvement of the sea ice classification accuracy. Therefore, this paper proposes a hyperspectral sea ice image classification method based on the spectral-spatial-joint feature with deep learning. The proposed method first extracts sea ice texture information by the gray-level co-occurrence matrix (GLCM). Then, it performs dimensionality reduction and a correlation analysis of the spectral information and spatial information of the unlabeled samples, respectively. It eliminates redundant information by extracting the spectral-spatial information of the neighboring unlabeled samples of the labeled sample and integrating the information with the spectral and texture data of the labeled sample to further enhance the quality of the labeled sample. Lastly, the three-dimensional convolutional neural network (3D-CNN) model is designed to extract the deep spectral-spatial features of sea ice. The proposed method combines relevant textural features and performs spectral-spatial feature extraction based on the 3D-CNN model by using a large amount of unlabeled sample information. In order to verify the effectiveness of the proposed method, sea ice classification experiments are carried out on two hyperspectral data sets: Baffin Bay and Bohai Bay. Compared with the CNN algorithm based on a single feature (spectral or spatial) and other CNN algorithms based on spectral-spatial features, the experimental results show that the proposed method achieves better sea ice classification (98.52% and 97.91%) with small samples. Therefore, it is more suitable for classifying hyperspectral sea ice images. [ABSTRACT FROM AUTHOR]

Today, more and more deep learning frameworks are being applied to hyperspectral image classification tasks and have achieved great results. However, such approaches are still hampered by long training times. Traditional spectral–spatial hyperspectral image classification only utilizes spectral features at the pixel level, without considering the correlation between local spectral signatures. Our article has tested a novel hyperspectral image classification pattern, using random-patches convolution and local covariance (RPCC). The RPCC is an effective two-branch method that, on the one hand, obtains a specified number of convolution kernels from the image space through a random strategy and, on the other hand, constructs a covariance matrix between different spectral bands by clustering local neighboring pixels. In our method, the spatial features come from multi-scale and multi-level convolutional layers. The spectral features represent the correlations between different bands. We use the support vector machine as well as spectral and spatial fusion matrices to obtain classification results. Through experiments, RPCC is tested with five excellent methods on three public data-sets. Quantitative and qualitative evaluation indicators indicate that the accuracy of our RPCC method can match or exceed the current state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Automatic weed detection and classification faces the challenges of large intraclass variation and high spectral similarity to other vegetation. With the availability of new high-resolution remote sensing data from various platforms and sensors, it is possible to capture both spectral and spatial characteristics of weed species at multiple scales. Effective multi-resolution feature learning is then desirable to extract distinctive intensity, texture and shape features of each category of weed to enhance the weed separability. We propose a feature extraction method using a Convolutional Neural Network (CNN) and superpixel based Local Binary Pattern (LBP). Both middle and high level spatial features are learned using the CNN. Local texture features from superpixel-based LBP are extracted, and are also used as input to Support Vector Machines (SVM) for weed classification. Experimental results on the hyperspectral and remote sensing datasets verify the effectiveness of the proposed method, and show that it outperforms several feature extraction approaches. [ABSTRACT FROM AUTHOR]

The efficient and accurate application of deep learning in the remote sensing field largely depends on the pre-processing technology of remote sensing images. Particularly, image fusion is the essential way to achieve the complementarity of the panchromatic band and multispectral bands in high spatial resolution remote sensing images. In this paper, we not only pay attention to the visual effect of fused images, but also focus on the subsequent application effectiveness of information extraction and feature recognition based on fused images. Based on the WorldView-3 images of Tongzhou District of Beijing, we apply the fusion results to conduct the experiments of object recognition of typical urban features based on deep learning. Furthermore, we perform a quantitative analysis for the existing pixel-based mainstream fusion methods of IHS (Intensity-Hue Saturation), PCS (Principal Component Substitution), GS (Gram Schmidt), ELS (Ehlers), HPF (High-Pass Filtering), and HCS (Hyper spherical Color Space) from the perspectives of spectrum, geometric features, and recognition accuracy. The results show that there are apparent differences in visual effect and quantitative index among different fusion methods, and the PCS fusion method has the most satisfying comprehensive effectiveness in the object recognition of land cover (features) based on deep learning. [ABSTRACT FROM AUTHOR]

In order to realize the automatic and accurate recognition of shipwreck targets in side-scan sonar (SSS) waterfall images, a pipeline that contains feature extraction, selection, and shipwreck recognition, an AdaBoost model was constructed by sample images. Shipwreck targets are detected quickly by a nonlinear matching model, and a shipwreck recognition in SSS waterfall images are given, and according to a wide set of combinations of different types of these individual procedures, the model is able to recognize the shipwrecks accurately. Firstly, two feature-extraction methods suitable for recognizing SSS shipwreck targets from natural sea bottom images were studied. In addition to these two typical features, some commonly used features were extracted and combined as comprehensive features to characterize shipwrecks from various feature spaces. Based on Independent Component Analysis (ICA), the preferred features were selected from the comprehensive features, which avoid dimension disaster and improved the correct recognition rate. Then, the Gentle AdaBoost algorithm was studied and used for constructing the shipwreck target recognition model using sample images. Finally, a shipwreck target recognition process for the SSS waterfall image was given, and the process contains shipwreck target fast detection by a nonlinear matching model and accurate recognition by the Gentle AdaBoost recognition model. The results show that the correct recognition rate of the model for the sample image is 97.44%, while the false positive rate is 3.13% and the missing detection rate is 0. This study of a measured SSS waterfall image confirms the correctness of the recognition process and model. [ABSTRACT FROM AUTHOR]

Coastal wetland vegetation is a vital component that plays an important role in environmental protection and the maintenance of the ecological balance. As such, the efficient classification of coastal wetland vegetation types is key to the preservation of wetlands. Based on its detailed spatial information, high spatial resolution imagery constitutes an important tool for extracting suitable texture features for improving the accuracy of classification. In this paper, a texture feature, Completed Local Binary Patterns (CLBP), which is highly suitable for face recognition, is presented and applied to vegetation classification using high spatial resolution Pléiades satellite imagery in the central zone of Yancheng National Natural Reservation (YNNR) in Jiangsu, China. To demonstrate the potential of CLBP texture features, Grey Level Co-occurrence Matrix (GLCM) texture features were used to compare the classification. Using spectral data alone and spectral data combined with texture features, the image was classified using a Support Vector Machine (SVM) based on vegetation types. The results show that CLBP and GLCM texture features yielded an accuracy 6.50% higher than that gained when using only spectral information for vegetation classification. However, CLBP showed greater improvement in terms of classification accuracy than GLCM for Spartina alterniflora. Furthermore, for the CLBP features, CLBP_magnitude (CLBP_m) was more effective than CLBP_sign (CLBP_s), CLBP_center (CLBP_c), and CLBP_s/m or CLBP_s/m/c. These findings suggest that the CLBP approach offers potential for vegetation classification in high spatial resolution images. [ABSTRACT FROM AUTHOR]

Recently, many researchers have been dedicated to using convolutional neural networks (CNNs) to extract global-context features (GCFs) for remote-sensing scene classification. Commonly, accurate classification of scenes requires knowledge about both the global context and local objects. However, unlike the natural images in which the objects cover most of the image, objects in remote-sensing images are generally small and decentralized. Thus, it is hard for vanilla CNNs to focus on both global context and small local objects. To address this issue, this paper proposes a novel end-to-end CNN by integrating the GCFs and local-object-level features (LOFs). The proposed network includes two branches, the local object branch (LOB) and global semantic branch (GSB), which are used to generate the LOFs and GCFs, respectively. Then, the concatenation of features extracted from the two branches allows our method to be more discriminative in scene classification. Three challenging benchmark remote-sensing datasets were extensively experimented on; the proposed approach outperformed the existing scene classification methods and achieved state-of-the-art results for all three datasets. [ABSTRACT FROM AUTHOR]