Search

Accurately estimating shrub biomass in arid and semi-arid regions is critical for understanding ecosystem productivity and carbon stocks at both local and global scales. Due to the short and sparse features of shrubs, capturing the shrub biomass accurately by satellite observations is challenging. Previous studies mostly used ground samples and satellite observations to estimate shrub biomass by establishing a direct connection, which was often hindered by the limited number of ground samples and spatial scale mismatch between samples and observations. Unmanned aerial vehicles (UAVs) provide opportunities to obtain more samples that are in line with the aspects of satellite observations (i.e., scale) for regional-scale shrub biomass estimations accurately with low costs. However, few studies have been conducted based on the air-space-ground-scale connection assisted by UAVs. Here we developed a framework for estimating 10 m shrub biomass at a regional scale by integrating ground measurements, UAV, Landsat, and Sentinel-1/2 observations. First, the spatial distribution map of shrublands and non-shrublands was generated in 2023 in the Helan Mountains of Ningxia province, China. This map had an F1 score of 0.92. Subsequently, the UAV-based shrub biomass map was estimated using an empirical model between the biomass and the crown area of shrubs, which was aggregated at a 10 m × 10 m grid to match the spatial resolution of Sentinel-1/2 images. Then, a regional-scale estimation model of shrub biomass was developed with a random forest regression (RFR) approach driven by ground biomass measurements, UAV-based biomass, and the optimal satellite metrics. Finally, the developed model was used to produce the biomass map of shrublands over the study area in 2023. The uncertainty of the resultant biomass map was characterized by the pixel-level standard deviation (SD) using the leave-one-out cross-validation (LOOCV) method. The results suggested that the integration of multi-scale observations from the ground, UAVs, and satellites provided a promising approach to obtaining the regional shrub biomass accurately. Our developed model, which integrates satellite spectral bands and vegetation indices (R2 = 0.62), outperformed models driven solely by spectral bands (R2 = 0.33) or vegetation indices (R2 = 0.55). In addition, our estimated biomass has an average uncertainty of less than 4%, with the lowest values (<2%) occurring in regions with high shrub coverage (>30%) and biomass production (>300 g/m2). This study provides a methodology to accurately monitor the shrub biomass from satellite images assisted by near-ground UAV observations as well as ground measurements. [ABSTRACT FROM AUTHOR]

Performing remote sensing scene classification (RSSC) directly on satellites can alleviate data downlink burdens and reduce latency. Compared to convolutional neural networks (CNNs), the all-adder neural network (A2NN) is a novel basic neural network that is more suitable for onboard RSSC, enabling lower computational overhead by eliminating multiplication operations in convolutional layers. However, the extensive floating-point data and operations in A2NNs still lead to significant storage overhead and power consumption during hardware deployment. In this article, a shared scaling factor-based de-biasing quantization (SSDQ) method tailored for the quantization of A2NNs is proposed to address this issue, including a powers-of-two (POT)-based shared scaling factor quantization scheme and a multi-dimensional de-biasing (MDD) quantization strategy. Specifically, the POT-based shared scaling factor quantization scheme converts the adder filters in A2NNs to quantized adder filters with hardware-friendly integer input activations, weights, and operations. Thus, quantized A2NNs (Q-A2NNs) composed of quantized adder filters have lower computational and memory overheads than A2NNs, increasing their utility in hardware deployment. Although low-bit-width Q-A2NNs exhibit significantly reduced RSSC accuracy compared to A2NNs, this issue can be alleviated by employing the proposed MDD quantization strategy, which combines a weight-debiasing (WD) strategy, which reduces performance degradation due to deviations in the quantized weights, with a feature-debiasing (FD) strategy, which enhances the classification performance of Q-A2NNs through minimizing deviations among the output features of each layer. Extensive experiments and analyses demonstrate that the proposed SSDQ method can efficiently quantize A2NNs to obtain Q-A2NNs with low computational and memory overheads while maintaining comparable performance to A2NNs, thus having high potential for onboard RSSC. [ABSTRACT FROM AUTHOR]

Unsupervised domain adaptation (UDA) methods have become a research hotspot in remote sensing scene classification to reduce dependence on labeled samples. However, most current methods focus on extracting domain invariant features, ignoring the problem of large intraclass differences and the imbalanced sample numbers between categories in remote sensing images. To address these issues, we propose a remote sensing scene domain adaptive method based on middle-layer feature extraction and nuclear norm maximization (MFE-NM). In the MFE module, the middle-layer features of the feature extractor are randomly extracted and processed. Since the receptive field of the middle-layer features is smaller and the resolution is higher, the effective use of the middle-layer features can reduce the impact of image feature heterogeneity caused by large intraclass differences in remote sensing images. In addition, it can be concluded that the constrained nuclear norm can simultaneously improve the prediction diversity and discriminability of the model through theoretical derivation. Therefore, the NM module is proposed to solve the problem of reduced prediction diversity caused by entropy minimization methods when dealing with scene classification problems with imbalanced sample numbers between categories. Extensive experiments and analyses on three public remote sensing datasets demonstrate the effectiveness and competitiveness of our proposed method. [ABSTRACT FROM AUTHOR]

In recent years, convolutional neural network (CNN)-based algorithms have been widely used in remote sensing image processing and show tremendous performance in a variety of application fields. However, large amounts of data and intensive computations make the deployment of CNN-based algorithms a challenging problem, especially for the spaceborne scenario where resources and power consumption are limited. To tackle this problem, this paper proposes an automatic CNN deployment solution on resource-limited field-programmable gate arrays (FPGAs) for spaceborne remote sensing applications. Firstly, a series of hardware-oriented optimization methods are proposed to reduce the complexity of the CNNs. Secondly, a hardware accelerator is designed. In this accelerator, a reconfigurable processing engine array with efficient convolutional computation architecture is used to accelerate CNN-based algorithms. Thirdly, to bridge the optimized CNNs and hardware accelerator, a compilation toolchain is introduced into the deployment solution. Through the automatic conversion from CNN models to hardware instructions, various networks can be deployed on hardware in real-time. Finally, we deployed an improved VGG16 network and an improved YOLOv2 network on Xilinx AC701 to evaluate the effectiveness of the proposed deployment solution. The experiments show that with only 3.407 W power consumption and 94 DSP consumption, our solution achieves 23.06 giga operations per second (GOPS) throughput in the improved VGG16 and 22.17 GOPS throughput in the improved YOLOv2. Compared to the related works, the DSP efficiency of our solution is improved by 1.3–2.7×. [ABSTRACT FROM AUTHOR]

Extensive convolutional neural network (CNN)-based methods have been widely used in remote sensing scene classification. However, the dense operation and huge memory storage of the state-of-the-art models hinder their deployment on low-power embedded devices. In this letter, we propose a mixed-precision quantization method to compress the model size without accuracy degradation. In this method, we propose a symmetric nonlinear quantization scheme to reduce the quantization error. A corresponding three-step training strategy is proposed to improve the performance of the quantized network. Finally, based on the proposed scheme and training strategy, we propose a neural architecture search (NAS)-based quantization bit-width search (NQBS) method. This method can automatically select a bit width for each quantized layer to obtain a mixed-precision network with an optimal model size. We apply the proposed method to the ResNet-34 and SqueezeNet networks and evaluate the quantized networks on the NWPU-RESISC45 data set. The experimental results show that the mixed-precision quantized networks under the proposed method strike a satisfying tradeoff between classification accuracy and model size. [ABSTRACT FROM AUTHOR]

Recently, remote sensing image scene classification technology has been widely applied in many applicable industries. As a result, several remote sensing image scene classification frameworks have been proposed; in particular, those based on deep convolutional neural networks have received considerable attention. However, most of these methods have performance limitations when analyzing images with large intraclass variations. To overcome this limitation, this letter presents the marginal center loss with an adaptive margin. The marginal center loss separates hard samples and enhances the contributions of hard samples to minimize the variations in features of the same class. Experimental results on public remote sensing image scene data sets demonstrate the effectiveness of our method. After the model is trained using the marginal center loss, the variations in the features of the same class are reduced. Furthermore, a comparison with state-of-the-art methods proves that our model has competitive performance in the field of remote sensing image scene classification. [ABSTRACT FROM AUTHOR]

Object detection in complex optical remote sensing images is a challenging problem due to the wide variety of scales, densities, and shapes of object instances on the earth surface. In this letter, we focus on the wide-scale variation problem of multiclass object detection and propose an effective object detection framework in remote sensing images based on YOLOv2. To make the model adaptable to multiscale object detection, we design a network that concatenates feature maps from layers of different depths and adopt a feature introducing strategy based on oriented response dilated convolution. Through this strategy, the performance for small-scale object detection is improved without losing the performance for large-scale object detection. Compared to YOLOv2, the performance of the proposed framework tested in the DOTA (a large-scale data set for object detection in aerial images) data set improves by 4.4% mean average precision without adding extra parameters. The proposed framework achieves real-time detection for $1024\times 1024$ image using Titan Xp GPU acceleration. 1 https://github.com/WenchaoliuMUC/Detection-of-Multiclass-Objects-in-Optical-Remote-Sensing-Images [ABSTRACT FROM AUTHOR]

In recent times, many efforts have been made to improve remote sensing image scene classification, especially using popular deep convolutional neural networks. However, most of these methods do not consider the specific scene orientation of the remote sensing images. In this letter, we propose the improved oriented response network (IORN), which is based on the ORN, to handle the orientation problem in remote sensing image scene classification. We propose average active rotating filters (A-ARFs) in the IORN. While IORNs are being trained, A-ARFs are updated by a method that is different from the ARFs of the ORN, without additional computations. This change helps IORN improve its ability to encode orientation information and speeds up optimization during training. We also propose Squeeze-ORAlign (S-ORAlign) by adding a squeeze layer to ORAlign of ORN. With the squeeze layer, S-ORAlign can address large-scale images, unlike ORAlign. An ablation study and comparison experiments are designed on a public remote sensing image scene classification data set. The experimental results demonstrate the effectiveness and better performance of the proposed model over that of other state-of-the-art models. [ABSTRACT FROM AUTHOR]

Ship detection is a challenging problem in complex optical remote-sensing images. In this letter, an effective ship detection framework in remote-sensing images based on the convolutional neural network is proposed. The framework is designed to predict bounding box of ship with orientation angle information. Note that the angle information which is added to bounding box regression makes bounding box accurately fit into the ship region. In order to make the model adaptable to the detection of multiscale ship targets, especially small-sized ships, we design the network with feature maps from the layers of different depths. The whole detection pipeline is a single network and achieves real-time detection for a $704 \times 704$ image with the use of Titan X GPU acceleration. Through experiments, we validate the effectiveness, robustness, and accuracy of the proposed ship detection framework in complex remote-sensing scenes. [ABSTRACT FROM PUBLISHER]

Sea–land segmentation is a key step for some important applications of panchromatic remote sensing image processing. However, robust and effective sea–land segmentation for high-resolution panchromatic remote sensing images is still a challenging problem. This letter presents an accurate and robust approach by integrating the improved multiscale normalized cut (IMNcut) method and improved Chan–Vese model for sea–land segmentation. At first, the image is downsampled and segmented into multiple regions by the IMNcut method. Next, the homogeneous regions are merged to obtain a coarse segmentation result. Finally, gray intensity and local entropy features are integrated as discriminants of the improved Chan–Vese model, which is used to obtain the final segmentation result through a low- to high-resolution segmentation scheme. Experimental results performed on several real data sets demonstrate the effectiveness of the proposed model in terms of visual and objective evaluations. [ABSTRACT FROM PUBLISHER]

In this paper, we mainly focused our research on northern Shaanxi district, which is a pilot area of the Grain for Green Project. We compared the spatial distribution patterns of croplands and their productivity for the past 20 years (from the end of the 1980s to 2010). Cropland dynamics for the past 20 years were interpreted from medium- and high-resolution remote sensing images (Landsat TM/ETM+). In addition, using the GLO-PEM and AGRO-VPM models with a medium resolution and long time series remote sensing dataset (AVHRR/MODIS), net primary productivity (NPP) and its relationship with cropland were estimated. Finally, the effect of cropland change on productivity was analyzed. The results show that during the first decade of the research period, cropland area and productivity in northern Shaanxi experienced a small boost, while in the latter decade, both cropland area and NPP were significantly reduced. The main cause of the increase in cropland was the reclamation of large area of grassland and unutilized land to meet the food demands of the local population as well as to compensate for the occupation of urban constructions. While the main cause of the decrease in cropland was the implementation of the Grain for Green Project. In addition, urbanization was also a key factor. Overall, during the past 20 years, the total area of cropland in northern Shaanxi decreased by 42.56%, and cropland NPP dropped by 41.90%. This study is of great importance for the assessment of regional cropland security, food security and scientific planning of regional land use. [ABSTRACT FROM AUTHOR]

In recent years, convolutional neural network (CNN)-based methods have been widely used for optical remote sensing object detection and have shown excellent performance. Some aerospace systems, such as satellites or aircrafts, need to adopt these methods to observe objects on the ground. Due to the limited budget of the logical resources and power consumption in these systems, an embedded device is a good choice to implement the CNN-based methods. However, it is still a challenge to strike a balance between performance and power consumption. In this paper, we propose an efficient hardware-implementation method for optical remote sensing object detection. Firstly, we optimize the CNN-based model for hardware implementation, which establishes a foundation for efficiently mapping the network on a field-programmable gate array (FPGA). In addition, we propose a hardware architecture for the CNN-based remote sensing object detection model. In this architecture, a general processing engine (PE) is proposed to implement multiple types of convolutions in the network using the uniform module. An efficient data storage and access scheme is also proposed, and it achieves low-latency calculations and a high memory bandwidth utilization rate. Finally, we deployed the improved YOLOv2 network on a Xilinx ZYNQ xc7z035 FPGA to evaluate the performance of our design. The experimental results show that the performance of our implementation on an FPGA is only 0.18% lower than that on a graphics processing unit (GPU) in mean average precision (mAP). Under a 200 MHz working frequency, our design achieves a throughput of 111.5 giga-operations per second (GOP/s) with a 5.96 W on-chip power consumption. Comparison with the related works demonstrates that the proposed design has obvious advantages in terms of energy efficiency and that it is suitable for deployment on embedded devices. [ABSTRACT FROM AUTHOR]

Recently, extensive convolutional neural network (CNN)-based methods have been used in remote sensing applications, such as object detection and classification, and have achieved significant improvements in performance. Furthermore, there are a lot of hardware implementation demands for remote sensing real-time processing applications. However, the operation and storage processes in floating-point models hinder the deployment of networks in hardware implements with limited resource and power budgets, such as field-programmable gate arrays (FPGAs) and application-specific integrated circuits (ASICs). To solve this problem, this paper focuses on optimizing the hardware design of CNN with low bit-width integers by quantization. First, a symmetric quantization scheme-based hybrid-type inference method was proposed, which uses the low bit-width integer to replace floating-point precision. Then, a training approach for the quantized network is introduced to reduce accuracy degradation. Finally, a processing engine (PE) with a low bit-width is proposed to optimize the hardware design of FPGA for remote sensing image classification. Besides, a fused-layer PE is also presented for state-of-the-art CNNs equipped with Batch-Normalization and LeakyRelu. The experiments performed on the Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset using a graphics processing unit (GPU) demonstrate that the accuracy of 8-bit quantized model drops by about 1%, which is an acceptable accuracy loss. The accuracy result tested on FPGA is consistent with that of GPU. As for the resource consumptions of FPGA, the Look Up Table (LUT), Flip-flop (FF), Digital Signal Processor (DSP), and Block Random Access Memory (BRAM) are reduced by 46.21%, 43.84%, 45%, and 51%, respectively, compared with that of floating-point implementation. [ABSTRACT FROM AUTHOR]