Search

Generative adversarial networks (or GANs) are a specific deep learning architecture often used for different usages, such as data generation or image-to-image translation. In recent years, this structure has gained increased popularity and has been used in different fields. One area of expertise currently in vogue is the use of GANs to produce synthetic aperture radar (SAR) data, and especially expand training datasets for SAR automatic target recognition (ATR). In effect, the complex SAR image formation makes these kind of data rich in information, leading to the use of deep networks in deep learning-based methods. Yet, deep networks also require sufficient data for training. However, contrary to optical images, we generally do not have a substantial number of available SAR images because of their acquisition and labelling cost; GANs are then an interesting tool. Concurrently, how to improve explainability for SAR ATR deep neural networks and how to make their reasoning more transparent have been increasingly explored as model opacity deteriorates trust of users. This paper aims at reviewing how GANs are used with SAR images, but also giving perspectives on how GANs could be used to improve interpretability and explainability of SAR classifiers. [ABSTRACT FROM AUTHOR]

The objectives of this paper are to investigate the trade-offs between a physically constrained neural network and a deep, convolutional neural network and to design a combined ML approach ("VarioCNN"). Our solution is provided in the framework of a cyberinfrastructure that includes a newly designed ML software, GEOCLASS-image (v1.0), modern high-resolution satellite image data sets (Maxar WorldView data), and instructions/descriptions that may facilitate solving similar spatial classification problems. Combining the advantages of the physically-driven connectionist-geostatistical classification method with those of an efficient CNN, VarioCNN provides a means for rapid and efficient extraction of complex geophysical information from submeter resolution satellite imagery. A retraining loop overcomes the difficulties of creating a labeled training data set. Computational analyses and developments are centered on a specific, but generalizable, geophysical problem: The classification of crevasse types that form during the surge of a glacier system. A surge is a glacial catastrophe, an acceleration of a glacier to typically 100–200 times its normal velocity. GEOCLASS-image is applied to study the current (2016-2024) surge in the Negribreen Glacier System, Svalbard. The geophysical result is a description of the structural evolution and expansion of the surge, based on crevasse types that capture ice deformation in six simplified classes. [ABSTRACT FROM AUTHOR]

Ground-penetrating radar (GPR) is an efficient and nondestructive geophysical method with great potential for detecting soil water content at the farmland scale. However, a key challenge in soil detection is obtaining soil water content rapidly and in real-time. In recent years, deep learning methods have become more widespread in the earth sciences, making it possible to use them for soil water content inversion from GPR data. In this paper, we propose a neural network framework GPRSW based on deep learning of GPR data. GPRSW is an end-to-end network that directly inverts volumetric soil water content (VSWC) through single-channel GPR data. Synthetic experiments show that GPRSW accurately identifies different VSWC boundaries in the model in time depth. The predicted VSWC and model fit well within 40 ns, with a maximum error after 40 ns of less than 0.10 cm3 × cm−3. To validate our method, we conducted GPR measurements at the experimental field of the Academy of Agricultural Sciences in Gongzhuling City, Jilin Province and applied GPRSW to VSWC measurements. The results show that predicted values of GPRSW match with field soil samples and are consistent with the overall trend of the TDR soil probe samples, with a maximum difference not exceeding 0.03 cm3 × cm−3. Therefore, our study shows that GPRSW has the potential to be applied to obtain soil water content from GPR data on farmland. [ABSTRACT FROM AUTHOR]

Visual SLAM (VSLAM) has been developing rapidly due to its advantages of low-cost sensors, the easy fusion of other sensors, and richer environmental information. Traditional visionbased SLAM research has made many achievements, but it may fail to achieve wished results in challenging environments. Deep learning has promoted the development of computer vision, and the combination of deep learning and SLAM has attracted more and more attention. Semantic information, as high-level environmental information, can enable robots to better understand the surrounding environment. This paper introduces the development of VSLAM technology from two aspects: traditional VSLAM and semantic VSLAM combined with deep learning. For traditional VSLAM, we summarize the advantages and disadvantages of indirect and direct methods in detail and give some classical VSLAM open-source algorithms. In addition, we focus on the development of semantic VSLAM based on deep learning. Starting with typical neural networks CNN and RNN, we summarize the improvement of neural networks for the VSLAM system in detail. Later, we focus on the help of target detection and semantic segmentation for VSLAM semantic information introduction. We believe that the development of the future intelligent era cannot be without the help of semantic technology. Introducing deep learning into the VSLAM system to provide semantic information can help robots better perceive the surrounding environment and provide people with higher-level help. [ABSTRACT FROM AUTHOR]

The detection of degraded soil distribution areas is an urgent task. It is difficult and very time consuming to solve this problem using ground methods. The modeling of degradation processes based on digital elevation models makes it possible to construct maps of potential degradation, which may differ from the actual spatial distribution of degradation. The use of remote sensing data (RSD) for soil degradation detection is very widespread. Most often, vegetation indices (indicative botany) have been used for this purpose. In this paper, we propose a method for constructing soil maps based on a multi-temporal analysis of the bare soil surface (BSS). It is an alternative method to the use of vegetation indices. The detection of the bare soil surface was carried out using the spectral neighborhood of the soil line (SNSL) technology. For the automatic recognition of BSS on each RSD image, computer vision based on deep machine learning (neural networks) was used. A dataset of 244 BSS distribution masks on 244 Landsat 4, 5, 7, and 8 scenes over 37 years was developed. Half of the dataset was used as a training sample (Landsat path/row 173/028). The other half was used as a test sample (Landsat path/row 174/027). Binary masks were sufficient for recognition. For each RSD pixel, value "1" was set when determining the BSS. In the absence of BSS, value "0" was set. The accuracy of the machine prediction of the presence of BSS was 75%. The detection of degradation was based on the average long-term spectral characteristics of the RED and NIR bands. The coefficient Cmean, which is the distance of the point with the average long-term values of RED and NIR from the origin of the spectral plane RED/NIR, was calculated as an integral characteristic of the mean long-term values. Higher long-term average values of spectral brightness served as indicators of the spread of soil degradation. To test the method of constructing soil degradation maps based on deep machine learning, an acceptance sample of 133 Landsat scenes of path/row 173/026 was used. On the territory of the acceptance sample, ground verifications of the maps of the coefficient Cmean were carried out. Ground verification showed that the values of this coefficient make it possible to estimate the content of organic matter in the plow horizon (R2 = 0.841) and the thickness of the humus horizon (R2 = 0.8599). In total, 80 soil pits were analyzed on an area of 649 ha on eight agricultural fields. Type I error (false positive) of degradation detection was 17.5%, and type II error (false negative) was 2.5%. During the determination of the presence of degradation by ground methods, 90% of the ground data coincided with the detection of degradation from RSD. Thus, the quality of machine learning for BSS recognition is sufficient for the construction of soil degradation maps. The SNSL technology allows us to create maps of soil degradation based on the long-term average spectral characteristics of the BSS. [ABSTRACT FROM AUTHOR]

Precision viticulture systems are essential for enhancing traditional intensive viticulture, achieving high-quality results, and minimizing costs. This study explores the integration of Unmanned Aerial Vehicles (UAVs) and artificial intelligence in precision viticulture, focusing on vine detection and vineyard zoning. Vine detection employs the YOLO (You Only Look Once) deep learning algorithm, achieving a remarkable 90% accuracy by analysing UAV imagery with various spectral ranges from various phenological stages. Vineyard zoning, achieved through the application of the K-means algorithm, incorporates geospatial data such as the Normalized Difference Vegetation Index (NDVI) and the assessment of nitrogen, phosphorus, and potassium content in leaf blades and petioles. This approach enables efficient resource management tailored to each zone's specific needs. The research aims to develop a decision-support model for precision viticulture. The proposed model demonstrates a high vine detection accuracy and defines management zones with variable weighting factors assigned to each variable while preserving location information, revealing significant differences in variables. The model's advantages lie in its rapid results and minimal data requirements, offering profound insights into the benefits of UAV application for precise vineyard management. This approach has the potential to expedite decision making, allowing for adaptive strategies based on the unique conditions of each zone. [ABSTRACT FROM AUTHOR]

Terrace detection and ridge extraction from high-resolution remote sensing imagery are crucial for soil conservation and grain production on sloping land. Traditional methods use low-to-medium resolution images, missing detailed features and lacking automation. Terrace detection and ridge extraction are closely linked, with each influencing the other's outcomes. However, most studies address these tasks separately, overlooking their interdependence. This research introduces a cutting-edge, multi-scale, and multi-task deep learning framework, termed DTRE-Net, designed for comprehensive terrace information extraction. This framework bridges the gap between terrace detection and ridge extraction, executing them concurrently. The network incorporates residual networks, multi-scale fusion modules, and multi-scale residual correction modules to enhance the model's robustness in feature extraction. Comprehensive evaluations against other deep learning-based semantic segmentation methods using GF-2 terraced imagery from two distinct areas were undertaken. The results revealed intersection over union (IoU) values of 85.18% and 86.09% for different terrace morphologies and 59.79% and 73.65% for ridges. Simultaneously, we have confirmed that the connectivity of results is improved when employing multi-task learning for ridge extraction compared to directly extracting ridges. These outcomes underscore DTRE-Net's superior capability in the automation of terrace and ridge extraction relative to alternative techniques. [ABSTRACT FROM AUTHOR]

This study investigates the potential of Sentinel-1 (S1) multi-temporal data for the early-season mapping of the rapeseed crop. Additionally, we explore the effectiveness of limiting the portion of a considered time series to map rapeseed fields. To this end, we conducted a quantitative analysis to assess several temporal windows (periods) spanning different phases of the rapeseed phenological cycle in the following two scenarios relating to the availability or constraints of providing ground samples for different years: (i) involving the same year for both training and the test, assuming the availability of ground samples for each year; and (ii) evaluating the temporal transferability of the classifier, considering the constraints of ground sampling. We employed two different classification methods that are renowned for their high performance in land cover mapping: the widely adopted random forest (RF) approach and a deep learning-based convolutional neural network, specifically the InceptionTime algorithm. To assess the classification outcomes, four evaluation metrics (recall, precision, F1 score, and Kappa) were employed. Using S1 time series data covering the entire rapeseed growth cycle, the tested algorithms achieved F1 scores close to 95% on same-year training and testing, and 92.0% when different years were used, both algorithms demonstrated robust performance. For early rapeseed detection within a two-month window post-sowing, RF and InceptionTime achieved F1 scores of 67.5% and 77.2%, respectively, and 79.8% and 88.9% when extended to six months. However, in the context of temporal transferability, both classifiers exhibited mean F1 scores below 50%. Notably, a 5-month time series, covering key growth stages such as stem elongation, inflorescence emergence, and fruit development, yielded a mean F1 score close to 95% for both algorithms when trained and tested in the same year. In the temporal transferability scenario, RF and InceptionTime achieved mean F1 scores of 92.0% and 90.0%, respectively, using a 5-month time series. Our findings underscore the importance of a concise S1 time series for effective rapeseed mapping, offering advantages in data storage and processing time. Overall, the study establishes the robustness of RF and InceptionTime in rapeseed detection scenarios, providing valuable insights for agricultural applications. [ABSTRACT FROM AUTHOR]

Building-change detection underpins many important applications, especially in the military and crisis-management domains. Recent methods used for change detection have shifted towards deep learning, which depends on the quality of its training data. The assembly of large-scale annotated satellite imagery datasets is therefore essential for global building-change surveillance. Existing datasets almost exclusively offer near-nadir viewing angles. This limits the range of changes that can be detected. By offering larger observation ranges, the scroll imaging mode of optical satellites presents an opportunity to overcome this restriction. This paper therefore introduces S2Looking, a building-change-detection dataset that contains large-scale side-looking satellite images captured at various off-nadir angles. The dataset consists of 5000 bitemporal image pairs of rural areas and more than 65,920 annotated instances of changes throughout the world. The dataset can be used to train deep-learning-based change-detection algorithms. It expands upon existing datasets by providing (1) larger viewing angles; (2) large illumination variances; and (3) the added complexity of rural images. To facilitate the use of the dataset, a benchmark task has been established, and preliminary tests suggest that deep-learning algorithms find the dataset significantly more challenging than the closest-competing near-nadir dataset, LEVIR-CD+. S2Looking may therefore promote important advances in existing building-change-detection algorithms. [ABSTRACT FROM AUTHOR]

In this paper, we present neural network methods for predicting uncertainty in atmospheric remote sensing. These include methods for solving the direct and the inverse problem in a Bayesian framework. In the first case, a method based on a neural network for simulating the radiative transfer model and a Bayesian approach for solving the inverse problem is proposed. In the second case, (i) a neural network, in which the output is the convolution of the output for a noise-free input with the input noise distribution; and (ii) a Bayesian deep learning framework that predicts input aleatoric and model uncertainties, are designed. In addition, a neural network that uses assumed density filtering and interval arithmetic to compute uncertainty is employed for testing purposes. The accuracy and the precision of the methods are analyzed by considering the retrieval of cloud parameters from radiances measured by the Earth Polychromatic Imaging Camera (EPIC) onboard the Deep Space Climate Observatory (DSCOVR). [ABSTRACT FROM AUTHOR]

Satellite image time series (SITS) is a sequence of satellite images that record a given area at several consecutive times. The aim of such sequences is to use not only spatial information but also the temporal dimension of the data, which is used for multiple real-world applications, such as classification, segmentation, anomaly detection, and prediction. Several traditional machine learning algorithms have been developed and successfully applied to time series for predictions. However, these methods have limitations in some situations, thus deep learning (DL) techniques have been introduced to achieve the best performance. Reviews of machine learning and DL methods for time series prediction problems have been conducted in previous studies. However, to the best of our knowledge, none of these surveys have addressed the specific case of works using DL techniques and satellite images as datasets for predictions. Therefore, this paper concentrates on the DL applications for SITS prediction, giving an overview of the main elements used to design and evaluate the predictive models, namely the architectures, data, optimization functions, and evaluation metrics. The reviewed DL-based models are divided into three categories, namely recurrent neural network-based models, hybrid models, and feed-forward-based models (convolutional neural networks and multi-layer perceptron). The main characteristics of satellite images and the major existing applications in the field of SITS prediction are also presented in this article. These applications include weather forecasting, precipitation nowcasting, spatio-temporal analysis, and missing data reconstruction. Finally, current limitations and proposed workable solutions related to the use of DL for SITS prediction are also highlighted. [ABSTRACT FROM AUTHOR]

The images of the Sentinel-2 constellation can help the verification process of farmers' declarations, providing, among other things, accurate spatial explicit maps of the agricultural land cover. The aim of the study is to design, develop, and evaluate two deep learning (DL) architectures tailored for agricultural land cover and crop type mapping. The focus is on a detailed class scheme encompassing fifteen distinct classes, utilizing Sentinel-2 imagery acquired on a monthly basis throughout the year. The study's geographical scope covers a diverse rural area in North Greece, situated within southeast Europe. These architectures are a Temporal Convolutional Neural Network (CNN) and a combination of a Recurrent and a 2D Convolutional Neural Network (R-CNN), and their accuracy is compared to the well-established Random Forest (RF) machine learning algorithm. The comparative approach is not restricted to simply presenting the results given by classification metrics, but it also assesses the uncertainty of the classification results using an entropy measure and the spatial distribution of the classification errors. Furthermore, the issue of sampling strategy for the extraction of the training set is highlighted, targeting the efficient handling of both the imbalance of the dataset and the spectral variability of instances among classes. The two developed deep learning architectures performed equally well, presenting an overall accuracy of 90.13% (Temporal CNN) and 90.18% (R-CNN), higher than the 86.31% overall accuracy of the RF approach. Finally, the Temporal CNN method presented a lower entropy value (6.63%), compared both to R-CNN (7.76%) and RF (28.94%) methods, indicating that both DL approaches should be considered for developing operational EO processing workflows. [ABSTRACT FROM AUTHOR]

Accurate location information has significant commercial and economic value as they are widely used in intelligent manufacturing, material localization and smart homes. Magnetic sequence-based approaches show great promise mainly due to their pervasiveness and stability. However, existing geomagnetic indoor localization methods are facing the problems of location ambiguity and feature extraction deficiency, which will lead to large localization errors. To address these issues, we propose a coarse-to-fine geomagnetic indoor localization method based on deep learning. First, a multidimensional geomagnetic feature extraction method is presented which can extract magnetic features from spatial and temporal aspects. Then, a hierarchical deep neural network model is devised to extract more accurate geomagnetic information and corresponding location clues for more accurate localization. Finally, localization is achieved through a particle filter combined with IMU localization. To evaluate the performance of the proposed methods, we carried out several experiments at three trial paths with two heterogeneous devices, Vivo X30 and Huawei Mate30. Experimental results demonstrate that the proposed algorithm can achieve more accurate localization performance than the state-of-the-art methods. Meanwhile, the proposed algorithm has low cost and good pervasiveness for different devices. [ABSTRACT FROM AUTHOR]

Data-driven deep learning has been well applied in radar target detection. However, the performance of the detection network is severely degraded when the detection scene changes, since the trained network with the data from one scene is not suitable for another scene with different data distribution. In order to address this problem, an adaptive network detector combined with scene classification is proposed in this paper. Aiming at maximizing the posterior probability of the feature vectors, the scene classification network is arranged to control the output ratio of a group of detection sub-networks. Due to the uncertainty of classification error rate in traditional machine learning, the classifier with a controllable false alarm rate is constructed. In addition, a new network training strategy, which freezes the parameters of the scene classification network and selectively fine-tunes the parameters of detection sub-networks, is proposed for the adaptive network structure. Comprehensive experiments are carried out to demonstrate that the proposed method guarantees a high detection probability when the detection scene changes. Compared with some classical detectors, the adaptive network detector shows better performance. [ABSTRACT FROM AUTHOR]

Soil degradation processes are widespread on agricultural land. Ground-based methods for detecting degradation require a lot of labor and time. Remote methods based on the analysis of vegetation indices can significantly reduce the volume of ground surveys. Currently, machine learning methods are increasingly being used to analyze remote sensing data. In this paper, the task is set to apply deep machine learning methods and methods of vegetation indices calculation to automate the detection of areas of soil degradation development on arable land. In the course of the work, a method was developed for determining the location of degraded areas of soil cover on arable fields. The method is based on the use of multi-temporal remote sensing data. The selection of suitable remote sensing data scenes is based on deep machine learning. Deep machine learning was based on an analysis of 1028 scenes of Landsats 4, 5, 7 and 8 on 530 agricultural fields. Landsat data from 1984 to 2019 was analyzed. Dataset was created manually for each pair of "Landsat scene"/"agricultural field number"(for each agricultural field, the suitability of each Landsat scene was assessed). Areas of soil degradation were calculated based on the frequency of occurrence of low NDVI values over 35 years. Low NDVI values were calculated separately for each suitable fragment of the satellite image within the boundaries of each agricultural field. NDVI values of one-third of the field area and lower than the other two-thirds were considered low. During testing, the method gave 12.5% of type I errors (false positive) and 3.8% of type II errors (false negative). Independent verification of the method was carried out on six agricultural fields on an area of 713.3 hectares. Humus content and thickness of the humus horizon were determined in 42 ground-based points. In arable land degradation areas identified by the proposed method, the probability of detecting soil degradation by field methods was 87.5%. The probability of detecting soil degradation by ground-based methods outside the predicted regions was 3.8%. The results indicate that deep machine learning is feasible for remote sensing data selection based on a binary dataset. This eliminates the need for intermediate filtering systems in the selection of satellite imagery (determination of clouds, shadows from clouds, open soil surface, etc.). Direct selection of Landsat scenes suitable for calculations has been made. It allows automating the process of constructing soil degradation maps. [ABSTRACT FROM AUTHOR]

The availability of freshwater is becoming a global concern. Because agricultural consumption has been increasing steadily, the mapping of irrigated areas is key for supporting the monitoring of land use and better management of available water resources. In this paper, we propose a method to automatically detect and map center pivot irrigation systems using U-Net, an image segmentation convolutional neural network architecture, applied to a constellation of PlanetScope images from the Cerrado biome of Brazil. Our objective is to provide a fast and accurate alternative to map center pivot irrigation systems with very high spatial and temporal resolution imagery. We implemented a modified U-Net architecture using the TensorFlow library and trained it on the Google cloud platform with a dataset built from more than 42,000 very high spatial resolution PlanetScope images acquired between August 2017 and November 2018. The U-Net implementation achieved a precision of 99% and a recall of 88% to detect and map center pivot irrigation systems in our study area. This method, proposed to detect and map center pivot irrigation systems, has the potential to be scaled to larger areas and improve the monitoring of freshwater use by agricultural activities. [ABSTRACT FROM AUTHOR]

The long-term spectral characteristics of the bare soil surface (BSS) in the BLUE, GREEN, RED, NIR, SWIR1, and SWIR2 Landsat spectral bands are poorly studied. Most often, the RED and NIR spectral bands are used to analyze the spatial heterogeneity of the soil cover; in our opinion, it is outmoded and seems unreasonable. The study of multi-temporal spectral characteristics requires the processing of big remote sensing data based on artificial intelligence in the form of convolutional neural networks. The analysis of BSS belongs to the direct methods of analysis of the soil cover. Soil degradation can be detected by ground methods (field reconnaissance surveys), modeling, or digital methods, and based on the remote sensing data (RSD) analysis. Ground methods are laborious, and modeling gives indirect results. RSD analysis can be based on the principles of calculation of vegetation indices (VIs) and on the BSS identification. The calculation of VIs also provides indirect information about the soil cover through the state of vegetation. BSS analysis is a direct method for analyzing soil cover heterogeneity. In this work, the informativeness of the long-term (37 years) average spectral characteristics of the BLUE, GREEN, RED, NIR, SWIR1 and SWIR2 bands of the Landsat 4–8 satellites for detecting areas of soil degradation with recognition of the BSS using deep machine learning methods was estimated. The objects of study are the spectral characteristics of kastanozems (dark chestnut soils) in the south of Russia in the territory of the Morozovsky district of the Rostov region. Soil degradation in this area is mainly caused by erosion. The following methods were used: retrospective monitoring of soil and land cover, deep machine learning using convolutional neural networks, and cartographic analysis. Six new maps of the average long-term spectral brightness of the BSS have been obtained. The information content of the BSS for six spectral bands has been verified on the basis of ground surveys. The informativeness was determined by the percentage of coincidences of degradation facts identified during the RSD analysis, and those determined in the field. It has been established that the spectral bands line up in the following descending order of information content: RED, NIR, GREEN, BLUE, SWIR1, SWIR2. The accuracy of degradation maps by band was determined as: RED—84.6%, NIR—82.9%, GREEN—78.0%, BLUE—78.0%, SWIR1—75.5%, SWIR2—62.2%. [ABSTRACT FROM AUTHOR]

Different satellite images may consist of variable numbers of channels which have different resolutions, and each satellite has a unique revisit period. For example, the Landsat-8 satellite images have 30 m resolution in their multispectral channels, the Sentinel-2 satellite images have 10 m resolution in the pan-sharp channel, and the National Agriculture Imagery Program (NAIP) aerial images have 1 m resolution. In this study, we propose a simple yet effective arithmetic deep model for multimodal temporal remote sensing image fusion. The proposed model takes both low- and high-resolution remote sensing images at t 1 together with low-resolution images at a future time t 2 from the same location as inputs and fuses them to generate high-resolution images for the same location at t 2 . We propose an arithmetic operation applied to the low-resolution images at the two time points in feature space to take care of temporal changes. We evaluated the proposed model on three modality pairs for multimodal temporal image fusion, including downsampled WorldView-2/original WorldView-2, Landsat-8/Sentinel-2, and Sentinel-2/NAIP. Experimental results show that our model outperforms traditional algorithms and recent deep learning-based models by large margins in most scenarios, achieving sharp fused images while appropriately addressing temporal changes. [ABSTRACT FROM AUTHOR]

This study investigates the feasibility of applying monoplotting to video data from a security camera and image data from an uncrewed aircraft system (UAS) survey to create a mapping product which overlays traffic flow in a university parking lot onto an aerial orthomosaic. The framework, titled VirtuaLot, employs a previously defined computer-vision pipeline which leverages Darknet for vehicle detection and tests the performance of various object tracking algorithms. Algorithmic object tracking is sensitive to occlusion, and monoplotting is applied in a novel way to efficiently extract occluding features from the video using a digital surface model (DSM) derived from the UAS survey. The security camera is also a low fidelity model not intended for photogrammetry with unstable interior parameters. As monoplotting relies on static camera parameters, this creates a challenging environment for testing its effectiveness. Preliminary results indicate that it is possible to manually monoplot between aerial and perspective views with high degrees of transition tilt, achieving coordinate transformations between viewpoints within one deviation of vehicle short and long axis measurements throughout 70.5% and 99.6% of the study area, respectively. Attempted automation of monoplotting on video was met with limited success, though this study offers insight as to why and directions for future work on the subject. [ABSTRACT FROM AUTHOR]

This article describes the development of a machine learning (ML)-based algorithm for snowfall retrieval (Snow retrievaL ALgorithm fOr gpM–Cross Track, SLALOM-CT), exploiting ATMS radiometer measurements and using the CloudSat CPR snowfall products as references. During a preliminary analysis, different ML techniques (tree-based algorithms, shallow and convolutional neural networks—NNs) were intercompared. A large dataset (three years) of coincident observations from CPR and ATMS was used for training and testing the different techniques. The SLALOM-CT algorithm is based on four independent modules for the detection of snowfall and supercooled droplets, and for the estimation of snow water path and snowfall rate. Each module was designed by choosing the best-performing ML approach through model selection and optimization. While a convolutional NN was the most accurate for the snowfall detection module, a shallow NN was selected for all other modules. SLALOM-CT showed a high degree of consistency with CPR. Moreover, the results were almost independent of the background surface categorization and the observation angle. The reliability of the SLALOM-CT estimates was also highlighted by the good results obtained from a direct comparison with a reference algorithm (GPROF). [ABSTRACT FROM AUTHOR]

Visual SLAM (VSLAM) has been developing rapidly due to its advantages of low-cost sensors, the easy fusion of other sensors, and richer environmental information. Traditional visionbased SLAM research has made many achievements, but it may fail to achieve wished results in challenging environments. Deep learning has promoted the development of computer vision, and the combination of deep learning and SLAM has attracted more and more attention. Semantic information, as high-level environmental information, can enable robots to better understand the surrounding environment. This paper introduces the development of VSLAM technology from two aspects: traditional VSLAM and semantic VSLAM combined with deep learning. For traditional VSLAM, we summarize the advantages and disadvantages of indirect and direct methods in detail and give some classical VSLAM open-source algorithms. In addition, we focus on the development of semantic VSLAM based on deep learning. Starting with typical neural networks CNN and RNN, we summarize the improvement of neural networks for the VSLAM system in detail. Later, we focus on the help of target detection and semantic segmentation for VSLAM semantic information introduction. We believe that the development of the future intelligent era cannot be without the help of semantic technology. Introducing deep learning into the VSLAM system to provide semantic information can help robots better perceive the surrounding environment and provide people with higher-level help.

Surface ocean dynamics play a key role in the Earth system, contributing to regulate its climate and affecting the marine ecosystem functioning. Dynamical processes occur and interact in the upper ocean at multiple scales, down to, or even less than, few kilometres. These scales are not adequately resolved by present observing systems, and, in the last decades, global monitoring of surface currents has been based on the application of geostrophic balance to absolute dynamic topography maps obtained through the statistical interpolation of along-track satellite altimeter data. Due to the cross-track distance and repetitiveness of satellite acquisitions, the effective resolution of interpolated data is limited to several tens of kilometres. At the kilometre scale, sea surface temperature pattern evolution is dominated by advection, providing indirect information on upper ocean currents. Computer vision techniques are perfect candidates to infer this dynamical information from the combination of altimeter data, surface temperature images and observing-system geometry. Here, we exploit one class of image processing techniques, super-resolution, to develop an original neural-network architecture specifically designed to improve absolute dynamic topography reconstruction. Our model is first trained on synthetic observations built from a numerical general-circulation model and then tested on real satellite products. Provided concurrent clear-sky thermal observations are available, it proves able to compensate for altimeter sampling/interpolation limitations by learning from primitive equation data. The algorithm can be adapted to learn directly from future surface topography, and eventual surface currents, high-resolution satellite observations. [ABSTRACT FROM AUTHOR]

Many natural disasters in South America are linked to meteorological phenomena. Therefore, forecasting and monitoring climatic events are fundamental issues for society and various sectors of the economy. In the last decades, machine learning models have been developed to tackle different issues in society, but there is still a gap in applications to applied physics. Here, different machine learning models are evaluated for precipitation prediction over South America. Currently, numerical weather prediction models are unable to precisely reproduce the precipitation patterns in South America due to many factors such as the lack of region-specific parametrizations and data availability. The results are compared to the general circulation atmospheric model currently used operationally in the National Institute for Space Research (INPE: Instituto Nacional de Pesquisas Espaciais), Brazil. Machine learning models are able to produce predictions with errors under 2 mm in most of the continent in comparison to satellite-observed precipitation patterns for different climate seasons, and also outperform INPE's model for some regions (e.g., reduction of errors from 8 to 2 mm in central South America in winter). Another advantage is the computational performance from machine learning models, running faster with much lower computer resources than models based on differential equations currently used in operational centers. Therefore, it is important to consider machine learning models for precipitation forecasts in operational centers as a way to improve forecast quality and to reduce computation costs. [ABSTRACT FROM AUTHOR]

Total Cloud Cover (TCC) retrieval from ground-based optical imagery is a problem that has been tackled by several generations of researchers. The number of human-designed algorithms for the estimation of TCC grows every year. However, there has been no considerable progress in terms of quality, mostly due to the lack of systematic approach to the design of the algorithms, to the assessment of their generalization ability, and to the assessment of the TCC retrieval quality. In this study, we discuss the optimization nature of data-driven schemes for TCC retrieval. In order to compare the algorithms, we propose a framework for the assessment of the algorithms' characteristics. We present several new algorithms that are based on deep learning techniques: A model for outliers filtering, and a few models for TCC retrieval from all-sky imagery. For training and assessment of data-driven algorithms of this study, we present the Dataset of All-Sky Imagery over the Ocean (DASIO) containing over one million all-sky optical images of the visible sky dome taken in various regions of the world ocean. The research campaigns that contributed to the DASIO collection took place in the Atlantic ocean, the Indian ocean, the Red and Mediterranean seas, and the Arctic ocean. Optical imagery collected during these missions are accompanied by standard meteorological observations of cloudiness characteristics made by experienced observers. We assess the generalization ability of the presented models in several scenarios that differ in terms of the regions selected for the train and test subsets. As a result, we demonstrate that our models based on convolutional neural networks deliver a superior quality compared to all previously published approaches. As a key result, we demonstrate a considerable drop in the ability to generalize the training data in the case of a strong covariate shift between the training and test subsets of imagery which may occur in the case of region-aware subsampling. [ABSTRACT FROM AUTHOR]

To precisely forecast downstream water levels in catchment areas during typhoons, the deep learning artificial neural networks were employed to establish two water level forecasting models using sequential neural networks (SNNs) and multiple-input functional neural networks (MIFNNs). SNNs, which have a typical neural network structure, are network models constructed using sequential methods. To develop a network model capable of flexibly consolidating data, MIFNNs are employed for processing data from multiple sources or with multiple dimensions. Specifically, when images (e.g., radar reflectivity images) are used as input attributes, feature extraction is required to provide effective feature maps for model training. Therefore, convolutional layers and pooling layers were adopted to extract features. Long short-term memory (LSTM) layers adopted during model training enabled memory cell units to automatically determine the memory length, providing more useful information. The Hsintien River basin in northern Taiwan was selected as the research area and collected relevant data from 2011 to 2019. The input attributes comprised one-dimensional data (e.g., water levels at river stations, rain rates at rain gauges, and reservoir release) and two-dimensional data (i.e., radar reflectivity mosaics). Typhoons Saola, Soudelor, Dujuan, and Megi were selected, and the water levels 1 to 6 h after the typhoons struck were forecasted. The results indicated that compared with linear regressions (REG), SNN using dense layers (SNN-Dense), and SNN using LSTM layers (SNN-LSTM) models, superior forecasting results were achieved for the MIFNN model. Thus, the MIFNN model, as the optimal model for water level forecasting, was identified. [ABSTRACT FROM AUTHOR]

In Earth observation (EO), large-scale land-surface dynamics are traditionally analyzed by investigating aggregated classes. The increase in data with a very high spatial resolution enables investigations on a fine-grained feature level which can help us to better understand the dynamics of land surfaces by taking object dynamics into account. To extract fine-grained features and objects, the most popular deep-learning model for image analysis is commonly used: the convolutional neural network (CNN). In this review, we provide a comprehensive overview of the impact of deep learning on EO applications by reviewing 429 studies on image segmentation and object detection with CNNs. We extensively examine the spatial distribution of study sites, employed sensors, used datasets and CNN architectures, and give a thorough overview of applications in EO which used CNNs. Our main finding is that CNNs are in an advanced transition phase from computer vision to EO. Upon this, we argue that in the near future, investigations which analyze object dynamics with CNNs will have a significant impact on EO research. With a focus on EO applications in this Part II, we complete the methodological review provided in Part I. [ABSTRACT FROM AUTHOR]

Deep learning (DL) has great influence on large parts of science and increasingly established itself as an adaptive method for new challenges in the field of Earth observation (EO). Nevertheless, the entry barriers for EO researchers are high due to the dense and rapidly developing field mainly driven by advances in computer vision (CV). To lower the barriers for researchers in EO, this review gives an overview of the evolution of DL with a focus on image segmentation and object detection in convolutional neural networks (CNN). The survey starts in 2012, when a CNN set new standards in image recognition, and lasts until late 2019. Thereby, we highlight the connections between the most important CNN architectures and cornerstones coming from CV in order to alleviate the evaluation of modern DL models. Furthermore, we briefly outline the evolution of the most popular DL frameworks and provide a summary of datasets in EO. By discussing well performing DL architectures on these datasets as well as reflecting on advances made in CV and their impact on future research in EO, we narrow the gap between the reviewed, theoretical concepts from CV and practical application in EO. [ABSTRACT FROM AUTHOR]

This work presents quantitative prediction of severity of the disease caused by Phytophthora infestans in potato crops using machine learning algorithms such as multilayer perceptron, deep learning convolutional neural networks, support vector regression, and random forests. The machine learning algorithms are trained using datasets extracted from multispectral data captured at the canopy level with an unmanned aerial vehicle, carrying an inexpensive digital camera. The results indicate that deep learning convolutional neural networks, random forests and multilayer perceptron using band differences can predict the level of Phytophthora infestans affectation on potato crops with acceptable accuracy. [ABSTRACT FROM AUTHOR]