Search

Neural architecture search (NAS) is a subset of automated machine learning that tries to find the best neural network to perform a given task. In this article, a network search space is defined and applied to perform the semantic segmentation of satellite imagery. Due to the spatial nature of the data, the search space uses cells that group parallel operations with kernels of different sizes, providing options to accommodate the neighborhood information required to perform a better classification. The architecture search space follows a UNet-like network. The proposed approach uses scaled sigmoid gates, a strategy for network pruning that was adapted to search for the best operations on the cell search space. The architecture achieved by the proposed approach uses wider kernels on lower resolution feature maps, which leads to the interpretation that some pixels required information from pixels farther away than expected. The resulting network was compared to a very similar UNet-like network that only used 3×3 convolutions. The resulting network shows slightly better results on the test set.

Conditional (cGAN)-based methods proposed so far for synthetic aperture radar (SAR)-to-optical image synthesis tend to produce noisy and unsharp optical outcomes. In this work, we propose the atrous-cGAN, a novel cGAN architecture that improves the SAR-to-optical image translation. The proposed generator and discriminator networks rely on atrous convolutions and incorporate an atrous spatial pyramid pooling (ASPP) module to enhance fine details in the generated optical image by exploiting spatial context at multiple scales. This letter reports experiments carried out to assess the performance of atrous-cGAN for the synthesis of Landsat-8 images from Sentinel-1A data based on three public data sets. The experimental analysis indicated that the atrous-cGAN consistently outperformed the classical pix2pix counterpart in terms of visual quality, similar to the true optical image, and as a feature learning tool for semantic segmentation.

Crop recognition in tropical regions is a challenging task because of the highly complex crop dynamics, with multiple crops per year. Nevertheless, most automatic methods proposed thus far are devoted to temperate areas where normally a single crop is cultivated along the crop year. This paper introduces convolutional recurrent networks for crop recognition in areas characterized by complex spatiotemporal dynamics typical of tropical agriculture, where a per date classification is required. The proposed networks consist of two sequential steps. First, a deep network simultaneously models spatial and temporal contexts. Second, a post-processing algorithm enforces prior knowledge about the crop dynamics in the target area based on the posterior probabilities computed in the first step. The paper proposes deep network architectures that join a fully convolutional network (FCN) for modeling spatial context at multiple levels and a bidirectional recurrent neural network to explore the temporal context. The recurrent network is configured as N-to-N, where N is the sequence length. This allows it to produce classification outcomes for the entire sequence of multi-temporal images using a single network. Different network designs are proposed based on three FCN architectures: U-Net, dense network, and Atrous Spatial Pyramid Pooling. A convolutional Long-Short-Term-Memory (ConvLSTM) accounts for sequence modeling, whereas the Most Likely Class Sequence (MLCS) algorithm is adopted for enforcing prior knowledge. The paper finally reports experiments conducted on Sentinel-1 data of two publicly available datasets from different tropical regions. The experimental results indicated that the proposed architectures outperformed state-of-the-art methods based on recurrent networks in terms of Overall Accuracy and per-class F1 score.

Early detection of deforestation processes is vital to maintain and regulate tropical rainforests, such as in the Amazon region. Most of them rely on optical imagery. Approaches based on Synthetic Aperture Radar (SAR) data are comparatively unexplored, in particular for deforestation detection in tropical rainforests. This work addresses this gap and evaluates Fully Convolutional Networks based on the U-Net, Res-Unet and Siamese Network, for deforestation detection using images from three different sensors, Landsat-8, Sentinel-2, and Sentinel-1. Experiments conducted on a dataset of the Amazon rainforest indicated that Fully Convolutional Networks working on Sentinel-1 data can achieve sufficient accuracy for detecting deforestation in tropical rainforests when clouds prevent the use of optical data11The source code is available in https://github.zcom/MabelOrtega/Comparison-of-Optical-and-SAR-data-for-deforestation-mapping-in-the-Amazon-Forest-with-FCN.

This paper introduces a novel hybrid N-to-N bidirectional ConvLSTM for multi-temporal crop recognition in areas characterized by highly complex crop dynamics as in tropical regions. The proposed method seeks to explore both spatial and temporal context of the data by adding a bidirectional ConvLSTM to every skip connection in the U-Net and applying the fully connected conditional random fields (CRF) methodology as a post-processing of the predictions. We evaluated our method on a publicly available tropical region dataset, from Sentinel-1 data, obtaining an average improvement of almost 4% in the Average F1 Score and 1.7% in Overall Accuracy, when comparing with a state-of-the art method.

A tecnica denominada de classificacao baseada em objetos, proposta por Baatz & Shape (2000), trata-se de uma abordagem de processamento de imagens em que a unidade primitiva e o objeto, composto de varios pixels. Softwares proprietarios como o eCognition® e de open source como o InterIMAGE realizam o processamento de imagens baseadas em objetos considerando o alto grau de relacoes mutuas e acoes em diferentes escalas, como informacoes de contexto, estrutura semântica e hierarquica. O objetivo principal da pesquisa foi demonstrar e avaliar resultados da integracao de sistemas open source com o sistema de classificacao denominado InterIMAGE. Os sistemas utilizados nessa pesquisa foram o sistema de gerenciamento de banco de dados objeto-relacional PostgreSQL/PostGIS Raster, biblioteca TerraLib, pacote computacional de sistema de informacoes geograficas QGIS e linguagem de programacao C++. Foi utilizada uma imagem do satelite GeoEye-1 de 2013 de uma area urbana do municipio de Goianesia no estado de Goias. Foi desenvolvida uma interface (API - Application Programming Interface) no sistema InterIMAGE para realizar a segmentacao multiresolucao em ambiente de banco de dados espaciais. A segmentacao processada utilizou-se da API com a imagem armazenada no PostgreSQL e em disco rigido, enquanto a classificacao foi efetuada somente no InterIMAGE. O indice Kappa foi utilizado para indicacao da acuracia dos resultados alcancados na classificacao, utilizando-se os parâmetros da segmentacao da API, obtendo-se um valor de 0,412. As regras da arvore de decisao devem ser modificadas para realizacao de novos experimentos visando verificar a influencia no processamento da classificacao no InterIMAGE. Apesar da ocorrencia de algumas confusoes tematicas no processo de classificacao, demonstrou-se a viabilidade de continuacao de desenvolvimento de aplicacoes de codigo aberto para o InterIMAGE.

This work follows a line of research dedicated to the parallelization of image segmentation algorithms on distributed computing environments, which is motivated by the increasing resolutions and availability of Remote Sensing (RS) images. Here we focus on region-growing segmentation, which is regarded as a time consuming and demanding approach in terms of computational resources. Its parallelization is a complex problem since it usually affects the final outcome in comparison to what would be delivered by a sequential solution. This is due to the fact that subdividing an image to perform segmentation of its tiles concurrently usually introduces undesirable artifacts near to the borders of the image tiles. Additional processing steps are then required to properly stitch together the segments alongside tiles borders in order to eliminate such artifacts. In this work we evaluated alternative implementations of a previously proposed region-growing distributed segmentation approach, which was originally built on top of the Hadoop distributed computing framework. We developed a new implementation of the approach, which was built with the Spark framework, and compared its performance with that of the original implementation. In this investigation RS images of various sizes were processed using different configurations of a physical computer cluster. We evaluated computational performances and accessed the differences among the segmentation outcomes generated by the alternative implementations. We also assessed the stability of the implementations by comparing the segmentations produced with different cluster configurations. Although the approach is, in principle, suitable to any region growing algorithm, the experiments were performed with a particular segmentation method, and the results showed that the Spark implementation consistently outperformed the Hadoop counterpart, bringing in most cases a significant improvement in terms of processing time. The experiment results also attested the stability of the distributed segmentation approach, as very similar results were produced with the alternative implementations, running on different cluster configurations.

Recently, recurrent neural networks have been proposed for crop mapping from multitemporal remote sensing data. Most of these proposals have been designed and tested in temperate regions, where a single harvest per season is the rule. In tropical regions, the favorable climate and local agricultural practices, such as crop rotation, result in more complex spatio-temporal dynamics, where the single harvest per season assumption does not hold. In this context, a demand arises for methods capable of recognizing agricultural crops at multiple dates along the multitemporal sequence. In the present work, we propose to adapt two recurrent neural networks, originally conceived for single harvest per season, for multidate crop recognition. In addition, we propose a novel multidate approach based on bidirectional fully convolutional recurrent neural networks. These three architectures were evaluated on public Sentinel-1 data sets from two tropical regions in Brazil. In our experiments, all methods achieved state-of-the-art accuracies with a clear superiority of the proposed architecture. It outperformed its counterparts in up to 3.8% and 7.4%, in terms of per-month overall accuracy, and it was the best performing method in terms of F1-score for most crops and dates on both regions.

Visual object tracking aims to deliver accurate estimates about the state of the target in a sequence of images or video frames. Nevertheless, tracking algorithms are sensitive to different kinds of image perturbations that frequently cause tracking failures. Indeed, tracking failures result from the insertion of imprecise target-related data into the trackers’ appearance models, which leads the trackers to lose the target or drift away from it. Here, we propose a tracking fusion approach, which incorporates feedback and re-initialization mechanisms to improve overall tracking performance. Our fusion technique, called SymTE-TR, enhances trackers’ overall performance by updating their appearances models with reliable information of the target’s states, while resets the imprecise trackers. We evaluated our approach on a facial video dataset, which characterizes a particular challenging tracking application under different imaging conditions. The experimental results indicate that our approach contributes to enhancing individual tracker performances by providing stable results across the video sequences and, consequently, contributes to stable overall tracking fusion performances.

Knowing the spatial distribution of endangered tree species in a forest ecosystem or forest remnants is a valuable information to support environmental conservation practices. The use of Unmanned Aerial Vehicles (UAVs) offers a suitable alternative for this task, providing very high-resolution images at low costs. In parallel, recent advances in the computer vision field have led to the development of effective deep learning techniques for end-to-end semantic image segmentation. In this scenario, the DeepLabv3+ is well established as the state-of-the-art deep learning method for semantic segmentation tasks. The present paper proposes and assesses the use of DeepLabv3+ for mapping the threatened Dipteryx alata Vogel tree, popularly also known as cumbaru. We also compare two backbone networks for feature extraction in the DeepLabv3+ architecture: the Xception and MobileNetv2. Experiments carried out on a dataset consisting of 225 UAV/RGB images of an urban area in Midwest Brazil demonstrated that DeepLabv3+ was able to achieve in mean overall accuracy and Fl-score above 90%, and IoU above 80%. The experimental analysis also pointed out that the MobileNetv2 backbone overcame its counterpart by a wide margin due to its comparatively simpler architecture in view of the available training data.

Advances in the development of Earth observation data acquisition systems have led to the continuously growing production of remote sensing datasets, for which timely analysis has become a major ch...

Spatially explicit information on urban tree species distribution is crucial for green infrastructure management in cities. This information is usually acquired with ground-based surveys, which are time-consuming and usually cover limited spatial extents. The combination of machine learning algorithms and remote sensing images has been hailed as a promising way to map tree species over broad areas. Recently, convolutional neural networks (CNNs), a type of deep learning method, have achieved outstanding results for tree species discrimination in various remote sensing data types. However, there is a lack of studies using CNN-based methods to produce tree species composition maps, particularly for tropical urban settings. Here, we propose a multi-task CNN to map tree species in a highly diverse neighborhood in Rio de Janeiro, Brazil. Our network architecture takes an aerial photograph (RGB bands and pixel size = 0.15 m) and delivers two outputs: a semantically segmented image and a distance map transform. In the former, all pixel positions are labeled, while in the latter, each pixel position contains the Euclidean distance to the crown boundary. We developed a post-processing approach that combines the two outputs, and we classified nine and five tree species with an average F1-score of 79.3 ± 8.6% and 87.6 ± 4.4%, respectively. Moreover, our post-processing approach produced a realistic tree species composition map by labeling only pixels of the target species with high class membership probabilities. Our results show the potential of CNNs and aerial photographs to map tree species in highly diverse tropical urban settings, providing valuable insights for urban forest management and green spaces planning.

This paper proposes a new distributed architecture for supervised classification of large volumes of earth observation data on a cloud computing environment. The architecture supports distributed execution, network communication, and fault tolerance in a transparent way to the user. The architecture is composed of three abstraction layers, which support the definition and implementation of applications by researchers from different scientific investigation fields. The implementation of architecture is also discussed. A software prototype (available online), which runs machine learning routines implemented on the cloud using the Waikato Environment for Knowledge Analysis (WEKA), a popular free software licensed under the GNU General Public License, is used for validation. Performance issues are addressed through an experimental analysis in which two supervised classifiers available in WEKA were used: random forest and support vector machines. This paper further describes how to include other classification methods in the available software prototype.

The rapid increase in the number of aerial and orbital Earth observation systems is generating a huge amount of remote sensing data that need to be readily transformed into useful information for policy and decision makers. This exposes an urgent demand for image interpretation tools that can deal efficiently with very large volumes of data. In this work, we introduce a set of methods that support distributed processing of georeferenced raster and vector data in a computer cluster, which may be a virtual cluster provided by cloud computing infrastructure services. The set of methods comprise a particular technique for indexing distributed georeferenced datasets, as well as strategies for distributing efficiently the processing of spatial context-aware operations. They provide the means for the development of scalable applications, capable of processing large volumes of geospatial data. We evaluated the proposed methods in a remote sensing image interpretation application, built on the MapReduce framework, and executed in a cloud computing infrastructure. The experimental results corroborate the capacity of the methods to support efficient handling of very large earth observation datasets.

This paper proposes a distributed strategy for region-growing segmentation of arbitrarily large images. The strategy is suited for most distributed architectures, including cloud-computing environments. It focuses on handling very large data and tackles the main problem of tiled-based segmentation: artifacts that occur in the segments adjacent to tile borders. Our proposal is to use a specific indexing mechanism and a hierarchical stitching method to suppress segmentation artifacts. The strategy was validated by a series of experiments carried out on a virtual cluster in a commercial cloud-computing infrastructure, over WorldView-2 images of different sizes. The experiments demonstrated that the proposed strategy is able to eliminate tile-based segmentation artifacts. The experimental analysis further showed that the strategy brings significant gains in terms of processing time, particularly for very large images, as more processing units are added to the distributed processing infrastructure.

This letter evaluates metaheuristics for the supervised parameter tuning of multiresolution-region-growing segmentation. Three groups of metaheuristics are tested in terms of convergence speed and solution quality. Generalized pattern search, mesh adaptive direct search, and Nelder–Mead represent the single-solution group. Differential evolution (DE) represents the population group. DE followed by each of the aforementioned single-solution metaheuristics represents the hybrid metaheuristic group. This letter reveals that the optimization objective functions typically have countless local minima, many of them leading to very poor solutions. Experiments on three data sets demonstrated that single-solution-based methods often lead to a solution with unacceptable quality. DE was less susceptible to be stuck in local minima when compared to single-solution methods, but it was slower in reaching the minima. Moreover, hybrid methods presented the best tradeoff between accuracy and convergence speed.

Agriculture monitoring is a key task for producers, governments and decision makers. The analysis of multitemporal remote sensing data provides a cost-effective way to perform this task. Recurrent Neural Networks (RNNs) have been successfully used in temporal modeling problems, while Convolutional Neural Networks (CNNS) are the state-of-the-art in image classification, mainly due to their ability to capture spatial context. In this work, we propose the use of a hybrid network architecture for crop mapping that combines RNNs and CNNs. We evaluate this architecture experimentally upon a Sentinel-1A database from a tropical region in Brazil. The ability of recurrent networks to model temporal context is compared with the conventional image stacking approach. The impact of using CNN learned features rather than context aware handcrafted features is also investigated. In our analysis the hybrid architecture achieved better average class accuracy than alternative approaches based on image stacking and GLCM features.

This work presents a dense fully convolutional architecture for crop type recognition from multitemporal RS images. Basically, we adapted a dense fully convolutional net to deal with stacks of multitemporal data. The proposed approach was tested upon a public dataset comprising two Sentinel-1A sequences from a tropical region in South America. We took as baseline a dense convolutional network designed for patch classification. Thematic and spatial accuracy, as well as the computational load were evaluated experimentally. The proposed architecture matched the baseline in terms of recognition rates and proved to be very efficient computationally in the inference phase.

Visual tracking is a challenging task due to a number of factors, such as occlusions, deformations, illumination variations and abrupt motion changes present in a video sequence. Generally, trackers are robust to some of these factors, but do not achieve satisfactory results when dealing with multiple factors at the same time. More robust results when multiple factors are present can be obtained by combining the results of different trackers. In this paper we propose a multiple tracker fusion method, named Symbiotic Tracker Ensemble with Feedback Learning (SymTE-FL), which combines the results of a set of trackers to produce a unified tracking estimate. The novelty of the method consists in providing feedback to the individual trackers, so that they can correct their own estimates, thus improving overall tracking accuracy. The proposal is validated by experiments conducted upon a publicly available database. The results show that the proposed method delivered in average more accurate tracking estimates than those obtained with individual trackers running independently and with the original approach.

This letter proposes a parallel version for graphics processing units (GPU) of a region-growing image segmentation algorithm widely used by the geographic object-based image analysis (GEOBIA) community. Initially, all image pixels are considered as seeds or primitive segments. Fine-grained parallel threads assigned to individual pixels merge adjacent segments iteratively always ensuring to minimize the overall heterogeneity increase. Besides spectral features the merging criterion considers morphological features that can be efficiently computed in the underlying GPU architecture. Two alternatives using different merging criteria are proposed and tested. An experimental analysis upon five different test images has shown that the parallel algorithm may run up to 19 times faster than its sequential counterpart.

Este artigo propõe um algoritmo paralelo de segmentação de imagens por crescimento de região voltado a Unidades de Processamento Gráfico (GPU). O algoritmo proposto deriva de um algoritmo sequencial largamente utilizado pela comunidade de Análise de Imagens de sensoriamento remoto Baseada em Objeto Geográfico (GEOBIA). Relativamente à versão sequencial propõem-se neste trabalho novos atributos para caracterização de heterogeneidade morfológica de segmentos, cujo cálculo pode ser realizado de modo mais eficiente em GPUs. Duas variantes do algoritmo paralelo com diferentes heurísticas para seleção dos segmentos adjacentes a serem fundidos a cada iteração são descritas. Visando explorar o potencial de GPUs para execução paralela de threads de baixa granularidade, o algoritmo proposto atribui uma thread para cada pixel da imagem, o que contribui ao mesmo tempo para uma distribuição mais uniforme da carga computacional entre os processadores da GPU. Uma detalhada análise experimental utilizando uma GPU convencional sobre quatro imagens de teste indicou acelerações superiores a 8 em relação ao algoritmo sequencial. This paper proposes a parallel image segmentation algorithm for Graphical Processing Units (GPU) that follows the region-growing paradigm. The proposal can be regarded as a parallel version of a sequential algorithm widely used by the Geographic Object Based Image Analysis community (GEOBIA). In relation to the sequential version this, work presents new attributes to characterize segments' morphological heterogeneity, whose computation can be performed by GPUs more efficiently than the original ones. Two variants of the parallel algorithm with distinct heuristics for the selection of adjacent segments to be merged in each iteration are described. Aiming at exploring the potential of GPUs for the parallel execution of fine grained threads, the new parallel method assigns a thread to each image pixel. This also contributes to better load balance among the GPU processors. A detailed experimental analysis using a simple GPU upon four different test images has shown that the parallel algorithm may run 8 times faster than its sequential counterpart or even more than that.

This work proposes and evaluates strategies based on Stacked Supervised Auto-Encoders (SSAE) for face representation in video surveillance applications. The study focuses on the identification task with a single sample per person (SSPP) in the gallery. Variations in terms of pose, facial expression, illumination and occlusion are approached in two ways. First, the SSAE extracts features from face images, which are robust to such variations. Second, we propose methods to exploit the multiple samples per persons probes (MSPPP) that can be extracted from video sequences. Three variants of the proposed method are compared upon HONDA/UCSD and VIDTIMIT video datasets. The experimental results demonstrate that strategies combining SSAE and MSPPP are able to outperform other SSPP methods, such a local binary patterns, in face recognition from video.

The rapid increase in the number and in the spatial resolution of aerial and orbital Earth observation systems is generating a huge amount of remote sensing data that need to be readily transformed into useful information for policy and decision makers. A possible approach to tackle the demand for image interpretation tools that can deal efficiently with very large volumes of data is to employ data analysis methods based on distributed computing. This paper presents an object-based, remote sensing image interpretation application executed over cloud-computing infrastructure. The application is implemented with InterCloud, a novel image interpretation platform designed to run on computer grids (physical clusters or cloud-computing infrastructure). The application described in this paper is a land cover/land use classification of a pansharpened GeoEye-1 image, with 19k by 23k pixels. The image covers an area of the municipality of Goianesia, in Goias State, Brazil. The site contains sparse urban areas intermixed with rural areas and natural patches of the Brazilian Cerrado biome. Eleven classes of objects, including urban, rural and Cerrado reminiscent targets were considered. In addition to the accuracies of the classification result, in this work we evaluate the scalability capability of InterCloud by performing different runs of the application with different configurations of the cloud infrastructure, in which we vary the number of computing nodes.

Motivated by the rapidly increase of remote sensing data over the last years, this paper presents a tool designed for distributed image segmentation. InterSeg is able to handle efficiently very large high-resolution images using scalable distributed segmentation methods over computer clusters. Through a graphic user interface, InterSeg hides the distributed processing complexity, allowing the user to easily perform distributed image segmentation on a cloud-computing environment. Experiments carried out with the tool attest its potential scalability and its capability to exploit distributed processing over many cluster nodes. Moreover, as an open-source solution, the software can be extended with different segmentation algorithms and methods.

This paper introduces the interpretation meta knowledge model devised for the InterCloud platform. InterCloud is a remote sensing image interpretation platform designed to run on computer clusters or on cloud computing infrastructure. The system is capable of distributing data processing tasks, such as segmentation, feature extraction and classification procedures over the processing elements of a computer grid in a transparent way to the user. Moreover, InterCloud can exploit the potential scalability offered by commercial cloud computing infrastructure services, enabling the interpretation of very large remote sensing datasets in an efficient way. The proposed meta model comprises two types of knowledge: declarative and procedural. The former describes the characteristics of the classes of objects expected to be found in the scene to be interpreted, and the relationships among those classes. The latter describes the functions and procedures that should be applied over the data in order to achieve the desired interpretation. In the proposed knowledge model, the user expresses declarative knowledge through the definition of an ontology, so-called descriptive ontology, which conveys the formal naming and definition of the properties and interrelationships of the object classes in a particular application. Procedural knowledge is expressed by the so-called task ontology, which is represented by a directed graph, in which the nodes represent operations over the input images or over the segments generated by segmentation operations. Besides segmentation, crisp or fuzzy classification operations can be defined by the user. The graph edges define the data flow between operations, which are triggered by the control process as soon as their inputs are produced by the preceding operations. In this paper we illustrate the main components of the meta knowledge model through a theoretical application.

Image segmentation is a critical step in image analysis, and usually involves a high computational cost, especially when dealing with large volumes of data. Given the significant increase in the spatial, spectral and temporal resolutions of remote sensing imagery in the last years, current sequential and parallel solutions fail to deliver the expected performance and scalability. This work proposes a scalable and efficient segmentation method, capable of handling efficiently very large high resolution images. The proposed solution is based on the MapReduce model, which offers a highly scalable and reliable framework for storing and processing massive data in cloud computing environments. The solution was implemented and validated using the Hadoop platform. Experimental results attest the viability of performing region growing segmentation in the MapReduce framework.

The Segmentation Parameter Tuner (SPT) is a tool designed for automatic tuning of segmentation parameters. In SPT, the goodness of a set of parameter values is given by the level of agreement between the segmentation result and a given reference (representing the desired outcome) quantified by a metric selected by the user (empirical discrepancy methods). This metric is used as the fitness function of an optimization algorithm that searches the parameter space for the minimum value, which is expected to correspond to the segmentation outcome most similar to the reference. SPT 3.1 offers many interesting features such as: five segmentation algorithms (for Optical, Hyperspectral and SAR images), four optimization algorithms (stochastic and direct search optimization methods) and seven discrepancy metrics (pixel and object-based). This paper describes the optimization procedure underlying SPT 3.1, the features added to this version as well as an experiment that illustrates the operation of the tool.