Search

Dwelling information is essential for humanitarian emergency response during or in the aftermath of disasters, especially in temporary settlement areas hosting forcibly displaced people. To map dwellings, the integration of very high-resolution remotely sensed imagery in computer vision models plays a key role. However, state-of-the-art deep learning models have two known downsides: 1) lack of generalization across space and time under changing scenes and object characteristics, and 2) extensive demand for annotated samples for training and validation. Both could pose a critical challenge during an emergency. To bypass this problem, this study deals with unsupervised domain adaptation for instance segmentation using a single-stage instance segmentation model, namely segmenting objects by location (SOLO). The goal is to adapt a SOLO model trained on a labeled source domain to detect dwellings in an unlabeled target domain. In this context, we study three domain adaptation techniques based on adversarial learning, domain discrepancy, and domain alignment mapping. We also propose domain similarity at different levels to understand its implication on domain adaptation. Experiments are conducted on very high-resolution satellite images obtained from four temporary settlement areas located in different countries and exhibiting various spatial characteristics. Analysis results show that in most source–target combinations unsupervised domain adaptation improves the performance by a large margin even surpassing a model trained with supervised learning. There is also an observed performance deviation among implemented strategies and different source–target dataset combinations. From the in-depth analysis of domain similarity at the image, object, and deep feature space levels, the former is more correlated with unsupervised domain adaptation performance.

Change detection from traditional 2D optical images has limited capability to model the changes in the height or shape of objects. Change detection using 3D point cloud from photogrammetry or LiDAR surveying can fill this gap by providing critical depth information. While most existing machine learning based 3D point cloud change detection methods are supervised, they severely depend on the availability of annotated training data, which is in practice a critical point. To circumnavigate this dependence, we propose an unsupervised 3D point cloud change detection method mainly based on self-supervised learning using deep clustering and contrastive learning. The proposed method also relies on an adaptation of deep change vector analysis to 3D point cloud via nearest point comparison. Experiments conducted on an aerial LiDAR survey dataset show that the proposed method obtains higher performance in comparison to the traditional unsupervised methods, with a gain of about 9% in mean accuracy (to reach more than 85%). Thus, it appears to be a relevant choice in scenario where prior knowledge (labels) is not ensured. The code will be made available at https://github.com/IdeGelis/torch-points3d-SSL-DCVA.

In this article, we develop a novel feature extraction method that combines two well-established mathematical morphology concepts: watersheds and morphological attribute profiles (APs). In order to extract spatial-spectral features from remote sensing data, APs were originally defined as sequences of filtering operators on inclusion trees, i.e., the max- and min-trees, computed from the input image. In this study, we extend the AP paradigm to the more general framework of hierarchical watersheds. Moreover, we explore the semantic knowledge provided by labeled training pixels during different phases of the watershed-AP construction, namely within the construction of hierarchical watersheds from the raw image and later within the filtering of the resulting hierarchy. We illustrate the relevance of the proposed method with two applications including land cover classification and building extraction using optical remote sensing images. Experimental results show that the new profiles outperform various existing features using two public datasets (Zurich and Vaihingen), thus providing another high potential feature extraction method within the AP family.

Despite the popularity of deep neural networks in various domains, the extraction of digital terrain models (DTMs) from airborne laser scanning (ALS) point clouds is still challenging. This might be due to the lack of the dedicated large-scale annotated dataset and the data-structure discrepancy between point clouds and DTMs. To promote data-driven DTM extraction, this article collects from open sources a large-scale dataset of ALS point clouds and corresponding DTMs with various urban, forested, and mountainous scenes. A baseline method is proposed as the first attempt to train a deep neural network to extract DTMs directly from ALS point clouds via rasterization techniques, coined DeepTerRa. Extensive studies with well-established methods are performed to benchmark the dataset and analyze the challenges in learning to extract DTM from point clouds. The experimental results show the interest of the agnostic data-driven approach, with submetric error level compared to methods designed for DTM extraction. The data and source code are available online at https://lhoangan.github.io/deepterra/ for reproducibility and further similar research.

Partiendo de diferentes ejemplos de géneros musicales y bailes de América latina, del caribe y de África como son la cumbia y la chilena afromexicanas, el gwoka guadalupano, el assiko de Senegal, la champeta colombiana y el n’dombolo congoleño, intentamos elaborar unas primeras pistas para diseñar nuevos puentes de comprensión entre África y América. La idea es intentar aplicar un breve análisis trasatlántico afrodiaspórico. Esta visión nos permite declinar una genealogía epistemológica entre África, América latina y el Caribe en cuanto a las llamadas “Américas Negras”, lo cual nos proporcionará otra lectura de los procesos de mestizajes culturales vistos muchas veces como una dilución de los orígenes.

The papers in this special section focus on the technology of urban remote sending. The rapid growth and the multiple changes of the urban environments pose unique challenges to cities across the globe. Due to the high rates of urbanization on our planet, it is often argued that the future of humanity will be decided in cities. This means that innovative solutions are required to develop new ideas and concepts to make cities resilient, sustainable, inclusive, and prosperous at the same time. To ensure sustainable urban development, however, the basis is first of all knowledge about our cities. Efficient monitoring is required of various aspects of the evolution of urban form, for assessing and forecasting interactions with other environments and resources, and for the development of applied solutions to the various urban challenges in terms of planning, safety, health, infrastructure, service provision, etc. Remote sensing is the tool to gain geoinformation in a consistent and systematic way, basically anywhere across the globe, and with it to fill in still existing large data gaps on cities on our planet. With the ever-growing number and types of sensors, recent advances in technologies (e.g., geospatial remote sensing, unmanned aerial vehicles, autonomous vehicles, hand-held devices), and large volumes of data available from volunteered geographical information, Internet of Things based systems, etc., urban sensing and modeling remains a thrilling field of research with a more promising outlook for developing practical solutions for such continuously evolving ecosystems.

Vehicle detection is an important but challenging problem in Earth observation due to the intricately small sizes and varied appearances of the objects of interest. In this paper, we use these issues to our advantage by considering them results of latent image augmentation. In particular, we propose using supervised contrastive loss in combination with a mutual guidance matching process to helps learn stronger object representations and tackles the misalignment of localization and classification in object detection. Extensive experiments are performed to understand the combination of the two strategies and show the benefits for vehicle detection on aerial and satellite images, achieving performance on par with state-of-the-art methods designed for small and very small object detection. As the proposed method is domain-agnostic, it might also be used for visual representation learning in generic computer vision problems.

Human activities in the sea, such as intensive fishing and exploitation of offshore wind farms, may impact negatively on the marine mega fauna. As an attempt to control such impacts, surveying, and tracking of marine animals are often performed on the sites where those activities take place. Nowadays, thank to high resolution cameras and to the development of machine learning techniques, tracking of wild animals can be performed remotely and the analysis of the acquired images can be automatized using state-of-the-art object detection models. However, most state-of-the-art detection methods require lots of annotated data to provide satisfactory results. Since analyzing thousands of images acquired during a flight survey can be a cumbersome and time consuming task, we focus in this article on the weakly supervised detection of marine animals. We propose a modification of the patch distribution modeling method (PaDiM), which is currently one of the state-of-the-art approaches for anomaly detection and localization for visual industrial inspection. In order to show its effectiveness and suitability for marine animal detection, we conduct a comparative evaluation of the proposed method against the original version, as well as other state-of-the-art approaches on two high-resolution marine animal image datasets. On both tested datasets, the proposed method yielded better F1 and recall scores (75% recall/41% precision, and 57% recall/60% precision, respectively) when trained on images known to contain no object of interest. This shows a great potential of the proposed approach to speed up the marine animal discovery in new flight surveys. Additionally, such a method could be adopted for bounding box proposals to perform faster and cheaper annotation within a fully-supervised detection framework.

In recent years, numerous deep learning techniques have been proposed to tackle the semantic segmentation of aerial and satellite images, increase trust in the leaderboards of main scientific contests and represent the current state-of-the-art. Nevertheless, despite their promising results, these state-of-the-art techniques are still unable to provide results with the level of accuracy sought in real applications, i.e., in operational settings. Thus, it is mandatory to qualify these segmentation results and estimate the uncertainty brought about by a deep network. In this work, we address uncertainty estimations in semantic segmentation. To do this, we relied on a Bayesian deep learning method, based on Monte Carlo Dropout, which allows us to derive uncertainty metrics along with the semantic segmentation. Built on the most widespread U-Net architecture, our model achieves semantic segmentation with high accuracy on several state-of-the-art datasets. More importantly, uncertainty maps are also derived from our model. While they allow for the performance of a sounder qualitative evaluation of the segmentation results, they also include valuable information to improve the reference databases.

Esta contribución parte de una experiencia de docencia acerca de las clases de literatura y civilización afrodiaspóricas de la sección de Lengua Española y Civilizaciones Hispánicas de la Université Gaston Berger de Saint Louis en Senegal. Al llegar a Senegal en el 2016 hizo falta escoger una obra para incluirla en el programa de estudio de la sección y fue casi naturalmente que escogí a Changó, el gran putas, como homenaje a su génesis que según se cuenta tuvo lugar en este mismo país. Partiré de la presentación de los currículos y orientaciones de las clases, de la visión didáctica de ellas para luego adentrarme en el análisis de la recepción que hicieron los estudiantes de dicha obra. Con este análisis pretendo mostrar otra recepción de la obra Changó, el gran putas en un contexto no diaspórico. Esta mirada desde las costas africanas, espero, revelará otras perspectivas de análisis y sobre todo mostrará la necesidad de desarrollar estudios afrodiaspóricos (África- Afrolatino-Caribe-América).

In the context of rapid urbanization, monitoring the evolution of cities is crucial. To do so, 3D change detection and characterization is of capital importance since, unlike 2D images, 3D data contain vertical information of utmost importance to monitoring city evolution (that occurs along both horizontal and vertical axes). Urban 3D change detection has thus received growing attention, and various methods have been published on the topic. Nevertheless, no quantitative comparison on a public dataset has been reported yet. This study presents an experimental comparison of six methods: three traditional (difference of DSMs, C2C and M3C2), one machine learning with hand-crafted features (a random forest model with a stability feature) and two deep learning (feed-forward and Siamese architectures). In order to compare these methods, we prepared five sub-datasets containing simulated pairs of 3D annotated point clouds with different characteristics: from high to low resolution, with various levels of noise. The methods have been tested on each sub-dataset for binary and multi-class segmentation. For supervised methods, we also assessed the transfer learning capacity and the influence of the training set size. The methods we used provide various kinds of results (2D pixels, 2D patches or 3D points), and each of them is impacted by the resolution of the PCs. However, while the performances of deep learning methods highly depend on the size of the training set, they seem to be less impacted by training on datasets with different characteristics. Oppositely, conventional machine learning methods exhibit stable results, even with smaller training sets, but embed low transfer learning capacities. While the main changes in our datasets were usually identified, there were still numerous instances of false detection, especially in dense urban areas, thereby calling for further development in this field. To assist such developments, we provide a public dataset composed of pairs of point clouds with different qualities together with their change-related annotations. This dataset was built with an original simulation tool which allows one to generate bi-temporal urban point clouds under various conditions.

We present a new approach for matching urban object instances across multiple ground-level images for the ultimate goal of city-scale mapping of objects with high positioning accuracy. What makes this task challenging is the strong change in view-point, different lighting conditions, high similarity of neighboring objects, and variability in scale. We propose to turn object instance matching into a learning task, where image-appearance and geometric relationships between views fruitfully interact. Our approach constructs a Siamese convolutional neural network that learns to match two views of the same object given many candidate image cut-outs. In addition to image features, we propose utilizing location information about the camera and the object to support image evidence via soft geometric constraints. Our method is compared to existing patch matching methods to prove its edge over state-of-the-art. This takes us one step closer to the ultimate goal of city-wide object mapping from street-level imagery to benefit city administration.

This article tackles the problem of detecting small objects in satellite or aerial remote sensing images by relying on super-resolution to increase image spatial resolution, thus the size and details of objects to be detected. We show how to improve the super-resolution framework starting from the learning of a generative adversarial network (GAN) based on residual blocks and then its integration into a cycle model. Furthermore, by adding to the framework an auxiliary network tailored for object detection, we considerably improve the learning and the quality of our final super-resolution architecture, and more importantly increase the object detection performance. Besides the improvement dedicated to the network architecture, we also focus on the training of super-resolution on target objects, leading to an object-focused approach. Furthermore, the proposed strategies do not depend on the choice of a baseline super-resolution framework, hence could be adopted for current and future state-of-the-art models. Our experimental study on small vehicle detection in remote sensing data conducted on both aerial and satellite images (i.e., ISPRS Potsdam and xView datasets) confirms the effectiveness of the improved super-resolution methods to assist with the small object detection tasks.

Object detection from aerial and satellite remote sensing images has been an active research topic over the past decade. Thanks to the increase in computational resources and data availability, deep learning-based object detection methods have achieved numerous successes in computer vision, and more recently in remote sensing. However, the ability of current detectors to deal with (very) small objects still remains limited. In particular, the fast detection of small objects from a large observed scene is still an open question. In this work, we address this challenge and introduce an enhanced one-stage deep learning-based detection model, called You Only Look Once (YOLO)-fine, which is based on the structure of YOLOv3. Our detector is designed to be capable of detecting small objects with high accuracy and high speed, allowing further real-time applications within operational contexts. We also investigate its robustness to the appearance of new backgrounds in the validation set, thus tackling the issue of domain adaptation that is critical in remote sensing. Experimental studies that were conducted on both aerial and satellite benchmark datasets show some significant improvement of YOLO-fine as compared to other state-of-the art object detectors.

The 6th biennial conference on object-based image analysis—GEOBIA 2016—took place in September 2016 at the University of Twente in Enschede, The Netherlands (see www [...]

The Geographic Object-Based Image Analysis (GEOBIA) paradigm relies strongly on the segmentation concept, i.e., partitioning of an image into regions or objects that are then further analyzed. Segmentation is a critical step, for which a wide range of methods, parameters and input data are available. To reduce the sensitivity of the GEOBIA process to the segmentation step, here we consider that a set of segmentation maps can be derived from remote sensing data. Inspired by the ensemble paradigm that combines multiple weak classifiers to build a strong one, we propose a novel framework for combining multiple segmentation maps. The combination leads to a fine-grained partition of segments (super-pixels) that is built by intersecting individual input partitions, and each segment is assigned a segmentation confidence score that relates directly to the local consensus between the different segmentation maps. Furthermore, each input segmentation can be assigned some local or global quality score based on expert assessment or automatic analysis. These scores are then taken into account when computing the confidence map that results from the combination of the segmentation processes. This means the process is less affected by incorrect segmentation inputs either at the local scale of a region, or at the global scale of a map. In contrast to related works, the proposed framework is fully generic and does not rely on specific input data to drive the combination process. We assess its relevance through experiments conducted on ISPRS 2D Semantic Labeling. Results show that the confidence map provides valuable information that can be produced when combining segmentations, and fusion at the object level is competitive w.r.t. fusion at the pixel or decision level.

Land cover mapping has benefited a lot from the introduction of the Geographic Object-Based Image Analysis (GEOBIA) paradigm, that allowed to move from a pixelwise analysis to a processing of elements with richer semantic content, namely objects or regions. However, this paradigm requires to define an appropriate scale, that can be challenging in a large-area study where a wide range of landscapes can be observed. We propose here to conduct the multiscale analysis based on hierarchical representations, from which features known as differential attribute profiles are derived over each single pixel. Efficient and scalable algorithms for construction and analysis of such representations, together with an optimized usage of the random forest classifier, provide us with a semi-supervised framework in which a user can drive mapping of elements such as Small Woody Features at a very large area. Indeed, the proposed open-source methodology has been successfully used to derive a part of the High Resolution Layers (HRL) product of the Copernicus Land Monitoring service, thus showing how the GEOBIA framework can be used in a big data scenario made of more than 38,000 Very High Resolution (VHR) satellite images representing more than 120 TB of data.

The theory of hierarchical image representations has been well-established in Mathematical Morphology, and provides a suitable framework to handle images through objects or regions taking into account their scale. Such approaches have increased in popularity and been favourably compared to treating individual image elements in various domains and applications. This survey paper presents the development of hierarchical image representations over the last 20 years using the framework of component trees. We introduce two classes of component trees, partitioning and inclusion trees, and describe their general characteristics and differences. Examples of hierarchies for each of the classes are compared, with the resulting study aiming to serve as a guideline when choosing a hierarchical image representation for any application and image domain.

The geographic object-based image analysis (GEOBIA) framework has gained increasing interest for the last decade. One of its key advantages is the hierarchical representation of an image, where object topological features can be extracted and modeled in the form of structured data. We thus propose to use a structured kernel relying on the concept of bag of subpaths to directly cope with such features. The kernel can be approximated using random Fourier features, allowing it to be applied on a large structure size (the number of objects in the structured data) and large volumes of data (the number of pixels or regions for training). With the so-called scalable bag of subpaths kernel (SBoSK), we also introduce a novel multi-source classification approach performing machine learning directly on a hierarchical image representation built from two images at different resolutions under the GEOBIA framework. Experiments run on an urban classification task show that the proposed approach run on a single image improves the classification overall accuracy in comparison with conventional approaches from 2% to 5% depending on the training set size and that fusing two images allows a supplementary 4% accuracy gain. Additional evaluations on public available large-scale datasets illustrate further the potential of SBoSK, with overall accuracy rates improvement ranging from 1% to 11% depending on the considered setup.

The rapidly increasing volume of visual Earth Observation data calls for effective content based image retrieval solutions, specifically tailored for their high spatial resolution and heterogeneous content. In this paper, we address this issue with a novel local implementation of the well-known morphological descriptors called pattern spectra. They are computationally efficient histogram-like structures describing the global distribution of arbitrarily defined attributes of connected image components. Besides employing pattern spectra for the first time in this context, our main contribution lies in their dense calculation, at a local scale, thus enabling their combination with sophisticated visual vocabulary strategies. The Merced Landuse/Landcover dataset has been used for comparing the proposed strategy against alternative global and local content description methods, where the introduced approach is shown to yield promising performances.