Search

We propose a deep fully convolutional neural network with a new type of layer, named median layer, to restore images contaminated by the salt-and-pepper (s&p) noise. A median layer simply performs median filtering on all feature channels. By adding this kind of layer into some widely used fully convolutional deep neural networks, we develop an end-to-end network that removes the extremely high-level s&p noise without performing any non-trivial preprocessing tasks, which is different from all the existing literature in s&p noise removal. Experiments show that inserting median layers into a simple fully-convolutional network with the L2 loss significantly boosts the signal-to-noise ratio. Quantitative comparisons testify that our network outperforms the state-of-the-art methods with a limited amount of training data. The source code has been released for public evaluation and use (https://github.com/llmpass/medianDenoise).

Accurately estimating the position of static objects, such as traffic lights, from the moving camera of a self-driving car is a challenging problem. In this work, we present a system that improves the localization of static objects by jointly-optimizing the components of the system via learning. Our system is comprised of networks that perform: 1) 5DoF object pose estimation from a single image, 2) association of objects between pairs of frames, and 3) multi-object tracking to produce the final geo-localization of the static objects within the scene. We evaluate our approach using a publicly-available data set, focusing on traffic lights due to data availability. For each component, we compare against contemporary alternatives and show significantly-improved performance. We also show that the end-to-end system performance is further improved via joint-training of the constituent models. Code is available at: https://github.com/MedChaabane/Static_Objects_Geolocalization.

With the increased spatial resolution of current sensor constellations, more details are captured about our changing planet, enabling the recognition of a greater range of land use/land cover classes. While pixeland object-based classification approaches are widely used for extracting information from imagery, recent studies have shown the importance of spatial contexts for discriminating more specific and challenging classes. This paper proposes a new compact representation for the fast query/classification of compound structures from very high resolution optical remote sensing imagery. This bag-of-features representation relies on the multiscale segmentation of the input image and the quantization of image structures pooled into visual word distributions for the characterization of compound structures. A compressed form of the visual word distributions is described, allowing adaptive and fast queries/classification of image patterns. The proposed representation and the query methodology are evaluated for the classification of the UC Merced 21-class data set, for the detection of informal settlements and for the discrimination of challenging agricultural classes. The results show that the proposed representation competes with state-of-the-art techniques. In addition, the complexity analysis demonstrates that the representation requires about 5% of the image storage space while allowing us to perform queries at a speed down to 1 s/ 1000 km 2 /CPU for 2-m multispectral data.

Building footprints (BFP) provide useful visual context for users of digital maps when navigating in space. This paper proposes a method for extracting and symbolizing building footprints from satellite imagery using a convolutional neural network (CNN). The CNN architecture outputs rotated rectangles, providing a symbolized approximation that works well for small buildings. Experiments are conducted on the four cities in the DeepGlobe Challenge dataset (Las Vegas, Paris, Shanghai, Khartoum). Our method performs best on suburbs consisting of individual houses. These experiments show that either large buildings or buildings without clear delineation produce weaker results in terms of precision and recall.

This chapter presents a selection of image representation methods for very high spatial resolution optical data acquired by agile satellite platforms. Each method aims at specific properties of the image information content and can be tailored to address unique features in spatial, temporal, and angular acquisitions. Techniques for the identification and characterization of surface structures and objects often employ spatial and spectral features best represented in panchromatic and multi-spectral images, respectively. In both cases, the vastness of the data space can only be addressed effectively by means of some data representation structure that organizes the image information content in meaningful ways. The latter suggest that a globally optimal representation of the object(s) of interest can be obtained through interactions with a scale space as opposed to single-scale information layer. Two examples, the Max-Tree and Alpha-Tree algorithms, are discussed in the context of interactive big data information mining. Optical and structural properties of the surface materials can be exploited by analyzing the tempo-angular domain by means of anisotropic decompositions, which rely on the availability of surface reflectance data and dense angular sampling. Bidirectional reflectance distribution functions of various materials are discussed in detail, showing that the temporal information should always be coupled to the corresponding angular component to make the best use of the available imagery.

Connective segmentation based on the definition of a dissimilarity measure on pairs of adjacent pixels is an appealing framework to develop new hierarchical segmentation methods. Usually, the dissimilarity is fully determined by the intensity values of the considered pair of adjacent pixels, so that it is independent of the values of the other image pixels. In this paper, we explore dissimilarity measures depending on the overall image content encapsulated in its local mutual information and show its invariance to information preserving transforms. This is investigated in the framework of the connective segmentation and constrained connectivity paradigms and leads to the concept of dependent connectivities. An efficient probability estimator based on depth functions is proposed to handle multi-dimensional images. Experiments conducted on hyper-spectral and multiangular remote sensing images highlight the robustness of the proposed approach., JRC.G.2-Global security and crisis management

Roofless buildings are encountered in case of conflicts and disasters as well as construction sites. A methodology for characterizing and counting roofless buildings in very high resolution optical images is presented. Using morphological transform to extract specific image components, the proposed method spatially aggregates them in a fuzzy logic framework. The result is a map of roofless building membership. This membership map is then used with partial ground truth to estimate the total number of roofless buildings. Finally, the methods are validated with a WorldView panchromatic image, where the membership map and the counting estimate show good accuracies., JRC.G.2-Global security and crisis management

The cognitive radio paradigm is based on the ability of sensing the radio environment in order to make informed decisions. This paper describes the effects of sensing on the cognitive radio channels capacity region. Sensing is modeled as a compression channel, which results in partial knowledge of the primary messages at the cognitive transmitter. This model enables to impose constraints on the sensing strategy. First, the dirty paper channel capacity is derived when the channel encoder knows partially the side information. Then, the capacity area of the Gaussian cognitive channel with partial information is derived. Finally, numerical results illustrate the capacity reduction associated with constrained sensing, in comparison to the capacity of the cognitive radio channel.

Satellite imagery is a valuable source of information for assessing damages in distressed areas undergoing a calamity, such as an earthquake or an armed conflict. However, the sheer amount of data required to be inspected for this assessment makes it impractical to do it manually. To address this problem, we present a semi-supervised learning framework for large-scale damage detection in satellite imagery. We present a comparative evaluation of our framework using over 88 million images collected from 4, 665 KM2 from 12 different locations around the world. To enable accurate and efficient damage detection, we introduce a novel use of hierarchical shape features in the bags-of-visual words setting. We analyze how practical factors such as sun, sensor-resolution, satellite-angle, and registration differences impact the effectiveness our proposed representation, and compare it to five alternative features in multiple learning settings. Finally, we demonstrate through a user-study that our semi-supervised framework results in a ten-fold reduction in human annotation time at a minimal loss in detection accuracy compared to manual inspection

During the 1990's, most work in Earth Observation (EO) data management focused on the design of catalogues and single mission (i.e., sensor) data repositories for enabling geospatial (i.e., boundin...

An interscale learning paradigm for global HR/VHR image information extraction is presented. The paradigm relies on the information matching between a priori global knowledge and features derived from high resolution imagery to perform adaptive high resolution land classification. Unlike traditional machine learning techniques, this strategy avoids the costly collection of local training datasets and the local parameter tuning and it enables the full automation of the information extraction process.

The location and identification of human settlements is a key information in any assessment related to human security and safety decision process, from the preparedness to and mitigation of natural hazards to postdisaster response and reconstruction. The human settlements can be defined as infrastructures build on earth surface for living, trading or any social activity. In order to cover and analyze large areas where scarce and heterogeneous data may exist, the automatic information extraction from Earth Observation is the only sustainable strategy. Several studies about the detection of built-up presence from high resolution satellite were performed during the past few years. Among the various methods, a built-up index aggregating anisotropic textural co-occurence measures (PanTex) was demonstrated to be a robust indicator, where 54 high resolution optical images representing cities distributed around the world are processed. The validation of the built-up presence index emphasized its robustness against the type of constructions, the sun angles and the seasons of acquisitions. In this paper, we propose an alternative procedure for the calculation of built-up presence index from panchromatic high resolution satellite images. While radiometric descriptors of human settlements are highly variable across the world and with illumination conditions, the shape of settlements often contains right angles. This property being more stable, we propose a built-up index based on the density of corners. The index is obtained in two steps: - corners are detected by multi scale Harris detector based on differential morphological decompositions; - corners are spatially aggregated to form a density of corners, which is the built-up index. The differential morphological decomposition is a scale-space representation of the image, where image elements are separated by their scales. Then, the Harris corner indicator, which is highly dependent on a scale parameter, can be adapted to a set of scales. The output of the corner detection is a set of points associated to a scale, which represents the right angles in the image. The corners associated to a scale in the range of admissible settlements dimensions are selected and spatially aggregated to derive a density of corners. The proposed index is extracted from various high resolution panchromatic images and it is compared to the PanTex. In the experimental section, the high correlation between both indicators proves the suitability of the proposed method for consistently detecting built-up presence. Moreover, it gives a new interpretation of the PanTex which is close to a density of corners. Such an observation is critical for understanding the variablity of the PanTex index in between dense urban and industrial areas (industrial areas being composed of large buildings mechanically contain less corners)., JRC.G.2-Global security and crisis management

Edges are crucial descriptors for image analysis and rely mostly on local spectral distances. In this paper, local spectral changes are assimilated to the local statistical dependences which are measured by the local mutual information. This metric is shown to be invariant to unknown bijective transforms which makes it a good candidate for analyzing consistently satellite images which are affected by uncontrolled spectral distortions. Such an edge property is assessed with WorldView-2 multi-angular sequence of images, where spectral distortions arise with far nadir view angles.

A new compact representation of differential morphological profile (DMP) vector fields is presented. It is referred to as the CSL model and is conceived to radically reduce the dimensionality of the DMP descriptors. The model maps three characteristic parameters, namely scale, saliency and level, into the RGB space through a HSV transform. The result is a a medium abstraction semantic layer used for visual exploration, image information mining and pattern classification. Fused with the PANTEX built-up presence index, the CSL model converges to an approximate building footprint representation layer in which color represents building class labels. This process is demonstrated on the first high resolution (HR) global human settlement layer (GHSL) computed from multi-modal HR and VHR satellite images. Results of the first massive processing exercise involving several thousands of scenes around the globe are reported along with validation figures., JRC.G.2-Global security and crisis management

The enumeration of the population remains a critical task in the management of refugee/IDP camps. Analysis of very high spatial resolution satellite data proofed to be an efficient and secure approach for the estimation of dwellings and the monitoring of the camp over time. In this paper we propose a new methodology for the automated extraction of features based on differential morphological decomposition segmentation for feature extraction and interactive training sample selection from the max-tree and min-tree structures. This feature extraction methodology is tested on a WorldView-2 scene of an IDP camp in Darfur Sudan. Special emphasis is given to the additional available bands of the WorldView-2 sensor. The results obtained show that the interactive image information tool is performing very well by tuning the feature extraction to the local conditions. The analysis of different spectral subsets shows that it is possible to obtain good results already with an RGB combination, but by increasing the number of spectral bands the detection of dwellings becomes more accurate. Best results were obtained using all eight bands of WorldView-2 satellite., JRC.G.2-Global security and crisis management

In this paper we present a fast, interactive method for collecting structural primitives from objects of interest contained within manually selected image regions. The input image is projected onto a Max-Tree and Min-Tree structure from which a pixel-to-node mapper marks the nodes of each tree that correspond to peak components explicitly contained within the selected window. In a pass through the selected nodes, an attribute vector is constructed from the pool of auxiliary data associated with each node separately. The set of all attribute vectors is mapped into a pre-computed multidimensional feature space from which a binary criterion is constructed to accept or reject the remaining image objects. The method is demonstrated in a real application on information extraction from very high resolution satellite imagery.

In this paper, mixed information similarity measure and a multidimensional density estimation method based on multivariate Edgeworth series expansion are proposed and assessed for the task of multi-temporal change detection. To unify mutual information and variational information, mixed information is proposed to quantify the degree of dependence between two random variables, which are intuitively appropriate for multi-temporal change detection. In the literature, Edgeworth series expansion is widely used in statistics and various engineering fields for one-dimensional density estimation. To compute the mixed information measure, multidimensional density estimation based on multivariate Edgeworth series expansion is proposed and evaluated. Two experiments on real SAR images and optical images are carried out to evaluate the performance of change detection. Experimental results confirm the promising capability of mixed information and the multivariate density estimation based on Edgeworth series expansion., JRC.DG.G.2-Global security and crisis management

Very High Resolution (VHR) satellite images are products which are very useful for assessing humanitarian situations following a disaster or a crisis. When disaster or conflict hits, generally several square kilometers on Earth surface are involved. Having a high resolution and an extended surface, the scenes, from which the information is extracted, become huge in the number of pixels. Built-up structures are the first visible objects from these images which give information about the situation. In this paper, an interactive image mining paradigm is presented for detecting structures of interest in gigapixel images. The proposed approach is tested for refugee tents detection in submetric panchromatic image., JRC.G.2-Global security and crisis management

A dissimilarity measure between adjacent pixels of an image is usually determined by the intensity values of these pixels and therefore does not depend on statistics computed over the whole image domain. In this paper, new dissimilarity measures exploiting image statistics are proposed. This is achieved by introducing the notion of dissimilarity function defined for every possible pair of intensity values. Necessary conditions for generating a valid dissimilarity function are provided and a series of functions integrating image statistics are presented. For example, the joint probability of adjacent pixel values leads to the notion of frequent connectivity while the notion of dependent connectivity relies on the local mutual information. The usefulness of the proposed approach is demonstrated by a series of experiments on satellite image data.

In this paper, a novel method for multi scale corner analysis and detection is presented. First, stateof-the-art Harris–Laplace corner detector is reminded, which benefits from linear scale-space analysis. Secondly, a non-linear scale-space transform, namely Differential Morphological Decomposition, is described. This multi-scale transform is used jointly with the Harris corner indicator to build a new multi scale corner detector. Both corner detectors are visually assessed on synthetic and satellite images, highlighting the advantages of such a method, JRC.G.2-Global security and crisis management

In this paper, we propose a novel approach for breaking up an image into multi-scale overlapping objects. The image is decomposed by granulometry or differential morphological pyramid, resulting in a discrete scale-space representation. Then, the scale-space transform is segmented by a region based method. Projecting the obtained scale-space partition into space constitutes the disintegrated image representation, which enables a multi-scale object based image description., JRC.DG.G.2-Global security and crisis management

Satellite Multitemporal Observations and Satellite Image Time Series are a new type of data sets acquired with optical and Synthetic Aperture Radar (SAR) technologies. Presently, due to the large number of Earth Observation satellites, every location on Earth can be observed very often; however, the observations in practical situations are usually multisensor. As a result, new models and algorithms are needed to automatically perform the data analysis. This paper presents two approaches for the analysis of multitemporal and multisensor data analysis. The first approach studies similarity measures derived from information measures and we assess their performance for multitemporal analysis. The temporal evolution of the scene changes is measured as a variation in the common and disjoint information measures between the different sensor data models. The second approach proposes the use of specific density estimation methods to reduce approximations effects for the detection of the multisensory changes. Usually the problem of change detection can be decomposed into two steps [1]: the generation of a change map based on the comparison of the 2 images using a similarity function, and a change classification performed either in un-supervised or supervised schema. In the former situation certain kinds of prior information about the scene, such as prior distribution assumption mode, is necessary. This paper will focus on an unsupervised approach of the generation of the indicator image in order to produce the binary change map. This approach is based on the comparison of local statistical similarity measures. In the literature, there several kinds of similarity measure, among which mutual information based similarity measures, have been proved to be more efficient. In [2], have been introduced several similarity measures between multi-sensor images. These measures are based on concepts such as statistical dependence or mutual information. The use of these measures allows for the design of image registration algorithms and automatic change detection techniques. And also in [3] are presented a new approach based on similarity measures. A series of such measures is employed for automatic change detection of optical and SAR images and a comparison of their performance is carried out to compare the performance of each similarity measures. And he concludes the CRA and the mutual information measures perform the best. Taking advantage of the Mutual Information, in (5) is proposed a pixel-based approach comparing the localized Mutual Information shared by two pixels. When the two pixels share a lot of Mutual Information, their location is considered unchanged. From this idea, in (4) it was introduced a new informational measure derived from the Mutual Information: the Mixed Information. This new informational measure quanti¿es in a unique framework the shared and different informations such that time separated pixels with different informations are set changed. The new method based on Mixed Information was also proffed to be invariant to nonlinear changes, thus we further investigate ist use for multi-sensor cahnge detectin. Both the methods of in [3] and [4] are based on distribution estimation as histograms. In [1] it is suggested that the histogram method should be avoided due to the need of a high number of samples for estimation. As alternative, a cumulant-based density estimation is used to derive the local distribution which is furtehr used to compare the similarity between two images with the Kullback-Leibler divergence. The marginal distributions of each image are used To overcome the shortcomings, this paper presents an alternative scheme which is an extension of the work in [4]., JRC.DG.G.2-Global security and crisis management

In this paper, we propose a method for predicting traffic values from spectrum measurements. The purpose is to obtain traffic information from the wireless environment for making informed decisions in order to balance traffic loads. We propose a supervised prediction scheme trained by real measurements, such that the method adapts to new radio contexts for new measurements. As a first step, features are extracted from spectrum measurements using probability distributions and a Principal Component Analysis. Then, the relevant features are selected based on a correlation criterion. In the third stage, the predictor, mapping the selected features to a traffic value, is trained using the k Nearest Neighbours algorithm. The predictor reliability is then evaluated using a cross validation. The results obtained with spectrum measurements realized on the GSM900 and the DCS1800 frequency bands show that the proposed scheme achieves a reliable traffic prediction.

Images may contain blemishes or artificial structures which come from the processing or directly from the sensors, that decrease the quality of the images and can lead to analysis and interpretation problems. We are interested in detecting these defects automatically. The aim of our method will be to look for artifacts as too regular or too irregular image structures that will be detected using the basic properties of Kolmogorov complexity.

In this paper, we propose a method for automatically extracting distinct spectral states present in spectrum power measurements. The purpose is to gain information on the wireless environment for constructing efficient policies and decisions in a flexible radio context. To achieve this, we perform unsupervised classification on data obtained from spectrum power measurements on the GSM 1800 downlink band. The number of spectral states having different spectral characteristics is automatically detected by a well-known efficient rate-distortion criterion. Spectral signals that constitute the power spectrum measurements are classified into these distinct spectral states through the k-means clustering algorithm. In order to eliminate redundancies and to decrease the computational complexity, Principal Component Analysis is performed on the set of spectral signals before classification. The time-averaged frequency profile of each spectral state can be considered as the quantization or compression of the spectral information contained by the spectral signal belonging to that spectral state. The preliminary results obtained on the GSM 1800 downlink spectrum measurements reveal that the proposed method successfully extracts the distinct spectral states of the wireless environment.

Derived from Information Theory, the Information Bottleneck principle enables to quantify and qualify the information contained in a signal. This paper presents an algorithm based on the Information Bottleneck principle to analyze Satellite Image Time Series (SITS). The method is composed of a parameter estimation and a model selection. This method has been previously applied to textural and radiometric parametric models and we propose here to extend it to take into account the geometry information. Two approaches are presented. In the first approach, each image of the SITS is segmented and the obtained regions are described by textural models. The Information Bottleneck method is further used to characterize the image segments of the SITS a spatio‐temporal way. In the second method, the geometrical information is extracted from a temporal adjacency graph of the spatial regions, and the radiometric and textural information is then extracted through the Information Bottleneck method. This approach leads to a temporal characterization of the spatial regions of the SITS.

Several techniques have been proposed in the last few years for the compression or hyperspectral and multispectral images. All of them tried to exploit, in various degrees, the particularities of these images, especially of the intraband and interband redundancies they exhibit. In this paper, the authors considered the compression of multitemporal and multispectral images and made a comparison between still image and video approaches for the compression of these images acquired by SPOT 1, 2 and 4. The still image compression is realized using the state-of-the-art JPEG2000 codec, while the video one uses an EZBC wavelet-based codec. Both spectral and temporal decorrelation approaches are discussed.