Search

In the context of urban 3D mapping, the 3D modelling step is a crucial operation, which can be very error prone, particularly when high fidelity and accuracy are required. While automatic reconstruction tools are way faster and less expensive than manual ones, these last are less sensible to the data defects.To combine all this complementary advantages, we propose a semi-automatic approach, where human operators will manually correct the result of an automatic reconstruction tool.While the automated reconstruction tool will be taken on the shelf, we have developed our own polyhedral modeler tool to make the correction step. In this article, we present this polyhedral modeler, which is based on face shifting, edge flipping and automatic topological events resolution.

Iconographic image collections are a cultural heritage that could reach a larger audience by proposing their immersive presentation in a 3D web application. Proposing a historical street view application, based on these historical images, raises issues such as the unavailability of historical 3D models of the scene and the heterogeneity and sparsity of these photographs. We propose to use the 3D city and terrain models of the current scene, as well as a 3D point cloud if available, to simultaneously reproject and blend many historical images using an image-based rendering approach. Our contributions raise significantly the number of projective textures blended per rendering pass (typically from 8 to 40) on triangular meshes (of the 3D city and terrain models) and on point clouds. As a first step to tackle diachrony artifacts, we also propose a simple point cloud classification to filter in the shader the points corresponding to building or terrain details from the points corresponding to transient objects.

This article presents a spatio-temporal web application dedicated to the co-exploitation of heterogeneous data spatialized in a common 3D environment, providing several paradigms for supporting their co-visualization and interactions within the 3D environment and across time. The relevance of this tool is demonstrated here with two use cases involving historians and sociologists with the common objective of better understanding the formation of the Parisian metropolis. The study focuses on the evolution of the city of Nanterre (Paris area), which underwent many changes in the 1950s, and in particular on shantytown areas. Through census as statistical data and aerial imagery as visual data, a group of historians and sociologists experimented the relevance of the joint exploitation of those heterogeneous data within the proposed spatio-temporal web application.

Archivists, historians and national mapping agencies, among others, are archiving large datasets of historical photographs. Nevertheless, the capturing devices used to acquire these images possessed a diversity of effects that influenced the quality of the final resulting picture, e.g. geometric distortion, chromatic aberration, depth of field variation, etc. This paper examines singularly the topic of geometric distortion for a co-visualization of historical photos within a 3D model of the photographed scene. A distortion function of an image is ordinarily estimated only on the image domain by adjusting its parameters to observations of point correspondences. This mathematical function may exhibit overfits, oscillations or may not be well defined outside of this domain. The contribution of this work is the description of a distortion model defined on the whole undistorted image plane. We extrapolate the distortion estimated only on the image domain and then transfer this distortion information to the view of the 3D scene. This enables to look at the scene through an estimated camera and zoom out to see the context around the original photograph with a well-defined and behaved distortion. These findings may be a significant addition to the overall purpose of creating innovative ways to examine and visualize old photographs.

In order to understand and explain urban climate, the visual analysis of urban climate data and their relationships with the urban morphology is at stake. This involves partly to co-visualize 3D field climate data, obtained from simulation, with urban 3D models. We propose two ways to visualize and navigate into simulated climate data in urban 3D models, using series of horizontal 2D planes and 3D point clouds. We then explore different parameters regarding transparency, 3D semiologic rules, filtering and animation functions in order to improve the visual analysis of climate data 3D distribution. To achieve this, we apply our propositions to the co-visualization of air temperature data with a 3D urban city model.

This paper deals with the distributed computation of Delaunay triangulations of massive point sets, mainly motivated by the needs of a scalable out-of-core surface reconstruction workflow from massive urban LIDAR datasets. Such a data often corresponds to a huge point cloud represented through a set of tiles of relatively homogeneous point sizes. This will be the input of our algorithm which will naturally partition this data across multiple processing elements. The distributed computation and communication between processing elements is orchestrated efficiently through an uncentralized model to represent, manage and locally construct the triangulation corresponding to each tile. Initially inspired by the star splaying approach, we review the Tile& Merge algorithm for computing Distributed Delaunay Triangulations on the cloud, provide a theoretical proof of correctness of this algorithm, and analyse the performance of our Spark implementation in terms of speedup and strong scaling in both synthetic and real use case datasets. A HPC implementation (e.g. using MPI), left for future work, would benefit from its more efficient message passing paradigm but lose the robustness and failure resilience of our Spark approach.

For a long time people have been interested in the past and history and how we can go back in time using a time machine. And while we cannot invent a time machine, at least not yet, we can create a virtual one. Our “virtual” time machine is an interactive web application that allows users to browse through and navigate within historical images (aerial/terrestrial photographs or postcards) that are projected on a 3D photogrammetric model (point cloud or 3D mesh), thus going back in time and interacting with historical 3D models and images. This was achieved by adopting a semiautomatic approach where the user identifies first 6 to 8 hints on the historical image and the photogrammetric model, then this information is used as an entry data to a photogrammetric software that computes the pose and orientation of the image. The purpose of this work, which is part of the ALEGORIA project, is to preserve cultural heritage, to give the users the opportunity to go back in time and study history of a place or simply discover how their hometown looked some years ago.

In this paper, we assume that augmented reality (AR) and mixed reality (MR) are relevant contexts for 3D urban geovisualization, especially in order to support the design of the urban spaces. We propose to design an in situ MR application, that could be helpful for urban designers, providing tools to interactively remove or replace buildings in situ. This use case requires advances regarding existing geovisualization methods. We highlight the need to adapt and extend existing 3D geovisualization pipelines, in order to adjust the specific requirements for AR/MR applications, in particular for data rendering and interaction. In order to reach this goal, we focus on and implement four elementary in situ and ex situ AR/MR experiments: each type of these AR/MR experiments helps to consider and specify a specific subproblem, i.e. scale modification, pose estimation, matching between scene and urban project realism, and the mix of real and virtual elements through portals, while proposing occlusion handling, rendering and interaction techniques to solve them.

This paper proposes a novel framework for the disocclusion of mobile objects in 3D LiDAR scenes aquired via street-based Mobile Mapping Systems (MMS). Most of the existing lines of research tackle this problem directly in the 3D space. This work promotes an alternative approach by using a 2D range image representation of the 3D point cloud, taking advantage of the fact that the problem of disocclusion has been intensively studied in the 2D image processing community over the past decade. First, the point cloud is turned into a 2D range image by exploiting the sensor’s topology. Using the range image, a semi-automatic segmentation procedure based on depth histograms is performed in order to select the occluding object to be removed. A variational image inpainting technique is then used to reconstruct the area occluded by that object. Finally, the range image is unprojected as a 3D point cloud. Experiments on real data prove the effectiveness of this procedure both in terms of accuracy and speed.

Mobile Mapping Systems (MMS) are now commonly acquiring lidar scans of urban environments for an increasing number of applications such as 3D reconstruction and mapping, urban planning, urban furniture monitoring, practicability assessment for persons with reduced mobility (PRM)... MMS acquisitions are usually huge enough to incur a usability bottleneck for the increasing number of non-expert user that are not trained to process and visualize these huge datasets through specific softwares. A vast majority of their current need is for a simple 2D visualization that is both legible on screen and printable on a static 2D medium, while still conveying the understanding of the 3D scene and minimizing the disturbance of the lidar acquisition geometry (such as lidar shadows). The users that motivated this research are, by law, bound to precisely georeference underground networks for which they currently have schematics with no or poor absolute georeferencing. A solution that may fit their needs is thus a 2D visualization of the MMS dataset that they could easily interpret and on which they could accurately match features with their user datasets they would like to georeference. Our main contribution is two-fold. First, we propose a 3D point cloud stylization for 2D static visualization that leverages a Principal Component Analysis (PCA)-like local geometry analysis. By skipping the usual and error-prone estimation of a ground elevation, this rendering is thus robust to non-flat areas and has no hard-to-tune parameters such as height thresholds. Second, we implemented the corresponding rendering pipeline so that it can scale up to arbitrary large datasets by leveraging the Spark framework and its Resilient Distributed Dataset (RDD) and Dataframe abstractions.

Despite of the popularity of Delauney structure for mesh generation, octree based approaches remain an interesting solution for a first step surface reconstruction. In this paper, we propose a generic framework for a octree cell based mesh generation. Its input is a set of Lidar-based 3D measurements or other inputs which are formulated as a set of mass functions that characterize the level of confidence on the occupancy of each octree’s leaf. The output is a binary segmentation of the space between occupied and empty areas by taking into account the uncertainty of data. To this end, the problem is then reduced to a global energy optimization framework efficiently optimized with a min-cut approach. We use the approach for producing a large scale surface reconstruction algorithm by merging data from ubiquitous sources like airborne, terrestrial Lidar data, occupancy map and extra cues. Once the surface is computed, a solution is proposed for texturing the mesh.

We propose an integrated bottom-up/top-down approach to road-marking extraction from image space. It is based on energy minimization using marked point processes. A generic road marking object model enable us to define universal energy functions that handle various types of road-marking objects (dashed-lines, arrows, characters, etc.). A RJ-MCMC sampler coupled with a simulated annealing is applied to find the configuration corresponding to the minimum of the proposed energy. We used input data measurements to guide the sampler process (data driven RJ-MCMC). The approach is enhanced with a model-driven kernel using preprocessed autocorrelation and inter-correlation of road-marking templates, in order to resolve type and transformation ambiguities. The method is generic and can be applied to detect road-markings in any orthogonal view produced from optical sensors or laser scanners from aerial or terrestrial platforms. We show the results an ortho-image computed from ground-based laser scanning.

This paper presents a full pipeline to extract mobile objects in images based on a simultaneous laser acquisition with a Velodyne scanner. The point cloud is first analysed to extract mobile objects in 3D. This is done using Dempster-Shafer theory and it results in weights telling for each points if it corresponds to a mobile object, a fixed object or if no decision can be made based on the data (unknown). These weights are projected in an image acquired simultaneously and used to segment the image between the mobile and the static part of the scene.

The 3D reconstruction of similar 3D objects detected in 2D faces a major issue when it comes to grouping the 2D detections into clusters to be used to reconstruct the individual 3D objects. Simple clustering heuristics fail as soon as similar objects are close. This paper formulates a framework to use the geometric quality of the reconstruction as a hint to do a proper clustering. We present a methodology to solve the resulting combinatorial optimization problem with some simplifications and approximations in order to make it tractable. The proposed method is applied to the reconstruction of 3D traffic signs from their 2D detections to demonstrate its capacity to solve ambiguities.

The paper presents an algorithm for reconstruction of 3D circle from its apparition in n images. It supposes that camera poses are known up to an uncertainty. They will be considered as observations and will be refined during the reconstruction process. First, circle apparitions will be estimated in every individual image from a set of 2D points using a constrained optimization. Uncertainty of 2D points are propagated in 2D ellipse estimation and leads to covariance matrix of ellipse parameters. In 3D reconstruction process ellipse and camera pose parameters are considered as observations with known covariances. A minimal parametrization of 3D circle enables to model the projection of circle in image without any constraint. The reconstruction is performed by minimizing the length of observation residuals vector in a non linear Gauss-Helmert model. The output consists in parameters of the corresponding circle in 3D and their covariances. The results are presented on simulated data.

Projective texturing is a commonly used image based rendering technique that enables the synthesis of novel views from the blended reprojection of nearby views on a coarse geometry proxy approximating the scene. When scene geometry is inexact, aliasing artefacts occur. This introduces disturbing artefacts in applications such as street-level immersive navigation in mobile mapping imagery, since a pixel-accurate modelling of the scene geometry and all its details is most of the time out of question. The filtered blending approach applies the necessary 1D low-pass filtering on the projective texture to trade out the aliasing artefacts at the cost of some radial blurring. This paper proposes extensions of the filtered blending approach. Firstly, we introduce Integral Radial Images that enable constant time radial box filtering and show how they can be used to apply box-filtered blending in constant time independently of the amount of depth uncertainty. Secondly, we show a very efficient application of filtered blending where the scene geometry is only given by a loose depth interval prior rather than an actual geometry proxy. Thirdly, we propose a silhouette-aware extension of the box-filtered blending that not only account for uncertain depth along the viewing ray but also for uncertain silhouettes that have to be blurred as well.

It may be argued that urban areas may now be modeled with sufficient details for realistic fly-through over the cities at a reasonable price point. Modeling cities at the street level for immersive street-view navigation is however still a very expensive (or even impossible) operation if one tries to match the level of detail acquired by street-view mobile mapping imagery. This paper proposes to leverage the richness of these street-view images with the common availability of nation-wide LOD1 3D city models, using an image-based rendering technique : projective multi-texturing. Such a coarse 3D city model may be used as a lightweight scene proxy of approximate coarse geometry. The images neighboring the interpolated viewpoint are projected onto this scene proxy using their estimated poses and calibrations and blended together according to their relative distance. This enables an immersive navigation within the image dataset that is perfectly equal to – and thus as rich as – original images when viewed from their viewpoint location, and which degrades gracefully in between viewpoint locations. Beyond proving the applicability of this preprocessing-free computer graphics technique to mobile mapping images and LOD1 3D city models, our contributions are three-fold. Firstly, image distortion is corrected online in the GPU, preventing an extra image resampling step. Secondly, externally-computed binary masks may be used to discard pixels corresponding to moving objects. Thirdly, we propose a shadowmap-inspired technique that prevents, at marginal cost, the projective texturing of surfaces beyond the first, as seen from the projected image viewpoint location. Finally, an augmented visualization application is introduced to showcase the proposed immersive navigation: images are unpopulated from vehicles using externally-computed binary masks and repopulated using a 3D visualization of a 2D traffic simulation.

Tree detection and reconstruction is of great interest in large-scale city modelling. In this paper, we present a marked point process model to detect single trees from airborne laser scanning (ALS) data. We consider single trees in ALS recovered canopy height model (CHM) as a realization of point process of circles. Unlike traditional marked point process, we sample the model in a constraint configuration space by making use of image process techniques. A Gibbs energy is defined on the model, containing a data term which judge the fitness of the model with respect to the data, and prior term which incorporate the prior knowledge of object layouts. We search the optimal configuration through a steepest gradient descent algorithm. The presented hybrid framework was test on three forest plots and experiments show the effectiveness of the proposed method.

Terrestrial geolocalized images are nowadays widely used on the Internet, mainly in urban areas, through immersion services such as Google Street View. On the long run, we seek to enhance the visualization of these images; for that purpose, radiometric corrections must be performed to free them from illumination conditions at the time of acquisition. Given the simultaneously acquired 3D geometric model of the scene with LIDAR or vision techniques, we face an inverse problem where the illumination and the geometry of the scene are known and the reflectance of the scene is to be estimated. Our main contribution is the introduction of a symbolic ray-tracing rendering to generate parametric images, for quick evaluation and comparison with the acquired images. The proposed approach is then based on an iterative estimation of the reflectance parameters of the materials, using a single rendering pre-processing. We validate the method on synthetic data with linear BRDF models and discuss the limitations of the proposed approach with more general non-linear BRDF models.

This paper proposes a method to estimate the local sharpness of an optical system through the wavelet-based analysis of a large set of images it acquired. Assuming a space-invariant distribution of image features, such as in the aerial photography context, the proposed approach produces a sharpness map of the imaging device over 16 × 16 pixels blocks that enables, for instance, the detection of optical defects and the qualification of the mosaicking of multiple sensor images into a larger composite image. The proposed analysis is based on accumulating of the edge maps corresponding to the first levels of the Haar Transform of each image of the dataset, following the intuition that statistically, each pixel will see the same image structures. We propose a calibration method to transform these accumulated edge maps into a sharpness map by approximating the local PSF (Point Spread Function) with a Gaussian blur.

Current Internet-inspired mapping data are in the form of street maps, orthophotos, 3D models or street-side images and serve to support mostly search and navigation. Yet the only mapping data that currently can really be searched are the street maps via their addresses and coordinates. The orthophotos, 3D models and street-side images represent predominantly "eye candy" with little added value to the Internet-user. We are interested in characterizing the elements of the urban space from imagery. In this paper we discuss the use of street side imagery and aerial imagery to develop descriptions of urban spaces, initially of building facades and roofs. We present methods (a) to segment facades using high-overlap street side facade images, (b) to map facades and facade details from vertical aerial images, and (c) to characterize roofs by their type and details, also from aerial photography. This paper describes a method of roof segmentation with the goal of assigning each roof to a specific architectural style. Questions of the use of the attic space, or the placement of solar panels, are of interest. It is of interest that roofs have recently been mapped using LiDAR point clouds. We demonstrate that aerial images are a useful and economical alternative to LiDAR for the characterization of building roofs, and that they also contain very valuable information about facades.

This paper presents a comparison of automatic photogrammetric techniques to terrestrial laser scanning for 3D modelling of complex interior spaces. We try to evaluate the automated photogrammetric techniques not only in terms of their geometric quality compared to laser scanning but also in terms of cost in money, acquisition and computational time. To this purpose we chose as test site a modern building’s stairway. APERO/MICMAC ( ©IGN )which is an Open Source photogrammetric software was used for the production of the 3D photogrammetric point cloud which was compared to the one acquired by a Leica Scanstation 2 laser scanner. After performing various qualitative and quantitative controls we present the advantages and disadvantages of each 3D modelling method applied in a complex interior of a modern building.

The librjmcmc is an open source C++ library that solves optimization problems using a stochastic framework. The library is primarily intended for but not limited to research purposes in computer vision, photogrammetry and remote sensing, as it has initially been developed in the context of extracting building footprints from digital elevation models using a marked point process of rectangles. It has been designed to be both highly modular and extensible, and have computational times comparable to a code specifically designed for a particular application, thanks to the powerful paradigms of metaprogramming and generic programming. The proposed stochastic optimization is built on the coupling of a stochastic Reversible-Jump Markov Chain Monte Carlo (RJMCMC) sampler and a simulated annealing relaxation. This framework allows, with theoretical guarantees, the optimization of an unrestricted objective function without requiring any initial solution. The modularity of our library allows the processing of any kind of input data, whether they are 1D signals (e.g. LiDAR or SAR waveforms), 2D images, 3D point clouds... The library user has just to define a few modules describing its domain specific context: the encoding of a configuration (e.g. its object type in a marked point process context), reversible jump kernels (e.g. birth, death, modifications...), the optimized energies (e.g. data and regularization terms) and the probabilized search space given by the reference process. Similar to this extensibility in the application domain, concepts are clearly and orthogonally separated such that it is straightforward to customize the convergence test, the temperature schedule, or to add visitors enabling visual feedback during the optimization. The library offers dedicated modules for marked point processes, allowing the user to optimize a Maximum A Posteriori (MAP) criterion with an image data term energy on a marked point process of rectangles.

The knowledge of the sky illumination is important for radiometric corrections and for computer graphics applications such as relighting or augmented reality. We propose an approach to compute environment maps, representing the sky radiance, from a set of ground-based images acquired by a panoramic acquisition system, for instance a mobile-mapping system. These images can be affected by important radiometric artifacts, such as bloom or overexposure. A Perez radiance model is estimated with the blue sky pixels of the images, and used to compute additive corrections in order to reduce these radiometric artifacts. The sky pixels are then aggregated in an environment map, which still suffers from discontinuities on stitching edges. The influence of the quality of estimated sky radiance on the simulated light signal is measured quantitatively on a simple synthetic urban scene; in our case, the maximal error for the total sensor radiance is about 10%.

This article describes a pipeline developed to automatically detect and correct motion blur due to the airplane motion in aerial images provided by a digital camera system with channel-dependent exposure times. Blurred images show anisotropy in their Fourier Transform coefficients that can be detected and estimated to recover the characteristics of the motion blur. To disambiguate the anisotropy produced by a motion blur from the possible spectral anisotropy produced by some periodic patterns present in a sharp image, we consider the phase difference of the Fourier Transform of two channel shot with different exposure times (i.e. with different blur extensions). This is possible because of the deep correlation between the three visible channels ensures phase coherence of the Fourier Transform coefficients in sharp images. In this context, considering the phase difference constitutes both a good detector and estimator of the motion blur parameters. In order to improve on this estimation, the phase difference is performed on local windows in the image where the channels are more correlated. The main lobe of the phase difference, where the phase difference between two channels is close to zero actually imitates an ellipse which axis ratio discriminates blur and which orientation and minor axis give respectively the orientation and the blur kernel extension of the long exposure-time channels. However, this approach is not robust to the presence in the phase difference of minor lobes due to phase sign inversions in the Fourier transform of the motion blur. They are removed by considering the polar representation of the phase difference. Based on the blur detection step, blur correction is eventually performed using two different approaches depending on the blur extension size: using either a simple frequency-based fusion for small blur or a semi blind iterative method for larger blur. The higher computing costs of the latter method make it only suitable for large motion blur, when the former method is not applicable.

Current 3D data capturing as implemented on for example airborne or mobile laser scanning systems is able to efficiently sample the surface of a city by billions of unselective points during one working day. What is still difficult is to extract and visualize meaningful information hidden in these point clouds with the same efficiency. This is where the FP7 IQmulus project enters the scene. IQmulus is an interactive facility for processing and visualizing big spatial data. In this study the potential of IQmulus is demonstrated on a laser mobile mapping point cloud of 1 billion points sampling ~ 10 km of street environment in Toulouse, France. After the data is uploaded to the IQmulus Hadoop Distributed File System, a workflow is defined by the user consisting of retiling the data followed by a PCA driven local dimensionality analysis, which runs efficiently on the IQmulus cloud facility using a Spark implementation. Points scattering in 3 directions are clustered in the tree class, and are separated next into individual trees. Five hours of processing at the 12 node computing cluster results in the automatic identification of 4000+ urban trees. Visualization of the results in the IQmulus fat client helps users to appreciate the results, and developers to identify remaining flaws in the processing workflow.

This article presents the empirical research carried out in the context of the multi-site EuroSDR project "Radiometric aspects of digital photogrammetric images" and provides highlights of the results. The investigations have considered the vicarious radiometric and spatial resolution validation and calibration of the sensor system, radiometric processing of the image blocks either by performing relative radiometric block equalization or into absolutely reflectance calibrated products, and finally aspects of practical applications on NDVI layer generation and tree species classification. The data sets were provided by Leica Geosystems ADS40 and Intergraph DMC and the participants represented stakeholders in National Mapping Authorities, software development and research. The investigations proved the stability and quality of evaluated imaging systems with respect to radiometry and optical system. The first new-generation methods for reflectance calibration and equalization of photogrammetric image block data provided promising accuracy and were also functional from the productivity and usability points of view. The reflectance calibration methods provided up to 5% accuracy without any ground reference. Application oriented results indicated that automatic interpretation methods will benefit from the optimal use of radiometrically accurate multi-view photogrammetric imagery.

We present an original multiview stereo reconstruction algorithm which allows the 3D-modeling of urban scenes as a combination of meshes and geometric primitives. The method provides a compact model while preserving details: Irregular elements such as statues and ornaments are described by meshes, whereas regular structures such as columns and walls are described by primitives (planes, spheres, cylinders, cones, and tori). We adopt a two-step strategy consisting first in segmenting the initial meshbased surface using a multilabel Markov Random Field-based model and second in sampling primitive and mesh components simultaneously on the obtained partition by a Jump-Diffusion process. The quality of a reconstruction is measured by a multi-object energy model which takes into account both photo-consistency and semantic considerations (i.e., geometry and shape layout). The segmentation and sampling steps are embedded into an iterative refinement procedure which provides an increasingly accurate hybrid representation. Experimental results on complex urban structures and large scenes are presented and compared to state-of-the-art multiview stereo meshing algorithms.

We propose an original hybrid modeling process of urban scenes that represents 3-D models as a combination of mesh-based surfaces and geometric 3-D-primitives. Meshes describe details such as ornaments and statues, whereas 3-D-primitives code for regular shapes such as walls and columns. Starting from an 3-D-surface obtained by multiview stereo techniques, these primitives are inserted into the surface after being detected. This strategy allows the introduction of semantic knowledge, the simplification of the modeling, and even correction of errors generated by the acquisition process. We design a hierarchical approach exploring different scales of an observed scene. Each level consists first in segmenting the surface using a multilabel energy model optimized by -expansion and then in fitting 3-D-primitives such as planes, cylinders or tori on the obtained partition where relevant. Experiments on real meshes, depth maps and synthetic surfaces show good potential for the proposed approach.