Search

Image-based 3D reconstruction has been employed in industrial metrology for micro-measurements and quality control purposes. However, generating a highly-detailed and reliable 3D reconstruction of non-collaborative surfaces is still an open issue. In this paper, a method for generating an accurate 3D reconstruction of non-collaborative surfaces through a combination of photogrammetry and photometric stereo is presented. On one side, the geometric information derived with photogrammetry is used in areas where its 3D measurements are reliable. On the other hand, the high spatial resolution capability of photometric stereo is exploited to acquire a finely detailed topography of the surface. Finally, three different approaches are proposed to fuse both geometric information and high frequency details. The proposed method is tested on six different non-collaborative objects with different surface characteristics. To evaluate the accuracy of the proposed method, a comprehensive cloud-to-cloud comparison between reference data and 3D points derived from the proposed fusion methods is provided. The experiments demonstrated that, despite correcting global deformation up to an average RMSE of less than 0.1 mm, the proposed method recovers the surface topography at the same high resolution as the photometric stereo.

The image orientation (or Structure from Motion – SfM) process needs well localized, repeatable and stable tie points in order to derive camera poses and a sparse 3D representation of the surveyed scene. The accurate identification of tie points in large image datasets is still an open research topic in the photogrammetric and computer vision communities. Tie points are established by firstly extracting keypoint using a hand-crafted feature detector and descriptor methods. In the last years new solutions, based on convolutional neural network (CNN) methods, were proposed to let a deep network discover which feature extraction process and representation are most suitable for the processed images. In this paper we aim to compare state-of-the-art hand-crafted and learning-based method for the establishment of tie points in various and different image datasets. The investigation highlights the actual challenges for feature matching and evaluates selected methods under different acquisition conditions (network configurations, image overlap, UAV vs terrestrial, strip vs convergent) and scene's characteristics. Remarks and lessons learned constrained to the used datasets and methods are provided.

Uncontrolled refraction of optical rays in underwater photogrammetry is known to reduce its accuracy potential. Several strategies have been proposed aiming at restoring the accuracy to levels comparable with photogrammetry applied in air. These methods are mainly based on rigours modelling of the refraction phenomenon or empirical iterative refraction corrections. The authors of this contribution have proposed two mitigation strategies of image residuals systematic patterns in the image plane: (i) empirical weighting of image observations as function of their radial position; (ii) iterative look-up table corrections computed in a squared grid. Here, a novel approach is developed. It explicitly takes into account the object point-to-camera distance dependent error introduced by refraction in multimedia photogrammetry. A polynomial correction function is iteratively computed to correct the image residuals clustered in radial slices in the image plane as function of the point-to-camera distance. The effectiveness of the proposed method is demonstrated by simulations that allow to: (i) separate the geometric error under investigation from other effects not easily modellable and (ii) have reliable reference data against which to assess the accuracy of the result., International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLIII-B2-2021, ISSN:1682-1750, ISSN:2194-9034, ISSN:1682-1777

3D digital reconstruction techniques are extensively used for quality control purposes. Among them, photogrammetry and photometric stereo methods have been for a long time used with success in several application fields. However, generating highly-detailed and reliable micro-measurements of non-collaborative surfaces is still an open issue. In these cases, photogrammetry can provide accurate low-frequency 3D information, whereas it struggles to extract reliable high-frequency details. Conversely, photometric stereo can recover a very detailed surface topography, although global surface deformation is often present. In this paper, we present the preliminary results of an ongoing project aiming to combine photogrammetry and photometric stereo in a synergetic fusion of the two techniques. Particularly, hereafter, we introduce the main concept design behind an image acquisition system we developed to capture images from different positions and under different lighting conditions as required by photogrammetry and photometric stereo techniques. We show the benefit of such a combination through some experimental tests. The experiments showed that the proposed method recovers the surface topography at the same high-resolution achievable with photometric stereo while preserving the photogrammetric accuracy. Furthermore, we exploit light directionality and multiple light sources to improve the quality of dense image matching in poorly textured surfaces.

In the last years, vision-based systems have flourished at an unprecedented pace, fuelled by developments in hardware components (higher resolution and higher sensitivity imaging sensors, smaller and smarter micro controllers, just to name a few), as well as in software or processing techniques, with AI (Artificial Intelligence) leading to a landmark revolution. Several disciplines have fostered and benefited from these advances, but, unfortunately, not always in a coordinated and cooperative way.When it comes to image-based sensing techniques, photogrammetry, computer vision and robotic vision have many contact points and overlapping areas. Yet, as for people of different cultures and languages, communicating among the three different communities can be very harsh and disorienting - especially for beginners and non-specialists.Driven by a strong educational and inclusive ambition, the LightCam project is funded by the ISPRS Education and Capacity Building Initiatives 2020 (ECB). The project’s ambition is to act as an interpreter and ease the dialog among the three actors, i.e. photogrammetry, computer vision and robotics. Two intermediation tools will be developed to serve this aim: (i) a dictionary of concepts, terminology and algorithms, in the form of a knowledge base website, and (ii) a code repository, where pieces of code for the conversion between different formulations implemented in available software solutions will be shared.

Systematic errors may result from the adoption of an incomplete functional model that is not able to properly incorporate all the effects involved in the image formation process. These errors very likely appear as systematic residual patterns in image observations and produce deformations of the photogrammetric model in object space. The Brown/Beyer model of self-calibration is often adopted in underwater photogrammetry, although it does not take into account the refraction introduced by the passage of the optical ray through different media, i.e. air and water. This reduces the potential accuracy of photogrammetry underwater. In this work, we investigate through simulations the depth-dependent systematic errors introduced by unmodelled refraction effects when both flat and dome ports are used. The importance of camera geometry to reduce the deformation in the object space is analyzed and mitigation measures to reduce the systematic patterns in image observations are investigated. It is shown how, for flat ports, the use of a stochastic approach, consisting in radial weighting of image observations, improves the accuracy in object space up to 50%. Iterative look-up table corrections are instead adopted to reduce the evident systematic residual patterns in the case of dome ports.

Although fully autonomous mapping methods are becoming more and more common and reliable, still the human operator is regularly employed in many 3D surveying missions. In a number of underwater applications, divers or pilots of remotely operated vehicles (ROVs) are still considered irreplaceable, and tools for real-time visualization of the mapped scene are essential to support and maximize the navigation and surveying efforts. For underwater exploration, image mosaicing has proved to be a valid and effective approach to visualize large mapped areas, often employed in conjunction with autonomous underwater vehicles (AUVs) and ROVs. In this work, we propose the use of a modified image mosaicing algorithm that coupled with image-based real-time navigation and mapping algorithms provides two visual navigation aids. The first is a classic image mosaic, where the recorded and processed images are incrementally added, named 2D sequential image mosaicing (2DSIM). The second one geometrically transform the images so that they are projected as planar point clouds in the 3D space providing an incremental point cloud mosaicing, named 3D sequential image plane projection (3DSIP). In the paper, the implemented procedure is detailed, and experiments in different underwater scenarios presented and discussed. Technical considerations about computational efforts, frame rate capabilities and scalability to different and more compact architectures (i.e. embedded systems) is also provided.

Among the existing Cultural Heritage settings, Underground Built Heritage (UBH) represents a peculiar case. The scarce or lack of knowledge and documentation of these spaces frequently limits their proper management, exploitation, and valorization. When mapping these environments for documentation purposes, the primary need is to achieve a complete, reliable, and adequate representation of the built spaces and their geometry. Terrestrial laser scanners were widely employed for this task, although the procedure is generally time-consuming and often lacks color information. Mobile Mapping Systems (MMSs) are nowadays fascinating and promising technologies for mapping underground structures, speeding up acquisition times. In this paper, mapping experiences (with two commercial tools and an in-house prototype) in UBH settings are presented, testing the different handheld mobile solutions to guarantee an accurate and reliable 3D digitization. Tests were performed in the selected case study of Camerano Caves (Italy), characterized by volumetric complexity, poor lighting conditions, and difficult accessibility. The aim of this research activity is not only to show the differences in the technological instruments used for 3D surveying, but rather to argue over the pros and cons of the systems, providing the community with best practices and rules for 3D data collection with handheld mobile systems. The experiments deliver promising results when compared with TLS data.

Augmented Reality (AR) is already transforming many fields, from medical applications to industry, entertainment and heritage. In its most common form, AR expands reality with virtual 3D elements, providing users with an enhanced and enriched experience of the surroundings. Until now, most of the research work focused on techniques based on markers or on GNSS/INS positioning. These approaches require either the preparation of the scene or a strong satellite signal to work properly. In this paper, we investigate the use of visual-based methods, i.e., methods that exploit distinctive features of the scene estimated with Visual Simultaneous Localization and Mapping (V-SLAM) algorithms, to determine and track the user position and attitude. The detected features, which encode the visual appearance of the scene, can be saved and later used to track the user in successive AR sessions. Existing AR frameworks like Google ARCore, Apple ARKit and Unity AR Foundation recently introduced visual-based localization in their frameworks, but they target mainly small scenarios. We propose a new Mobile Augmented Reality (MAR) methodology that exploits OPEN-V-SLAM to extend the application range of Unity AR Foundation and better handle large-scale environments. The proposed methodology is successfully tested in both controlled and real-case large heritage scenarios. Results are available also in this video: https://youtu.be/Q7VybmiWIuI.

Photogrammetry needs known geometric elements to provide metric traceable measurements. These known elements can be a distance between two three-dimensional object points or two camera stations, or a combination of known coordinates and/or angles to solve the seven degrees of freedom that lead to rank deficiency of the normal-equation matrix. In this paper we present a novel approach for scaling and levelling to the local vertical direction an underwater photogrammetric survey. The developed methodology is based on a portable low-cost device designed and realized by the authors that uses depth measurements from a high resolution pressure sensor. The prototype consists of a data logger featuring a pressure sensor synchronized with a digital camera in its underwater pressure housing. The modular design, with optical communication and synchronization, provides great flexibility not requiring the camera housing to undergo any hardware modifications. The proposed methodology allows for a full 3D levelling transformation comprising two angles, a vertical translation and a scale factor and can work for surveying scenes extending horizontally, vertically or both. The paper presents the theoretical principles, an overview of the developed system together with preliminary calibration results. Tests in a lake and at sea are reported. An accuracy better than 1:5000 on the length measurement was achieved in calm water conditions.

Mobile and handheld mapping systems are becoming widely used nowadays as fast and cost-effective data acquisition systems for 3D reconstruction purposes. While most of the research and commercial systems are based on active sensors, solutions employing only cameras and photogrammetry are attracting more and more interest due to their significantly minor costs, size and power consumption. In this work we propose an ARM-based, low-cost and lightweight stereo vision mobile mapping system based on a Visual Simultaneous Localization And Mapping (V-SLAM) algorithm. The prototype system, named GuPho (Guided Photogrammetric System), also integrates an in-house guidance system which enables optimized image acquisitions, robust management of the cameras and feedback on positioning and acquisition speed. The presented results show the effectiveness of the developed prototype in mapping large scenarios, enabling motion blur prevention, robust camera exposure control and achieving accurate 3D results.

The Azure Window was a natural arch situated in the west coast of Gozo (Maltese Archipelago) that collapsed in March 2017. We employ a Diver Propulsion Vehicle-mounted camera system to capture data for the 3D-modelling of this collapsed arch via photogrammetry. We demonstrate use of this method to document complex underwater geomorphology spread across a large area, and draw up a geomorphic assessment of the site and collapse event on the basis of this 3D-model. The methodology enables a reconstruction and understanding of the collapse event. On account of the high-resolution attained, we are able to cross-match the principal submerged components with their pre-collapse location; this enables an understanding of the dynamics of the collapse, confirmation of rock break-up along existing joints, and mapping the distribution of the rock and debris from the collapse event. We conclude that the key stages in the collapse of the Azure Window entailed erosion at the base of the pillar, leading to the latter's collapse in the southwest direction, breaking into two main sections that separated along the lithological boundary. We also find clear evidence that separation of some sections of the pillar followed pre-existing joints. The bridge collapsed vertically upon loss of support from the pillar, breaking into two main components and many other fragments. We also document further changes at the site post-collapse. We show that this approach can be utilised to understand and characterise such events even when significant time has elapsed since collapse, and rocks have already undergone erosion and significant marine growth., 33 pages, 14 figures, accepted for publication in Geomorphology

The Special Issue “Underwater 3D recording and modelling” is focused onchallenges for 3D modeling and ways to overcome them in the underwater environment. Given that existing sensors and algorithms are not optimized, nor present the best possible solutions for the harsh conditions of the submerged environment, new techniques and methods need to be developed. During the last years, we have witnessed groundbreaking technological developments, which allow underwater documentation with unprecedented accuracy and detail. Photogrammetry‐based approaches coupled withvirtual and augmented reality (VR/AR) applications are becoming infused ininterdisciplinary research in topics such as archeology, biology, industry. [...]

During underwater investigations, whatever the mission objective and the type of vehicle, obstacles detection and avoidance are essential tasks. They can either represent a target of interest that is the object of the mission or, on the contrary, represent obstacles that can hinder or affect the navigation of the vehicle. The underwater optical cameras that are usually fitted to underwater vehicles only offer a narrow field of view. The absorption of electromagnetic waves in the first few meters and the diffusion of light by the particles limit the use of these sensors to only a few meters range. The use of acoustic sensors, such as the forward looking sonar (FLS), is then necessary to enlarge the volume in which a target can be detected during the progression of the vehicle. Traditionally, sonars featured mechanical rotating parts, but lately bidirectional forward looking sonar, which directly produces a 2D image of the area, are becoming more and more common. Although these sonars can operate at frequency higher than 1MHz, their spatial resolution remains much lower if compared to current optical sensors and can be insufficient to identify and characterize a target. The combination of these two sensors in an operational scenario is essential to take advantage of each technology. In this paper we describe a low cost, multi-sensor, underwater survey solution for the identification, tracking, and 3D mapping of targets. After a description of the architecture of the opto-acoustics data acquisition and processing platform, we will focus on the calibration of the rigid transformation between the two sensors.

Underwater photogrammetry is increasingly being used by marine ecologists because of its ability to produce accurate, spatially detailed, non-destructive measurements of benthic communities, coupled with affordability and ease of use. However, independent quality control, rigorous imaging system set-up, optimal geometry design and a strict modeling of the imaging process are essential to achieving a high degree of measurable accuracy and resolution. If a proper photogrammetric approach that enables the formal description of the propagation of measurement error and modeling uncertainties is not undertaken, statements regarding the statistical significance of the results are limited. In this paper, we tackle these critical topics, based on the experience gained in the Moorea Island Digital Ecosystem Avatar (IDEA) project, where we have developed a rigorous underwater photogrammetric pipeline for coral reef monitoring and change detection. Here, we discuss the need for a permanent, underwater geodetic network, which serves to define a temporally stable reference datum and a check for the time series of photogrammetrically derived three-dimensional (3D) models of the reef structure. We present a methodology to evaluate the suitability of several underwater camera systems for photogrammetric and multi-temporal monitoring purposes and stress the importance of camera network geometry to minimize the deformations of photogrammetrically derived 3D reef models. Finally, we incorporate the measurement and modeling uncertainties of the full photogrammetric process into a simple and flexible framework for detecting statistically significant changes among a time series of models., Remote Sensing, 12 (18), ISSN:2072-4292

The VAST (valorisation of history and landscape) project (http://vast.fbk.eu/) was part of the initiatives promoted by the Autonomous Province of Trento (Italy) for the commemoration of the 100th Anniversary of the First World War (WWI) outbreak. The project was primarily aimed to document and promote, through 3D digitization approaches and communication material, the memory of sites, theatre of the world conflict. The Italian Trento’s province had been under the Austro–Hungarian Empire until the end of the WWI, during which represented a crucial and bloody war front between the Austrian and Italian territories. The region was constellated of military fortresses, trenches and tunnels, most of them now ruined and at risk to slowly disappear. 3D surveying and modelling techniques have been exploited to produce 3D digital models of structures and objects, along with virtual tours, communication material and a WebGIS of the area. All the products are available on the web for valorisation, educational and communication purposes.

The paper reports some comparisons between commercial software able to automatically process image datasets for 3D reconstruction purposes. The main aspects investigated in the work are the capability to correctly orient large sets of image of complex environments, the metric quality of the results, replicability and redundancy. Different datasets are employed, each one featuring a diverse number of images, GSDs at cm and mm resolutions, and ground truth information to perform statistical analyses of the 3D results. A summary of (photogrammetric) terms is also provided, in order to provide rigorous terms of reference for comparisons and critical analyses.

Underwater photogrammetry, like its counterpart in 'air', has gained an increasing diffusion thanks to the availability of easy-to-use, fast and often quite inexpensive software applications. Moreover, underwater equipment that allows the use of digital cameras normally designed to work in air also in water are largely available. However, for assuring accurate and reliable 3D modelling results a profound knowledge of the employed devices as well as physical and geometric principle is even more crucial than in air. This study aims to take a step forward in understanding the effect of underwater ports in front of the photographic lens. In particular, the effect of dome or flat ports on image quality in 3D modelling applications is investigated. Experiments conducted in a semi submerged indust rial structure show that the tested flat port performs worse than the dome, providing higher image residuals and lower precision and accuracy in object space. A significant different quality per colour channel is also observed and its influence on achievable processing results is discussed.

Portable mobile mapping systems (PMMSs) are an emerging technology, increasingly employed in different application fields. Their distinctive feature is the ability to acquire 3D information on-the-move, and, unlike vehicle-based MMSs, the possibility to efficiently document narrow, indoor and confined environments. The present chapter provides an overview of PMMSs, with a taxonomy of the available platforms and a description of hardware components and data processing algorithms. A review about 3D quality assessment and typical applications is also reported, with the aim of providing readers with the basic knowledge and understanding of potentialities and challenges of this promising technology.

International audience; Augmented Reality (AR) is already transforming many fields, from medical applications to industry, entertainment and heritage. In its most common form, AR expands reality with virtual 3D elements, providing users with an enhanced and enriched experience of the surroundings. Until now, most of the research work focused on techniques based on markers or on GNSS/INS positioning. These approaches require either the preparation of the scene or a strong satellite signal to work properly. In this paper, we investigate the use of visual-based methods, i.e., methods that exploit distinctive features of the scene estimated with Visual Simultaneous Localization and Mapping (V-SLAM) algorithms, to determine and track the user position and attitude. The detected features, which encode the visual appearance of the scene, can be saved and later used to track the user in successive AR sessions. Existing AR frameworks like Google ARCore, Apple ARKit and Unity AR Foundation recently introduced visual-based localization in their frameworks, but they target mainly small scenarios. We propose a new Mobile Augmented Reality (MAR) methodology that exploits OPEN-V-SLAM to extend the application range of Unity AR Foundation and better handle large-scale environments. The proposed methodology is successfully tested in both controlled and real-case large heritage scenarios. Results are available also in this video: https://youtu.be/Q7VybmiWIuI. a) b) c) Figure 1. The three large-scale scenarios used in the paper and the AR results based on markerless smartphone solution: (a) historical photographs of the city of Trento, (b) the remains of the underground roman city in Trento and (c) the pile dwelling site of Fiavè.

An underwater imaging system with camera and lens behind a flat port does not behave as a standard pinhole camera with additional parameters. Indeed, whenever the entrance pupil of the lens is not in contact with the flat port, the standard photogrammetric model is not suited anymore and an extended mathematical model that considers the different media would be required. Therefore, when dealing with flat ports, the use of the classic photogrammetric formulation represents a simplification of the image formation phenomenon, clearly causing a degradation in accuracy. Furthermore, flat ports significantly change the characteristics of the enclosed imaging device and negatively affect the image quality, introducing heavy curvilinear distortions and optical aberrations. With the aim of mitigating the effect of systematic errors introduced by a combination of (i) image quality degradation, induced by the flat ports, and (ii) a non-rigorous modelling of refraction, this paper presents a stochastic model for image observations that penalises those that are more affected by aberrations and departure from the pinhole model. Experiments were carried out at sea and in pools showing that the use of the proposed stochastic model is beneficial for the final accuracy with improvements up to 50 %.

International audience; The main purpose of this paper is to show that photogrammetric bundle adjustment computations can be sequentially organized into modules. Furthermore, the chain rule can be used to simplify the computation of the analytical Jacobians needed by the adjustment. Novel projection models can be flexibly evaluated by inserting, modifying, or swapping the order of selected modules. As a proof of concept, two variants of the pin-hole projection model with Brown lens distortion were implemented in the open-source Damped Bundle Adjustment Toolbox (DBAT) and applied to simulated and calibration data for a non-conventional lens system. The results show a significant difference for the simulated, error-free, data but not for the real calibration data. The current flexible implementation incurs a performance loss. However, in cases where flexibility is more important, the modular formulation should be a useful tool to investigate novel sensors, data processing techniques, and refractive models.

Metrology is fundamental in all the applications that require to qualify, verify and validate measured data according to standards or, in other words, to assess their compliance with predefined tolerances. At sea, metrology is commonly associated with the process of measuring underwater structures, mainly pipeline elements widely used in offshore industry. Subsea operations are very expensive; optimizing time and money resources are the core factors driving innovation in the subsea metrology industry. In this study, the authors investigate the use of state-of-art vision-based algorithms, i.e. ORB-SLAM2 and Visual Odometry, as a navigation tool to assist and control a Remotely Operated Vehicle (ROV) while performing subsea metrology operations. In particular, the manuscript will focus on methods for assessing the accuracy of both trajectory and tie points provided by the tested approaches and evaluating whether the preliminary real time reconstruction meets the tolerances defined in typical subsea metrology scenarios., International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 42 (2/W10), ISSN:1682-1750, ISSN:2194-9034, ISSN:1682-1777

The paper presents the first results of an interdisciplinary project related to the 3D documentation, dissemination, valorization and digital access of archeological sites. Beside the mere 3D documentation aim, the project has two goals: (i) to easily explore and share via web references and results of the interdisciplinary work, including the interpretative process and the final reconstruction of the remains; (ii) to promote and valorize archaeological areas using reality-based 3D data and Virtual Reality devices. This method has been verified on the ruins of the archeological site of Pausilypon, a maritime villa of Roman period (Naples, Italy). Using Unity3D, the virtual tour of the heritage site was integrated and enriched with the surveyed 3D data, text documents, CAAD reconstruction hypotheses, drawings, photos, etc. In this way, starting from the actual appearance of the ruins (panoramic images), passing through the 3D digital surveying models and several other historical information, the user is able to access virtual contents and reconstructed scenarios, all in a single virtual, interactive and immersive environment. These contents and scenarios allow to derive documentation and geometrical information, understand the site, perform analyses, see interpretative processes, communicate historical information and valorize the heritage location.

Photogrammetry is currently facing some challenges and changes mainly related to automation, ubiquitous processing and variety of applications. Within an ISPRS Scientific Initiative a team of researchers from USAL, UCLM, FBK and UNIBO have developed an open photogrammetric tool, called GRAPHOS (inteGRAted PHOtogrammetric Suite). GRAPHOS allows to obtain dense and metric 3D point clouds from terrestrial and UAV images. It encloses robust photogrammetric and computer vision algorithms with the following aims: (i) increase automation, allowing to get dense 3D point clouds through a friendly and easy-to-use interface; (ii) increase flexibility, working with any type of images, scenarios and cameras; (iii) improve quality, guaranteeing high accuracy and resolution; (iv) preserve photogrammetric reliability and repeatability. Last but not least, GRAPHOS has also an educational component reinforced with some didactical explanations about algorithms and their performance. The developments were carried out at different levels: GUI realization, image pre-processing, photogrammetric processing with weight parameters, dataset creation and system evaluation. The paper will present in detail the developments of GRAPHOS with all its photogrammetric components and the evaluation analyses based on various image datasets. GRAPHOS is distributed for free for research and educational needs.