Search

Urban visual localization is the process of determining the pose (position and attitude) of the imaging sensor (or platform) with the help of existing geo-referenced data. This task is critical and challenging for many applications, such as autonomous navigation, virtual and augmented reality, and robotics, due to the dynamic and complex nature of urban environments that may obstruct Global Navigation Satellite Systems (GNSS) signals. This paper proposes a block-wise matching strategy for urban visual localization by using geo-referenced Google Street View (GSV) panoramas as the database. To determine the pose of the monocular query images collected from a moving vehicle, neighboring GSVs should be found to establish the correspondence through image-wise and block-wise matching. First, each query image is semantically segmented and a template containing all permanent objects is generated. The template is then utilized in conjunction with a template matching approach to identify the corresponding patch from each GSV image within the database. Through the conversion of the query template and corresponding GSV patch into feature vectors, their image-wise similarity is computed pairwise. To ensure reliable matching, the query images are temporally grouped into query blocks, while the GSV images are spatially organized into GSV blocks. By using the previously computed image-wise similarities, we calculate a block-wise similarity for each query block with respect to every GSV block. A query block and its corresponding GSV blocks of top-ranked similarities are then input into a photogrammetric triangulation or structure from motion process to determine the pose of every image in the query block. A total of three datasets, consisting of two public ones and one newly collected on the Purdue campus, are utilized to demonstrate the performance of the proposed method. It is shown it can achieve a meter-level positioning accuracy and is robust to changes in acquisition conditions, such as image resolution, scene complexity, and the time of day.

A widely used method for estimating fine scale long-term spatial variation in air pollution, especially for epidemiology studies, is land use regression (LUR) modeling using fixed off-road monitors. More recently, LUR models have been developed using data from mobile monitors that repeatedly measure road pollutants and mixed-effects modeling. Here, nitrogen dioxide (NO2) and black carbon (BC) predictions from two independent models were compared across streets (defined as 30–60 m road segments) (N = 30,312) and residences (N = 76,752) in Copenhagen, Denmark. The first model was Google Street View (GSV)-based mixed-effects LUR models (Google-MM) that predicted 2019 mean NO2 and BC levels, and the second was European-wide (EUW) LUR models that predicted annual mean 2010 levels at 100 m spatial resolution. Across street segments, the Spearman correlation coefficient between the 2019 NO2 from Google-MM-LUR and 2010 NO2 from EUW-LUR was 0.66, while at residences, this was 0.60. For BC, these were 0.51 across street segments and 0.40 at the residential level. The ratio of percentile 97.5 to 2.5 for NO2 across the study area streets using Google-MM NO2 was 4.5, while using EUW-LUR, this was 2.1. These NO2 ratios at residences were 3.1 using Google-MM LUR, and 1.7 using EUW-LUR. Such ratios for BC across street segments were 3.4 using Google-MM LUR and 2.3 using EUW-LUR, while at the residential level, they were 2.4 and 1.9, respectively. In conclusion, Google-MM-LUR NO2 for 2019 was moderately correlated with EUW-LUR NO2 developed in 2010 across Copenhagen street segments and residences. For BC, while Google-MM-LUR was moderately correlated with EUW-LUR across Copenhagen streets, the correlation was lower at the residential level. Overall, Google-MM-LUR revealed larger spatial contrasts than EUW-LUR.

With Electric Vehicles (EV) emerging as the dominant form of green transport in the UK, it is critical that we better understand existing infrastructures in place to support the uptake of these vehicles. In this multi-disciplinary paper, we demonstrate a novel end-to-end workflow using deep learning to perform automated surveys of urban areas to identify residential properties suitable for EV charging. A unique dataset comprised of open source Google Street View images was used to train and compare three deep neural networks and represents the first attempt to classify residential driveways from streetscape imagery. We demonstrate the full system workflow on two urban areas and achieve accuracies of 87.2% and 89.3% respectively. This proof of concept demonstrates a promising new application of deep learning in the field of remote sensing, geospatial analysis, and urban planning, as well as a major step towards fully autonomous artificially intelligent surveying techniques of the built environment.

This study takes one step further to complement the application of a method for mapping informal green spaces (IGSs) using an efficient combination of open-source data with simple tools and algorithms. IGSs are unofficially recognized by the government as vegetation spaces designed for recreation, gardening, and forestry in urban areas. Due to the economic crisis, many formal green spaces such as urban parks and garden projects have been postponed, while IGSs have significant potential as green space retrofits. However, because they are small and spatially continuous and cannot be fully detected via airborne surveys, they are surveyed in small areas and neglected by government and city planners. Therefore, in this research, we combined the use of Google Street View (GSV) data with machine learning to develop a survey method that can be used to survey a wide area at once. Deeplab V3+ was used to segment the semantics based on the model created using 1000 labelled photos, with an accuracy rate of nearly 65%. Applying this method gave high accuracy in Ichikawa, Japan, with 3029 photos, and matched the results of a field survey in a previous study. In contrast, low accuracy was seen in Ho Chi Minh City, with 204 photos, where the quality of the GSV data was considerably lower.

Extreme heat exposure has severe negative impacts on humans, and the issue is exacerbated by climate change. Estimating spatial heat stress such as mean radiant temperature (MRT) is currently difficult to apply at city scale. This study constructed a method for estimating the MRT of street canyons using Google Street View (GSV) images and investigated its large-scale spatial patterns at street level. We used image segmentation using deep learning to calculate the view factor (VF) and project panorama into fisheye images. We calculated sun paths to estimate MRT using panorama images from Google Street View. This paper shows that regression analysis can be used to validate between estimated short-wave, long-wave radiation and the measurement data at seven field measurements in the clear-sky (0.97 and 0.77, respectively). Additionally, we compared the calculated MRT and land surface temperature (LST) from Landsat 8 on a city scale. As a result of investigating spatial patterns of MRT in Seoul, South Korea, we found that a high MRT of street canyons (>59.4 °C) is mainly distributed in open space areas and compact low-rise density buildings where the sky view factor is 0.6–1.0 and the building view factor (BVF) is 0.35–0.5, or west-east oriented street canyons with an SVF of 0.3–0.55. However, high-density buildings (BVF: 0.4–0.6) or high-density tree areas (Tree View Factor, TVF: 0.6–0.99) showed low MRT (

Collecting neighborhood data can both be time- and resource-intensive, especially across broad geographies. In this study, we leveraged 1.4 million publicly available Google Street View (GSV) images from Utah to construct indicators of the neighborhood built environment and evaluate their associations with 2017–2019 health outcomes of approximately one-third of the population living in Utah. The use of electronic medical records allows for the assessment of associations between neighborhood characteristics and individual-level health outcomes while controlling for predisposing factors, which distinguishes this study from previous GSV studies that were ecological in nature. Among 938,085 adult patients, we found that individuals living in communities in the highest tertiles of green streets and non-single-family homes have 10–27% lower diabetes, uncontrolled diabetes, hypertension, and obesity, but higher substance use disorders—controlling for age, White race, Hispanic ethnicity, religion, marital status, health insurance, and area deprivation index. Conversely, the presence of visible utility wires overhead was associated with 5–10% more diabetes, uncontrolled diabetes, hypertension, obesity, and substance use disorders. Our study found that non-single-family and green streets were related to a lower prevalence of chronic conditions, while visible utility wires and single-lane roads were connected with a higher burden of chronic conditions. These contextual characteristics can better help healthcare organizations understand the drivers of their patients’ health by further considering patients’ residential environments, which present both risks and resources.

The recovery of a built heritage and specifically of singular buildings is a key aspect of local development. The aim of this study was to understand the influence of these regenerations on their environment by transforming adjacent businesses and initiating parallel processes of gentrification and local development. The renewed attraction of these new businesses to the area can result in increased employment and production. The methodology used was based on self-organizing maps of neural networks with matrix architecture and competitive learning. Through the analysis of neural networks, we were able to identify common relationships and behaviors in commercial properties which are adjacent to singular buildings and that share common patterns and characteristics or attributes. The singular buildings analyzed are located along the Spanish Mediterranean coast in the cities of Almería, Barcelona, and Valencia. The results obtained were based on the following hypotheses: occupancy model and the classification based on total occupancy, total variation in occupancy, and the most common types of usage of a given ground floor commercial property. Among the conclusions, we highlight the existence of commercial premises that display anti-cyclical economic behavior and the presence of commercial premises considered to be “unfortunate” or with low potential.

Building databases are important assets when estimating and planning for national energy savings from energy retrofitting. However, databases often lack information on building characteristics needed to determine the feasibility of specific energy conservation measures. In this paper, machine learning methods are used to enrich the Swedish database of Energy Performance Certificates with building characteristics relevant for a chosen set of energy retrofitting packages. The study is limited to the Swedish multifamily building stock constructed between 1945 and 1975, as these buildings are facing refurbishment needs that advantageously can be combined with energy retrofitting. In total, 514 ocular observations were conducted in Google Street View of two building characteristics that were needed to determine the feasibility of the chosen energy retrofitting packages: (i) building type and (ii) suitability for additional façade insulation. Results showed that these building characteristics could be predicted with an accuracy of 88.9% and 72.5% respectively. It could be concluded that machine learning methods show promising potential to enrich building databases with building characteristics relevant for energy retrofitting, which in turn can improve estimations of national energy savings potential.

(1) Background: Evidence-based policymaking requires data about the local population’s socioeconomic status (SES) at detailed geographical level, however, such information is often not available, or is too expensive to acquire. Researchers have proposed solutions to estimate SES indicators by analyzing Google Street View images, however, these methods are also resource-intensive, since they require large volumes of manually labeled training data. (2) Methods: We propose a methodology for automatically computing surrogate variables of SES indicators using street images of parked cars and deep multiple instance learning. Our approach does not require any manually created labels, apart from data already available by statistical authorities, while the entire pipeline for image acquisition, parked car detection, car classification, and surrogate variable computation is fully automated. The proposed surrogate variables are then used in linear regression models to estimate the target SES indicators. (3) Results: We implement and evaluate a model based on the proposed surrogate variable at 30 municipalities of varying SES in Greece. Our model has R 2 = 0.76 and a correlation coefficient of 0.874 with the true unemployment rate, while it achieves a mean absolute percentage error of 0.089 and mean absolute error of 1.87 on a held-out test set. Similar results are also obtained for other socioeconomic indicators, related to education level and occupational prestige. (4) Conclusions: The proposed methodology can be used to estimate SES indicators at the local level automatically, using images of parked cars detected via Google Street View, without the need for any manual labeling effort.

Traditional methods of detecting and mapping utility poles are inefficient and costly because of the demand for visual interpretation with quality data sources or intense field inspection. The advent of deep learning for object detection provides an opportunity for detecting utility poles from side-view optical images. In this study, we proposed using a deep learning-based method for automatically mapping roadside utility poles with crossarms (UPCs) from Google Street View (GSV) images. The method combines the state-of-the-art DL object detection algorithm (i.e., the RetinaNet object detection algorithm) and a modified brute-force-based line-of-bearing (LOB, a LOB stands for the ray towards the location of the target [UPC at here] from the original location of the sensor [GSV mobile platform]) measurement method to estimate the locations of detected roadside UPCs from GSV. Experimental results indicate that: (1) both the average precision (AP) and the overall accuracy (OA) are around 0.78 when the intersection-over-union (IoU) threshold is greater than 0.3, based on the testing of 500 GSV images with a total number of 937 objects; and (2) around 2.6%, 47%, and 79% of estimated locations of utility poles are within 1 m, 5 m, and 10 m buffer zones, respectively, around the referenced locations of utility poles. In general, this study indicates that even in a complex background, most utility poles can be detected with the use of DL, and the LOB measurement method can estimate the locations of most UPCs.

Hemispherical (fisheye) photography is a well-established approach for estimating the sky view factor (SVF). High-resolution urban models from LiDAR and oblique airborne photogrammetry can provide continuous SVF estimates over a large urban area, but such data are not always available and are difficult to acquire. Street view panoramas have become widely available in urban areas worldwide: Google Street View (GSV) maintains a global network of panoramas excluding China and several other countries; Baidu Street View (BSV) and Tencent Street View (TSV) focus their panorama acquisition efforts within China, and have covered hundreds of cities therein. In this paper, we approach this issue from a big data perspective by presenting and validating a method for automatic estimation of SVF from massive amounts of street view photographs. Comparisons were made with SVF estimates derived from two independent sources: a LiDAR-based Digital Surface Model (DSM) and an oblique airborne photogrammetry-based 3D city model (OAP3D), resulting in a correlation coefficient of 0.863 and 0.987, respectively. The comparisons demonstrated the capacity of the proposed method to provide reliable SVF estimates. Additionally, we present an application of the proposed method with about 12,000 GSV panoramas to characterize the spatial distribution of SVF over Manhattan Island in New York City. Although this is a proof-of-concept study, it has shown the potential of the proposed approach to assist urban climate and urban planning research. However, further development is needed before this approach can be finally delivered to the urban climate and urban planning communities for practical applications.