Search

Visual saliency modeling of driving scenes is an important research direction in intelligent driving, especially in the areas of assisted driving and automatic driving. The existing visual saliency modeling methods for static and virtual scenes cannot adapt to the real-time, dynamic and task-driven characteristics of road scenes in real driving environments. Building a visual saliency model of dynamic road scenes in real driving environments is a challenge for current research. Starting from the characteristics of driving environment and driver's visual cognitive law, this paper extracts low-level visual features, high-level visual features and dynamic visual features of road scenes, and combines two influencing factors of speed and road curvature to build a visual saliency calculation model of driving scenes based on logistic regression model (LR). In this paper, the AUC value is used to evaluate the model, and the results showed an accuracy of 90.43%, which is significant advantage over traditional algorithms.

There are significant nonlinear intensity differences between multi-modal images. Moreover, the noise in these images will cause image degradation. Therefore, the automatic registration of multi-modal images is a challenging task. To address the two problems, this paper proposes a multi-modal image automatic registration method, which is divided into two stages: pre-registration and fine registration. In the pre-registration stage, an improved SIFT algorithm is used to roughly align multi-modal images. In the fine registration stage, the block Harris detector is first used to extract evenly distributed feature points on the pre-registered reference image. Then, the structure information in the multi-modal images is captured by the anisotropic structure tensor to construct a feature descriptor, which is robust to noise. Furthermore, a similarity criterion named TOMI (tensor orientation and mutual information) is proposed combining the tensor orientation parallelism and gradient mutual information. Finally, Multi-modal images (including Optical, LiDAR, SAR, and Map data) are used to evaluate the proposed algorithm. The experimental results show that the method proposed in this paper is robust to nonlinear intensity differences and noise, and the matching effect is superior.

Using deep convolutional neural network (CNN) to intelligently extract buildings from remote sensing images is of great significance for digital city construction, disaster detection and land management. The color difference between multi-temporal remote sensing images will lead to the decrease of generalization ability of building semantic segmentation model. In view of this, this paper proposes the attention-guided color consistency adversarial network (ACGAN). The algorithm takes the reference color style images and the images to be corrected in the same area and different phases as the training set and adopts the consistency adversarial network with the U-shaped attention mechanism to train the color consistency model. In the prediction stage, this model converts the hue of the images to that of the reference color style image, which is based on the reasoning ability of the deep learning model, instead of the corresponding reference color style image. This model transforms the hue of the images to be corrected into that of the reference color style images. This stage is based on the reasoning ability of the deep learning model, and the corresponding reference color style image is no longer needed. In order to verify the effectiveness of the algorithm, firstly, we compare the algorithm of this paper with the traditional image processing algorithm and other consistency adversarial network. The results show that the images after ACGAN color consistency processing are more similar to that of the reference color style images. Secondly, we carried out the building semantic segmentation experiment on the images processed by the above different color consistency algorithms, which proved that the method in this paper is more conducive to the impro-vement of the generalization ability of multi-temporal remote sensing image semantic segmentation model.

Traffic plays a vital role in people's life and social economy.Vehicle speed detection is an important part of intelligent transportation system.This paper focus on the vehicle speed detection based on multi-source data from autonomous unmanned aerial vehicle (UAV).Firstly,we build a multi-source data acquisition system on UAV for visible image and thermal infrared image.Secondly,we utilize "You only look once" (YOLO),which is a deep learning framework for vehicle detection.Finally,we track the vehicle based on Kalman filter and calculated the vehicle speed according to the result of vehicle tracking.This paper adopts the UAV platform to increase the flexibility.While the use of multi-source data improves the accuracy of the vehicle detection and tracks the vehicle in different illumination.The result of experiments shows that the strategy is effective and robust,which provides an efficient and flexible monitoring mode for traffic management department.

The existing three-dimensional (3D) visualization of flood disaster scenes has problems such as weak immersion and lack of mobility.This paper combines mobile virtual reality (VR) with 3D visualization of flood disaster scenes,and proposes a construction optimization and interaction method for flood disaster scenes based on mobile VR.Firstly,a mobile VR scene construction scheme under a plugin-free B/S architecture is proposed.Second,certain key technologies for scene optimization are discussed,including diverse modes of scene data organization,optimization control of scene data volume and adaptive scheduling of mobile scenes.Finally,some interaction and analysis schemes based on gaze,such as scene roaming exploration and interactive flood information query,are designed.A test platform was developed and a case experiment analysis was carried out.The experiment results demonstrate that the flood disaster scene constructed by the methods addressed in this paper can be displayed smoothly and explored interactively on smart phones.

This paper is the third part of the three dimensional digital photogrammetry intelligent conformation under the machine vision parameter:the direct 3D imaging mechanism of the light field and the 3D intelligent conformation benchmark of digital photogrammetry.In this paper,the moving object’s capture takes 3-3-2 information processing as the starting point,the basic principle of using biological compound eye three-dimensional imaging,however,because the traditional bionic vision by the hardware constraints,this paper obtained the theory based on bionic compound eye 3-3-2 information,the use of a new generation of micro lens array light field camera technology,put forward a method of extracting depth information of the scene to single light field images,and its application to absolute depth measurement.The light field image an important characteristic is that the intensity and direction of different light information recording into the camera,this paper proposes the use of light information for multi depth focusing,the focusing points in different levels of cost calculation,determine the object image plane,finally construct the depth map object point based on the location of the imaging plane.Compared with the traditional light depth estimation,the depth map obtained by this method can greatly improve the resolution,and does not depend on the number of micro lens in the light field.In order to verify the effectiveness of the proposed method,this paper will compare the depth calculation using the algorithm proposed in this paper and the existing depth information algorithm.

This paper focuses on intelligent photogrammetry deep learning:deep residual method.Salient object detection aims at identifying the visually interesting object regions that are consistent with human perception.Multispectral remote sensing images provide rich radiometric information in revealing the physical properties of the observed objects,therefore promise a great potential in salient object detection tasks.Conventional salient object detection methods often employ handcrafted features to predict saliency by evaluating the pixel-wise or superpixel-wise similarity.With the recent emergence of deep learning based approaches,in particular,fully convolutional neural networks,there has been profound progress in visual saliency detection.However,this success has not been extended to multispectral remote sensing images,and existing multispectral salient object detection methods are still mainly based on handcrafted features,essentially due to the difficulties in image acquisition and labeling.In this paper,we propose a novel deep residual network based on a top-down model,which is trained in an end-to-end manner to tackle the above issues in multispectral salient object detection.Our model effectively exploits the saliency cues at different levels of the deep residual network.To overcome the limited availability of remote sensing images in training of our deep residual network,we also introduce a new spectral image reconstruction model that can generate multispectral images from RGB images.Our extensive experimental evaluations using both multispectral and RGB salient object detection datasets demonstrate a significant performance improvement of more than 10% compared with the state-of-the-art methods.

Accuracy is a key factor in high-resolution remote sensing and photogrammetry.The factors that affect accuracy are imaging system errors and data processing errors.Because of the complexity of aerial camera system errors,this paper focuses on the design of digital aerial camera system,to reduce the system error and provide data procession fundamentally.There are many kinds of digital aerial camera system at present,but lacking a unified physical model,which causes the system to be built in multi-camera and multi-rigid model.Such system is complex,costly,and difficult to describe,and is easily affected by factors such as vibration and temperature,so the installed accuracy can only reach millimeter level.For this reason,this paper proposes the unified physical structure of digital aerial camera,which imitates the theory of out-of-field multi-CCD,in-field multi-CCD,and once-imaging and twice-imaging digital camera systems.Considering this,the spatial-temporal representation of the variable baseline-height ratio is established.From the variable baseline-height ratio,we can link the opto-mechanical spatial parameters with the elevation accuracy,so that to achieve connection between the surface elevation with opto-mechanical structural parameter; further designing the twice-imaging digital camera prototype system and the wideband limb imaging spectrometer,which provides prototype for transformation from the current multi-rigid,one-time imaging aerial camera to single rigid structure.Our research has laid a theoretical foundation and prototype references for the construction and industrialization of digital aerial system.

Because of the lack of relationships between matching line and adjacent lines in the process of individual line matching, and the weak reliability of the individual line descriptor facing on discontinue texture, this paper presents a reliable line matching algorithm for stereo images with topological relationship. The algorithm firstly generates grouped line pairs from lines extracted from the reference image and searching image according to the basic topological relationships such as distance and angle between the lines. Then it takes the grouped line pairs as matching primitives, and matches these grouped line pairs by using epipolar constraint, homography constraint, quadrant constraint and gray correlation constraint of irregular triangle in order. And finally, it resolves the corresponding line pairs into two pairs of corresponding individual lines, and obtains one to one matching results after the post-processing of integrating, fitting, and checking. This paper adopts digital aerial images and close-range images with typical texture features to deal with the parameter analysis and line matching, and the experiment results demonstrate that the proposed algorithm in this paper can obtain reliable line matching results.

With the impetus of ICT, cartography faces the unprecedented challenges. The paper discusses the problems and changes of cartography facing the digital technology, analyzes the constraints of traditional cartography that are mainly delimited by 2D physical paper map. Diverseness of modern cartography shows various map products, and the paper illustrates the digital freedom in information space of modern cartography from eight aspects, including physical reality VS virtual reality, paper map VS digital map, superficial visualization VS inner visualization and so on. Modern cartography encounters the new development opportunities and fresh demands in digital era, and it's necessary to extend the framework of cartography and to assimilate newly sprouted things to promote the rejuvenation of cartography.

This paper introduces the extended collinearity equations into aerial triangulation of the vehicle-borne multi-camera rig images to improve the positioning accuracy. The cameras in a multi-camera rig are rigidly fixed and the relative position and orientation parameters among the mono cameras of multi-camera rig can be calibrated accurately, so the extended collinearity equations can be used to extend the imaging unit from a mono camera image to multiple images of multi-camera rigs. Compared with the existing spherical models, including the spherical ideal model and the spherical rigorous model, the extended collinearity equations used in this paper avoid the spherical projection error and fusion error caused by the misalignment of projection centers. Experimental results show that the method omits the processing procedure of model projection and fusion of overlap areas, which avoids the precision loss and complicated processing procedure, and finally obtains more robustness triangulation net, more precise and robust position accuracy.

Geometric calibration of non-metric digital camera, including the accurate determination of interior orientation elements and distortion parameters, directly affects the ultimate accuracy of its using in aerial photogrammetry. In this paper, a laboratory geometric calibration system of non-metric digital camera is developed. A collimator and a star tester are used as the target generator in this system. After measuring locations of targets in CCD and corresponding angles of parallel lights, high precision geometric calibration can be accomplished. Two kinds of mainstream digital aerial cameras, Canon EOS 5D Mark Ⅱ and Hasselblad H3D, are used to take experiments based on this system and the results are analyzed. Experiment results show that the method of this paper is correct and the calibration precision is reliable.

Simultaneous localization and mapping (SLAM) has broad application prospects in many fields due to its high energy efficiency and low power consumption. However, there are still some problems in the traditional SLAM system: the key frame in the traditional visual odometer does not contain semantic information, the image information obtained by the mobile robot is relatively single, and the key frame in the actual scene always contains a large number of mismatched points and dynamic points. In response to the above problems, this paper proposes a new idea of semantic SLAM mapping technology. First, in order to find the correct and corresponding feature points, abandon the interference of dynamic points and mismatched points simultaneously, a method for judging the feature state of adjacent frames based on the Lucas-Kanade optical flow method is proposed, and this function is regarded as a new feature. The thread is added to the visual odometry part of ORB-SLAM3 to complete the optimization and improvement of part of the traditional SLAM framework. Secondly, in view of the problem that the image frame obtained by the front-end visual odometer of the traditional SLAM system does not contain any semantic information, the target detection algorithm based on YOLOV4 and the Mask R-CNN semantic segmentation algorithm fused with fully connected conditional random field CRF are used to compare ORB-SLAM3. The key frame image processing of the robot effectively improves the perception of the indoor environment of smart devices such as robots.

To solve the problem of matching difficulty caused by the significant nonlinear grayscale differences between SAR and optical images, this paper proposes a fast matching algorithm based on image structural properties named SOFM(SAR-to-optical fast matching algorithm).The traditional methods based on image grayscale are generally difficult to resist the nonlinear grayscale differences between SAR and optical images, but the geometric constructs and shape features can exist stably among different types of images, so in our the proposed method both the magnitude and orientation information of image gradient are used to build a geometric structural feature descriptor named AWOG(angular weighted orientated gradients), then based on the template matching strategy, the sum of squared difference of the descriptors is used to define the similarity metric for matching and then the image matching function expressed in the frequency domain is given. A complete set of image matching process is established based on SOFM, and has been validated using multiple pairs of SAR and optical images, the results show that the proposed method can effectively resist the nonlinear grayscale differences between SAR and optical images, and outperforms the traditional classical image grayscale-based methods and existing image structural-based methods in matching performance and precision.

Remote sensing image retrieval (RSIR) aims to find relevant images of a query image from a remote sensing image retrieval dataset. But the similarity between a query image and a retrieval image is generally used and the relationship among images on the retrieval dataset is neglected during the retrieval process. To deal with the problem, this paper presents a new retrieval method based on ant colony optimization (ACO) for RSIR. First, our method uses the pheromone to represent the similarity between images on the retrieval dataset; then the pheromone matrix is updated by ACO. Finally, the pheromone of images is used to improve the performance of RSIR. Meanwhile, an improved weighted image-to-class distance is used to measure the similarity between two images for further improving the retrieval performance. Extensive experiments are conducted on two publicly available remote sensing image databases, UCMD and PatternNet. Compared with the state-of-the-art methods, the proposed method can achieve better retrieval results.

The result of point cloud semantic segmentation includes the recognition of multiple objects in the scene, which is an important part of 3D scene information extraction. It also plays a key role in many fields such as smart cities. Since the large amount of data and high scene complexity, however, most existing methods can only extract a limited type of objects with a relatively low recognition rate from laser scanning data. This paper proposes a hierarchical multiple object automatic extraction framework combining residual learning and Markov random field (MRF) optimization in a 3D LiDAR point cloud scene. The framework first filters raw data into ground points and non-ground points; buildings are extracted from non-ground points to reduce scene complexity; then the residual learning network is adopted to pointwise classify the remaining point clouds; finally, a MRF-based optimization is used for post-processing and improving the accuracy of point cloud classification. In order to evaluate the effectiveness and robustness of the proposed method, experiments were performed on three outdoor large-scale point cloud scenarios. Experimental results show that the proposed method can perform effective semantic segmentation on various types of point cloud scenes, with (94.6%, 96.8%, 95.7%), (88.5%, 90.5%, 89.2%) and (95.3%, 95.2%, 95.3%), respectively. In addition, compared with existing advanced methods, it is shown that our method significantly improves the performance of semantic segmentation.

Aiming at the problems of nonlinear gray difference, speckle noise and different imaging viewpoints in SAR and optical image registration, this paper presents a SAR and optical images registration using uniform distribution and structure description-based ASIFT. In the proposed algorithm, firstly, the guided scale space is constructed by guided filter to achieve noise suppression and edge preservation. In the feature extraction stage, the phase congruency is utilized due to the nonlinear gray difference, and combined with scale space gridding to extract from images for the uniform feature points. In the feature description stage, the consistency gradient magnitude and orientation of SAR and optical image are calculated by extended phase congruency method, which improves the accuracy of the main orientation and descriptor. At last, Optimal-RANSAC is used to establish feature descriptor matching to achieve effective registration. The simulation experiment and analysis on four pairs of real images show that the proposed algorithm has more accurate registration accuracy than SAR-SIFT and traditional ASIFT.

Traditional fuzzy C-means clustering (FCM) only considers the gray features of image in image segmentation, which results in unsatisfactory segmentation results when the algorithm is applied to high-resolution remote sensing image segmentation. In order to solve this problem, a new method of superpixel segmentation method of high-resolution remote sensing image based on fuzzy C-means clustering is proposed in this paper. Firstly, watershed transform algorithm is used to generate multiple superpixels, and then the similarity of spectrum features among superpixels are compared. Finally, these superpixels are merged by a FCM method combined with spectrum features. Four sets of remote sensing images of different scenes were selected in the experiment, and the experimental results were evaluated by combining qualitative and quantitative methods. The experimental results show that the method can effectively improve accuracy of the segmentation and achieve better visual effect of the segmentation.

As the extension of image stitching, video stitching plays an important role in scene monitoring, target recognition and so on. Traditional video stitching methods are mostly suitable for the videos with large overlapping regions and only geometric features of images are considered in feature matching. When dealing with traffic monitoring videos, existing methods often leads to stitching failure or large distortion because of the overlap region between different cameras is small and the angle between the main optical axes is large. Hence, an image semantic segmentation and stitching method of traffic monitoring video is proposed in this paper. First, the edge angular second-order difference histogram algorithm is proposed to recognize the orthophoto image automatically in the multi-video intersection area, and the orthophoto image is used as the stitching background image. Second, the orthophoto image and video image are segmented semantically based on fully convolutional network (FCN), and traffic thematic features are extracted separately. Finally, the traffic thematic features are used as constraints for feature point matching, and each traffic monitoring image is matched to the orthophoto image to realize regional video stitching. The experimental of real video data from a city in Shandong Province show that the proposed method obtain better stitching images for monitoring videos with smaller overlap areas, and effectively improve the accuracy of feature point matching.

Image matching is the process of obtaining corresponding points between two or more overlapping images by a specific algorithm. It is the critical step in the low-altitude photogrammetric data processing. The quality and efficiency of matching directly affect the subsequent data processing and the quality of mapping product generation. Therefore, image matching is one of the hot topics in the field of low-altitude photogrammetry and many relevant algorithms have been proposed. In this paper, the research status and prospect of image matching in low-altitude photogrammetry are described systematically. Firstly, the categories of image matching are summarized and can be generally divided into gray- and feature-based matching. We focus on feature-based image matching, e.g., point, line, and region-based features extraction and the relevant descriptors and similarity measures are described in detail. Besides, the latest image matching algorithms based on deep learning are listed, and the image matching methods involved in data fusion of various sensors on low-altitude platforms are mentioned.

The dynamic foreground target analysis of surveillance video is an important basis for security construction in safe cities and smart campuses. The dynamic attributes can be assigned to static geographic data by integrating surveillance video with geospatial data. For the integration of traditional surveillance video and geographic information data, only video data is projected into geospatial, which causes storage issue and difficult in understanding video content. The model that integrates geographic information and foreground dynamic targets, and integrates surveillance video with geographic information is proposed in this paper. And the dynamic foreground target and the tracking trajectory in the image are projected into the geospatial by the derived mapping model. The multi-layer fusion mode is utilized to distribute the video dynamic summary content in geospatial space.

Pan-sharpening is a process of obtaining a high spatial and spectral multispectral image (HMS) by combining a low resolution multispectral image (LMS) with a high resolution panchromatic image (PAN). In this paper, a Pan-sharpening method called PAIHS is proposed. It is based on adaptive intensity-hue-saturation (AIHS) transformation, variational Pan-sharpening framework and two assumptions:①pan-sharpened image and original multispectral image (MS) have the same spectral information; ②pan-sharpened image and PAN image contain the same geometric information. The suitable objective function was established, and optimized by particle swarm optimization (PSO) to obtain the optimal control parameters and minimum value, which corresponds to the best Pan-sharpening quality. The experimental results show that the proposed method has high efficiency and reliability, and the obtained performance index is also better than some of the current mainstream fusion methods.

Simultaneous localization and mapping (SLAM) is a research hotspot in fields of photogrammetry, computer vision and robotics, and has been widely applied in mobile mapping system, robots, driverless car, etc. This paper presents a fully automated feature based SLAM solution for a panoramic imaging system consisted of multi-camera rig. First, we developed a fisheye camera calibration model for guaranteeing high accurate coordinate transformation between the fisheye camera and the panoramic camera. Second, we imbedded the panoramic camera model into the SLAM process including initialization, local map building, key frame selection, graph optimization and bundle adjustment. In addition, we developed the algorithm in the processes of feature matching, bundle adjustment, frame tracking considering the disadvantages from the large image distortion and long baseline of the panoramic camera system. Experiments are executed on two data sets with more than 8000 panoramic images. Results show that the proposed panoramic SLAM solution achieves automatic camera localization and map construction, and the localization accuracy approaches the GPS reference. With respect to the mainstream SLAM systems based on conventional cameras, such as Mono-SLAM, Stereo-SLAM and RGB-D SLAM, our proposed panoramic SLAM system could serve as a beneficial supplement and supplies a cheap solution for GPS denied localization problem.

Precision and high-resolution global DEM composes the fundamental basis of the spatial data infrastructure for a variety of applications. To obtain high precision global DEM using domestic high-resolution satellite images, is one of the most important requirements for the construction of global geographic information resources of China. However, due to the complexity of the structure of terrain surface, a single set of parameters is not capable to handle the intricacies of the ground characteristics, which leads to decreasing of accuracies. Aiming at solving this problem, this paper proposes an adaptive surface filter for automatic terrain filtering and Poisson terrain editing for post processing and quality control. The two contributions have been extended to the software framework, LINK, for the global DEM production. Experiments and production works by several departments, including the Sichuan, Heilongjiang, Shanxi, Hainan, Chongqing Bureau of Surveying, Mapping and Geoinformation, have revealed that the DEM quality satisfies with the standard set, including various difficult scenarios, such as building high-rise areas, forest and water body. The quality and efficiency of the proposed method exceed other competitive solutions, which indicates a strong potential for the on-going global DEM production of China.

This paper proposes a method of semi-automatic right-angle building extraction from very high resolution remote sensing imagery, based on graph cuts with the multi-stars shape constraint and regularization. The framework consists of the following steps:Firstly, the image block containing the target building is obtained by manual interaction. Next, the image block is preprocessed by bilateral filtering. Then the graph cuts with the star shape constraint is used to obtain the building objects.Finally, building object is regularized into real regular shape through corner detection and linear fitting. The experiments performed on two different region and spatial resolution aerial imageries demonstrate the stability and accuracy of the proposed method.

The calibration of the external parameters of the laser scanner is the precondition and guarantee for obtain high-precision 3D geographic data. Most of the traditional calibration methods require setting up a special calibration field, manual collection of checkpoints, or the amount of calculation in the process of solving is large. Based on this, an automatic calibration method is proposed in this paper, by collecting point cloud data in the same area with different vehicle directions, extracting planar features data and automating registration of these planar features data, through the co-calibration of planar features of different angles.The proposed method realizes the coincidence of point clouds collected by different vehicles in three-dimensional space and finally completes the calibration of the system external parameters. The results show that the method is automatic for the calibration of the external parameters of the vehicle laser scanning system, reduces the need for manual participation, and achieves high precision.

Jitter is a crucial error source that undermines the geometric and radiometric performance of satellite imagery.In order to enhance the feasibility and trust of retrieving jitter from imagery,this paper proposes an intelligent jitter detection method using parallax observation based on phase correlation with local frequency analysis.Through connecting structural image representation with phase correlation,the local and global phase information is fully utilized to reduce the influence of radiometric variation,noise and other interferences on subpixel translation estimation.The subpixel phase correlation is integrated into the framework of jitter detection using parallax observation,which ensures the accurate and robust dense matching and achieves effective jitter detection.The effectiveness of the proposed method is demonstrated by method comparison using images with realistic radiometric difference as well as jitter detection test using ZY-3 images.The promising results indicate that,the subpixel phase correlation method with local frequency analysis outperforms other state-of-the-art methods in the presence of radiometric difference and noise,achieving a matching accuracy of 0.05 pixels,and the proposed jitter detection method effectively estimates the jitter information of ZY-3 satellite.

Traditional target detection methods based on sliding window search paradigm and hand-craft based features are difficult to be applied to the multi-class target detection of very-high-resolution remote sensing images. In this paper,we proposed a deformable convolutional networks based multi-class target detection method by introducing deformable convolution layer and deformable RoI (Region-of-Interest) pooling layer. Specially,our method consists of two sub networks:a region proposal network aims to predict candidate regions from several layers with different filter size,and a region classification network for discrimination and regression. The quantitative comparison results on the challenging NWPU VHR-10 data set,large-scale Google Earth images, GF-2 and JL-1 images show that our method is more accurate and robust than existing algorithms.

Man-made object detection is important for object detection from remote sensing images.In this paper we propose a variational approach for man-made object detection which formulates the man-made object detection problem as a problem of variational energy optimization.In this method,an image is firstly segmented into superpixels,and the saliency map by combining image features such as texture,color and gradient is computed.In second step,we construct an energy function with saliency,area,edge,texture and intensity variance constrains.The energy function is solved via variational method to obtain the foreground,which is the detected man-made objects.The proposed approach on several remote sensing images is evaluated and compared with the C-V model,MRF model and deep learning based semantic segmentation.Experimental results show that the proposed approach can effectively detect man-made objects on remote sensing images with low false alarm and false negatives rates.The comparison and analysis with deep learning based method are also presented.

This paper introduces some of the image processing techniques developed in the Canada Research Chair in Advanced Geomatics Image Processing Laboratory (CRC-AGIP Lab) and in the Department of Geodesy and Geomatics Engineering (GGE) at the University of New Brunswick (UNB),Canada.The techniques were developed by innovatively utilizing the characteristics of the available very high resolution remote sensing optical images to solve important problems or create new applications in photogrammetry and remote sensing.The techniques to be introduced are:automated image fusion (UNB-PanSharp),satellite image online mapping,street view technology,moving vehicle detection using single set satellite imagery,supervised image segmentation,image matching in smooth areas,and change detection using images from different viewing angles.

The diversity of means of obtaining space information and the tight constraints of the strict mathematical rules of data processing are the new contradictions associated with the development of the new aeronautics and astronautics technology.For example,push broom,variable angle swing staring,zoom imaging,aerial platform attitude,large angle flight,high overlap and short baseline effect,which bring severe challenges to convergence,efficiency,accuracy and anti-interference.Based on the principle of the bionic machine parallax angle and the essence of pyramidal projection of the aerial space platform to the surface,the mathematical expression of polar coordinates is introduced.This paper has explored the solution to the causes of the high resolution image sparsity ill conditioned singularity and nonconvergence,built a set of mathematical models for the polar coordinates processing of the parallax angular vector,and formed the polar information theory of spatial information initially.This method has been improved in the range of accuracy,efficiency and anti-interference order of magnitude in close-range photogrammetry and the free net bundle adjustment model,and publish open source code in the world more than three years,which has a good reaction.The effectiveness is verified in the aero photogrammetry and absolute network adjustment model experiment,and the performance is better than the Descartes coordinate processing method.Finally,the high-order solution characteristics of various applications and spaceflight platforms are given,which is expected to lay the foundation for the construction of the new polar coordinates system for aerospace multi-scale all attitude spatial information (acquisition organization management storage processing application).

In this paper, a framework of change detection based on adaptive distance and fuzzy topology (FATCD) is proposed for synthetic aperture radar (SAR) imagery. FATCD integrates the characteristics of differenced image and can overcome the limitations of fuzzy C-means (FCM) type algorithms. The framework includes two key steps. First, a new adaptive method is employed to calculate the distances from samples to cluster centers using an adaptive distance function. As a result, the formula of pixel membership evaluation is modified, and the accuracy of the obtained fuzzy membership degree is improved. Then, fuzzy topology is integrated into the maximum membership rule to improve the traditional defuzzification method. In virtue of the above two points, FATCD can enhance the change detection performance of FCM-type algorithms. Experimental results on two different SAR images confirm the effectiveness of the proposed technique.

High resolution remote sensing imagery scene classification is important for automatic complex scene recognition, which is the key technology for military and disaster relief, etc. In this paper, we propose a novel joint multi-scale convolution neural network (JMCNN) method using a limited amount of image data for high resolution remote sensing imagery scene classification. Different from traditional convolutional neural network, the proposed JMCNN is an end-to-end training model with joint enhanced high-level feature representation, which includes multi-channel feature extractor, joint multi-scale feature fusion and Softmax classifier. Multi-channel and scale convolutional extractors are used to extract scene middle features, firstly. Then, in order to achieve enhanced high-level feature representation in a limit dataset, joint multi-scale feature fusion is proposed to combine multi-channel and scale features using two feature fusions. Finally, enhanced high-level feature representation can be used for classification by Softmax. Experiments were conducted using two limit public UCM and SIRI datasets. Compared to state-of-the-art methods, the JMCNN achieved improved performance and great robustness with average accuracies of 89.3% and 88.3% on the two datasets.

Color design in administrative maps should incorporate and balance area configuration, color harmony, and users' purposes. Based on visual perceptual rules, this paper quantifies color harmony, color contrast and perceptual balance in coloring administrative maps, and a model is suggested to evaluate the coloring quality after color template is selected. Then a backtracking method based on area balance is proposed to compute colored areas. Experiments show that this method can well meet visual perceptual rules while coloring administrative maps, and can be used for later map design.

The segmentation of point clouds obtained by light detection and ranging (LiDAR) systems is a critical step for many tasks,such as data organization,reconstruction and information extraction.In this paper,we propose a bilevel progressive optimization algorithm based on the local differentiability.First,we define the topological relation and distance metric of points in the framework of Riemannian geometry,and in the point-based level using k-means method generates over-segmentation results,e.g.super voxels.Then these voxels are formulated as nodes which consist a minimal spanning tree.High level features are extracted from voxel structures,and a graph-based optimization method is designed to yield the final adaptive segmentation results.The implementation experiments on real data demonstrate that our method is efficient and superior to state-of-the-art methods.

With the development of 3D laser scanning technology,the demand for 3D modeling of point cloud data is increasing.As one of the core technologies of 3D modeling,surface reconstruction technology has an extremely widespread application prospect in reverse engineering,computer vision,computer graphics and virtual reality technology.This paper presents a surface reconstruction algorithm based on three-dimensional Delaunay triangulation,it is essentially a greedy algorithm,and combined with the idea of surface region growing algorithm.Manifold surface composed of a selection of explicit triangles can be reconstructed by this algorithm under a certain topological limit,the best triangles are selected from the preconstructed 3D Delaunay triangulation according to the appropriate triangle selection criteria by this algorithm,and added to the growing surface one by one.Compared with the current mainstream implicit surface reconstruction algorithm,this method has the advantages of less parameter dependency and no need to calculate normal line,and it can reconstruct the point cloud model of terrain scanning,building scanning and fine scanning.Using this algorithm to reconstruct the surface of a variety of point cloud models,experimental results show that the quality of the surface generated by the algorithm is choiceness,and the efficiency of reconstruction is faster,this surface reconstruction algorithm has stronger practicability and can be better applied in the field of 3D modeling.

Multimodal image registration method based on feature matching can't satisfy the demands of pixel level registration precision.This paper proposes a multimodal image registration algorithm considering grayscale and gradient information.The nonparametric registration model based on Markov random field (MRF) makes full use of the image information of multimodal image to measure the similarity which considers the grayscale and the gradient statistical information are considered,and the value space is discretized to improve the convergence speed.The algorithm is validated both qualitatively and quantitatively demonstrating its potentials on three groups of multimodal image registration experiments.The result indicates that the proposed algorithm is superior to the polynomial model registration based on manual selection and the multimodal image registration only with gray information only.At the same time,this algorithm has some applicability for multimodal image registration of large deformation.In terms of spatial accuracy,the average registration error is less than 1 pixel and the maximum registration error is less than 2 pixels.

Traditional building extraction from very high resolution remote sensing optical imagery is limited by low precision and incomplete boundary. In this paper, a high-resolution remote sensing image building extraction method based on deep learning is proposed. Firstly, Principal Component Analysis is used to pre-train network structure in an unsupervised way and obtain the characteristics of remote sensing image. Secondly, an adaptive pooling model is proposed to reduce the feature information loss in the pooling process. The texture features are extracted by non-subsampled contour wave transformation and introduced to the network to improve the building extraction. Finally, the obtained image features are inputted into the softmax classifier for classification and building extraction results. A typical experiment areas selected. The comparison with typical building extraction method, the experimental results shows that the proposed method can extract the buildings with higher accuracy, especially the clearer and more complete boundary.

For a large collection of satellite images,because of the differences in season,light,atmospheric conditions and so on,adjacent satellite images may have different color and there will be obviously "seam-line" in mosaic satellite image.In this paper,a method of satellite image color normalization was proposed,it was implemented to correct the color of image automatically based on better color low resolution satellite imagery which covered the mapping area.We selected a large collection and long time interval GF-1 satellite images as experiment data,results show that the mosaic image have a good visual effect,the color transition of two images smoothly,statistical indicators of overlap area obviously improved,all of those verified the validity and reliability of the algorithm.

Studies based on object-based image analysis (OBIA) representing the paradigm shift in remote sensing image change detection (CD) have achieved remarkable progress in the last decade.Their aim has been developing more intelligent interpretation analysis methods in the future.The prediction effect and performance stability of random forest (RF),as a new kind of machine learning algorithm,are better than many single predictors and integrated forecasting method. This paper presents a novel RF OBIA method for high resolution remote sensing image CD that makes full use of the advantages of RF and OBIA. Firstly,the entropy rate segmentation algorithm is used to segment the image for the purpose of measuring the homogeneity of super-pixels. Then the optimal image segmentation result is obtained from the evaluation index of the optimal super-pixel number.Afterwards,the spectral features and Gabor features of each super-pixelareextracted and used as feature datasets for the training of RF model.On the basis of the initial pixel-level CD result,the changed and unchanged samples are automatically selected and used to build the classifier model in order to get the final object-level CD result.Experimental results on Quickbird,IKONOS and SPOT-5 multi-spectral images show that the proposed method out performs the compared methods in the accuracy of CD.

This paper is divided into three parts. The first part of the article is a brief review of the developmental history of ancient and modern cartography in China. It shows that China is one of the earliest countries in the world to create maps. China played an important role in the development of cartography in the world, and has made significant contributions to the development of cartography. Up till the Ming Dynasty in 15th Century, China's map and cartography had always been ahead of the Western countries, but fell behind only after the Qing Dynasty. The second part of the article describes the rapid development of China's mapping and cartography after the founding of the "New China", including the tremendous achievements in the areas of mapping and creation of national topographic maps, thematic maps and atlases, remote sensing cartography, computer-aided cartography, multimedia electronic maps, mobile communication maps, online maps, and theoretical research of cartography. China has caught up and reached the world's advanced level in cartography. The third part It proposes the direction and adjusted roles of mapping and cartography in China in the new century, and analyzes the opportunities, challenges and prospects of cartography in the era of big data, internet and artificial intelligence. Via visualization of 3D dynamic mapping, big data can show the spatial pattern and regional differentiation as well as temporal and spatial dynamic change of subjects and phenomena, and then make analysis and evaluation, forecast and early warning, zoning and layout, planning and design, management and regulation. Therefore cartography can play an important role in the era of big data. In the future, internet will become the main platform for map creation and application. Maps will be more popular, personalized, intelligent and practical.

Due to the noise sensitivity and limited directivity of extracted edge presented in current edge detection algorithms, it is difficult to effectively extract the edges with any direction between the two regions with similar intensities from SAR images. In this paper, an edge extraction algorithm of SAR image using two symmetric windows is proposed. The position of grid point of any pixel in SAR image is employed as a symmetric center, then symmetric windows in any direction are constructed by the symmetric center. The kernel function of distances from the central pixel to pixels in each window is defined, then the weighted averages of pixel spectral measurements are calculated for the two symmetric windows, respectively. The absolute value of difference between above averages is employed as edge degree of the central pixel on this direction, and maximum degree from all directions is selected to judge the pixel as an edge pixel. To delete error edge pixels extracted, post-processing procedure based on filtering operation is designed. The interferences of speckle noises are overcome to some extent and edges in random direction are extracted accurately by the proposed algorithm, respectively, which is shown by qualitative and quantitative analysis of the edge extraction results from analog images.

Image matching based on feature was one of practical methods in UAV image matching. Since the conventional methods of image registration mainly used gray image as input that it could not take color features into account to distinguish the identical point. To address this problem, this paper designed a matching algorithm combined color invariant with AKAZE feature, which overcame the shortcoming of ignoring color information in traditional UAV image matching. Then gray level transformation was utilized to reduce the number of feature points and remain its reliability. Experimental results demonstrate that the proposed method can find the identical point accurately and enhance the efficiency.

Image segmentation is the basic and key step of object-oriented remote sensing image analysis. Conventional image segmentation method is sensitive to image noise and hard to determine the correct segmentation scale. To solve these problems, a novel image segmentation method by combining superpixels with minimum spanning tree was proposed in this paper. First, the image is over-segmented by simple linear iterative clustering algorithm to obtain superpixels. Then, superpixels are firstly clustered by regionalization with dynamically constrained agglomerative clustering and partitioning algorithm using the initial segmentation number and the sum of squared deviations (SSD), local variance (LV), rate of LV change (ROC-LV) index of graphs corresponding to the segmentation number are obtained. So the suitable image segmentation number is determined according to the SSD, LV, ROC-LV index of graphs corresponding to segmentation number. Finally, superpixels are reclustered by regionalization with dynamically constrained agglomerative clustering and partitioning algorithm based on the suitable segmentation number. The experimental results showed that the proposed method can obtain good segmentation results.

The purpose of geometric matching is to extract the geometric transformation parameters between the corresponding images. It is widely used in photogrammetric mapping, deformation detection, and flying platform's posture analysis, etc. In this paper, a new image matching method which is different from the traditional features based image matching algorithm is proposed, it takes all the pixels of the corresponding images to participate the matching procedure and calculate the geometric parameters by least square criterion. The principle of the algorithm, including the gray corresponding equation, the information quantity inequation and procedure of least square solution are expressed. Particularly, the wavelet analysis for gray signal and calculating the information quantity by signal to noise ratio are discussed in detail. For verifying the theory and algorithm, a series of sequential images taking from a video camera mounted on a helicopter are selected for experiment. The results of two typical models according to the relative orientation elements model and parallax grid model are given, and the experimental results show that the proposed algorithm is feasible and effective. The comparison of APM with ordinary features based method by the information quantity inequation is given in the conclusion.

In this paper, a new cloud removal method in remote sensing images is adopted.Based on the theory of compressive sensing,this method combines K-SVD dictionary learning with the orthogonal matching pursuit(OMP) algorithm of sparse representation.At the same time, a specific sorting rule is added to the process of dictionary atoms training, so that each image dictionary has its own image properties while its atoms also have a similar arrangement order to reduce the interference between image differences.In this method,the good effect of reconstruction of the contaminated region by clouds and shadows in remote sensing images is achieved.To illustrate the performance of the proposed method,experiments on two sets of data of multitemporal ZY-3 images at the same area are discussed.

With the rise of virtual reality (VR) technology, panoramic images are used more widely, which obtained by multi-camera stitching and take advantage of homography matrix and image transformation, however, this method will destroy the collinear condition, make it's difficult to 3D reconstruction and other work. This paper proposes a new method for cylindrical panoramic image mosaic, which set the number of mosaic camera, imaging focal length, imaging position and imaging attitude, simulate the mapping process of multi-camera and construct cylindrical imaging equation from 3D points to 2D image based on photogrammetric collinearity equations. This cylindrical imaging equation can not only be used for panoramic stitching, but also be used for precision analysis, test results show: ①this method can be used for panoramic stitching under the condition of multi-camera and incline imaging; ②the accuracy of panoramic stitching is affected by 3 kinds of parameter errors including focus, displacement and rotation angle, in which focus error can be corrected by image resampling, displacement error is closely related to object distance and rotation angle error is affected mainly by the number of cameras.

In order to solve the problem of texture optimization in 3D city reconstruction by using multi-lens oblique images, the paper presents a method of seamless texture model reconstruction. At first, it corrects the radiation information of images by camera response functions and image dark channel. Then, according to the corresponding relevance between terrain triangular mesh surface model to image, implements occlusion detection by sparse triangulation method, and establishes the triangles' texture list of visible. Finally, combines with triangles' topology relationship in 3D triangular mesh surface model and means and variances of image, constructs a graph-cuts-based texture optimization algorithm under the framework of MRF(Markov random filed), to solve the discrete label problem of texture optimization selection and clustering, ensures the consistency of the adjacent triangles in texture mapping, achieves the seamless texture reconstruction of city. The experimental results verify the validity and superiority of our proposed method.

The automatic identification of pedestrian's navigation state is a difficult problem in pedestrian navigation research. It is important to improve the precision feedback and navigation performance of pedestrian navigation services, and few researches have been done in this field. This paper proposes a pedestrian navigation state matching algorithm based on finite state machine (FSM). The main idea of this method is to identify the pedestrian navigation state on the basis of recognizing pedestrian's actions. The pedestrian's action characteristics are recognized by using multiple sensor data collected by Google glass and mobile phone. Then, the pedestrian navigation states are divided into familiar, unfamiliar and lost state. The state transition model is established according to the FSM theory, and the pedestrian navigation state matching algorithm based on the model is designed. Finally, this algorithm is implemented, and experiments are conducted to validate its effectiveness. Experimental results show that the proposed algorithm can reach a good precision of recognizing the state transitions during pedestrian navigation process, and especially the accuracy of recognizing lost state achieves 90%.

In this paper an automated camera self-calibration method based on large scale aerial images coplanarity constraint is proposed. The method uses stereo images corresponding points to compute camera inner parameters and the distortion coefficients based on coplanarity constraint. Firstly, aerial images corresponding points are matched and stereo images are constructed. Then the relative orientation of the camera is resolved by using the direct method and the iterative optimization based on the coplanarity constraint. Finally, the least square optimization method is used to calculate the inner parameters and the distortion coefficients of the camera. For large image the distortion of image boundary is more severe than center, in order to compute parameters more precisely mesh partition in arbitrary area of image is proposed. Compared with classical techniques which use expensive equipment or calibration field, the proposed technique can reflect the actual inner parameters of the camera in the real environment. Using all the corresponding points avoid the influence of choosing different feature points or control points on the calibration accuracy.Experimental results indicate that it is effective and flexible.The accuracy of the method is equivalent to that of calibration field, which can accurately reflect the camera parameters and the distortion coefficients of the image acquired by aerial photogrammetry.