Search

To optimize growth and management, precision agriculture relies on a deep understanding of agricultural dynamics, particularly crop water status analysis. Leveraging unmanned aerial vehicles, we can efficiently acquire high-resolution spatiotemporal samples by utilizing remote sensors. However, non-linear relationships among data features, localized within specific subgroups, frequently emerge in agricultural data. Interpreting these complex patterns requires sophisticated analysis due to the presence of noise, high variability, and non-stationarity behavior in the collected samples. Here, we introduce Local Biplot, a methodological framework tailored for discerning meaningful data patterns in non-stationary contexts for precision agriculture. Local Biplot relies on the well-known uniform manifold approximation and projection method, such as UMAP, and local affine transformations to codify non-stationary and non-linear data patterns while maintaining interpretability. This lets us find important clusters for transformation and projection within a single global axis pair. Hence, our framework encompasses variable and observational contributions within individual clusters. At the same time, we provide a relevance analysis strategy to help explain why those clusters exist, facilitating the understanding of data dynamics while favoring interpretability. We demonstrated our method's capabilities through experiments on both synthetic and real-world datasets, covering scenarios involving grass and rice crops. Moreover, we use random forest and linear regression models to predict water status variables from our Local Biplot-based feature ranking and clusters. Our findings revealed enhanced clustering and prediction capability while emphasizing the importance of input features in precision agriculture. As a result, Local Biplot is a useful tool to visualize, analyze, and compare the intricate underlying patterns and internal structures of complex agricultural datasets. [ABSTRACT FROM AUTHOR]

Object detection based on RGB and infrared images has emerged as a crucial research area in computer vision, and the synergy of RGB-Infrared ensures the robustness of object-detection algorithms under varying lighting conditions. However, the RGB-IR image pairs captured typically exhibit spatial misalignment due to sensor discrepancies, leading to compromised localization performance. Furthermore, since the inconsistent distribution of deep features from the two modalities, directly fusing multi-modal features will weaken the feature difference between the object and the background, therefore interfering with the RGB-Infrared object-detection performance. To address these issues, we propose an adaptive dual-discrepancy calibration network (ADCNet) for misaligned RGB-Infrared object detection, including spatial discrepancy and domain-discrepancy calibration. Specifically, the spatial discrepancy calibration module conducts an adaptive affine transformation to achieve spatial alignment of features. Then, the domain-discrepancy calibration module separately aligns object and background features from different modalities, making the distribution of the object and background of the fusion feature easier to distinguish, therefore enhancing the effectiveness of RGB-Infrared object detection. Our ADCNet outperforms the baseline by 3.3% and 2.5% in m A P 50 on the FLIR and misaligned M3FD datasets, respectively. Experimental results demonstrate the superiorities of our proposed method over the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Optical and SAR image registration is the primary procedure to exploit the complementary information from the two different image modal types. Although extensive research has been conducted to narrow down the vast radiometric and geometric gaps so as to extract homogeneous characters for feature point matching, few works have considered the registration issue for non-flat terrains, which will bring in more difficulties for not only sparse feature point matching but also outlier removal and geometric relationship estimation. This article addresses these issues with a novel and effective optical-SAR image registration framework. Firstly, sparse feature points are detected based on the phase congruency moment map of the textureless SAR image (SAR-PC-Moment), which helps to identify salient local regions. Then a template matching process using very large local image patches is conducted, which increases the matching accuracy by a significant margin. Secondly, a mutual verification-based initial outlier removal method is proposed, which takes advantage of the different mechanisms of sparse and dense matching and requires no geometric consistency assumption within the inliers. These two procedures will produce a putative correspondence feature point (CP) set with a low outlier ratio and high reliability. In the third step, the putative CPs are used to segment the large input image of non-flat terrain into dozens of locally flat areas using a recursive random sample consensus (RANSAC) method, with each locally flat area co-registered using an affine transformation. As for the mountainous areas with sharp elevation variations, anchor CPs are first identified, and then optical flow-based pixelwise dense matching is conducted. In the experimental section, ablation studies using four precisely co-registered optical-SAR image pairs of flat terrain quantitatively verify the effectiveness of the proposed SAR-PC-Moment-based feature point detector, big template matching strategy, and mutual verification-based outlier removal method. Registration results on four 1 m-resolution non-flat image pairs prove that the proposed framework is able to produce robust and quite accurate registration results. [ABSTRACT FROM AUTHOR]

The Gaofen-1 (GF-1) and Gaofen-6 (GF-6) satellites have acquired many GF-1 and GF-6 wide-field-view (WFV) images. These images have been made available for free use globally. The GF-1 WFV (GF-1) and GF-6 WFV (GF-6) images have rational polynomial coefficients (RPCs). In practical applications, RPC corrections of GF-1 and GF-6 images need to be completed using the rational function model (RFM). However, can the accuracy of the rational function model satisfy practical application requirements? To address this issue, a geometric accuracy method was proposed in this paper to evaluate the accuracy of the RFM of GF-1 and GF-6 images. First, RPC corrections were completed using the RFM and refined RFM, respectively. The RFM was constructed using the RPCs and Shuttle Radar Topography Mission (SRTM) 90 m DEM. The RFM was refined via affine transformation based on control points (CPs), which resulted in a refined RFM. Then, an automatic matching method was proposed to complete the automatic matching of GF-1/GF-6 images and reference images, which enabled us to obtain many uniformly distributed CPs. Finally, these CPs were used to evaluate the geometric accuracy of the RFM and refined RFM. The 14th-layer Google images of the corresponding area were used as reference images. In the experiments, the advantages and disadvantages of BRIEF, SIFT, and the proposed method were first compared. Then, the values of the root mean square error (RSME) of 10,561 Chinese, French, and Brazilian GF-1 and GF-6 images were calculated and statistically analyzed, and the local geometric distortions of the GF-1 and GF-6 images were evaluated; these were used to evaluate the accuracy of the RFM. Last, the accuracy of the refined RFM was evaluated using the eight GF-1 and GF-6 images. The experimental results indicate that the accuracy of the RFM for most GF-1 and GF-6 images cannot meet the actual use requirement of being better than 1.0 pixel, the accuracy of the refined RFM for GF-1 images cannot meet practical requirement of being better than 1.0 pixel, and the accuracy of the refined RFM for most GF-6 images meets the practical requirement of being better than 1.0 pixel. However, the RMSE values that meet the requirements are between 0.9 and 1.0, and the geometric accuracy can be further improved. [ABSTRACT FROM AUTHOR]

Operational forest monitoring often requires fine-detail information in the form of an orthomosaic, created by stitching overlapping nadir images captured by aerial platforms such as drones. RGB drone sensors are commonly used for low-cost, high-resolution imaging that is conducive to effective orthomosaicking, but only capture visible light. Thermal sensors, on the other hand, capture long-wave infrared radiation, which is useful for early pest detection among other applications. However, these lower-resolution images suffer from reduced contrast and lack of descriptive features for successful orthomosaicking, leading to gaps or swirling artifacts in the orthomosaic. To tackle this, we propose a thermal orthomosaicking workflow that leverages simultaneously acquired RGB images. The latter are used for producing a surface mesh via structure from motion, while thermal images are only used to texture this mesh and yield a thermal orthomosaic. Prior to texturing, RGB-thermal image pairs are co-registered using an affine transformation derived from a machine learning technique. On average, the individual RGB and thermal images achieve a mutual information of 0.2787 after co-registration using our technique, compared to 0.0591 before co-registration, and 0.1934 using manual co-registration. We show that the thermal orthomosaic generated from our workflow (1) is of better quality than other existing methods, (2) is geometrically aligned with the RGB orthomosaic, (3) preserves radiometric information (i.e., surface temperatures) from the original thermal imagery, and (4) enables easy transfer of downstream tasks—such as tree crown detection from the RGB to the thermal orthomosaic. We also provide an open-source tool that implements our workflow to facilitate usage and further development. [ABSTRACT FROM AUTHOR]

In close-range or unmanned aerial vehicle (UAV) photogrammetry, Schneider concentric circular coded targets (SCTs), which are public, are widely used for image matching and as ground control points. GSI point-distributed coded targets (GCTs), which are only mainly applied in a video-simultaneous triangulation and resection system (V-STARS), are non-public and rarely applied in UAV photogrammetry. In this paper, we present our innovative detailed solution to identify GCTs. First, we analyze the structure of a GCT. Then, a special 2D P2-invariant of five coplanar points derived from cross ratios is adopted in template point registration and identification. Finally, the affine transformation is used for decoding. Experiments indoors—including different viewing angles ranging from 0° to 80° based on 6 mm-diameter GCTs, smaller 3 mm-diameter GCTs, and different sizes mixed—and outdoors with challenging scenes were carried out. Compared with V-STARS, the results show that the proposed method can preserve the robustness and achieves a high accuracy rate in identification when the viewing angle is not larger than 65° through indoor experiments, and the proposed method can achieve approximate or slightly weaker effectiveness than V-STARS on the whole. Finally, we attempted to extend and apply the designed GCTs in UAV photogrammetry for a preliminary experiment. This paper demonstrates that GCTs can be designed, printed, and identified easily through our method. It is expected that the proposed method may be helpful when applied to image matching, camera calibration, camera orientation, or 3D measurements or serving as control points in UAV photogrammetry for scenarios with complex structures in the future. [ABSTRACT FROM AUTHOR]

Due to the inherent periodicity of the angle, the direction of the object detected by the current rotating object detection algorithm is fuzzy. In order to solve this problem, this paper proposes a rotating object direction estimation method based on a neural network, which determines the unique direction of the object by predicting the direction vector of the object. Firstly, we use the two components (sin θ , cos θ) of the direction vector and the length and width parameters of the object to express the object model. Secondly, we construct a neural network model to predict the parameters used to express the object model. However, there is a constraint that the sum of the squares of the two components of the direction vector of the object is equal to 1. Because each output element of the neural network is independent, it is difficult to learn the constrained data between such neurons. Therefore, the function transformation model is designed, and the network transformation layer is added. Finally, affine transformation is used to transform the object parameters and carry out regression calculation, so as to detect the object and determine the direction of the object at the same time. This paper uses three sets of data to carry out the experiment, which are DOTA 1.5, HRSC, and UCAS-AOD data sets. It can be seen from the experimental results that for the object with correct ground truth, the proposed method can not only locate the object but also estimate the direction of the object accurately. [ABSTRACT FROM AUTHOR]

Reliable feature matching plays an important role in the fields of computer vision and photogrammetry. Due to the complex transformation model caused by photometric and geometric deformations, and the limited discriminative power of local feature descriptors, initial matches with high outlier ratios cannot be addressed very well. This study proposes a reliable outlier-removal algorithm by combining two affine-invariant geometric constraints. First, a very simple geometric constraint, namely, CAI (collinear affine invariance) has been implemented, which is based on the observation that the collinear property of any two points is invariant under affine transformation. Second, after the first-step outlier removal based on the CAI constraint, the SOV (scale-orientation voting) scheme was then adopted to remove remaining outliers and recover the lost inliers, in which the peaks of both scale and orientation voting define the parameters of the geometric transformation model. Finally, match expansion was executed using the Delaunay triangulation of refined matches. By using close-range (rigid and non-rigid images) and UAV (unmanned aerial vehicle) datasets, comprehensive comparison and analysis are conducted in this study. The results demonstrate that the proposed outlier-removal algorithm achieves the best overall performance when compared with RANSAC-like and local geometric constraint-based methods, and it can also be applied to achieve reliable outlier removal in the workflow of SfM-based UAV image orientation. [ABSTRACT FROM AUTHOR]

The traditional template matching strategy of optical and synthetic aperture radar (SAR) is sensitive to the nonlinear transformation between two images. In some cases, the optical and SAR image pairs do not conform to the affine transformation condition. To address this issue, this study presents a novel template matching strategy which uses the One-Class Support Vector Machine (SVM) to remove outliers. First, we propose a method to construct the similarity map dataset using the SEN1-2 dataset for training the One-Class SVM. Second, a four-step strategy for optical and SAR image registration is presented in this paper. In the first step, the optical image is divided into some grids. In the second step, the strongest Harris response point is selected as the feature point in each grid. In the third step, we use Gaussian pyramid features of oriented gradients (GPOG) descriptor to calculate the similarity map in the search region. The trained One-Class SVM is used to remove outliers through similarity maps in the fourth step. Furthermore, the number of improve matches (NIM) and the rate of improve matches (RIM) are designed to measure the effect of image registration. Finally, this paper designs two experiments to prove that the proposed strategy can correctly select the matching points through similarity maps. The experimental results of the One-Class SVM in dataset show that the One-Class SVM can select the correct points in different datasets. The image registration results obtained on the second experiment show that the proposed strategy is robust to the nonlinear transformation between optical and SAR images. [ABSTRACT FROM AUTHOR]

Feature matching is a key method of feature-based image registration, which refers to establishing reliable correspondence between feature points extracted from two images. In order to eliminate false matchings from the initial matchings, we propose a simple and efficient method. The key principle of our method is to maintain the topological and affine transformation consistency among the neighborhood matches. We formulate this problem as a mathematical model and derive a closed solution with linear time and space complexity. More specifically, our method can remove mismatches from thousands of hypothetical correspondences within a few milliseconds. We conduct qualitative and quantitative experiments on our method on different types of remote-sensing datasets. The experimental results show that our method is general, and it can deal with all kinds of remote-sensing image pairs, whether rigid or non-rigid image deformation or image pairs with various shadow, projection distortion, noise, and geometric distortion. Furthermore, it is two orders of magnitude faster and more accurate than state-of-the-art methods and can be used for real-time applications. [ABSTRACT FROM AUTHOR]

This study presents an optimal weighting approach for combined image matching of high-resolution satellite stereo images (HRSI). When the rational polynomial coefficients (RPCs) for a pair of stereo images are available, some geometric constraints can be combined in image matching equations. Combining least squares image matching (LSM) equations with geometric constraints equations necessitates determining the appropriate weights for different types of observations. The common terms between the two sets of equations are the image coordinates of the corresponding points in the search image. Considering the fact that the RPCs of a stereo pair are produced in compliance with the coplanarity condition, geometric constraints are expected to play an important role in the image matching process. In this study, in order to control the impacts of the imposed constraint, optimal weights for observations were assigned through equalizing their average redundancy numbers. For a detailed assessment of the proposed approach, a pair of CARTOSAT-1 sub-images, along with their precise RPCs, were used. On top of obtaining different matching results, the dimension of the error ellipses of the intersection points in the object space were compared. It was shown through analysis that the geometric mean of the semi-minor and semi-major axis by our method was reduced 0.17 times relative to the unit weighting approach. [ABSTRACT FROM AUTHOR]

The mosaicking of Unmanned Aerial Vehicle (UAV) imagery usually requires information from additional sensors, such as Global Position System (GPS) and Inertial Measurement Unit (IMU), to facilitate direct orientation, or 3D reconstruction approaches (e.g., structure-from-motion) to recover the camera poses. In this paper, we propose a novel mosaicking method for UAV imagery in which neither direct nor indirect orientation procedures are required. Inspired by the embedded deformation model, a widely used non-rigid mesh deformation model, we present a novel objective function for image mosaicking. Firstly, we construct a feature correspondence energy term that minimizes the sum of the squared distances between matched feature pairs to align the images geometrically. Secondly, we model a regularization term that constrains the image transformation parameters directly by keeping all transformations as rigid as possible to avoid global distortion in the final mosaic. Experimental results presented herein demonstrate that the accuracy of our method is twice as high as an existing (purely image-based) approach, with the associated benefits of significantly faster processing times and improved robustness with respect to reference image selection. [ABSTRACT FROM AUTHOR]

The fusion of image data from multiple sensors is crucial for many applications. However, there are significant nonlinear intensity deformations between images from different kinds of sensors, leading to matching failure. To address this need, this paper proposes an effective coarse-to-fine matching method for multimodal remote sensing images (3MRS). In the coarse matching stage, feature points are first detected on a maximum moment map calculated with a phase congruency model. Then, feature description is conducted using an index map constructed by finding the index of the maximum value in all orientations of convolved images obtained using a set of log-Gabor filters. At last, several matches are built through image matching and outlier removal, which can be used to estimate a reliable affine transformation model between the images. In the stage of fine matching, we develop a novel template matching method based on the log-Gabor convolution image sequence and match the template features with a 3D phase correlation matching strategy, given that the initial correspondences are achieved with the estimated transformation. Results show that compared with SIFT, and three state-of-the-art methods designed for multimodal image matching, PSO-SIFT, HAPCG, and RIFT, only 3MRS successfully matched all six types of multimodal remote sensing image pairs: optical–optical, optical–infrared, optical–depth, optical–map, optical–SAR, and day–night, with each including ten different image pairs. On average, the number of correct matches (NCM) of 3MRS was 164.47, 123.91, 4.88, and 4.33 times that of SIFT, PSO-SIFT, HAPCG, and RIFT for the successfully matched image pairs of each method. In terms of accuracy, the root-mean-square error of correct matches for 3MRS, SIFT, PSO-SIFT, HAPCG, and RIFT are 1.47, 1.98, 1.79, 2.83, and 2.45 pixels, respectively, revealing that 3MRS got the highest accuracy. Even though the total running time of 3MRS was the longest, the efficiency for obtaining one correct match is the highest considering the most significant number of matches. The source code of 3MRS and the experimental datasets and detailed results are publicly available. [ABSTRACT FROM AUTHOR]

Time delay and integration (TDI) charge-coupled device (CCD) is an image sensor for capturing images of moving objects at low light levels. This study examines the model construction of stitched TDI CCD original multi-slice images. The traditional approaches, for example, include the image-space-oriented algorithm and the object-space-oriented algorithm. The former indicates concise principles and high efficiency, whereas the panoramic stitching images lack the clear geometric relationships generated from the image-space-oriented algorithm. Similarly, even though the object-space-oriented algorithm generates an image with a clear geometric relationship, it is time-consuming due to the complicated and intensive computational demands. In this study, we developed a multi-slice satellite images stitching and geometric model construction method. The method consists of three major steps. First, the high-precision reference data assist in block adjustment and obtain the original slice image bias-corrected RFM to perform multi-slice image block adjustment. The second process generates the panoramic stitching image by establishing the image coordinate conversion relationship from the panoramic stitching image to the original multi-slice images. The final step is dividing the panoramic stitching image uniformly into image grids and employing the established image coordinate conversion relationship and the original multi-slice image bias-corrected RFM to generate a virtual control grid to construct the panoramic stitching image RFM. To evaluate the performance, we conducted experiments using the Tianhui-1(TH-1) high-resolution image and the Ziyuan-3(ZY-3) triple liner-array image data. The experimental results show that, compared with the object-space-oriented algorithm, the stitching accuracy loss of the generated panoramic stitching image was only 0.2 pixels and that the mean value was 0.799798 pixels, achieving the sub-pixel stitching requirements. Compared with the object-space-oriented algorithm, the RFM positioning difference of the panoramic stitching image was within 0.3 m, which achieves equal positioning accuracy. [ABSTRACT FROM AUTHOR]

This study used close-range multispectral imagery over cucumber plants inside a commercial greenhouse to detect powdery mildew due to Podosphaera xanthii. It was collected using a MicaSense® RedEdge camera at 1.5 m over the top of the plant. Image registration was performed using Speeded-Up Robust Features (SURF) with an affine geometric transformation. The image background was removed using a binary mask created with the aligned NIR band of each image, and the illumination was corrected using Cheng et al.'s algorithm. Different features were computed, including RGB, image reflectance values, and several vegetation indices. For each feature, a fine Gaussian Support Vector Machines algorithm was trained and validated to classify healthy and infected pixels. The data set to train and validate the SVM was composed of 1000 healthy and 1000 infected pixels, split 70–30% into training and validation datasets, respectively. The overall validation accuracy was 89, 73, 82, 51, and 48%, respectively, for blue, green, red, red-edge, and NIR band image. With the RGB images, we obtained an overall validation accuracy of 89%, while the best vegetation index image was the PMVI-2 image which produced an overall accuracy of 81%. Using the five bands together, overall accuracy dropped from 99% in the training to 57% in the validation dataset. While the results of this work are promising, further research should be considered to increase the number of images to achieve better training and validation datasets. [ABSTRACT FROM AUTHOR]

Cucumber powdery mildew, which is caused by Podosphaera xanthii, is a major disease that has a significant economic impact in cucumber greenhouse production. It is necessary to develop a non-invasive fast detection system for that disease. Such a system will use multispectral imagery acquired at a close range with a camera attached to a mobile cart's mechanic extension. This study evaluated three image registration methods applied to non-georeferenced multispectral images acquired at close range over greenhouse cucumber plants with a MicaSense® RedEdge camera. The detection of matching points was performed using Speeded-Up Robust Features (SURF), and outliers matching points were removed using the M-estimator Sample Consensus (MSAC) algorithm. Three geometric transformations (affine, similarity, and projective) were considered in the registration process. For each transformation, we mapped the matching points of the blue, green, red, and NIR band images into the red-edge band space and computed the root mean square error (RMSE in pixel) to estimate the accuracy of each image registration. Our results achieved an RMSE of less than 1 pixel with the similarity and affine transformations and of less than 2 pixels with the projective transformation, whatever the band image. We determined that the best image registration method corresponded to the affine transformation because the RMSE is less than 1 pixel and the RMSEs have a Gaussian distribution for all of the bands, but the blue band. [ABSTRACT FROM AUTHOR]

Intelligent detection and recognition of ships from high-resolution remote sensing images is an extraordinarily useful task in civil and military reconnaissance. It is difficult to detect ships with high precision because various disturbances are present in the sea such as clouds, mist, islands, coastlines, ripples, and so on. To solve this problem, we propose a novel ship detection network based on multi-layer convolutional feature fusion (CFF-SDN). Our ship detection network consists of three parts. Firstly, the convolutional feature extraction network is used to extract ship features of different levels. Residual connection is introduced so that the model can be designed very deeply, and it is easy to train and converge. Secondly, the proposed network fuses fine-grained features from shallow layers with semantic features from deep layers, which is beneficial for detecting ship targets with different sizes. At the same time, it is helpful to improve the localization accuracy and detection accuracy of small objects. Finally, multiple fused feature maps are used for classification and regression, which can adapt to ships of multiple scales. Since the CFF-SDN model uses a pruning strategy, the detection speed is greatly improved. In the experiment, we create a dataset for ship detection in remote sensing images (DSDR), including actual satellite images from Google Earth and aerial images from electro-optical pod. The DSDR dataset contains not only visible light images, but also infrared images. To improve the robustness to various sea scenes, images under different scales, perspectives and illumination are obtained through data augmentation or affine transformation methods. To reduce the influence of atmospheric absorption and scattering, a dark channel prior is adopted to solve atmospheric correction on the sea scenes. Moreover, soft non-maximum suppression (NMS) is introduced to increase the recall rate for densely arranged ships. In addition, better detection performance is observed in comparison with the existing models in terms of precision rate and recall rate. The experimental results show that the proposed detection model can achieve the superior performance of ship detection in optical remote sensing image. [ABSTRACT FROM AUTHOR]

This paper studies inverse synthetic aperture radar (ISAR) image matching and three-dimensional (3D) scattering imaging based on extracted dominant scatterers. In the condition of a long baseline between two radars, it is easy for obvious rotation, scale, distortion, and shift to occur between two-dimensional (2D) radar images. These problems lead to the difficulty of radar-image matching, which cannot be resolved by motion compensation and cross-correlation. What is more, due to the anisotropy, existing image-matching algorithms, such as scale invariant feature transform (SIFT), do not adapt to ISAR images very well. In addition, the angle between the target rotation axis and the radar line of sight (LOS) cannot be neglected. If so, the calibration result will be smaller than the real projection size. Furthermore, this angle cannot be estimated by monostatic radar. Therefore, instead of matching image by image, this paper proposes a novel ISAR imaging matching and 3D imaging based on extracted scatterers to deal with these issues. First, taking advantage of ISAR image sparsity, radar images are converted into scattering point sets. Then, a coarse scatterer matching based on the random sampling consistency algorithm (RANSAC) is performed. The scatterer height and accurate affine transformation parameters are estimated iteratively. Based on matched scatterers, information such as the angle and 3D image can be obtained. Finally, experiments based on the electromagnetic simulation software CADFEKO have been conducted to demonstrate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

A large number of evenly distributed conjugate points (CPs) in entirely overlapping regions of the images are required to achieve successful co-registration between very-high-resolution (VHR) remote sensing images. The CPs are then used to construct a non-linear transformation model that locally warps a sensed image to a reference image's coordinates. Piecewise linear (PL) transformation is largely exploited for warping VHR images because of its superior performance as compared to the other methods. The PL transformation constructs triangular regions on a sensed image from the CPs by applying the Delaunay algorithm, after which the corresponding triangular regions in a reference image are constructed using the same CPs on the image. Each corresponding region in the sensed image is then locally warped to the regions of the reference image through an affine transformation estimated from the CPs on the triangle vertices. The warping performance of the PL transformation shows reliable results, particularly in regions inside the triangles, i.e., within the convex hulls. However, the regions outside the triangles, which are warped when the extrapolated boundary planes are extended using CPs located close to the regions, incur severe geometric distortion. In this study, we propose an effective approach that focuses on the improvement of the warping performance of the PL transformation over the external area of the triangles. Accordingly, the proposed improved piecewise linear (IPL) transformation uses additional pseudo-CPs intentionally extracted from positions on the boundary of the sensed image. The corresponding pseudo-CPs on the reference image are determined by estimating the affine transformation from CPs located close to the pseudo-CPs. The latter are simultaneously used with the former to construct the triangular regions, which are enlarged accordingly. Experiments on both simulated and real datasets, constructed from Worldview-3 and Kompsat-3A satellite images, were conducted to validate the effectiveness of the proposed IPL transformation. That transformation was shown to outperform the existing linear/non-linear transformation models such as an affine, third and fourth polynomials, local weighted mean, and PL. Moreover, we demonstrated that the IPL transformation improved the warping performance over the PL transformation outside the triangular regions by increasing the correlation coefficient values from 0.259 to 0.304, 0.603 to 0.657, and 0.180 to 0.338 in the first, second, and third real datasets, respectively. [ABSTRACT FROM AUTHOR]

Infrared and visible image registration is a very challenging task due to the large geometric changes and the significant contrast differences caused by the inconsistent capture conditions. To address this problem, this paper proposes a novel affine and contrast invariant descriptor called maximally stable phase congruency (MSPC), which integrates the affine invariant region extraction with the structural features of images organically. First, to achieve the contrast invariance and ensure the significance of features, we detect feature points using moment ranking analysis and extract structural features via merging phase congruency images in multiple orientations. Then, coarse neighborhoods centered on the feature points are obtained based on Log-Gabor filter responses over scales and orientations. Subsequently, the affine invariant regions of feature points are determined by using maximally stable extremal regions. Finally, structural descriptors are constructed from those regions and the registration can be implemented according to the correspondence of the descriptors. The proposed method has been tested on various infrared and visible pairs acquired by different platforms. Experimental results demonstrate that our method outperforms several state-of-the-art methods in terms of robustness and precision with different image data and also show its effectiveness in the application of trajectory tracking. [ABSTRACT FROM AUTHOR]