Search

The task of 3D reconstruction of urban targets holds pivotal importance for various applications, including autonomous driving, digital twin technology, and urban planning and development. The intricate nature of urban landscapes presents substantial challenges in attaining 3D reconstructions with high precision. In this paper, we propose a semantically aware multi-view 3D reconstruction method for urban applications which incorporates semantic information into the technical 3D reconstruction. Our research primarily focuses on two major components: sparse reconstruction and dense reconstruction. For the sparse reconstruction process, we present a semantic consistency-based error filtering approach for feature matching. To address the challenge of errors introduced by the presence of numerous dynamic objects in an urban scene, which affects the Structure-from-Motion (SfM) process, we propose a computation strategy based on dynamic–static separation to effectively eliminate mismatches. For the dense reconstruction process, we present a semantic-based Semi-Global Matching (sSGM) method. This method leverages semantic consistency to assess depth continuity, thereby enhancing the cost function during depth estimation. The improved sSGM method not only significantly enhances the accuracy of reconstructing the edges of the targets but also yields a dense point cloud containing semantic information. Through validation using architectural datasets, the proposed method was found to increase the reconstruction accuracy by 32.79% compared to the original SGM, and by 63.06% compared to the PatchMatch method. Therefore, the proposed reconstruction method holds significant potential in urban applications.

Compared to wide-field telescopes, small-field detection systems have higher spatial resolution, resulting in stronger detection capabilities and higher positioning accuracy. When detecting by small fields in synchronous orbit, both space debris and fixed stars are imaged as point targets, making it difficult to distinguish them. In addition, with the improvement in detection capabilities, the number of stars in the background rapidly increases, which puts higher requirements on recognition algorithms. Therefore, star detection is indispensable for identifying and locating space debris in complex backgrounds. To address these difficulties, this paper proposes a real-time star extraction method based on adaptive filtering and multi-frame projection. We use bad point repair and background suppression algorithms to preprocess star images. Afterwards, we analyze and enhance the target signal-to-noise ratio (SNR). Then, we use multi-frame projection to fuse information. Subsequently, adaptive filtering, adaptive morphology, and adaptive median filtering algorithms are proposed to detect trajectories. Finally, the projection is released to locate the target. Our recognition algorithm has been verified by real star images, and the images were captured using small-field telescopes. The experimental results demonstrate the effectiveness of the algorithm proposed in this paper. We successfully extracted hip-27066 star, which has a magnitude of about 12 and an SNR of about 1.5. Compared with existing methods, our algorithm has advantages in both recognition rate and false-alarm rate, and can be used as a real-time target recognition algorithm for space-based synchronous orbit detection payloads.

With the development of space technology, deep learning methods, with their excellent generalization ability, are increasingly applied in various space activities. The space object data is difficult to obtain, which greatly limits its application in space activities. The images of the existing public spacecraft dataset are mostly rendered, which not only lack physical meaning but also have limited data. In this paper, we propose an effective construction procedure to build a space object dataset based on STK, which can help to break the limitations of deep learning methods in space activities. Firstly, based on STK, we conduct orbit simulation for 24 space targets and establish the simulation dataset; secondly, we use 600 images of 6 typical targets and label them to build a real-shot validation dataset. Finally, the constructed space object dataset based on STK is verified to be effective through six semantic segmentation networks, which can be used to train the real spacecraft’s semantic segmentation. Lots of experiments show that the accuracy of migrating the training results of the simulation dataset to the real shooting dataset is slightly reduced, but the mPA is still greater than 85%. In particular, after adding orbital physics simulation data, the accuracy of six semantic segmentation methods is generally improved. Therefore, the STK-based physical simulation of orbit is an effective method for space object dataset construction.

Non-cooperative spacecraft pose acquisition is a challenge in on-orbit service (OOS), especially for targets with unknown structures. A method for the pose measurement of non-cooperative spacecrafts based on the collaboration of binocular and time-of-flight (TOF) cameras is proposed in this study. The joint calibration is carried out to obtain the transformation matrix from the left camera coordinate system to the TOF camera system. The initial pose acquisition is mainly divided into feature point association and relative motion estimation. The initial value and key point information generated in stereo vision are yielded to refine iterative closest point (ICP) frame-to-frame registration. The final pose of the non-cooperative spacecraft is determined through eliminating the cumulative error based on the keyframes in the point cloud process. The experimental results demonstrate that the proposed method is able to track the target spacecraft during aerospace missions, which may provide a certain reference value for navigation systems.

It is difficult to determine the accurate pose of non-cooperative space targets in on-orbit servicing (OOS). The visual camera is easily affected by the extreme light environment in space, and the scanning lidar will have motion distortion when the target moves at high speed. Therefore, we proposed a non-cooperative target pose-estimation system combining a registration and a mapping algorithm using a TOF camera. We first introduce the projection model of the TOF camera and proposed a new calibration method. Then, we introduce the three modules of the proposed method: the TOF data preprocessing module, the registration module and the model mapping module. We assembled the experimental platform to conduct semi-physical experiments; the results showed that the proposed method has the smallest translation error 8 mm and Euler angle error 1° compared with other classical methods. The total time consumption is about 100 ms, and the pose tracking frequency can reach 10 Hz. We can conclude that the proposed pose-estimation scheme can achieve the high-precision pose estimation of non-cooperative targets and meet the requirements necessary for aerospace applications.