Search

Abstract Point cloud registration technology, by precisely aligning repair components with the original artifacts, can accurately record the geometric shape of cultural heritage objects and generate three-dimensional models, thereby providing reliable data support for the digital preservation, virtual exhibition, and restoration of cultural relics. However, traditional point cloud registration methods face challenges when dealing with cultural heritage data, including complex morphological and structural variations, sparsity and irregularity, and cross-dataset generalization. To address these challenges, this paper introduces an innovative method called Enhancing Point Cloud Registration with Transformer (EPCRT). Firstly, we utilize local geometric perception for positional encoding and combine it with a dynamic adjustment mechanism based on local density information and geometric angle encoding, enhancing the flexibility and adaptability of positional encoding to better characterize the complex local morphology and structural variations of artifacts. Additionally, we introduce a convolutional-Transformer hybrid module to facilitate interactive learning of artifact point cloud features, effectively achieving local–global feature fusion and enhancing detail capture capabilities, thus effectively handling the sparsity and irregularity of artifact point cloud data. We conduct extensive evaluations on the 3DMatch, ModelNet, KITTI, and MVP-RG datasets, and validate our method on the Terracotta Warriors cultural heritage dataset. The results demonstrate that our method has significant performance advantages in handling the complexity of morphological and structural variations, sparsity and irregularity of relic data, and cross-dataset generalization.

Abstract Utilizing Generative Adversarial Networks (GANs) to generate 3D representations of the Terracotta Warriors offers a novel approach for the preservation and restoration of cultural heritage. Through GAN technology, we can produce complete 3D models of the Terracotta Warriors’ faces, aiding in the repair of damaged or partially destroyed figures. This paper proposes a distillation model, DIGAN, for generating 3D Terracotta Warrior faces. By extracting knowledge from StyleGAN2, we train an innovative 3D generative network. G2D, the primary component of the generative network, produces detailed and realistic 2D images. The 3D generator modularly decomposes the generation process, covering texture, shape, lighting, and pose, ultimately rendering 2D images of the Terracotta Warriors’ faces. The model enhances the learning of 3D shapes through symmetry constraints and multi-view data, resulting in high-quality 2D images that closely resemble real faces. Experimental results demonstrate that our method outperforms existing GAN-based generation methods.

Abstract Digital modeling is an essential means for preserving and passing down historical culture within cultural heritage. Point cloud registration technology, by aligning point cloud data captured from multiple perspectives, enhances the accuracy of reconstructing the complex structures of artifacts and buildings and provides a reliable digital foundation for their protection, exhibition, and research. Due to the challenges posed by complex morphology, noise, and missing data when processing cultural heritage data, this paper proposes a point cloud registration method based on the Diffusion Transformer (PointDT). Compared to traditional methods, the Diffusion Transformer can better capture both the global features and local structures of point cloud data, more accurately capturing the geometric and semantic information of the target point cloud, thereby achieving precise digital reconstruction. In this study, we trained our method using indoor datasets such as 3DMatch and large-scale outdoor datasets like KITTI, and validated it on various cultural heritage datasets, including those of the Terracotta Warriors and heritage buildings. The results demonstrate that this method not only significantly improves accuracy but also shows advantages in computational efficiency.

Abstract Due to the antiquity and difficulty of excavation, the Terracotta Warriors have suffered varying degrees of damage. To restore the cultural relics to their original appearance, utilizing point clouds to repair damaged Terracotta Warriors has always been a hot topic in cultural relic protection. The output results of existing methods in point cloud completion often lack diversity. Probability-based models represented by Denoising Diffusion Probabilistic Models have recently achieved great success in the field of images and point clouds and can output a variety of results. However, one drawback of diffusion models is that too many samples result in slow generation speed. Toward this issue, we propose a new neural network for Terracotta Warriors fragments completion. During the reverse diffusion stage, we initially decrease the number of sampling steps to generate a coarse result. This preliminary outcome undergoes further refinement through a multi-scale refine network. Additionally, we introduce a novel approach called Partition Attention Sampling to enhance the representation capabilities of features. The effectiveness of the proposed model is validated in the experiments on the real Terracotta Warriors dataset and public dataset. The experimental results conclusively demonstrate that our model exhibits competitive performance in comparison to other existing models.

The domain of remote sensing image processing has witnessed remarkable advancements in recent years, with deep convolutional neural networks (CNNs) establishing themselves as a prominent approach for building segmentation. Despite the progress, traditional CNNs, which rely on convolution and pooling for feature extraction during the encoding phase, often fail to precisely delineate global pixel interactions, potentially leading to the loss of vital semantic details. Moreover, conventional CNN-based segmentation models frequently neglect the nuanced semantic differences between shallow and deep features during the decoding phase, which can result in subpar feature integration through rudimentary addition or concatenation techniques. Additionally, the unique boundary characteristics of buildings in remote sensing images, which offer a rich vein of prior information, have not been fully harnessed by traditional CNNs. This paper introduces an innovative approach to building segmentation in remote sensing images through a prior-guided dual branch multi-feature fusion network (PDBMFN). The network is composed of a prior-guided branch network (PBN) in the encoding process, a parallel dilated convolution module (PDCM) designed to incorporate prior information, and a multi-feature aggregation module (MAM) in the decoding process. The PBN leverages prior region and edge information derived from superpixels and edge maps to enhance edge detection accuracy during the encoding phase. The PDCM integrates features from both branches and applies dilated convolution across various scales to expand the receptive field and capture a more comprehensive semantic context. During the decoding phase, the MAM utilizes deep semantic information to direct the fusion of features, thereby optimizing segmentation efficacy. Through a sequence of aggregations, the MAM gradually merges deep and shallow semantic information, culminating in a more enriched and holistic feature representation. Extensive experiments are conducted across diverse datasets, such as WHU, Inria Aerial, and Massachusetts, revealing that PDBMFN outperforms other sophisticated methods in terms of segmentation accuracy. In the key segmentation metrics, including mIoU, precision, recall, and F1 score, PDBMFN shows a marked superiority over contemporary techniques. The ablation studies further substantiate the performance improvements conferred by the PBN’s prior information guidance and the efficacy of the PDCM and MAM modules.

Accurate and robust 3D human modeling from a single image presents significant challenges. Existing methods have shown potential, but they often fail to generate reconstructions that match the level of detail in the input image. These methods particularly struggle with loose clothing. They typically employ parameterized human models to constrain the reconstruction process, ensuring the results do not deviate too far from the model and produce anomalies. However, this also limits the recovery of loose clothing. To address this issue, we propose an end-to-end method called IHRPN for reconstructing clothed humans from a single 2D human image. This method includes a feature extraction module for semantic extraction of image features. We propose an image semantic feature extraction aimed at achieving pixel model space consistency and enhancing the robustness of loose clothing. We extract features from the input image to infer and recover the SMPL-X mesh, and then combine it with a normal map to guide the implicit function to reconstruct the complete clothed human. Unlike traditional methods, we use local features for implicit surface regression. Our experimental results show that our IHRPN method performs excellently on the CAPE and AGORA datasets, achieving good performance, and the reconstruction of loose clothing is noticeably more accurate and robust.

Abstract The Mexican Hat wavelet (MHW) is strictly derived from the heat kernel by taking its negative first-order derivative with respect to time t. As a solution to the heat equation that the heat kernel has a clear initial condition, the Laplace–Beltrami operator. Although the MHW descriptor can effectively characterize the model information, but it has poor robustness to the model with scale transformation, and the feature description performance is affected to some extent. Following a popular mathematical method, in this paper, we bases on the MHW to study scaling invariance and proposes a new shape descriptor, the scale-invariant Mexican Hat wavelet (SIMHW), which by logarithmic sampling and Fourier transform that obtains the expression of SIMHW in Fourier domain. The experimental results show that SIMHW has finer information description ability and stronger recognition ability, and has better robustness to various non-rigid transformations. It can correctly calculate the similarity between 3D shapes and realize the effective shape retrieval.

Abstract Rock-art has been scratched, carved, and pecked into rock panels all over the world resulting in a huge number of engraved figures on natural rock surfaces that record ancient human life and culture. To preserve and recognize these valuable artifacts of human history, 2D digitization of rock surfaces has become a suitable approach due to the development of powerful 2D image processing techniques in recent years. In this article, we present a novel systematical framework for the segmentation of different petroglyph figures from 2D high-resolution images. The novel boundary enhancement with Gaussian loss (BEGL) function is proposed aiming at refining and smoothing the rock-arts boundaries in the basic UNet architecture. Several experiments on the 3D-pitoti dataset demonstrate that our proposed approach can achieve more accurate boundaries and superior results compared with other loss functions. The comprehensive framework of petroglyph segmentation from 2D high-resolution images provides the foundation for recognizing multiple petroglyph marks. The framework can then be extended to other cultural heritage digital protection domain easily.

Abstract To obtain a higher simplification rate while retaining geometric features, a simplification framework for the point cloud is proposed. Firstly, multi-angle images of the original point cloud are obtained with a virtual camera. Then, feature lines of each image are extracted by deep neural network. Furthermore, according to the proposed mapping relationship between the acquired 2D feature lines and original point cloud, feature points of the point cloud are extracted automatically. Finally, the simplified point cloud is obtained by fusing feature points and simplified non-feature points. The proposed simplification method is applied to four data sets and compared with the other six algorithms. The experimental results demonstrate that our proposed simplification method has the superiority in terms of both retaining geometric features and high simplification rate.

Disentanglement research is a critical and important issue in the field of image editing. In order to perform disentangled editing on images generated by generative models, this paper presents an unsupervised, model-agnostic, two-stage trained editing framework. This work addresses the problem of discovering interpretable, disentangled directions of edited image attributes in the latent space of generative models. This effort's primary objective was to address the limitations discovered in previous research, mainly (a) the discovered editing directions are interpretable but significantly entangled, i.e., changes to one attribute affect the others and (b) Prior research has utilized direction discovery and direction disentanglement separately, and they can't work synergistically. More specifically, this paper proposes a two-stage training method that discovers the editing direction with semantics, perturbs the dimension of the direction vector, adjusts it with a penalty mechanism, and makes the editing direction more disentangled. This allows easy distinguishable image editing, such as age and facial expressions in facial images. Experimentally compared to other methods, the proposed method outperforms them both qualitatively and quantitatively in terms of interpretability, disentanglement, and distinguishability of the generated images. The implementation of our method is available at https://github.com/ydniuyongjie/twoStageForFaceEdit.

Abstract Geometry images parameterise a mesh with a square domain and store the information in a single chart. A one-to-one correspondence between the 2D plane and the 3D model is convenient for processing 3D models. However, the parameterised vertices are not all located at the intersection of the gridlines the existing geometry images. Thus, errors are unavoidable when a 3D mesh is reconstructed from the chart. In this paper, we propose parameterise surface onto a novel geometry image that preserves the constraint of topological neighbourhood information at integer coordinate points on a 2D grid and ensures that the shape of the reconstructed 3D mesh does not change from supplemented image data. We find a collection of edges that opens the mesh into simply connected surface with a single boundary. The point distribution with approximate blue noise spectral characteristics is computed by capacity-constrained delaunay triangulation without retriangulation. We move the vertices to the constrained mesh intersection, adjust the degenerate triangles on a regular grid, and fill the blank part by performing a local affine transformation between each triangle in the mesh and image. Unlike other geometry images, the proposed method results in no error in the reconstructed surface model when floating-point data are stored in the image. High reconstruction accuracy is achieved when the xyz positions are in a 16-bit data format in each image channel because only rounding errors exist in the topology-preserving geometry images, there are no sampling errors. This method performs one-to-one mapping between the 3D surface mesh and the points in the 2D image, while foldovers do not appear in the 2D triangular mesh, maintaining the topological structure. This also shows the potential of using a 2D image processing algorithm to process 3D models.

Current development of Pakistan’s economy, transportation and industry with the improvement of urbanization, environmental pollution problems have gradually become prominent, but this is contrary to people’s vision of pursuing a high-quality life. Now the problem of haze, photochemical problems in the air, and global warming is already a key issue of global concern. This is focused on the ambient air quality of Lahore city of Pakistan. The study reveals that the particulate matter in the Lahore season (PM2.5, PM10) exceeds Pakistan’s National Environmental Quality Standards (NEQS). Correlation study suggests the positive correlation between the particulate matter and other mass concentration particles like Ozone (O3), Nitrogen Oxide (NO), Sulphur Dioxide (SO2). Higher values of CO/NO suggest that mobile sources are one of the major factors of this increase in NO. Further estimation of backward trajectory is done by the Hybrid-Single Particle Lagrangian Integrated Trajectory (HYSPLIT) model which provides the path of those particles in the last year period and the source of origin is from Afghanistan. This study provides in depth analysis of all factors of air pollutants by correlation between those factors. Prediction of future concentration of PM2.5 is predicted using the Seasonal Autoregressive Integrated Moving Average (SARIMA) model which gives the increasing value of PM2.5 in next year and provides the lowest and highest predicts (more than $100~\mu \text{g}/\text{m}^{3}$ ).

Transferring a photo to a stylised image with beautiful texture has become one of the most popular topics in computer vision and the application of image processing. Controlling the stroke size of the texture is one of the challenging problems in this task. Recent representative methods for such problem introduce a pyramid model to regulate receptive fields in the network. Meanwhile, dilated convolutions are proved to be a very efficient way to adjust receptive fields without losing resolution. By combining the advantages of both approaches and making special optimisation for VGG19 model for style transfer tasks, the authors propose to exploit dilated convolutions to extract texture information endowing the network with stroke controllable. Several sets of contrast experiments were conducted and results show that their algorithm can generate more attractive stylisation images and control stroke size flexibly. It demonstrates the superiority of applying dilated convolutions as a texture extraction method for maintaining more texture information and controlling stroke size.

We present a simple pairwise matching method for 3D fragment reassembly that uses boundary curves and concave-convex patches to accelerate and optimize the matching. Given the boundary curves of two fracture surfaces that completely or partially coincide, we can quickly exclude the fracture surface pairs with low boundary curve similarities, which can substantially reduce the computational cost of the subsequent patch matching, where we extract and delineate concave-convex patches of the selected fracture surfaces. A modified iterative closest point algorithm is applied on these concave-convex patches to refine the alignment. Finally, we determine the matched 3D fragments according to the overlap ratio of their fracture surfaces. The results of experiments on real-world examples demonstrate that our proposed algorithm is both accurate and efficient.

With the rapid development of 3D scanners, the cultural heritage artifacts can be stored as a point cloud and displayed through the Internet. However, due to natural and human factors, many cultural relics had some surface damage when excavated. As a result, the holes caused by these damages still exist in the generated point cloud model. This work proposes a multi-scale upsampling GAN (MU-GAN) based framework for completing these holes. Firstly, a 3D mesh model based on the original point cloud is reconstructed, and the method of detecting holes is presented. Secondly, the point cloud patch contains hole regions and is extracted from the point cloud. Then the patch is input into the MU-GAN to generate a high-quality dense point cloud. Finally, the empty areas on the original point cloud are filled with the generated dense point cloud patches. A series of real-world experiments are conducted on real scan data to demonstrate that the proposed framework can fill the holes of 3D heritage models with grained details. We hope that our work can provide a useful tool for cultural heritage protection.

This study proposes a blocks matching method based on contour curves and feature regions that improve the matching precision and speed with which rigid blocks with a specified thickness in point clouds are matched. The method comprises two steps: coarse matching and fine matching. In the coarse matching step, the rigid blocks are first segmented into a series of surfaces and the fracture surfaces are distinguished. Then, the contour curves of the fracture surfaces are extracted using an improved boundary growth method and the rigid blocks are coarsely matched with them. In the fine matching step, feature regions are first extracted from the fracture surfaces. Then, the centroid of each feature region is calculated and the fine matching of rigid blocks with the centroid sets is completed using an improved iterative closest point (ICP) algorithm. The improved ICP algorithm integrates the rotation angle constraint and dynamic iteration coefficient into a probability ICP algorithm, which significantly improves matching precision and speed. Experiments conducted using public blocks and Terracotta Warriors blocks indicate that the proposed method carries out rigid blocks matching more accurately and rapidly than various conventional methods.

Automatically selecting a set of representative views of a 3D virtual cultural relic is crucial for constructing wisdom museums. There is no consensus regarding the definition of a good view in computer graphics; the same is true of multiple views. View-based methods play an important role in the field of 3D shape retrieval and classification. However, it is still difficult to select views that not only conform to subjective human preferences but also have a good feature description. In this study, we define two novel measures based on information entropy, named depth variation entropy and depth distribution entropy. These measures were used to determine the amount of information about the depth swings and different depth quantities of each view. Firstly, a canonical pose 3D cultural relic was generated using principal component analysis. A set of depth maps obtained by orthographic cameras was then captured on the dense vertices of a geodesic unit-sphere by subdividing the regular unit-octahedron. Afterwards, the two measures were calculated separately on the depth maps gained from the vertices and the results on each one-eighth sphere form a group. The views with maximum entropy of depth variation and depth distribution were selected, and further scattered viewpoints were selected. Finally, the threshold word histogram derived from the vector quantization of salient local descriptors on the selected depth maps represented the 3D cultural relic. The viewpoints obtained by the proposed method coincided with an arbitrary pose of the 3D model. The latter eliminated the steps of manually adjusting the model’s pose and provided acceptable display views for people. In addition, it was verified on several datasets that the proposed method, which uses the Bag-of-Words mechanism and a deep convolution neural network, also has good performance regarding retrieval and classification when dealing with only four views.

As an essential step in the restoration of Terracotta Warriors, the results of fragments classification will directly affect the performance of fragments matching and splicing. However, most of the existing methods are based on traditional technology and have low accuracy in classification. A practical and effective classification method for fragments is an urgent need. In this case, an attention-based multi-scale neural network named AMS-Net is proposed to extract significant geometric and semantic features. AMS-Net is a hierarchical structure consisting of a multi-scale set abstraction block (MS-BLOCK) and a fully connected (FC) layer. MS-BLOCK consists of a local-global layer (LGLayer) and an improved multi-layer perceptron (IMLP). With a multi-scale strategy, LGLayer can parallel extract the local and global features from different scales. IMLP can concatenate the high-level and low-level features for classification tasks. Extensive experiments on the public data set (ModelNet40/10) and the real-world Terracotta Warrior fragments data set are conducted. The accuracy results with normal can achieve 93.52% and 96.22%, respectively. For real-world data sets, the accuracy is best among the existing methods. The robustness and effectiveness of the performance on the task of 3D point cloud classification are also investigated. It proves that the proposed end-to-end learning network is more effective and suitable for the classification of the Terracotta Warrior fragments.

As one of China′s most precious cultural relics, the excavation and protection of the Terracotta Warriors pose significant challenges to archaeologists. A fairly common situation in the excavation is that the Terracotta Warriors are mostly found in the form of fragments, and manual reassembly among numerous fragments is laborious and time-consuming. This work presents a fracture-surface-based reassembling method, which is composed of SiamesePointNet, principal component analysis (PCA), and deep closest point (DCP), and is named SPPD. Firstly, SiamesePointNet is proposed to determine whether a pair of point clouds of 3D Terracotta Warrior fragments can be reassembled. Then, a coarse-to-fine registration method based on PCA and DCP is proposed to register the two fragments into a reassembled one. The above two steps iterate until the termination condition is met. A series of experiments on real-world examples are conducted, and the results demonstrate that the proposed method performs better than the conventional reassembling methods. We hope this work can provide a valuable tool for the virtual restoration of three-dimension cultural heritage artifacts.

The computer-aided craniofacial reconstruction (CFR) technique has been widely used in the fields of criminal investigation, archaeology, anthropology and cosmetic surgery. The evaluation of craniofacial reconstruction results is important for improving the effect of craniofacial reconstruction. Here, we used the sparse principal component analysis (SPCA) method to evaluate the similarity between two sets of craniofacial data. Compared with principal component analysis (PCA), SPCA can effectively reduce the dimensionality and simultaneously produce sparse principal components with sparse loadings, thus making it easy to explain the results. The experimental results indicated that the evaluation results of PCA and SPCA are consistent to a large extent. To compare the inconsistent results, we performed a subjective test, which indicated that the result of SPCA is superior to that of PCA. Most importantly, SPCA can not only compare the similarity of two craniofacial datasets but also locate regions of high similarity, which is important for improving the craniofacial reconstruction effect. In addition, the areas or features that are important for craniofacial similarity measurements can be determined from a large amount of data. We conclude that the craniofacial contour is the most important factor in craniofacial similarity evaluation. This conclusion is consistent with the conclusions of psychological experiments on face recognition and our subjective test. The results may provide important guidance for three- or two-dimensional face similarity evaluation, analysis and face recognition.

Novel and effective engines for data analysis in integrated information systems are urgently required by diverse applications, in which massive business data can be analyzed to enable the capturing of various situations in real time. The performance of existing engines has limited capacity of data processing in distributed computing. Although Complex Event Processing (CEP) has enhanced the capacity of data analysis in information systems, it is still a big challenging task since events are rapidly increasing in diverse applications. In this paper, a lightweight intelligent data analysis system with a novel CEP engine named LIDA-E is introduced, which employs the knowledge base with rules and an event processing algorithm for analysis. Event models as well as operators support the rules for event selection and aggregation. These operators and rules have been utilized for constructing new CEP system architecture which combines expressiveness and efficiency in analysis. It adopts the agents and filter conception explicitly to provide the event transmission mechanism efficiently. Finally, the comparison between the proposed engine and the existing engine shows that LIDA-E has 48.65% averagely reduced time cost in different tests. The experimental results demonstrate that the developed architecture has better performance in both transmitting and analyzing a large number of events.

The kudzu root is one of the earliest medicinal plants listed in traditional Chinese medicine. In this paper, chemical compositions changes of kudzu roots from one year old to five years old were analyzed with respect to puerarin, acid-insoluble polysaccharides, acid-soluble polysaccharides, reducing sugar, protein, free amino acids, and lipid. In addition, the puerarin content was determined by high performance liquid chromatography (HPLC) method. The results showed that acid-soluble polysaccharides content of kudzu root increased with each growth period. In contrast, the acid-insoluble polysaccharides decreased significantly. The contents of reducing sugar and puerarin in kudzu root decreased significantly during its growth period. Beyond that, the contents of protein, free amino acids, and lipid in kudzu root ranged from 31.8 to 45.8 g/kg, 2.21 to 4.33 g/kg, and 32.2 to 76.9 g/kg, respectively. The trend of protein content coincided with the total content of free amino acids, in contrast to lipid. This paper provides a set of data and the select of kudzu root for the processing and development of new products of kudzu root.

That researchers in the field of computer games usually find it is difficult to simulate the motion of actual 3D model trees lies in the fact that the tree model itself has very complicated structure, and many sophisticated factors need to be considered during the simulation. Though there are some works on simulating 3D tree and its motion, few of them are used in computer games due to the high demand for real-time in computer games. In this paper, an approach of animating trees in computer games based on a novel tree model representation—Ball B-Spline Curves (BBSCs) are proposed. By taking advantage of the good features of the BBSC-based model, physical simulation of the motion of leafless trees with wind blowing becomes easier and more efficient. The method can generate realistic 3D tree animation in real-time, which meets the high requirement for real time in computer games.

We present a new point matching method to overcome the dense point-to-point alignment of scanned 3D faces. Instead of using the rigid spatial transformation in the traditional iterative closest point (ICP) algorithm, we adopt the thin plate spline (TPS) transformation to model the deformation of different 3D faces. Because TPS is a non-rigid transformation with good smooth property, it is suitable for formulating the complex variety of human facial morphology. A closest point searching algorithm is proposed to keep one-to-one mapping, and to get good efficiency the point matching method is accelerated by a KD-tree method. Having constructed the dense point-to-point correspondence of 3D faces, we create 3D face morphing and animation by key-frames interpolation and obtain realistic results. Comparing with ICP algorithm and the optical flow method, the presented point matching method can achieve good matching accuracy and stability. The experiment results have shown that our method is efficient for dense point objects registration.

As the numbers of 3D models available grow in many application fields, there is an increasing need for a search method to help people find them. Unfortunately, traditional search techniques are not always effective for 3D data. In this paper, we describe a novel method of interactive 3D model retrieval with building blocks. First, by using a cube block as the baseblock in a 3D virtual space, we may construct the query model with human-computer interaction method. Then through retrieving the polygon model of the database generated by the voxel model, we may get retrieval results in real time. Experiments are conducted to evaluate the performance of the proposed method.