Search

Background Purpose Methods Results Conclusion Forty to fifty percent of women and 13%–22% of men experience an osteoporosis‐related fragility fracture in their lifetimes. After the age of 50 years, the risk of hip fracture doubles in every 10 years. x‐Ray based DXA is currently clinically used to diagnose osteoporosis and predict fracture risk. However, it provides only 2‐D representation of bone and is associated with other technical limitations. Thus, alternative methods are needed.To develop and evaluate an ultra‐low dose (ULD) hip CT‐based automated method for assessment of volumetric bone mineral density (vBMD) at proximal femoral subregions.An automated method was developed to segment the proximal femur in ULD hip CT images and delineate femoral subregions. The computational pipeline consists of deep learning (DL)‐based computation of femur likelihood map followed by shape model‐based femur segmentation and finite element analysis‐based warping of a reference subregion labeling onto individual femur shapes. Finally, vBMD is computed over each subregion in the target image using a calibration phantom scan. A total of 100 participants (50 females) were recruited from the Genetic Epidemiology of COPD (COPDGene) study, and ULD hip CT imaging, equivalent to 18 days of background radiation received by U.S. residents, was performed on each participant. Additional hip CT imaging using a clinical protocol was performed on 12 participants and repeat ULD hip CT was acquired on another five participants. ULD CT images from 80 participants were used to train the DL network; ULD CT images of the remaining 20 participants as well as clinical and repeat ULD CT images were used to evaluate the accuracy, generalizability, and reproducibility of segmentation of femoral subregions. Finally, clinical CT and repeat ULD CT images were used to evaluate accuracy and reproducibility of ULD CT‐based automated measurements of femoral vBMD.Dice scores of accuracy (n = 20), reproducibility (n = 5), and generalizability (n = 12) of ULD CT‐based automated subregion segmentation were 0.990, 0.982, and 0.977, respectively, for the femoral head and 0.941, 0.970, and 0.960, respectively, for the femoral neck. ULD CT‐based regional vBMD showed Pearson and concordance correlation coefficients of 0.994 and 0.977, respectively, and a root‐mean‐square coefficient of variation (RMSCV) (%) of 1.39% with the clinical CT‐derived reference measure. After 3‐digit approximation, each of Pearson and concordance correlation coefficients as well as intraclass correlation coefficient (ICC) between baseline and repeat scans were 0.996 with RMSCV of 0.72%. Results of ULD CT‐based bone analysis on 100 participants (age (mean ± SD) 73.6 ± 6.6 years) show that males have significantly greater (p < 0.01) vBMD at the femoral head and trochanteric regions than females, while females have moderately greater vBMD (p = 0.05) at the medial half of the femoral neck than males.Deep learning, combined with shape model and finite element analysis, offers an accurate, reproducible, and generalizable algorithm for automated segmentation of the proximal femur and anatomic femoral subregions using ULD hip CT images. ULD CT‐based regional measures of femoral vBMD are accurate and reproducible and demonstrate regional differences between males and females. [ABSTRACT FROM AUTHOR]

While current facial re‐ageing methods can produce realistic results, they purely focus on the 2D age transformation. In this work, we present an approach to transform the age of a person in both facial appearance and shape across different ages while preserving their identity. We employ an α‐(de)blending diffusion network with an age‐to‐α transformation to generate coarse structure changes, such as wrinkles. Additionally, we edit biophysical skin properties, including melanin and hemoglobin, to simulate skin color changes, producing realistic re‐ageing results from ages 10 to 80 years. We also propose a geometric neural network that alters the coarse scale facial geometry according to age, followed by a lightweight and efficient network that adds appropriate skin displacement on top of the coarse geometry. Both qualitative and quantitative comparisons show that our method outperforms current state‐of‐the‐art approaches. [ABSTRACT FROM AUTHOR]

• A mathematical model for inverse kinematics and shape modeling of continuum robots is presented. • The inverse kinematics algorithm performs very well in the accuracy and efficiency by simulation. • Shape modeling of continuum robots is verified by prototype experiment. A mathematical model for the inverse kinematics and shape modeling of continuum robots is presented in this paper. This method simplifies the forward kinematics derivation process by introducing a rotation axis. In the inverse kinematics solution, an iterative approach is employed to try different initial values, and the optimal initial value is obtained by computing the condition number of the Jacobian matrix. Simulation experiments are verified that optimized initial values can effectively enhance the efficiency of inverse kinematics solution. Based on the Euler-Bernoulli beam theory, the shape modeling of continuum robots is derived, and the mathematical model of the relationship between robot shape and end loads is established. Finally, the mathematical model is verified through prototype experiments, demonstrating that the established shape modeling can achieve accurate simulation of deformation within a range where the bending angle is less than 90°, with errors within 10 %. [ABSTRACT FROM AUTHOR]

Shape modeling is an indispensable task for spacecraft exploration of small planetary bodies. Traditional image-based techniques, such as stereo-photogrammetry or structure-from-motion + multi-view stereo, and stereo-photoclinometry, typically use a large number of images taken under favorable conditions for fine shape modeling, often requiring a long time for data acquisition and processing. Here, a novel neural implicit method, encoded by fully connected neural networks, is proposed for shape modeling using a sparse image set. The positions of surrounding points (SPs) with multi-scale receptive fields of a given input point are used as additional inputs for the network training, providing neighboring information. For fine-scale terrain features, a mask-based classification sampling strategy is developed to mitigate over-smoothing encountered by neural implicit methods. The effectiveness of our method is validated on two asteroids of distinct shapes, Itokawa and Ryugu, using 52 and 70 images, respectively. Comparative experiments demonstrate that the mask-based strategy, combined with the SPs configuration, accelerates network convergence for extracting fine surface details while minimizing the occurrence of artifacts. The proposed method can generate comprehensive shape models even in regions with restricted camera coverage, and the resulting models are consistent with those from traditional methods using larger image sets. Besides, the training process is executed in an end-to-end fashion, requiring limited manual intervention, and our method can readily be applied to other small planetary bodies. The source code and generated shape models in this study are available at https://doi.org/10.5281/zenodo.10909252. [ABSTRACT FROM AUTHOR]

We present CharacterMixer, a system for blending two rigged 3D characters with different mesh and skeleton topologies while maintaining a rig throughout interpolation. CharacterMixer also enables interpolation during motion for such characters, a novel feature. Interpolation is an important shape editing operation, but prior methods have limitations when applied to rigged characters: they either ignore the rig (making interpolated characters no longer posable) or use a fixed rig and mesh topology. To handle different mesh topologies, CharacterMixer uses a signed distance field (SDF) representation of character shapes, with one SDF per bone. To handle different skeleton topologies, it computes a hierarchical correspondence between source and target character skeletons and interpolates the SDFs of corresponding bones. This correspondence also allows the creation of a single "unified skeleton" for posing and animating interpolated characters. We show that CharacterMixer produces qualitatively better interpolation results than two state‐of‐the‐art methods while preserving a rig throughout interpolation. Project page: https://seanxzhan.github.io/projects/CharacterMixer. [ABSTRACT FROM AUTHOR]

Cardiac shape modeling is a useful computational tool that has provided quantitative insights into the mechanisms underlying dysfunction in heart disease. The manual input and time required to make cardiac shape models, however, limits their clinical utility. Here we present an end-to-end pipeline that uses deep learning for automated view classification, slice selection, phase selection, anatomical landmark localization, and myocardial image segmentation for the automated generation of three-dimensional, biventricular shape models. With this approach, we aim to make cardiac shape modeling a more robust and broadly applicable tool that has processing times consistent with clinical workflows. Cardiovascular magnetic resonance (CMR) images from a cohort of 123 patients with repaired tetralogy of Fallot (rTOF) from two internal sites were used to train and validate each step in the automated pipeline. The complete automated pipeline was tested using CMR images from a cohort of 12 rTOF patients from an internal site and 18 rTOF patients from an external site. Manually and automatically generated shape models from the test set were compared using Euclidean projection distances, global ventricular measurements, and atlas-based shape mode scores. The mean absolute error (MAE) between manually and automatically generated shape models in the test set was similar to the voxel resolution of the original CMR images for end-diastolic models (MAE = 1.9 ± 0.5 mm) and end-systolic models (MAE = 2.1 ± 0.7 mm). Global ventricular measurements computed from automated models were in good agreement with those computed from manual models. The average mean absolute difference in shape mode Z-score between manually and automatically generated models was 0.5 standard deviations for the first 20 modes of a reference statistical shape atlas. Using deep learning, accurate three-dimensional, biventricular shape models can be reliably created. This fully automated end-to-end approach dramatically reduces the manual input required to create shape models, thereby enabling the rapid analysis of large-scale datasets and the potential to deploy statistical atlas-based analyses in point-of-care clinical settings. Training data and networks are available from cardiacatlas.org.

Three-dimensional models, reconstructed from real-life objects, are extensively used in virtual and mixed reality technologies. In this paper we propose an approach to 3D model reconstruction via inverse procedural modeling and describe two variants of this approach. The first option is to fit a set of input parameters using a genetic algorithm. The second option allows us to significantly improve precision by using gradients within the memetic algorithm, differentiable rendering, and differentiable procedural generators. We demonstrate the results of our work on different models, including trees, which are complex objects that most existing methods cannot reconstruct. In our work, we see two main contributions. First, we propose a method to join differentiable rendering and inverse procedural modeling. This gives us the ability to reconstruct 3D models more accurately than existing approaches when few input images are available, even for a single image. Second, we combine both differentiable and non-differentiable procedural generators into a single framework that allows us to apply inverse procedural modeling to fairly complex generators. We show that both variants of our approach can be useful: the differentiable one is more precise but puts limitations on the procedural generator, while the one based on genetic algorithms can be used with any existing generator. The proposed approach uses information about the symmetry and structure of the object to achieve high-quality reconstruction from a single image. [ABSTRACT FROM AUTHOR]

Reconstructing 3D hair is challenging due to its complex micro‐scale geometry, and is of essential importance for the efficient creation of high‐fidelity virtual humans. Existing hair capture methods based on multi‐view stereo tend to generate results that are noisy and inaccurate. In this study, we propose a refinement method for hair geometry by incorporating the gradient of strands into the computation of their position. We formulate a gradient integration strategy for hair strands. We evaluate the performance of our method using a synthetic multi‐view dataset containing four hairstyles, and show that our refinement produces more accurate hair geometry. Furthermore, we tested our method with a real image input. Our method produces a plausible result. Our source code is publicly available at https://github.com/elerac/strand_integration. [ABSTRACT FROM AUTHOR]

Sketch‐based shape modeling aims to bridge the gap between 2D drawing and 3D modeling by providing an intuitive and accessible approach to create 3D shapes from 2D sketches. However, existing methods still suffer from limitations in reconstruction quality and multi‐view interaction friendliness, hindering their practical application. This paper proposes a faithful and user‐friendly iterative solution to tackle these limitations by learning geometry‐aligned deep implicit functions from one or multiple sketches. Our method lifts 2D sketches to volume‐based feature tensors, which align strongly with the output 3D shape, enabling accurate reconstruction and faithful editing. Such a geometry‐aligned feature encoding technique is well‐suited to iterative modeling since features from different viewpoints can be easily memorized or aggregated. Based on these advantages, we design a unified interactive system for sketch‐based shape modeling. It enables users to generate the desired geometry iteratively by drawing sketches from any number of viewpoints. In addition, it allows users to edit the generated surface by making a few local modifications. We demonstrate the effectiveness and practicality of our method with extensive experiments and user studies, where we found that our method outperformed existing methods in terms of accuracy, efficiency, and user satisfaction. The source code of this project is available at https://github.com/LordLiang/GA‐Sketching. [ABSTRACT FROM AUTHOR]

Various terrain modelling methods have been proposed for the past decades, providing efficient and often interactive authoring tools. However, they seldom include any notion of style, which is critical for designers in the entertainment industry. We introduce a new generative network method that bridges the gap between automatic terrain synthesis and authoring, providing a versatile set of authoring tools allowing spatialised style. We build upon the StyleGAN2 architecture and extend it with authoring tools. Given an input sketch or existing elevation map, our method generates a terrain with features that can be authored, enhanced, and augmented using interactive brushes and style manipulation tools. The strength of our approach lies in the versatility and interoperability of the different tools. We validate our method quantitatively with drainage calculation against other previous techniques and qualitatively by asking users to follow a prompt or freely create a terrain. [ABSTRACT FROM AUTHOR]

Procedural models enable the generation of a large amount of diverse shapes by varying the parameters of the model. However, writing a procedural model for replicating a collection of reference shapes is difficult, requiring much inspection of the original and replicated shapes during the development of the model. In this paper, we introduce a data‐guided method for aiding a programmer in creating a procedural model to replicate a collection of reference shapes. The user starts by writing an initial procedural model, and the system automatically predicts the model parameters for reference shapes, also grouping shapes by how well they are approximated by the current procedural model. The user can then update the procedural model based on the given feedback and iterate the process. Our system thus automates the tedious process of discovering the parameters that replicate reference shapes, allowing the programmer to focus on designing the high‐level rules that generate the shapes. We demonstrate through qualitative examples and a user study that our method is able to speed up the development time for creating procedural models of 2D and 3D man‐made shapes. [ABSTRACT FROM AUTHOR]

A least square support vector machine (SVM) model is proposed for shape modeling of slot antennas. The slot image is mapped into the electromagnetic response by the SVM model. A modified shapechanging technique is also proposed to describe the antenna geometry by the quadratic uniform B-spline curve and generate the slot images. In the model, the histogram of oriented gradients feature is extracted from the slot images to show the appearance and shape of the slot. The relationship between the histogram of oriented gradient features and the electromagnetic responses is preliminarily built on SVM and the transfer function. Then a radial basis function network is used for error correction. The effectiveness of the proposed model is confirmed with an example of a tri-band microstrip-fed slot antenna. Compared with the convolutional neural network (CNN), the feature extracted by CNN is substituted by the histogram of oriented gradients feature, and the proposed model shows the same accuracy and the improvement of training efficiency. [ABSTRACT FROM AUTHOR]

Implicit surface generation based on the interpolation of surface points is one of the well-known modeling methods in the area of computer graphics. Several methods for the implicit surface reconstruction from surface points have been proposed on the basis of radial basis functions, a weighted sum of local functions, splines, wavelets, and combinations of them. However, if the surface points contain errors or are sparsely distributed, irregular components, such as curvature-shaped redundant bulges and unexpectedly generated high-frequency components, are commonly seen. This paper presents a framework for restoring irregular components generated on and around surfaces. Users are assumed to specify local masks that cover irregular components and parameters that determine the degree of restoration. The algorithm in this paper removes the defects based on the user-specific masks and parameters. Experiments have shown that the proposed methods can effectively remove redundant protrusions and jaggy noise. [ABSTRACT FROM AUTHOR]

We present an efficient new method for computing Mandelbrot‐like fractals (Julia sets) that approximate a user‐defined shape. Our algorithm is orders of magnitude faster than previous methods, as it entirely sidesteps the need for a time‐consuming numerical optimization. It is also more robust, succeeding on shapes where previous approaches failed. The key to our approach is a versor‐modulus analysis of fractals that allows us to formulate a novel shape modulus function that directly controls the broad shape of a Julia set, while keeping fine‐grained fractal details intact. Our formulation contains flexible artistic controls that allow users to seamlessly add fractal detail to desired spatial regions, while transitioning back to the original shape in others. No previous approach allows Mandelbrot‐like details to be "painted" onto meshes. [ABSTRACT FROM AUTHOR]

The density-equalizing map, a technique developed for cartogram creation, has been widely applied to data visualization but only for 2D applications. In this work, we propose a novel method called the volumetric density-equalizing reference map for computing density-equalizing map for volumetric domains. Given a prescribed density distribution in a volumetric domain in R3, the proposed method continuously deforms the domain, with different volume elements enlarged or shrunk according to the density distribution. With the aid of the proposed method, medical and sociological data can be visualized via deformations of 3D objects. The method can also be applied to adaptive remeshing and shape modeling. Furthermore, by exploiting the time-dependent nature of the proposed method, applications to shape morphing can be easily achieved. Experimental results are presented to demonstrate the effectiveness of the proposed method.

Abstract Background Cardiac shape modeling is a useful computational tool that has provided quantitative insights into the mechanisms underlying dysfunction in heart disease. The manual input and time required to make cardiac shape models, however, limits their clinical utility. Here we present an end-to-end pipeline that uses deep learning for automated view classification, slice selection, phase selection, anatomical landmark localization, and myocardial image segmentation for the automated generation of three-dimensional, biventricular shape models. With this approach, we aim to make cardiac shape modeling a more robust and broadly applicable tool that has processing times consistent with clinical workflows. Methods Cardiovascular magnetic resonance (CMR) images from a cohort of 123 patients with repaired tetralogy of Fallot (rTOF) from two internal sites were used to train and validate each step in the automated pipeline. The complete automated pipeline was tested using CMR images from a cohort of 12 rTOF patients from an internal site and 18 rTOF patients from an external site. Manually and automatically generated shape models from the test set were compared using Euclidean projection distances, global ventricular measurements, and atlas-based shape mode scores. Results The mean absolute error (MAE) between manually and automatically generated shape models in the test set was similar to the voxel resolution of the original CMR images for end-diastolic models (MAE = 1.9 ± 0.5 mm) and end-systolic models (MAE = 2.1 ± 0.7 mm). Global ventricular measurements computed from automated models were in good agreement with those computed from manual models. The average mean absolute difference in shape mode Z-score between manually and automatically generated models was 0.5 standard deviations for the first 20 modes of a reference statistical shape atlas. Conclusions Using deep learning, accurate three-dimensional, biventricular shape models can be reliably created. This fully automated end-to-end approach dramatically reduces the manual input required to create shape models, thereby enabling the rapid analysis of large-scale datasets and the potential to deploy statistical atlas-based analyses in point-of-care clinical settings. Training data and networks are available from cardiacatlas.org.

Three-dimensional models, reconstructed from real-life objects, are extensively used in virtual and mixed reality technologies. In this paper we propose an approach to 3D model reconstruction via inverse procedural modeling and describe two variants of this approach. The first option is to fit a set of input parameters using a genetic algorithm. The second option allows us to significantly improve precision by using gradients within the memetic algorithm, differentiable rendering, and differentiable procedural generators. We demonstrate the results of our work on different models, including trees, which are complex objects that most existing methods cannot reconstruct. In our work, we see two main contributions. First, we propose a method to join differentiable rendering and inverse procedural modeling. This gives us the ability to reconstruct 3D models more accurately than existing approaches when few input images are available, even for a single image. Second, we combine both differentiable and non-differentiable procedural generators into a single framework that allows us to apply inverse procedural modeling to fairly complex generators. We show that both variants of our approach can be useful: the differentiable one is more precise but puts limitations on the procedural generator, while the one based on genetic algorithms can be used with any existing generator. The proposed approach uses information about the symmetry and structure of the object to achieve high-quality reconstruction from a single image.

We introduce an approach for converting pixel art into high‐quality vector images. While much progress has been made on automatic conversion, there is an inherent ambiguity in pixel art, which can lead to a mismatch with the artist's original intent. Further, there is room for incorporating aesthetic preferences during the conversion. In consequence, this work introduces an interactive framework to enable users to guide the conversion process towards high‐quality vector illustrations. A key idea of the method is to cast the conversion process into a spring‐system optimization that can be influenced by the user. Hereby, it is possible to resolve various ambiguities that cannot be handled by an automatic algorithm. [ABSTRACT FROM AUTHOR]

We propose a 3D+time framework for modeling dynamic sequences of 3D facial shapes, representing realistic non‐rigid motion during a performance. Our work extends neural 3D morphable models by learning a motion manifold using a transformer architecture. More specifically, we derive a novel transformer‐based autoencoder that can model and synthesize 3D geometry sequences of arbitrary length. This transformer naturally determines frame‐to‐frame correlations required to represent the motion manifold, via the internal self‐attention mechanism. Furthermore, our method disentangles the constant facial identity from the time‐varying facial expressions in a performance, using two separate codes to represent neutral identity and the performance itself within separate latent subspaces. Thus, the model represents identity‐agnostic performances that can be paired with an arbitrary new identity code and fed through our new identity‐modulated performance decoder; the result is a sequence of 3D meshes for the performance with the desired identity and temporal length. We demonstrate how our disentangled motion model has natural applications in performance synthesis, performance retargeting, key‐frame interpolation and completion of missing data, performance denoising and retiming, and other potential applications that include full 3D body modeling. [ABSTRACT FROM AUTHOR]

3D‐aware generative models have demonstrated their superb performance to generate 3D neural radiance fields (NeRF) from a collection of monocular 2D images even for topology‐varying object categories. However, these methods still lack the capability to separately control the shape and appearance of the objects in the generated radiance fields. In this paper, we propose a generative model for synthesizing radiance fields of topology‐varying objects with disentangled shape and appearance variations. Our method generates deformable radiance fields, which builds the dense correspondence between the density fields of the objects and encodes their appearances in a shared template field. Our disentanglement is achieved in an unsupervised manner without introducing extra labels to previous 3D‐aware GAN training. We also develop an effective image inversion scheme for reconstructing the radiance field of an object in a real monocular image and manipulating its shape and appearance. Experiments show that our method can successfully learn the generative model from unstructured monocular images and well disentangle the shape and appearance for objects (e.g., chairs) with large topological variance. The model trained on synthetic data can faithfully reconstruct the real object in a given single image and achieve high‐quality texture and shape editing results. [ABSTRACT FROM AUTHOR]

This paper addresses the design of surface shells as assemblies of tileable, flat geometric patterns with predictable performance in response to mechanical stimuli. We design a family of tileable and fabricable patterns represented as triangle meshes, which can be assembled for creating surface tessellations. First, a regular recursive subdivision of the planar space generates different geometric configurations for candidate patterns, having interesting and varied aesthetic properties. Then, a refinement step addresses manufacturability by solving for non-manifold configurations and sharp angles that would produce disconnected or weak patterns. We devise a strategy for creating continuous variations on the geometry of individual patterns, in both aesthetics and behavior, to enrich the catalog of available designs. Finally, we simulate our patterns to evaluate their mechanical response when loaded in different scenarios targeting out-of-plane bending. Through a simple browsing interface, we show that our patterns span a variety of different bending behaviors. The result is a catalog of patterns with varied aesthetics and predefined mechanical behavior, to use for the direct design of mechanical metamaterials. To assess the feasibility of our design-to-fabricate approach, we show fabricated 3D objects with different curvatures, and compare physical and simulated experiments. [Display omitted] • Tileable geometric patterns, generated through recursive subdivision and refinement for manufacturability. • The patterns exhibit diverse mechanical behaviors computed in standard out-of-plane bending scenarios. • Strategy for domain augmentation by variating continuous parameters. • Navigation of a design catalog to select desirable aesthetic and mechanical qualities. • Design-to-fabricate approach validated through digital fabrication of 3D objects in comparison with simulations. [ABSTRACT FROM AUTHOR]

An effective convolutional neural network (CNN) with the transfer function (TF) is proposed for shape modeling of electromagnetic (EM) behaviors of microstrip filters. The input of CNN is the images of metallic strips instead of the geometric parameters. To define the training samples, a one-to-one relation between the strip contour and the knot positions is built with a shape-changing technique based on cubic spline interpolation. The proposed model is confirmed with an example of a microstrip/coplanar waveguide (CPW) ultrawideband (UWB) filter. Compared with the parametric artificial neural network (ANN) and the shape ANN, the proposed model shows the improvement of design flexibility and the expansion of the solution domain. [ABSTRACT FROM AUTHOR]

An object shape information plays a vital role in many computer applications. Among these applications, some tasks can allow object shape analysis directly solve the problem. Thus, how to extract shape features and model the shape is a crucial issue. This paper proposes a new shape modeling method utilizing the centered convolutional deep Boltzmann machine to model two-dimensional (2D) shape. The proposed method employs a probabilistic generative model based on deep Boltzmann machine and convolution computation to extract both local and global features of the shape within one framework. In addition, we also propose a bidirectional inference-based training algorithm coupled with the centering method to better learn the probability distribution of shapes without the need for a pre-training procedure. Our experimental results show that the proposed model can achieve best shape modeling performance in qualitative and quantitative evaluation compared with other probabilistic generative models, including the restricted Boltzmann machine, convolutional restricted Boltzmann machine, and deep Boltzmann machine. [ABSTRACT FROM AUTHOR]

We present a reliable method to generate planar meshes of nonlinear rational triangular elements. The elements are guaranteed to be valid, i.e. defined by injective rational functions. The mesh is guaranteed to conform exactly, without geometric error, to arbitrary rational domain boundary and feature curves. The method generalizes the recent Bézier Guarding technique, which is applicable only to polynomial curves and elements. This generalization enables the accurate handling of practically important cases involving, for instance, circular or elliptic arcs and NURBS curves, which cannot be matched by polynomial elements. Furthermore, although many practical scenarios are concerned with rational functions of quadratic and cubic degree only, our method is fully general and supports arbitrary degree. We demonstrate the method on a variety of test cases. [ABSTRACT FROM AUTHOR]

Neural implicit fields are quickly emerging as an attractive representation for learning based techniques. However, adopting them for 3D shape modeling and editing is challenging. We introduce a method for Editing Implicit Shapes Through Part Aware GeneraTion, permuted in short as SPAGHETTI. Our architecture allows for manipulation of implicit shapes by means of transforming, interpolating and combining shape segments together, without requiring explicit part supervision. SPAGHETTI disentangles shape part representation into extrinsic and intrinsic geometric information. This characteristic enables a generative framework with part-level control. The modeling capabilities of SPAGHETTI are demonstrated using an interactive graphical interface, where users can directly edit neural implicit shapes. Our code, editing user interface demo and pre-trained models are available at github.com/amirhertz/spaghetti. [ABSTRACT FROM AUTHOR]

The so‐called motorcycle graph has been employed in recent years for various purposes in the context of structured and aligned block decomposition of 2D shapes and 2‐manifold surfaces. Applications are in the fields of surface parametrization, spline space construction, semi‐structured quad mesh generation, or geometry data compression. We describe a generalization of this motorcycle graph concept to the three‐dimensional volumetric setting. Through careful extensions aware of topological intricacies of this higher‐dimensional setting, we are able to guarantee important block decomposition properties also in this case. We describe algorithms for the construction of this 3D motorcycle complex on the basis of either hexahedral meshes or seamless volumetric parametrizations. Its utility is illustrated on examples in hexahedral mesh generation and volumetric T‐spline construction. [ABSTRACT FROM AUTHOR]

Immersive environments with head mounted displays (HMD) and hand-held controllers, either in Virtual or Augmented Reality (VR/AR), offer new possibilities for the creation of artistic 3D content. Some of them are exploited by mid-air drawing applications: the user's hand trajectory generates a set of stylized curves or ribbons in space, giving the impression of painting or drawing in 3D. We propose a method to extend this approach to the sketching of surfaces with a VR controller. The idea is to favor shape exploration, offering a tool, where the user creates a surface just by painting ribbons. These ribbons are not constrained to form patch boundaries for example or to completely cover the shape. They can be very sparse, disordered, overlap or not, intersect or not. The shape is computed simultaneously, starting with the first piece of ribbon drawn by the user and continuing to evolve in real-time as long as the user continues sketching. Our method involves minimizing an energy function based on the projections of the ribbon strokes on a proxy surface by taking the controller's orientations into account. The current implementation considers elevation surfaces. In addition to many examples, we evaluate the time performance of the dynamic shape modeling with respect to an increasing number of input ribbon strokes. Finally, we present images of an artistic creation that combines stylized curve drawings in VR with our surface sketching tool created by a professional artist. [Display omitted] • A novel VR sketch-based surface modeling tool is presented. • Ribbon strokes are drawn as if imitating the artist's gesture while painting. • On-the-fly creation and updating of surfaces from disordered ribbon strokes. • Can process sparse, disordered, possibly overlapping ribbon strokes. • Interactive frame rates give immediate visual feedback on the sketching gestures. [ABSTRACT FROM AUTHOR]

Time-varying connectivity of triangle mesh sequences leads to substantial difficulties in their processing. Unlike editing sequences with constant connectivity, editing sequences with varying connectivity requires addressing the problem of temporal correspondence between the frames of the sequence. We present a method for time-consistent editing of triangle mesh sequences with varying connectivity using sparse temporal correspondence, which can be obtained using existing methods. Our method includes a deformation model based on the usage of the sparse temporal correspondence, which is suitable for the temporal propagation of user-specified deformations of the edited surface with respect to the shape and true topology of the surface while preserving the individual connectivity of each frame. Since there is no other method capable of comparable types of editing on time-varying meshes, we compare our method and the proposed deformation model with a baseline approach and demonstrate the benefits of our framework. [Display omitted] • This paper presents a method for editing mesh sequences with varying connectivity. • The proposed method exploits sparse temporal correspondences. • Dual quaternions are used to represent the transformations. [ABSTRACT FROM AUTHOR]

Statistical shape models are an essential tool for various tasks in medical image analysis, including shape generation, reconstruction and classification. Shape models are learned from a population of example shapes, which are typically obtained through segmentation of volumetric medical images. In clinical practice, highly anisotropic volumetric scans with large slice distances are prevalent, e.g., to reduce radiation exposure in CT or image acquisition time in MR imaging. For existing shape modeling approaches, the resolution of the emerging model is limited to the resolution of the training shapes. Therefore, any missing information between slices prohibits existing methods from learning a high-resolution shape prior. We propose a novel shape modeling approach that can be trained on sparse, binary segmentation masks with large slice distances. This is achieved through employing continuous shape representations based on neural implicit functions. After training, our model can reconstruct shapes from various sparse inputs at high target resolutions beyond the resolution of individual training examples. We successfully reconstruct high-resolution shapes from as few as three orthogonal slices. Furthermore, our shape model allows us to embed various sparse segmentation masks into a common, low-dimensional latent space — independent of the acquisition direction, resolution, spacing, and field of view. We show that the emerging latent representation discriminates between healthy and pathological shapes, even when provided with sparse segmentation masks. Lastly, we qualitatively demonstrate that the emerging latent space is smooth and captures characteristic modes of shape variation. We evaluate our shape model on two anatomical structures: the lumbar vertebra and the distal femur, both from publicly available datasets. • A novel continuous shape model enables training on sparse, anisotropic volumetric segmentation masks. • High-resolution shape reconstruction from various sparse inputs. • Unification of sparse segmentations in a meaningful, low-dimensional joint latent space. • Discriminative latent space allows for the classification of knee osteoarthritis from sparse inputs. [ABSTRACT FROM AUTHOR]

It is a challenging task to transform style of 3D shapes for generating diverse outputs with learning-based methods. The reasons include two folds: (1) the lack of training data with different styles and (2) multi-modal information of 3D shapes which are hard to disentangle. In this work, a multi-view-based neural network model is proposed to learn style transformation while preserving contents of 3D shapes from unpaired domains. Given two sets of shapes in different style domains, such as Japanese chairs and Ming chairs, multi-view representations of each shape are calculated, and style transformation between these two sets is learnt based on these representations. This multi-view representation not only preserves the structural details of a 3D shape, but also ensures the richness of the training data. At test stage, transformed maps are generated with the trained network by the combination of the extracted style/content features from multi-view representation and new style features. Then, transformed maps are consolidated into a 3D point cloud by solving a domain-stability optimization problem. Depth maps from all viewpoints are fused to obtain a shape whose style is similar to the target shape. Experimental results demonstrate that the proposed method outperforms the baselines and state-of-the-art approaches on style transformation. [ABSTRACT FROM AUTHOR]

Purpose: The shape of the mandible has been analyzed in a variety of fields, whether to diagnose conditions like osteoporosis or osteomyelitis, in forensics, to estimate biological information such as age, gender, and race or in orthognathic surgery. Although the methods employed produce encouraging results, most rely on the dry bone analyses or complex imaging techniques that, ultimately, hamper sample collection and, as a consequence, the development of large-scale studies. Thus, we proposed an objective, repeatable, and fully automatic approach to provide a quantitative description of the mandible in orthopantomographies (OPGs). Methods: We proposed the use of a deep convolutional neural network (CNN) to localize a set of landmarks of the mandible contour automatically from OPGs. Furthermore, we detailed four different descriptors for the mandible shape to be used for a variety of purposes. This includes a set of linear distances and angles calculated from eight anatomical landmarks of the mandible, the centroid size, the shape variations from the mean shape, and a group of shape parameters extracted with a point distribution model. Results: The fully automatic digitization of the mandible contour was very accurate, with a mean point to the curve error of 0.21 mm and a standard deviation comparable to that of a trained expert. The combination of the CNN and the four shape descriptors was validated in the well-known problems of forensic sex and age estimation, obtaining 87.8% of accuracy and a mean absolute error of 1.57 years, respectively. Conclusion: The methodology proposed, including the shape model, can be valuable in any field that requires a quantitative description of the mandible shape and a visual representation of its changes such as clinical practice, surgery management, dental research, or legal medicine. [ABSTRACT FROM AUTHOR]

We propose an approach to construct realistic 3D facial morphable models (3DMM) that allows an intuitive facial attribute editing workflow. Current face modeling methods using 3DMM suffer from a lack of local control. We thus create a 3DMM by combining local part‐based 3DMM for the eyes, nose, mouth, ears, and facial mask regions. Our local principal component analysis (PCA)‐based approach uses a novel method to select the best eigenvectors from the local 3DMM to ensure that the combined 3DMM is expressive, while allowing accurate reconstruction. We provide different editing paradigms, all designed from the analysis of the data set. Some use anthropometric measurements from the literature and others allow the user to control the dominant modes of variation extracted from the data set. Our part‐based 3DMM is compact, yet accurate, and compared to other 3DMM methods, it provides a new trade‐off between local and global control. We tested our approach on a data set of 135 scans used to derive the 3DMM, plus 19 scans that served for validation. The results show that our part‐based 3DMM approach has excellent generative properties and allows the user intuitive local control. [ABSTRACT FROM AUTHOR]

[Display omitted] • l -cysteine was used as pre-treatment to break disulphide bonds. • The shape of wool yarns was changed by tyrosine grafting. • Laccase was used as catalyst for the tyrosine grafting. The shape of wool yarns was changed by laccase-assisted grafting of tyrosine. Prior to tyrosine grafting a cysteine pre-treatment was optimized aiming to increase the amount of thiol reaction groups available. The best operational conditions for laccase-assisted tyrosine grafting were: i) pre-treatment with cysteine (2.2 mM) in a solution of 20 % ethanol, 15 % propylene glycol and 0.5 % benzyl alcohol, pH = 10, 40 °C; ii) tyrosine grafting with 3.0 mM tyrosine, 18 U/mL laccase, pH = 5, 40 °C. The shape modification was evaluated by number of curly twists determination on the grafted yarn samples. The thermal and mechanical properties of the grafted wool yarns was evaluated by TGA, DSC and breaking strength determination. The amount of free thiols and weight gain were assessed aiming to infer the role of the cysteine pre-treatment on the final tyrosine grafting and shape modification. The laccase-assisted grafting of tyrosine onto wool yarns have influenced the thermal and mechanical properties of the yarns however without compromising their structural integrity for the final application purposes. The developed methodology to impart new shape to wool yarns is presented herein as an environmentally friendly alternative to chemical methods. The new findings revealed great potentialities for application in similar fibers like hair. [ABSTRACT FROM AUTHOR]

This article presents a rapid parametric design system of the rotary kinetic sculpture. The multilevel skeletons enable users to model propeller-like units rapidly, arrange and deform them collaboratively, and generate transmission mechanisms automatically without any specialized machine knowledge. Experimental results show that our system can help users get diverse rotary kinetic sculptures effectively. [ABSTRACT FROM AUTHOR]

A common approach to automatic quad layout generation on surfaces is to, in a first stage, decide on the positioning of irregular layout vertices, followed by finding sensible layout edges connecting these vertices and partitioning the surface into quadrilateral patches in a second stage. While this two‐step approach reduces the problem's complexity, this separation also limits the result quality. In the worst case, the set of layout vertices fixed in the first stage without consideration of the second may not even permit a valid quad layout. We propose an algorithm for the creation of quad layouts in which the initial layout vertices can be adjusted in the second stage. Whenever beneficial for layout quality or even validity, these vertices may be moved within a prescribed radius or even be removed. Our algorithm is based on a robust quantization strategy, turning a continuous T‐mesh structure into a discrete layout. We show the effectiveness of our algorithm on a variety of inputs. [ABSTRACT FROM AUTHOR]

This paper proposes a passive multi‐view system for human body shape reconstruction, namely RHF‐Human, to overcome several challenges including accurate calibration and stereo matching in self‐occluded and low‐texture skin regions. The reconstruction process includes four steps: capture, multi‐view camera calibration, dense reconstruction, and meshing. The capture system, which consists of 90 digital single‐lens reflex cameras, is single‐shot to avoid nonrigid deformation of the human body. Two technical contributions are made: (1) a two‐step robust multi‐view calibration approach that improves calibration accuracy and saves calibration time for each new human body acquired and (2) an accurate PatchMatch multi‐view stereo method for dense reconstruction to perform correct matching in self‐occluded and low‐texture skin regions and to reduce the noise caused by body hair. Experiments on models of various genders, poses, and skin with different amounts of body hair show the robustness of the proposed system. A high‐fidelity human body shape dataset with 227 models is constructed, and the average accuracy is within 1.5 mm. The system provides a new scheme for the accurate reconstruction of nonrigid human models based on passive vision and has good potential in fashion design and health care. [ABSTRACT FROM AUTHOR]

Extending the ARAP energy with a term that depends on the face normal, energy minimization becomes an effective stylization tool for shapes represented as meshes. Our approach generalizes the possibilities of Cubic Stylization: the set of preferred normals can be chosen arbitrarily from the Gauss sphere, including semi‐discrete sets to model preference for cylinder‐ or cone‐like shapes. The optimization is designed to retain, similar to ARAP, the constant linear system in the global optimization. This leads to convergence behavior that enables interactive control over the parameters of the optimization. We provide various examples demonstrating the simplicity and versatility of the approach. [ABSTRACT FROM AUTHOR]

Parametric shapes model objects as programs producing a geometry based on a few semantic degrees of freedom, called hyper-parameters. These shapes are the typical output of non-destructive modeling, CAD modeling or rigging. However they suffer from the core issue of being manipulated only indirectly, through a series of values rather than the geometry itself. In this paper, we introduce an amendment process of the underlying direct acyclic graph (DAG) of a parametric shape. This amendment enables a local differentiation of the shape w.r.t. its hyper-parameters that we leverage to provide interactive direct manipulation of the output. By acting on the shape synthesis process itself, our method is agnostic to the variations of the connectivity and topology that may occur in its output while changing the input hyper-parameters. Furthermore, our method is oblivious to the internal logic of the DAG nodes. We illustrate our approach on a collection of examples combining the typical nodes found in modern parametric modeling packages - such as deformation, booleans and surfacing operators - for which our method provides the user with inverse control over the hyper-parameters through a brush stroke metaphor. [ABSTRACT FROM AUTHOR]

In this article we introduce a differentiable rendering module which allows neural networks to efficiently process 3D data. The module is composed of continuous piecewise differentiable functions defined as a sensor array of cells embedded in 3D space. Our module is learnable and can be easily integrated into neural networks allowing to optimize data rendering towards specific learning tasks using gradient based methods in an end-to-end fashion. Essentially, the module's sensor cells are allowed to transform independently and locally focus and sense different parts of the 3D data. Thus, through their optimization process, cells learn to focus on important parts of the data, bypassing occlusions, clutter, and noise. Since sensor cells originally lie on a grid, this equals to a highly non-linear rendering of the scene into a 2D image. Our module performs especially well in presence of clutter and occlusions as well as dealing with non-linear deformations to improve classification accuracy through proper rendering of the data. In our experiments, we apply our module in various learning tasks and demonstrate that using our rendering module we accomplish efficient classification, localization, and segmentation tasks on 2D/3D cluttered and non-cluttered data. [ABSTRACT FROM AUTHOR]

In this paper, we leverage advances in neural networks towards forming a neural rendering for controllable image generation, and thereby bypassing the need for detailed modeling in conventional graphics pipeline. To this end, we present Neural Graphics Pipeline (NGP), a hybrid generative model that brings together neural and traditional image formation models. NGP decomposes the image into a set of interpretable appearance feature maps, uncovering direct control handles for controllable image generation. To form an image, NGP generates coarse 3D models that are fed into neural rendering modules to produce view‐specific interpretable 2D maps, which are then composited into the final output image using a traditional image formation model. Our approach offers control over image generation by providing direct handles controlling illumination and camera parameters, in addition to control over shape and appearance variations. The key challenge is to learn these controls through unsupervised training that links generated coarse 3D models with unpaired real images via neural and traditional (e.g., Blinn‐Phong) rendering functions, without establishing an explicit correspondence between them. We demonstrate the effectiveness of our approach on controllable image generation of single‐object scenes. We evaluate our hybrid modeling framework, compare with neural‐only generation methods (namely, DCGAN, LSGAN, WGAN‐GP, VON, and SRNs), report improvement in FID scores against real images, and demonstrate that NGP supports direct controls common in traditional forward rendering. Code is available at http://geometry.cs.ucl.ac.uk/projects/2021/ngp. [ABSTRACT FROM AUTHOR]

Statistical shape models (SSMs) have been established as robust anatomical priors for medical image segmentation, registration and anatomy modelling. To construct an SSM which accurately models the inter-subject anatomical variations, it is crucial to compute accurate shape correspondence between the training samples. To achieve this goal, the state-of-the-art shape correspondence computation methods always require tedious segmentation of the training images, while they seldom pay enough attention to the correspondence accuracy of key anatomical landmarks like the bone joints, vessel bifurcations, etc. As a result, the computation of shape correspondence is time-consuming and the correspondence accuracy is imperfect. To solve these problems, this paper proposes a novel shape correspondence computation approach which eliminates the need for image segmentation by registering an organ shape template to the training images. This method allows the human expert to specify key anatomical landmarks in the training images to define the correspondence of the crucial landmarks. An intensity-and-landmark-combined strategy is implemented to utilized both the image intensity and expert landmarks to obtain accurate shape correspondence. This method is evaluated for the construction of head anatomy SSM and spine SSM based on computed tomography (CT) images. The SSMs constructed using the proposed method demonstrates better shape correspondence accuracy than other state-of-the-arts correspondence methods. In particular, this method obtains pixel-level surface correspondence accuracy (1.38 mm) for the skull and sub-pixel level accuracy (0.92 mm) for the spine. The generalisability and specificity of the SSMs constructed using our method are also superior to SSMs constructed using other compared correspondence methods. With this method, we propose a novel approach which takes less human intervention and produces higher quality SSM with better shape modelling accuracy.

The identification and annotation of buildings has long been a tedious and expensive part of high-precision vector map production. The deep learning techniques such as fully convolution network (FCN) have largely promoted the accuracy of automatic building segmentation from remote sensing images. However, compared with the deep-learning-based building segmentation methods that greatly benefit from data-driven feature learning, the building boundary vector representation generation techniques mainly rely on handcrafted features and high human intervention. These techniques continue to employ manual design and ignore the opportunity of using the rich feature information that can be learned from training data to directly generate vectorized boundary descriptions. Aiming to address this problem, we introduce PolygonCNN, a learnable end-to-end vector shape modeling framework for generating building outlines from aerial images. The framework first performs an FCN-like segmentation to extract initial building contours. Then, by encoding the vertices of the building polygons along with the pooled image features extracted from segmentation step, a modified PointNet is proposed to learn shape priors and predict a polygon vertex deformation to generate refined building vector results. Additionally, we propose 1) a simplify-and-densify sampling strategy to generate homogeneously sampled polygon with well-kept geometric signals for shape prior learning; and 2) a novel loss function for estimating shape similarity between building polygons with vastly different vertex numbers. The experiments on over 10,000 building samples verify that PolygonCNN can generate building vectors with higher vertex-based F1-score than the state-of-the-art method, and simultaneously well maintains the building segmentation accuracy achieved by the FCN-like model. [ABSTRACT FROM AUTHOR]

3D models have been widely used in a great variety of fields. The verification and debugging of the algorithms on 3D models are critical. MeshLab provides a series of infrastructure for 3D shape analysis with an open architecture. This paper summarizes the key technologies involved in the development of 3D based on MeshLab, including adding customized menus and batch importing data. This paper presents an algorithm for traversing arbitrary order adjacent mesh primitives based on the Depth-first algorithm and design pattern. The test result shows that the proposed algorithm is superior to the traditional method in the aspect of cyclomatic complexity. The novel algorithm has succeeded in performing arbitrary order adjacent operations on a series of challenging meshes. [ABSTRACT FROM AUTHOR]

Representing 3D geometry for different tasks, e.g. rendering and reconstruction, is an important goal in different fields, such as computer graphics, computer vision and robotics. Robotic applications often require perception of object shape information extracted from sensory data that can be noisy and incomplete. This is a challenging task and in order to facilitate analysis of new methods and comparison of different approaches for shape modeling (e.g. surface estimation), completion and exploration, we provide real sensory data acquired from exploring various objects of different complexities. The dataset includes visual and tactile readings in the form of 3D point clouds obtained using two different robot setups that are equipped with visual and tactile sensors. During data collection, the robots touch the experiment objects in a predefined manner at various exploration configurations and gather visual and tactile points in the same coordinate frame based on calibration between the robots and the used cameras. The goal of this exhaustive exploration procedure is to sense unseen parts of the objects which are not visible to the cameras, but can be sensed via tactile sensors activated at touched areas. The data was used for shape completion and modeling via Implicit Surface representation and Gaussian-Process-based regression, in the work “Object shape estimation and modeling, based on sparse Gaussian process implicit surfaces, combining visual data and tactile exploration” [3], and also used partially in “Enhancing visual perception of shape through tactile glances” [4], both studying efficient exploration of objects to reduce number of touches.

Purpose: Statistical shape modeling provides a powerful tool for describing and analyzing human anatomy. By linearly combining the variance of the shape of a population of a given anatomical entity, statistical shape models (SSMs) identify its main modes of variation and may approximate the total variance of that population to a selected threshold, while reducing its dimensionality. Even though SSMs have been used for over two decades, they lack in characterization of their goodness of prediction, in particular when defining whether these models are actually representative for a given population.Methods: The current paper presents, to the authors' knowledge, the most extent lower limb anatomy shape model considering the pelvis, femur, patella, tibia, fibula, talus, and calcaneum to date. The present study includes the segmented training shapes (n = 542) obtained from 271 lower limb CT scans. The different models were evaluated in terms of accuracy, compactness, generalizability as well as specificity.Results: The size of training samples needed in each model so that it can be considered population covering was estimated to approximate around 200 samples, based on the generalizability properties of the different models. Simultaneously differences in gender and patterns in left-right asymmetry were identified and characterized. Size was found to be the most pronounced sexual discriminator whereas intra-individual variations in asymmetry were most pronounced at the insertion site of muscles.Conclusion: For models aimed at population covering descriptive studies, the number of training samples required should amount a sizeable 200 samples. The geometric morphometric method for sex discrimination scored excellent, however, it did not largely outperformed traditional methods based on discrete measures.