Search

Generating talking lips in sync with input speech has the potential to enhance speech communication and enable novel applications. This paper presents a system that can generate accurate 3D talking lips, readily applicable to unseen subjects and different languages. The developed head-mounted facial acquisition device and automated data processing pipeline can generate precise landmarks while mitigating the difficulty of acquiring 3D facial data. Our system consists of three stages to generate accurate lip movements. In the first stage, the fine-tuned Wav2Vec2.0+Transformer captures long-range audio context dependencies. In the second stage, we propose the Viseme Fixing method, which significantly improves lip accuracy at/b//p//m//f/ phonemes. In the last stage, we innovatively use the structural relationship between the inner and outer lips and learn to map the outer lip landmarks to the inner lip landmarks. Subjective evaluations show that the generated talking lips match the input audio significantly. We demonstrate two applications that animate 2D face videos and 3D face models using our landmarks. The precise lip landmarks allow the generated animations to exceed the results of state-of-the-art methods. [Display omitted] • Generate accurate 3D talking lips for unseen subjects and different languages. • Develop a head-mounted facial acquisition device to acquire precise 3D facial data. • Propose Viseme Fixing algorithm to enhance the lip accuracy of /b//p//m//f/ visemes. • Make two applications that bridge 3D landmarks to talking videos and talking meshes. [ABSTRACT FROM AUTHOR]

This paper proposes a new approach that acquires camera parameters and generates an integrated 3D joint using an RGB-D camera network distributed in an arbitrary location in space. The proposed technique consists of three steps. In the first step, camera parameters are calculated using partial joints as feature points. The internal and external parameters between cameras are not calculated using a specially manufactured calibration plate. In the second step, a 3D joint set is estimated by integrating the joints obtained from each camera using the calculated camera parameters. At the same time, a 3D volumetric model in the form of a point cloud is reconstructed. The third step consists of a joint correction algorithm. The resultant 3D joint with high reliability is estimated by correcting the position of the joint using the 3D point cloud reconstructed previously. The generated 3D joint can accurately express the shape and movement of 3D human. The estimated 3D joint was compared with the joint measured using a motion capture device to evaluate its performance. The temporal standard deviation between two measurements shows very low value from 1.966 mm to 7.99 mm. [Display omitted] • Performing camera calibration using incomplete or partial joints obtained from images output from cameras installed in irregular positions. • Temporal improvement of the precision of transformation parameters obtained through camera calibration. • 3D joint generation while minimizing the error of pose estimation algorithm. • Correction of 3d joint using 3d point cloud registration. • Simultaneous 3D reconstruction of 3D indoor space and humans. [ABSTRACT FROM AUTHOR]

To display images on panels and screens of different dimensions, there is a need to create algorithms that help adjust them to desired sizes through image retargeting (IR). In this sense, choosing the right algorithms seems to be a challenge. From this perspective, this paper was to propose a method for the assessment of retargeted images using deep learning (DL) models. To this end, the Conditional Random Fields as Recurrent Neural Networks (CRF-RNN) approach was applied to find image pixels in terms of importance and, the You-Only-Look-Once (YOLO) model was employed to identify semantic objects of images. As well, image patch similarity was computed based on Siamese Neural Network (SNN), and output image distortion was recognized by the CNN model. With reference to three well-known databases, viz., MIT RetargetMe, NRID and CUHK, the assessment results of the proposed algorithm demonstrated its superior usage as compared to the existing ones. [Display omitted] • Operating an R-CNN model to extract image importance maps. • Using a pre-trained YOLO to extract regions containing important image objects. • Training SNN to compute corresponding patch-similarity of original and retargeted images. • Training regression-based CNN to measure distortion of retargeted images based on pixel displacement patterns. • Merging measures using Gaussian Process Regression (GPR) and calculating the final measure. [ABSTRACT FROM AUTHOR]

Understanding and analyzing univariate distributions of data in terms of their shapes as well as their specific characteristics, regarding gaps, spikes, or outliers, is crucial in many scientific disciplines. In this paper, we propose a design space composed of the visual channels position and color for representing accumulated distributions. The designs are a mixture of color-coded stripes with density lines. The width and coloring of the stripes is based on the applied binning technique. In a crowd-sourced experiment we explore a subspace, called the AccuStripes (i.e., "accumulated stripes") design space, consisting of nine representations. These AccuStripes designs integrate three composition strategies (color only, overlay, filled curve) with three binning techniques, one uniform (UB) and two adaptive methods, namely Bayesian Blocks (BB) and Jenks' Natural Breaks (NB). We evaluate the accuracy, efficiency, and confidence ratings of the nine AccuStripes designs for structural estimation and comparison tasks. Across all study tasks, the overlay composition was found to be most accurate and preferred by observers. Furthermore, the results demonstrate that while no binning method performed best in both identification and comparison, detection of structures using adaptive binning was the most accurate one. For validation we compared the best AccuStripes' design, i.e., the overlay composition, to line charts. Our results show that the AccuStripes' design outperformed the line charts in accuracy for all study tasks. [Display omitted] • Novel design space to represent and compare univariate data distributions. • A subspace is introduced combining color-coded stripes with density lines. • The stripes' width and coloring are defined by adaptive and uniform binning methods. • Evaluation and validation of the subspace through two crowdsource studies. [ABSTRACT FROM AUTHOR]

Point cloud registration aligns point clouds through transformation or deformation. The existing advanced registration methods perform poorly in registering under various perturbations, especially for large rotations and small overlaps. Moreover, these methods cannot achieve rigid and non-rigid registrations simultaneously. In this paper, we propose a two-step generative optimization (GO) method to achieve registration, which alleviates the constraints of transformation representation and completely circumvents the matching of correspondences, making it suitable for rigid registration with large rotations and small overlaps, as well as for non-rigid registration. GO achieves registration by generating a pseudo point cloud approximating the target point cloud. The pseudo point cloud is iteratively deformed to approach the target point cloud through an estimation step (ES) and an updating step (US). The ES step generates initial pseudo point clouds in three different views and estimates their deformation via the extracted features decoding the similarity between the pseudo point clouds and the target point cloud. The US step calculates regressors to update the current estimations, generating new pseudo point clouds closer to the target point cloud. The final pseudo point clouds generated are averaged to attain the registration result. We evaluate the performance of GO for rigid and non-rigid registrations through the comparison with seven deep learning-based methods on eight datasets, consisting of both 3D and 2D data and ranging from synthetic to real scenes. Our experimental results demonstrate the high robustness and stability of GO in rigid registrations and illustrate the feasibility and high accuracy for non-rigid registrations. [Display omitted] Given a source point cloud S and a target point cloud M, GO aims to register S to M by iteratively generating different pseudo point clouds P t S, P t F, P t O, and making each generated pseudo point cloud approach the target model M to mimic the deformation of the source model S. The registration result S T is the average of the generated pseudo point clouds at the final iteration. Specifically, the GO iteration alternates between performing an estimation (ES) step and an updating (US) step. The former aims to estimate the coordinates of the newly generated pseudo point clouds in different views via the extracted feature of the existing pseudo point clouds. The latter intends to update the existing pseudo point clouds for generating new pseudo point clouds by making the current estimation approach the target point cloud. The generated pseudo point clouds are then used to estimate the coordinates of new pseudo point clouds in the next ES step. GO terminates the iteration once the training error is below a threshold. And the generated pseudo point clouds are averaged to attain the final registration result simultaneously. • A novel two-step generative optimization (GO) is proposed for rigid and non-rigid registration. • GO outperforms both traditional and deep learning methods in robustness and accuracy. • The generation of pseudo models expedites the convergence of GO. [ABSTRACT FROM AUTHOR]

Point cloud registration precision degrades when the scene contains symmetric structures or repeated patches. Also, the common operation of continuous downsampling in existing registration methods causes the loss of detailed information and leads to further registration precision degradation. In this paper, we propose a high precision point cloud registration network (HPNet) to learn discriminative features by considering both multi-scale information and position information. We first propose a multi-scale feature extraction module similar to the feature pyramid that allows the extracted features to contain multi-scale information. Then, an information interaction is performed on the features to learn global contextual information by using Transformer with a hybrid position encoding, which takes into account both absolute and relative positions of points. Finally, the features obtained from the information interaction module are directly used to predict the point correspondences. Comprehensive experiments on 3DMatch, 3DLoMatch, ModelNet, and ModelLoNet datasets demonstrate the state-of-the-art performance of the proposed method and the implementation of HPNet in SLAM shows its effectiveness in real application. [Display omitted] • Point cloud registration loses detail information during continuous downsampling. • Feature fusion of low-level detail information and high-level semantic information. • Hybrid position encoding consider both absolute and relative positions. positions. • Point cloud registration algorithms applied to SLAM. [ABSTRACT FROM AUTHOR]

Real point cloud registration, involving homologous and cross-source 3D data, poses significant challenges such as partial overlap, high noise, density disparities, and scale variations. In this paper, we introduce a framework called OEFM to guide transform estimation on the basis of feature metrics through overlap estimation with overlap region center weighting, which estimates transform parameters by minimizing feature projection error on overlap region features. Our proposed approach utilizes an overlap filtering module with a center-weighting mechanism to filter overlap points and feeds these points into a feature metric framework with a forward–backward transformation to estimate the transformation parameters. The rationale behind our approach is that only overlapping regions are useful for point cloud registration, while non-overlapping regions are anomalous. Furthermore, our proposed approach requires no correspondence search, making it robust to partial overlaps, large noise, density differences, and scale variations. We show that our approach achieves the highest registration recall for 3DMatch and 3DCSR in extensive experiments on both homologous and cross-source datasets. • We present an efficient unsupervised training module for transform estimation with forward–backward transformation, which could enhance the performance of unsupervised training as well as transform estimation. • We provide a robust point cloud registration technique combined with feature matching, without using the common correspondence search. [ABSTRACT FROM AUTHOR]

The ability to ease operation and real-time feedback to image editing has attracted increasing attention in our everyday lives. Existing edit propagation approaches typically formulate the color editing task as a quadratic energy optimization problem, which is computationally expensive and requires a lot of user interactions. To address this problem, in this paper, we propose a novel edit propagation method based on color palette. We first extract a color palette from the input image, and then calculate the mixing weights of the image pixels with respect to the extracted palette. Finally, we solve for an edited palette according to user edits, to propagate local edits to the whole image. Our main contributions include formulating the edit propagation as a palette-based optimization problem that can be solved efficiently, and requiring only fewer user interactions than existing methods. We have demonstrated our method for color editing on a wide range of examples. Compared to existing methods, our approach is more efficient and friendly for novice users due to its ease of interaction. [Display omitted] • We presented a palette-based edit propagation for pixel-level color editing. • We developed a graphical user interface (GUI) that is easy to use and learn. • Our approach is efficient and requires fewer interactions than previous methods. [ABSTRACT FROM AUTHOR]

Simulating multi-body dynamics with both rigid and flexible parts and with frictional contacts is a hard problem. We solve this by expressing the couplings between the bodies as position level constraints. The implicit treatment of the constraint directions gives us improved stability over velocity based methods. Then by employing regularization of nonlinear constraints and a convex minimization formulation, we bridge constraint-based methods to traditional force-based methods. In fact, the former are just a dual variables formulation of the latter. We solve this dual problem using position based dynamics (PBD). We show how PBD is a completely valid modeling technique and we extend it with an accurate contact and Coulomb friction model. We further show for the first time how the same solver can be used to simulate both rigid and deformable solids with two way coupling. For the soft bodies we introduce a novel form of linear finite elements expressed as constraints, that is more accurate than PBD mass-spring systems. More of our results include the energy conserving Newmark integrator and the accelerated Jacobi solver suitable for parallel architectures. Note that this paper is an extended and revised version of the conference paper published in [1]. [ABSTRACT FROM AUTHOR]

Digital images are everywhere, from social media to news and scientific papers. This paper describes an extensive user study to evaluate the ability of an average individual to spot edited images. By design, our study avoids lucky guesses. After observing an image, subjects were asked if it is authentic or not. Whenever a subject indicated that an image has been altered, (s)he had to provide evidence to support the answer by pointing at the suspected region in the image. We collected 17,208 individual answers from 393 volunteers, using 177 images selected from public forensic databases. Our results indicate that the average individual is not good at distinguishing original from edited images, answering correctly on 58% of all images, and only identifying the modified ones 46.5% of the time. This performance is superior to random guessing, but poor compared to results achieved by computational techniques. [ABSTRACT FROM AUTHOR]

With recent technological advancements in sports broadcasting, remote spectators are presented with an enriched experience. These enriched experiences include additional content such as statistics and graphics that support game understanding. However, these technological advancements are often not accessible to on-site sports spectators. In this paper, we explore the opportunities of using situated visualization to enrich on-site sports spectating. Situated visualization techniques allow us to display information in a spatial context to its physical reference and have the potential to close the gap between on-site spectating and content access. With regards to this, we developed a framework for situated visualization focusing on sports spectating. We identified components needed for such a use case and developed two novel situated visualization approaches based on the proposed framework: (1) situated broadcast-styled visualization and (2) situated infographics. To evaluate the visualization approaches and explore user preferences, we conducted a lab study and a subsequent on-site study in a stadium. [Display omitted] • Conceptual framework describing components of on-site situated sports spectating. • Main components of sports situated visualization are Canvas, Content and Context. • Framework led to situated broadcast-style visualization and situated infographics. • Positive responses towards situated infographics and an improved game understanding. • No increase in mental demand recorded for situated infographics despite the benefits. [ABSTRACT FROM AUTHOR]

3D reconstruction from a single image is one of the core computer vision problems. Thanks to the development of deep learning, 3D reconstruction of a single image has demonstrated impressive progress in recent years. Existing researches use Chamfer distance as a loss function to guard the training of the neural network. However, the Chamfer loss will give equal weights to all points inside the 3D point clouds. It tends to sacrifice fine-grained and thin structures to avoid incurring a high loss, which will lead to visually unsatisfactory results. This paper proposes a framework that can recover a detailed three-dimensional point cloud from a single image by focusing more on boundaries (edge and corner points). Experimental results demonstrate that the proposed method outperforms existing techniques significantly, both qualitatively and quantitatively, and has fewer training parameters. [Display omitted] • Reconstructing the full 3D shape of an object from a single image. • Using a differentiable point cloud projection module to get the projected images. • Increasing visual reconstruction quality. [ABSTRACT FROM AUTHOR]

Point set registration has been actively studied in computer vision and graphics. Optimization algorithms are at the core of solving registration problems. Traditional optimization approaches are mainly based on the gradient of objective functions. The derivation of objective functions makes it challenging to find optimal solutions for complex optimization models, especially for those applications where accuracy is critical. Learning-based optimization is a novel approach to address this problem, which learns the gradient direction from datasets. However, many learning-based optimization algorithms learn gradient directions via a single feature extracted from the dataset, which will cause the updating direction to be vulnerable to perturbations around the data, thus falling into a bad stationary point. This paper proposes the General Discriminative Optimization (GDO) method that updates a gradient path automatically through the trade-off among contributions of different features on updating gradients. We illustrate the benefits of GDO with tasks of 3D point set registrations and show that GDO outperforms the state-of-the-art registration methods in terms of accuracy and robustness to perturbations. [Display omitted] • Cast the point sets registration as a learning-based optimization problem. • Utilize features of point sets to update gradients without the Hessian matrix. • Collaborate the different features to reduce the influence of perturbations. • Outperform the other advanced registration methods on accuracy and robustness. [ABSTRACT FROM AUTHOR]

Virtual reality allows for highly immersive simulated experiences and interaction with virtual objects. However, virtual objects do not have real masses. Providing the sense of mass for virtual objects using un-grounded haptic interfaces has proven to be a complicated task in virtual reality. This paper proposes using a physically-based virtual hand with improved co-location and a complementary vibrotactile effect on the index fingertip to give the sensation of mass to objects in virtual reality. The vibrotactile feedback is proportional to the balanced forces acting on the virtual object and is modulated based on the object's velocity. For evaluating this method, we set an experiment in a virtual environment where participants wear a VR headset and attempt to pick up and move different virtual objects using a virtual physically-based hand while a voice-coil actuator attached to their index fingertip provides the vibrotactile feedback. Our experiments indicate that the virtual hand and our vibration effect give the ability to discriminate and perceive the mass of virtual objects. [Display omitted] [ABSTRACT FROM AUTHOR]

Neural Style Transfer (NST) aims to render a content image with the style of another image in the feature space of a Convolution Neural Network (CNN). A fundamental concept of NST is to define the features extracted from a CNN as a distribution so that the style similarity can be computed by measuring the distance between distributions. Conceptually, Wasserstein Distance (WD) is ideal for measuring the distance between distributions as it theoretically guarantees the similarity of style distributions with the WD between them equaling 0. However, due to the high computation cost of WD, previous WD-based methods either oversimplify the style distribution or only use a lower bound of WD, therefore, losing the theoretical guarantee of WD. In this paper, we propose a new style loss based on Sliced Wasserstein Distance (SWD), which has a theoretical approximation guarantee. Besides, an adaptive sampling algorithm is also proposed to further improve the style transfer results. Experiment results show that the proposed method improves the similarity of style distributions, and such improvements result in visually better style transfer results. [Display omitted] • A new style loss with a theoretical guarantee on the similarity of style distributions. • An adaptive sampling algorithm, which allows an unbalanced optimal transport. • A new quantitative evaluation method for the stylized quality. [ABSTRACT FROM AUTHOR]

Decomposing a complex object into simple components is a fundamental problem in geometry processing. Existing methods for decomposing point clouds rely on local or global features of an object, which leads to over-segmentation or unnatural component boundaries. In this paper, we propose a novel method for decomposing the point cloud by using internal and external critical points. First, we propose a novel shrinking strategy to build the global skeleton topology, from which we can extract internal critical points for locating components. External critical points are selected from the ridge and valley points for component segmentation. Then, we apply the constraint of internal and external critical points to decompose an object into semantic components by skeleton-based piecewise labeling. Experimental results demonstrate that our method is effective in decomposing 3D point clouds and is robust to limited noise and incomplete data. [Display omitted] • A shrinking-based skeleton extraction method is proposed to reserve sharp features. • A point cloud decomposition based on global and local features is presented. • Our framework has good efficiency and robustness in point cloud decomposition. [ABSTRACT FROM AUTHOR]

Skeleton creation is an important phase in the character animation pipeline. However, handcrafting skeleton takes extensive labor time and domain knowledge. Automatic skeletonization provides a solution. However, most of the current approaches are far from real-time and lack the flexibility to control the skeleton complexity. In this paper, we present an efficient skeletonization method, which can be seamlessly integrated into the sketch-based modeling process in real-time. The method contains three steps: (i) local sub-skeleton extraction; (i i) sub-skeleton connection; and (i i i) global skeleton refinement. Firstly, the local skeleton is extracted from processed polygon stroke and forms a subpart along with the sub-mesh. Then, local sub-skeletons are connected according to the intersecting relationships and modeling sequence of subparts. Lastly, a global refinement method is proposed to gives users coarse-to-fine control on the connected skeleton. We demonstrate the effectiveness of our method on a variety of examples created from both novices and professionals. [Display omitted] • A real-time skeletonization algorithm for immediately constructing an animatable skeleton from user sketch. • A flexible solution for skeleton complexity control. • An easy-to-use system for creating animatable models featured by sketch-based shape modeling and automatic rigging. [ABSTRACT FROM AUTHOR]

Implicit curve and surface reconstruction has wide applications in computer graphics, which attracts great attention of many researchers. In this paper, we propose the relaxed implicit randomized algebraic reconstruction technique (RIR-ART) for curve and surface reconstruction. The RIR-ART method generates a sequence of curves and surfaces by adjusting the control coefficients with the relaxation parameters iteratively. It is proved that the sequence of curves and surfaces generated by the RIR-ART method with the suitable relaxation parameters converge to the least-norm results in terms of the mean square error. The RIR-ART method can eliminate the extra zero-level sets effectively during its iterative process, and converge faster than the implicit progressive-iterative approximation method (Hamza et al. 2020). Numerical examples present the efficiency and effectiveness of the proposed method. • The RIR-ART method converges in terms of the mean square error. • The extra zero-level sets are eliminated by the RIR-ART method effectively. • The RIR-ART method converges faster than the I-PIA method for the reconstruction. [ABSTRACT FROM AUTHOR]

The electrovibration tactile display could render the tactile feeling of different textured surfaces by generating the frictional force through voltage modulation. When a user is sliding his/her finger on the display surface, he/she can feel the frictional texture. However, it is not trivial to prepare and fine-tune the appropriate frictional signals for haptic design and texture simulation. In this paper, we present a deep-learning-based framework to generate the frictional signals from the textured images of fabric materials. The generated frictional signal can be used for the tactile rendering on the electrovibration tactile display. Leveraging GANs (Generative Adversarial Networks), our system could generate the displacement-based data of frictional coefficients for the tactile display to simulate the tactile feedback of different fabric materials. Our experimental results show that the proposed generative model could generate the frictional-coefficient signals visually and statistically close to the ground-truth signals. The following user studies on fabric-texture simulation show that users could not discriminate the generated and the ground-truth frictional signals being rendered on the electrovibration tactile display, suggesting the effectiveness of our deep-frictional-signal-generation model. [Display omitted] • The deep-learning-based image-to-friction generation framework for tactile simulation of fabric material. • The augmented visual-to-frictional database based on HapTex for image-to-friction generation. • The technical experiment of frictional-coefficient signal generation evidencing the performance of the proposed generative model. • The user-perception experiment validating the effectiveness of generated signals for tactile simulation of fabrics on the electrovibration tactile display. [ABSTRACT FROM AUTHOR]

This paper presents a novel pipeline that improves the efficiency of narrowband surface reconstruction for particle-based fluids. Different from previous narrow band methods, we only construct the scalar field in the outer surface region. A method with robust criteria for detecting the outer surface vertices of the scalar field is then proposed to eliminate the redundant inner vertices, leading to a significant reduction in the computational complexity and memory consumption. The presented pipeline is fully parallel with a parallel compact memory scheme to avoid race conditions. Several experiments are conducted to compare our present method with the state-of-the-art approach, and the results indicate that our method saves nearly half of the computing time and ten percent of the memory consumption. [Display omitted] [ABSTRACT FROM AUTHOR]

Pose tracking and geometry reconstruction are greatly significant for the high-level perceptual understanding and close-proximity operations of dynamic and geometrically unknown non-cooperative targets in space. However, the performance degrades severely under the commonly outlier-contaminated and corrupted measurements acquired from Time-of-Flight (ToF) cameras. In this paper, we are motivated to investigate the framework of both robust pose tracking and reliable geometry reconstruction. For the pose tracking, we propose an improved robust Iterative Closest Point (ICP) method based on adaptive Iteratively Reweighted Least Squares (IRLS), which can gradually jump out of the local minima in a naturally coarse-to-fine fashion. Besides, we put forward a hybrid feature-free loop closure detection approach to efficiently eliminate the accumulated error, meanwhile avoiding the ambiguity caused by the symmetrical structure. Regrading the geometry reconstruction, we present an explicit and general geometry uncertainty description, incorporated into a mixture-based probabilistic fusion method, to cope with the reconstruction defects. The experimental results show that our pose tracking and geometry reconstruction methods can achieve better performance in terms of robustness and accuracy. [Display omitted] • An adaptive IRLS-ICP method is proposed to boost the robustness of pose estimation. • A hybrid loop closure approach is proposed to avoid the geometry ambiguity. • A probabilistic fusion method integrated with a novel uncertainty model is presented. [ABSTRACT FROM AUTHOR]

Surfaces evolving through time are hard to analyze without prior knowledge of the represented deforming shape. Topological noise introduced by inaccuracies during surface capture and reconstruction renders the estimation of time-consistent correspondence an ill-posed problem. It has previously been shown that this problem may be mitigated by tracking the volume within the surface rather than the surface itself. Assuming that the captured phenomenon does not involve sudden appearance or disappearance of volume, it is possible to establish a bijective correspondence of volume elements, which in turn may be used for various purposes, such as extracting a canonical shape and guiding the computation of surface correspondence. In this paper, we discuss an improved volume tracking methodology that provides better results by modifying the energy formulation and by incorporating a notion of volume element affinity, combined with an improved optimization strategy. We also discuss two metrics that, when combined, quantify the quality of the tracking results. [Display omitted] • We present an improved pipeline for tracking volume within time-varying meshes. • ARAP based smoothness energy achieves results more robust to global rigid motion. • Weighting based on similarity of motion reduces tracking inconsistencies. • We outperform previous work on majority of tested datasets based on proposed metrics. [ABSTRACT FROM AUTHOR]

Surface reconstruction is a fundamental problem in 3D graphics. In this paper, we propose a learning-based approach for implicit surface reconstruction from raw point clouds without normals. Our method is inspired by Gauss Lemma in potential energy theory, which gives an explicit integral formula for the indicator functions. We design a novel deep neural network to perform surface integral and learn the modified indicator functions from un-oriented and noisy point clouds. We concatenate features with different scales for accurate point-wise contributions to the integral. Moreover, we propose a novel Surface Element Feature Extractor to learn local shape properties. Experiments show that our method generates smooth surfaces with high normal consistency from point clouds with different noise scales and achieves state-of-the-art reconstruction performance compared with current data-driven and non-data-driven approaches. [Display omitted] • A learning-based approach for surface reconstruction from raw point clouds. • A novel Surface Element Feature Extractor to learn local shape properties. • Aggregating the point-wise contributions of Gauss Lemma in the network. • Generating smooth surfaces with high normal consistency. [ABSTRACT FROM AUTHOR]

Our goal is to automatically generate CAD 3D models from 2D sketches as part of a design chain where models should be procedural, containing features arranged in a model tree and linked to suitable datums. Current procedural models capture much about the design intent and are easy to edit, but must be created from scratch during the detailed design state—given conceptual sketches as used by designers in the early part of the design process, current sketch-based modeling approaches only output explicit models. Thus, we describe an approach to extract high-level information directly from 2D engineering wireframe sketches and use it to complete a CSG feature tree, which serves as a model tree for a procedural 3D CAD model. Our method extracts procedural model information directly from 2D sketches in the form of a set of features, plus a set of datums and relationships between these features. We detect and analyze features of 2D sketches in isolation, and define the CSG feature tree by the parent–child relationships between features, and combine this information to obtain a complete and consistent CSG feature tree that can be transferred to a 3D modeler, which reconstructs the model. This paper focuses on how to extract the feature datums and the extrusion operation from an input 2D sketch. [Display omitted] • We create a complete CAD model tree from a vectorized sketch of a polyhedral shape • The method extracts the cross-section and auxiliary reference system of each feature • The approach works when line drawing is a connected graph or disconnected subgraphs • Output serves as proof of concept to demonstrate that it can build a procedural model [ABSTRACT FROM AUTHOR]

We analyze the performance of a reduced-order simulation of geometric meta-materials based on zigzag patterns using a simplified representation. As geometric meta-materials we denote planar cellular structures which can be fabricated in 2d and bent elastically such that they approximate doubly-curved 2-manifold surfaces in 3d space. They obtain their elasticity attributes mainly from the geometry of their cellular elements and their connections. In this paper we focus on cells build from so-called zigzag springs. The physical properties of the base material (i.e., the physical substance) influence the behavior as well, but we essentially factor them out by keeping them constant. The simulation of such complex geometric structures comes with a high computational cost, thus we propose an approach to reduce it by abstracting the zigzag cells by a simpler model and by learning the properties of their elastic deformation behavior. In particular, we analyze the influence of the sampling of the full parameter space and the expressiveness of the reduced model compared to the full model. Based on these observations, we draw conclusions on how to simulate such complex meso-structures with simpler models. [Display omitted] • We present a reduced-order simulation of complex geometric meta-materials. • The deformation behavior is encoded into the parameters of a simplified model. • We analyze the space of samples from which this information can be learned. • The simplified model can then be used for the task of shape optimization. • The simplification yields better performance compared to the original structure. [ABSTRACT FROM AUTHOR]

In recent years, Topology Optimization (TO) gained interest in the scientific community. It assists in finding the best arrangement of material in a design volume. The classical approach named "Solid Isotropic Material with Penalization" (SIMP) associates a fictitious density to each finite element in the domain. While SIMP is described as an implicit approach which can lead to problems with dimensionality of variables, explicit methods adopt a geometric projection of simple elements (eg.: bars) to reduce the number of design variables. This simplifies the geometric interpretation of the optimal architecture. The major explicit methods were recently unified into a general framework, Generalized Geometric Projection (GGP). Currently it is quite challenging to take into account manufacturing constraints in the topology optimization design phase. Therefore this paper presents an application of the GGP Method to the design of products made by Additive Layer Manufacturing (ALM). Every printed layer constitutes a geometric element, involving design variables relative to position and width. Specific constraints of ALM, including bridge length and overhang angle, can be easily monitored by exploiting the geometric features of the combined elements. Examples in two dimensions will be reported, analyzing two academic benchmark problems. A comparison to other proven techniques is also detailed. An mean difference of 7.7% is observed for solutions with only overhang angle constraint, while a mean difference of 11% is observed for solutions with overhang angle and bridge length constraint. The presented work integrates design and manufacturing, directly identifying the path of the printed layers. [Display omitted] • Single framework that includes most of the 3D printing geometric constraints. • Includes print orientation, overhang angle and bridge length. • Unique geometric primitive to mimic additive layer manufacturing processes. • Combination of topology optimization and print layer optimization (in 2D only). • Also provides optimized print direction and print height per component. • Easy parametrization of hyperparameters to obtain a range of solutions. [ABSTRACT FROM AUTHOR]

The use of technology in cultural heritage sites for end-users, like virtual audio-visual reconstructions, has become a common solution to improve the communication and perception between these spaces and their visitors. Since humans perceive the world with different senses and in real-time, to evoke more than one or two senses at a time can bring benefits for the user perception. Accordingly, some very different implementations unleashing multisensory experiences have been made. Aiming to understand how different stimuli are being evoked in multisensory experiences to enhance cultural heritage experiences, and how these implementations are being evaluated, this paper presents a systematic review of technological multisensory applications in cultural heritage. Thus, the collected and analyzed data, focused on technologies used, purpose of the experience, stimuli explored, evaluation process, main findings obtained, and limitations found, will provide valuable information for further implementations. The overall results unleash the wide diversity available for multisensory implementations, from technological solutions to available content for users. Covering the pros and cons of such diversity, this study sustains the use of multisensory applications in cultural heritage as a powerful tool to enrich users' visits. [Display omitted] • Systematic review on Multisensory VR and AR implementations in Cultural Heritage. • Analysis of evoked stimuli and of what technologies were used to evoke each stimulus. • Detailed analysis of the purpose of each implementation and of conducted evaluations. • Mention to studies' main findings and main limitations found. • Discussion related to analyzed multisensory studies. [ABSTRACT FROM AUTHOR]

Creating realistic characters that can react to the users' or another character's movement can benefit computer graphics, games and virtual reality hugely. However, synthesizing such reactive motions in human–human interactions is a challenging task due to the many different ways two humans can interact. While there are a number of successful researches in adapting the generative adversarial network (GAN) in synthesizing single human actions, there are very few on modeling human–human interactions. In this paper, we propose a semi-supervised GAN system that synthesizes the reactive motion of a character given the active motion from another character. Our key insights are two-fold. First, to effectively encode the complicated spatial–temporal information of a human motion, we empower the generator with a part-based long short-term memory (LSTM) module, such that the temporal movement of different limbs can be effectively modeled. We further include an attention module such that the temporal significance of the interaction can be learned, which enhances the temporal alignment of the active–reactive motion pair. Second, as the reactive motion of different types of interactions can be significantly different, we introduce a discriminator that not only tells if the generated movement is realistic or not, but also tells the class label of the interaction. This allows the use of such labels in supervising the training of the generator. We experiment with the SBU, the HHOI and the 2C datasets. The high quality of the synthetic motion demonstrates the effective design of our generator, and the discriminability of the synthesis also demonstrates the strength of our discriminator. [Display omitted] • A generative adversarial network to synthesize reactive motion in an interaction. • A generator implemented with an attentive part-aware recurrent network. • A dual discriminator design with a binary classifier and a multi-class classifier to improve the motion quality. [ABSTRACT FROM AUTHOR]

Learning new representations of 3D point clouds is an active research area in 3D vision, as the order-invariant point cloud structure still presents challenges for the design of neural network architectures. Recent work explored learning global, local, or multi-scale features for point clouds. However, none of the earlier methods focused on capturing contextual shape information by analyzing local orientation distributions of points. In this paper, we use point orientation distributions around a point in order to obtain an expressive local neighborhood representation for point clouds. We achieve this by dividing the spherical neighborhood of a given point into predefined cone volumes, and statistics inside each volume are used as point features. In this way, a local patch can be represented not only by the selected point's nearest neighbors, but also by considering a point density distribution defined along multiple orientations around the point. We are then able to construct an orientation distribution function (ODF) neural network that makes use of an ODFBlock which relies on MLP (multi-layer perceptron) layers. The new ODFNet model achieves state-of-the-art accuracy for object classification on ModelNet40 and ScanObjectNN datasets, and segmentation on ShapeNet and S3DIS datasets. [Display omitted] • ODFs which incorporate the directional information inside a neighborhood is defined. • ODFNet, for classification and segmentation of point clouds is presented. • ODFs provide fully rotation invariance or invariance in x–y plane. • ODFNet obtained SoTA accuracy on ModelNet40, Shapenet, S3DIS and ScanObjectNN. [ABSTRACT FROM AUTHOR]

This paper presents an Augmented Reality (AR)-based network cabling tutoring system that trains users how to interconnect cables within a Virtual Local Area Network (VLAN) on a physical switching rack. AR arrows combined with text-based instruction and a checklist provided assistance during practical learning. When learners made a mistake, an Intelligent Tutoring System (ITS) identified the source of the mistake and provided real-time feedback using text, AR and text-to-speech mechanisms. The design was motivated by the human-cognitive architecture and its five evolutionary principles (proposed by Sweller and Sweller (2006)). Users performed four consecutive network cabling training tasks with assistance from our ITS. We found that users made fewer errors when the AR cues, text-based instruction and checklist solutions were replaced with summarised information and then removed completely in the final task compared to those who used an identical system with fixed instruction. Cognitive Load Theory (CLT) explains our results by suggesting that the instructional mechanisms become redundant as knowledge increases. Implications of the study are discussed as well as how AR can help facilitate knowledge transfer in the network cabling domain. [Display omitted] • Several cognitive load considerations for AR-based network training are proposed. • A novel AR training system combining fading with an ITS is proposed. • Our AR training system reduces mistakes during network security training. • In network security training, AR should be faded out as expertise increases. [ABSTRACT FROM AUTHOR]

This paper presents the results of SHREC'21 track: Quantifying Shape Complexity. Our goal is to investigate how good the submitted shape complexity measures are (i.e. with respect to ground truth) and investigate the relationships between these complexity measures (i.e. with respect to correlations). The dataset consists of three collections: 1800 perturbed cube and sphere models classified into 4 categories, 50 shapes inspired from the fields of architecture and design classified into 2 categories, and the data from the Princeton Segmentation Benchmark, which consists of 19 natural object categories. We evaluate the performances of the methods by computing Kendall rank correlation coefficients both between the orders produced by each complexity measure and the ground truth and between the pair of orders produced by each pair of complexity measures. Our work, being a quantitative and reproducible analysis with justified ground truths, presents an improved means and methodology for the evaluation of shape complexity. [Display omitted] • Two novel datasets with associated ground truth are introduced. • Segmentation is used as a proxy task for measuring complexity. • The performance of 2D/3D complexity measures are systematically compared/evaluated. [ABSTRACT FROM AUTHOR]

Deep learning (DL) models have achieved impressive performance in various domains such as medicine, finance, and autonomous vehicle systems with advances in computing power and technologies. However, due to the black-box structure of DL models, the decisions of these learning models often need to be explained to end-users. Explainable Artificial Intelligence (XAI) provides explanations of black-box models to reveal the behavior and underlying decision-making mechanisms of the models through tools, techniques, and algorithms. Visualization techniques help to present model and prediction explanations in a more understandable, explainable, and interpretable way. This survey paper aims to review current trends and challenges of visual analytics in interpreting DL models by adopting XAI methods and present future research directions in this area. We reviewed literature based on two different aspects, model usage and visual approaches. We addressed several research questions based on our findings and then discussed missing points, research gaps, and potential future research directions. This survey provides guidelines to develop a better interpretation of neural networks through XAI methods in the field of visual analytics. [Display omitted] • A comprehensive survey for visual analytics, particularly those that adopted explainable artificial intelligence (XAI) methods, in interpreting Neural Networks is conducted. • We reviewed the literature based on model usage and visual approaches. • We concluded some visual approaches commonly used to support the illustration of XAI methods for various types of data and machine learning models; however, a generic approach is needed for the field. • We listed several future work including data manipulations, scalability, and bias in data representation, and generalizable real-time visualizations integrating XAI. [ABSTRACT FROM AUTHOR]

Instance segmentation is an essential part of image semantic understanding. In this paper we propose a novel cascade framework for instance segmentation. Unlike some existing methods that only output contour discrete coordinates, our approach obtains a vectorized representation of contours using periodic B-splines. In order to make better use of geometry and appearance information, we consider the global and local features of objects and introduce two types of graph structures, the star graph and circular graph, for feature extraction. Thereby, we develop a neural network, termed the mix network, to better exploit extracted features. Specifically, we first regress the spline control points to an object boundary via the mix network, then perform spline sampling to obtain the initial predictions of contours, and finally deform the predicted contours to the real contours of the objects. In addition, we add a regularization to further constrain the fairness of contour splines. Experiments show that our approach achieves 34.6% in mask mAP, Mean Average Precision, with a ResNet-101-FPN-DCN backbone on the challenging COCO benchmark, which significantly improves the performance of contour-based methods. [Display omitted] • We design graph structures and mix networks to use global and local features. • Fairness regularization gives constrains on object contours. • Our approach achieves superior results among contour-based methods. [ABSTRACT FROM AUTHOR]

Acceleration techniques for Rendering in general, and Ray-Casting in particular, have been the subject of much research in Computer Graphics. Most efforts have been focused on new data structures for efficient ray/scene traversal and intersection. In this paper, we propose an acceleration technique that approximates rendering and that is built around a new feature-based clustering approach. The technique starts preprocessing the scene by grouping elements according to their features using a set of channels based on an information theory-based approach. Then, at run-time, a rendering strategy uses that clustering information to reconstruct the final image, by deciding which areas could take advantage of the coherence in the features and thus, could be interpolated; and which areas require more involved calculations. This process starts with a low-resolution render that is iteratively refined up to the desired resolution by reusing previously computed pixels. Our experimental results show a significant speedup of an order of magnitude, depending on the complexity of the per-pixel calculations, the screen size of the objects, and the number of clusters. Rendering quality and speed directly depend on the number of clusters and the number of steps performed during the reconstruction procedure, and both can easily be set by the user. Our findings show that feature-based clustering can significantly impact rendering speed if samples are chosen to enable interpolation of smooth regions. Our technique, thus, accelerates a range of popular and costly techniques, ranging from texture mapping up to complex ambient occlusion, soft and hard shadow calculations, and it can even be used in conjunction with more traditional acceleration methods. [Display omitted] • Mesh-clustering based Information Theory tools to exploit feature similarity. • User-definable parameters, decoupling them from the actual rendering process. • Multi-pass rendering based on reuse and interpolation of previous results. • Controlling mechanism to guarantee Ray-casting as a lower bound to rendering speed. • Our technique can accommodate both static and animated scenes as well. [ABSTRACT FROM AUTHOR]

3D tooth segmentation is a challenging fundamental problem in computer-aided design (CAD) systems for orthodontic treatment planning. Existing methods of tooth segmentation cannot meet the requirements for both efficiency and accuracy. This paper presents a coarse-to-fine 3D tooth segmentation algorithm, called C2F-3DToothSeg, to improve tooth segmentation. The algorithm consists of two steps: extraction of the coarse gum line and optimization of this line. A concavity-sensitive harmonic field is searched in a local region of entire dental meshes to produce a series of isolines. The best isoline is selected as the coarse gum line, which is then optimized by a heuristic feature line extraction scheme to yield an accurate gum line. Extensive visual and numerical experiments demonstrate that our C2F-3DToothSeg has better overall performance in accuracy and efficiency compared to other existing methods directly applicable in orthodontic CAD systems. A point on the tooth surface is loosely found to locate the center zone of each individual tooth via a simple intuitive single click of the operation. The experiments also show that our method is robust to different complex dental models; for example, it copes with bulge bubbles, decay and cavity issues, and dental accessories. [Display omitted] • Coarse-to-fine 3D tooth segmentation. • Combining the concavity-aware harmonic field and heuristic feature line extraction. • The concavity-aware harmonic field could get a closed coarse gum line to ensure our segmentation algorithm robust. • The heuristic feature line extraction algorithm can optimize the closed coarse gum line for precise segmentation. • The algorithm is robust to handle very complex and abnormal cases. [ABSTRACT FROM AUTHOR]

The representation of consistent mixed reality (XR) environments requires adequate real and virtual illumination composition in real-time. Estimating the lighting of a real scenario is still a challenge. Due to the ill-posed nature of the problem, classical inverse-rendering techniques tackle the problem for simple lighting setups. However, those assumptions do not satisfy the current state-of-art in computer graphics and XR applications. While many recent works solve the problem using machine learning techniques to estimate the environment light and scene's materials, most of them are limited to geometry or previous knowledge. This paper presents a CNN-based model to estimate complex lighting for mixed reality environments with no previous information about the scene. We model the environment illumination using a set of spherical harmonics (SH) environment lighting, capable of efficiently represent area lighting. We propose a new CNN architecture that inputs an RGB image and recognizes, in real-time, the environment lighting. Unlike previous CNN-based lighting estimation methods, we propose using a highly optimized deep neural network architecture, with a reduced number of parameters, that can learn high complex lighting scenarios from real-world high-dynamic-range (HDR) environment images. We show in the experiments that the CNN architecture can predict the environment lighting with an average mean squared error (MSE) of 7.85 × 10−4 when comparing SH lighting coefficients. We validate our model in a variety of mixed reality scenarios. Furthermore, we present qualitative results comparing relights of real-world scenes. [Display omitted] • Automatic end-to-end method to estimate the environment lighting in XR applications. • A CNN architecture that learns a latent-space of the environment lighting. • A methodology to generate egocentric mixed-reality-views from HDR panoramas. • Real-time lighting estimation that does not make assumptions about the XR scene. [ABSTRACT FROM AUTHOR]

Streaming data applications are becoming more common due to the ability of different information sources to continuously capture or produce data, such as sensors and social media. Although there are recent advances, most visualization approaches, particularly Dimensionality Reduction (DR) techniques, cannot be directly applied in such scenarios due to the transient nature of streaming data. A few DR methods currently address this limitation using online or incremental strategies, continuously updating the visualization as data is received. Despite their relative success, most impose the need to store and access the data multiple times to produce a complete projection, not being appropriate for streaming where data continuously grow. Others do not impose such requirements but cannot update the position of the data already projected, potentially resulting in visual artifacts. This paper presents Xtreaming , a novel incremental DR technique that continuously updates the visual representation to reflect new emerging structures or patterns without visiting the high-dimensional data more than once. Our tests show that in streaming scenarios where data is not fully stored in-memory, Xtreaming is competitive in terms of quality compared to other streaming and incremental techniques while being orders of magnitude faster. [Display omitted] • A novel incremental Dimensionality Reduction technique for streaming scenarios. • A re-projection strategy to update visual representation without revisiting input data. • A visualization that presents better trade-off between quality and runtime compared to other approaches. [ABSTRACT FROM AUTHOR]

• MolVA is the Workshop on Molecular Graphics and Visual Analysis of Molecular Data. • MolVA accepts short and full papers addressing challenges in the visualization of molecular data. • This special section contains a full paper presented at MolVA 2023. [Display omitted] [ABSTRACT FROM AUTHOR]

Cluster interpretation after dimensionality reduction (DR) is a ubiquitous part of exploring multidimensional datasets. DR results are frequently represented by scatterplots, where spatial proximity encodes similarity among data samples. In the literature, techniques support the understanding of scatterplots' organization by visualizing the importance of the features for cluster definition with layout enrichment strategies. However, current approaches usually focus on global information, hampering the analysis whenever the focus is to understand the differences among clusters. Thus, this paper introduces a methodology to visually explore DR results and interpret clusters' formation based on contrastive analysis. We also introduce a bipartite graph to visually interpret and explore the relationship between the statistical variables employed to understand how the data features influence cluster formation. Our approach is demonstrated through case studies, in which we explore two document collections related to news articles and tweets about COVID-19 symptoms. Finally, we evaluate our approach through quantitative results to demonstrate its robustness to support multidimensional analysis. • A new methodology to find contrastive features in datasets. • Hypothesis testing employed to find descriptive terms in document collections. • Novel visualization techniques to interpret contrastive information of multidimensional datasets. [ABSTRACT FROM AUTHOR]

Real-time rendering of pixel-accurate hard shadows in 3D scenes is still a challenging problem for shadow generation techniques. In this paper, we present the concept of non-overlapping geometric shadow map (NOGSM), which enables the construction of high-quality hard shadows from static lights for any point in the scene. The NOGSM is created in a preprocessing phase, in which the scene's geometry is looked at from the light's point of view. All hidden surfaces are removed through total elimination of hidden polygons plus clipping of the partially hidden polygons. The resulting list of non-overlapping projected polygons is then stored in a multi-grid data structure. Our solution generates shadows with far better quality than standard shadow mapping techniques, while maintaining real time performance, and using a smaller memory footprint than compression techniques. [Display omitted] • Geometric hidden surface removal. • Geometric shadow mapping with non-overlapping polygons. • Real time alias-free hard shadows. • Highly detailed shadows of complex scenes in real time. • Sharp shadows on large scenes avoiding depth bias and resolution problems. • Non-overlapping geometry, instead of depth texture. [ABSTRACT FROM AUTHOR]

3D CAD models play an important role in large-scale engineering projects. The increasing demand for highly-detailed datasets presents a challenge for efficient storage, transmission, and rendering. To reduce dataset size, 3D shape matching techniques have been proposed to find repeated triangle meshes, but are strongly dependent on surface triangulation. Meanwhile, existing shape registration techniques are not well suited for the 3D CAD domain. In this paper, we present a deep learning-based framework that uses point clouds to identify repeated instances of triangle meshes (a single 3D CAD model component) overcoming the limitations of previous work and guaranteeing an upper bound on any geometric errors. The framework combines PointNet++ for shape classification with a registration procedure based on Principal Component Analysis and the Adam optimizer. The resulting affine transformation can be used to efficiently instantiate repeated CAD geometries. Using the proposed framework, we were able to reduce a real-world 3D CAD model to 2.61% of its original size, while preserving its geometric accuracy and improving rendering performance. [Display omitted] • Guarantee of an upper bound on any geometric errors. • Independent of vertices ordering and mesh topology. • Generalization for any kind of geometry. • Well suited for the 3D CAD domain. • Accurate and consistent 3D shape registration. [ABSTRACT FROM AUTHOR]

• A fast and unified microstructure analyzer that computes mechanical properties based on asymptotic homogenization. • A mechanical property profile (MPP) for the microstructure that depicts five commonly used mechanical properties in a radar chart, in addition to the spatial distribution of effective Young's modulus surface and the worst-case von Mises stress distribution. • A joint reinforcement algorithm for enhancing the strength of truss-based microstructures. [Display omitted] Microstructure is the microscopic scale of a material, which can strongly influence physical properties, such as elasticity, strength, hardness, etc. Recently, many research efforts have been made to design the topology of microstructures carefully and govern their material properties according to the specific application of these materials in industrial practice. However, most works target specific properties that are tailored to particular applications. An integral image for one microstructure concerning multiple mechanical properties is in high demand. In this paper, we present a microstructure analyzer based on asymptotic homogenization. A mechanical property profile (MPP) is proposed to depict commonly used mechanical properties, including Young's modulus, shear modulus, Poisson's ratio, compressive strength, shear strength, the spatial distribution of effective Young's modulus surface, and the worst-case von Mises stress distribution. Specifically, we utilize a radar chart to visualize the five scalar properties to help with an intuitive understanding of the microstructure. Moreover, we apply joints reinforcement on truss-based microstructures to enhance their strength. The results show that our analyzer and MPPs have the potential to help users understand and select the desired microstructure from alternatives and help them improve specific microstructure properties in microstructural design. [ABSTRACT FROM AUTHOR]

• An efficient radial-basis-function guided segmentation method is proposed to generate smooth and contiguous model partition based on surface attributes, which serves as the initial segmentation result. • A differential evolution based smoothing algorithm is introduced to optimize the cut interfaces of segmented parts constrained by the prescribed minimal printing thickness. • A refinement method for part extractability is designed to detect the infeasible segmented parts and compute division planes to further sub-divide them to ensure the extractability of all split parts. • The proposed technique is applied on a variety of models, and the experimental results shows its advantages over prior methods. Additionally, printed physical objects are provided as a proof of its reliability on current mainstream printers. [Display omitted] Volume segmentation based on surface attributes is an essential problem in multi-material fabrication and model packing. In practice, current mainstream fabrication techniques have difficulties in yielding models with diverse surface attributes in one pass owing to their craft limitations, making model segmentation a sensible choice for model realization. Partitioning 3D objects into single-attribute volumetric parts prevents fabricating models with different material in a single printing procedure, whereas the arisen challenge is to determine a reliable segmentation solution that is able to handle complicated models in various use scenarios. To achieve this goal, we propose a novel volume partition algorithm generating feasible volumetric parts, each of which is affiliated with one single surface attribute. Our technique enables model segmentation with least conflict and constrained wall thickness so that each volumetric segment can be realized independently by 3D-printing. Generally, it starts with computing a partition proposal guided by radial-based-function iso-surface, then optimizes segmentation interface with a prescribed minimal printing thickness to produce high-quality surface for every volumetric part, and finally splits unextractable volumetric parts into smaller sub-volumes to ensure assemblability of the whole model. As previous methods do not work well in optimizing segment interface for printing, we propose a differential evolution based smoothing algorithm to generate smooth and continuous interface, declining the risk of collision between adjacent volumetric parts. Extensive experimental results are provided in this paper to demonstrate the effectiveness and quality of our proposed technique, showing its advantages on model manufacture over prior methods. [ABSTRACT FROM AUTHOR]

• A multigranular model of co-adaptation with a focus on making co-adaptation testable. • Learning phases: define testable goals for user and system to structure design. • Adaptation processes: actors adapt knowledge representation models. • Interaction dynamics: applications enable adaptation in learning-teaching dynamics. • A discussion of actor expectations towards adaptation. • An overview of research opportunities in co-adaptive analytics. [Display omitted] Mixed-initiative visual data analysis processes are characterized by the co-adaptation of users and systems over time. As the analysis progresses, both actors – users and systems – gather information, update their analysis behavior, and work on different tasks towards their respective goals. In this paper, we contribute a multigranular model of co-adaptive visual analysis that is centered around incremental learning goals derived from a hierarchical taxonomy of learning goals from pedagogy. Our model captures how both actors adapt their data-, task-, and user/system-models over time. We characterize interaction patterns in terms of the dynamics of learning and teaching that drive adaptation. To demonstrate our model's applicability, we outline aspects of co-adaptation in related models of visual analytics and highlight co-adaptation in existing applications. We further postulate a set of expectations towards adaptation in mixed-initiative processes and identify open research questions and opportunities for future work in co-adaptation. [ABSTRACT FROM AUTHOR]

• We present a benchmark of cultural heritage objects for the evaluation of 3D shape retrieval methods. • We propose two semantically different challenges: retrieval by shape and retrieval by culture. • Ten teams around the world presented techniques to address the challenges, mainly using learning approaches. • Our experiments and results show that learning methods on image-based multi-view representation are suitable for tackling 3D retrieval in a cultural heritage domain. [Display omitted] This paper presents the methods and results of the SHREC'21 track on a dataset of cultural heritage (CH) objects. We present a dataset of 938 scanned models that have varied geometry and artistic styles. For the competition, we propose two challenges: the retrieval-by-shape challenge and the retrieval-by-culture challenge. The former aims at evaluating the ability of retrieval methods to discriminate cultural heritage objects by overall shape. The latter focuses on assessing the effectiveness of retrieving objects from the same culture. Both challenges constitute a suitable scenario to evaluate modern shape retrieval methods in a CH domain. Ten groups participated in the challenges: thirty runs were submitted for the retrieval-by-shape task, and twenty-six runs were submitted for the retrieval-by-culture task. The results show a predominance of learning methods on image-based multi-view representations to characterize 3D objects. Nevertheless, the problem presented in our challenges is far from being solved. We also identify the potential paths for further improvements and give insights into the future directions of research. [ABSTRACT FROM AUTHOR]

• Extending the IMF definition to study accessibility of support structures in the near-net shape generated by an additive process. • Generalizing the IMF for a multi-axis machining setup with multiple fixturing devices/configurations. • Developing a build orientation optimization for hybrid AM-then-SM processes based on Pareto optimality of total support volume and secluded support volume. • Developing an automated support removal planning for a general multi-axis machining setup based on IMF and maximal removable volume. • Demonstrating the efficacy of the proposed strategies through studies in 2D and 3D. [Display omitted] Parts fabricated by additive manufacturing (AM) are often fabricated first as a near-net shape, a combination of intended nominal geometry and sacrificial support structures, which need to be removed in a subsequent post-processing stage using subtractive manufacturing (SM). In this paper, we present a framework for optimizing the build orientation with respect to removability of support structures. In particular, given a general multi-axis machining setup and sampled build orientations, we define a Pareto-optimality criterion based on the total support volume and the "secluded" support volume defined as the support volume that is not accessible by a given set of machining tools. Since total support volume mainly depends on the build orientation and the secluded volume is dictated by the machining setup, in many cases the two objectives are competing and their trade-off needs to be taken into account. The accessibility analysis relies on the inaccessibility measure field (IMF), which is a continuous field in the Euclidean space that quantifies the inaccessibility of each point given a collection of tools and fixturing devices. The value of IMF at each point indicates the minimum possible volumetric collision between objects in relative motion including the part, fixtures, and the tools, over all possible tool orientations and sharp points on the tool. We also propose an automated support removal planning algorithm based on IMF, where a sequence of actions are provided in terms of the fixturing devices, cutting tools, and tool orientation at each step. In our approach, each step is chosen based on the maximal removable volume to iteratively remove accessible supports. The effectiveness of the proposed approach is demonstrated through benchmark examples in 2D and realistic examples in 3D. [ABSTRACT FROM AUTHOR]

• A new benchmark of protein surfaces equipped with three physicochemical properties. • Protein surfaces are retrieved and classified with respect to different ground truths. • Analysis of the retrieval and classification performances of the methods that participated in SHREC 2021. • A thorough Investigation of how different physicochemical properties impact the performance of retrieval methods. [Display omitted] This paper presents the methods that have participated in the SHREC 2021 contest on retrieval and classification of protein surfaces on the basis of their geometry and physicochemical properties. The goal of the contest is to assess the capability of different computational approaches to identify different conformations of the same protein, or the presence of common sub-parts, starting from a set of molecular surfaces. We addressed two problems: defining the similarity solely based on the surface geometry or with the inclusion of physicochemical information, such as electrostatic potential, amino acid hydrophobicity, and the presence of hydrogen bond donors and acceptors. Retrieval and classification performances, with respect to the single protein or the existence of common sub-sequences, are analysed according to a number of information retrieval indicators. [ABSTRACT FROM AUTHOR]

• 3D Shape Retrieval Challenge 2021 at 3DOR'21 Track on Skeleton-based Hand Gesture Recognition in the Wild. • New gesture dataset with 180 gestures sequences and 18 gestures dictionary. • Contest with 4 groups presenting their gesture recognition methods. • Report of results and performances for all the methods. [Display omitted] Gesture recognition is a fundamental tool to enable novel interaction paradigms in a variety of application scenarios like Mixed Reality environments, touchless public kiosks, entertainment systems, and more. Recognition of hand gestures can be nowadays performed directly from the stream of hand skeletons estimated by software provided by low-cost trackers (Ultraleap) and MR headsets (Hololens, Oculus Quest) or by video processing software modules (e.g. Google Mediapipe). Despite the recent advancements in gesture and action recognition from skeletons, it is unclear how well the current state-of-the-art techniques can perform in a real-world scenario for the recognition of a wide set of heterogeneous gestures, as many benchmarks do not test online recognition and use limited dictionaries. This motivated the proposal of the SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild. For this contest, we created a novel dataset with heterogeneous gestures featuring different types and duration. These gestures have to be found inside sequences in an online recognition scenario. This paper presents the result of the contest, showing the performances of the techniques proposed by four research groups on the challenging task compared with a simple baseline method. [ABSTRACT FROM AUTHOR]

[Display omitted] Motion Blur is an important effect of photo-realistic rendering. Distribution ray tracing can simulate motion blur very well by integrating light, both over the spatial and the temporal domain. However, increasing the problem by the temporal dimension entails many challenges, particularly in cinematic multi-bounce path tracing of complex scenes, where heavy-weight geometry with complex lighting and even offscreen elements contribute to the final image. In this paper, we propose the Motion DAG (Directed Acyclic Graph), a novel data structure that filters an entire animation sequence of an object, both in the spatial and temporal domain. These Motion DAGs interleave a temporal interval binary tree for filtering time consecutive data and a sparse voxel octree (SVO), which simplifies spatially nearby data. Motion DAGs are generated in a pre-process and can be easily integrated in a conventional physically based path tracer. Our technique is designed to target motion blur of small objects, where coarse representations are sufficient. Specifically, in this scenario our results show that it is possible to significantly reduce both, memory consumption and render time. [ABSTRACT FROM AUTHOR]

• A simple and effective ridge valley point recognition method is present, which greatly reduces the number of potential feature points and improves the robustness of the recognition. • A freeform feature enhancement algorithm based on position and normal constraints is proposed. By different settings, several enhancement effects can be realized, such as preserving the original shape as much as possible, sharpen it or recovering a totally sharp edge. • A point cloud parameterization method based on feature line guidance is introduced to reduce the number of unknowns by 2/3 and eliminate the lateral sliding of points. [Display omitted] Ridge-valley features are important elements of a model. To recognize these features from point cloud, this paper introduces a new criterion named Extremal Point Distance (EPD) to greatly reduce the number of potential feature points and locate feature position more accurately. On this basis, a feature enhancement algorithm is proposed. The algorithm adjusts the coordinates of feature regions by minimizing a linear objective function consisting of expected position and normal, which can ensure the accurate sampling of feature position. We also present a parameterization method to eliminate the lateral sliding of feature points and reduce the number of unknowns in the objective function. Since the EPD criterion only depends on the changing trend, rather than the absolute value of the curvature, our algorithm can infer the expected position and normal with a large neighborhood radius, which makes it robust to noise. Experiments show that our algorithm can adjust the feature amplitude and sharpness freely, and achieve satisfactory results in feature recognition, feature enhancement and sharp feature restoration. [ABSTRACT FROM AUTHOR]