Search

Search and retrieval remains a major research topic in several domains, including computer graphics, computer vision, engineering design, etc. A search engine requires primarily an input search query and a database of items to search from. In engineering, which is the primary context of this paper, the database consists of 3D CAD models, such as washers, pistons, connecting rods, etc. A query from a user is typically in the form of a sketch, which attempts to capture the details of a 3D model. However, sketches have certain typical defects such as gaps, over-drawn portions (multi-strokes), etc. Since the retrieved results are only as good as the input query, sketches need cleaning-up and enhancement for better retrieval results. In this paper, a deep learning approach is proposed to improve or clean the query sketches. Initially, sketches from various categories are analysed in order to understand the many possible defects that may occur. A dataset of cleaned-up or enhanced query sketches is then created based on an understanding of these defects. Consequently, an end-to-end training of a deep neural network is carried out in order to provide a mapping between the defective and the clean sketches. This network takes the defective query sketch as the input and generates a clean or an enhanced query sketch. Qualitative and quantitative comparisons of the proposed approach with other state-of-the-art techniques show that the proposed approach is effective. The results of the search engine are reported using both the defective and enhanced query sketches, and it is shown that using the enhanced query sketches from the developed approach yields improved search results. [Display omitted] • The first learning-based strategy to clean rough query sketches of 3D CAD models • Introduces SketchCleanNet — an end-to-end image translation scheme • SketchCleanNet aims to understand the mapping between rough sketches and clean query images • A novel scheme to calculate the loss is introduced • Dataset Contribution: The resulting enhanced query sketch dataset is made available publicly. • This paper will significantly contribute to the research community and give researchers opportunities to develop new algorithms for search and retrieval of 3D mechanical components. [ABSTRACT FROM AUTHOR]

• Foreword to the 16th Eurographics Symposium on 3D Object Retrieval (3DOR2023). • Research into methods and evaluation techniques for shape retrieval are presented. • It contains 9 full technical papers and 3 SHREC full papers. [ABSTRACT FROM AUTHOR]

Probabilistic summarization is the process of creating compact statistical representations of the original data. It is used for data reduction, and to facilitate efficient post-hoc visualization for large-scale patch-based data generated in parallel numerical simulation. To ensure high reconstruction accuracy, existing methods typically merge and repartition data patches stored across multiple processor cores, which introduces time-consuming processing. Therefore, this paper proposes a novel probabilistic summarization method for large-scale patch-based scientific data. It considers neighborhood statistical properties by importance-driven sampling guided by the information entropy, thus eliminating the requirement of patch merging and repartitioning. In addition, the reconstruction value of a given spatial location is estimated by coupling the statistical representations of each data patch and the sampling results, thereby maintaining high reconstruction accuracy. We demonstrate the effectiveness of our method using five datasets, with a maximum grid size of one billion. The experimental results show that the method presented in this paper reduced the amount of data by about one order of magnitude. Compared with the current state-of-the-art methods, our method had higher reconstruction accuracy and lower computational cost. [Display omitted] • A novel probabilistic summarization method for large-scale patch-based scientific data. • The neighborhood statistical properties are considered by importance-driven sampling guided by the information entropy. • Compared with the current state-of-the-art methods, our method had higher reconstruction accuracy and lower computational cost. [ABSTRACT FROM AUTHOR]

The research topic of estimating hand pose from the images of hand-object interaction has the potential for replicating natural hand behavior in many practical applications of virtual reality and robotics. However, the intricacy of hand-object interaction combined with mutual occlusion, and the need for physical plausibility, brings many challenges to the problem. This paper provides a comprehensive survey of the state-of-the-art deep learning-based approaches for estimating hand pose (joint and shape) in the context of hand-object interaction. We discuss various deep learning-based approaches to image-based hand tracking, including hand joint and shape estimation. In addition, we review the hand-object interaction dataset benchmarks that are well-utilized in hand joint and shape estimation methods. Deep learning has emerged as a powerful technique for solving many problems including hand pose estimation. While we cover extensive research in the field, we discuss the remaining challenges leading to future research directions. [Display omitted] • Deep learning is effectively used for estimating hand pose from images. • The correlation between a hand and an object helps in estimating hand-object pose. • Hand model helps estimate hand shape, but it restricts within the model's prior. • Implicit function methods have emerged in hand-object pose estimation. [ABSTRACT FROM AUTHOR]

Regolith sampling is one of the core missions in deep space exploration. The design, optimization, and fabrication of samplers are challenging tasks to meet the requirements of deep space exploration, often necessitating complex modeling with computer-aided design tools and demanding the expertise of experienced space engineers with lengthy design iterations. We propose an interactive design framework where designers collaborate with optimization tool to streamline the design process. With the operator adjusting the design goals, we introduce Bayesian optimization to automatically suggest the next sets of parameters to explore. This approach is suitable for optimization scenarios when the design goals cannot be well established as analytical functions and fewer design iterations are required. In this paper, we design and optimize the core structure of the sampler under both stress analysis and discrete element analysis, considering lower stress, greater sampling volume per unit power consumption, and smaller size. Both simulation and physical experimental results show that the design proposed by our framework outperforms existing designs with a small number of design iterations. [Display omitted] • A computational design framework for searching forms of planetary regolith samplers. • The introduction of Bayesian optimization to reduce the cost in optimizing the shape in terms of efficient interaction with granular material. • Simulation and physical experiments to validate the proposed design from our framework. [ABSTRACT FROM AUTHOR]

We present a particle-based smoothed particle hydrodynamics (SPH) framework for simulating hyperelastic materials with anisotropic stiffness models. While most elastic simulations predominantly rely on mesh-based approaches, such as the Finite Element method, the relationship between Lamé's first parameter and Poisson's ratio complicates the strict enforcement of volume conservation, making it challenging to stabilize simulations for common biological tissues like fat and muscle. In this paper, we couple an implicit divergence-free SPH solver with particle-based deformation gradient computation and apply various elastic energy functions to achieve incompressible elastic simulations. The incompressibility of elastic objects and collisions between different bodies are managed by the implicit SPH algorithm. We further incorporate anisotropic energy functions, constructed from the extrapolation of Cauchy–Green invariants, to introduce anisotropic properties to the objects. By integrating activation and contraction coefficients into the energy functions, particles can simulate muscle contractions and lift heavy objects. Our method can effectively represent elastic objects with varying mechanical properties across different directions and be further employed to mimic muscle contractions. Experiments demonstrate that our approach provides realistic simulations for a wide range of animal and human body movements. [Display omitted] • Leveraging a Lagrangian-based approach for the simulation of anisotropic elasticity. • Integration of Smoothed Particle Hydrodynamics with anisotropic energy functions for advanced modeling. • Adaptable simulation of muscle contraction, effectively mimicking a diverse range of movement behaviors. • Enforcing strict incompressibility in the simulation of muscle-like tissues, ensuring highly accurate representations. [ABSTRACT FROM AUTHOR]

Arbitrary style transfer has broad application prospects and important research values, and it is a research hotspot in the field of computer vision. Many studies have shown that arbitrary style transfer achieves remarkable success. However, existing methods may produce artifacts and sometimes lead to distortion of the content structure. To overcome this limitation, this paper proposes a novel Attention-wise and Covariance-Matching Module (ACMM) that preserves the content structure well without unpleasant artifacts. First, our method uses global attention covariance matching to match the global information of style features with content features, thereby obtaining pleasing stylized images. Then, to enable the model to better match the global statistics, a histogram loss is introduced to improve the saturation and stability of the resulting color. Our method can preserve the content structure; so that appearance transfer can be achieved with simple adjustments to the model. The effectiveness of the proposed method is demonstrated through qualitative and quantitative experiments with comparisons with some state-of-the-art arbitrary style transfer methods. [Display omitted] • The Attention-wise and Covariance-Matching Module can preserve content structure well without unpleasant artifacts. • The histogram loss can improve the color saturation of the generated results and enhances the stability of the results. • The proposed framework can be extended to achieve photorealistic style transfer. • The visual performance of the proposed framework is comparable to other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Virtual simulators play a significant role in training surgeons for endoscopic procedures, improving their abilities in real-world scenarios while reducing patient exposure. However, due to the complexity of the organs' behavior, building realistic virtual simulators is still a challenge. In this paper, we propose an approach to modeling the endoscope and its interaction with the colon with a focus on compatibility between colon simulation, endoscope simulation, and user interaction. Instead of using a traditional physics-based approach, which is computationally expensive, we rely on efficient position-based methods that were originally built for video games, where high accuracy is traded for interactive plausibility. We propose a Cosserat rod constraint to model the endoscope and designed a user-centered interface that allows the simulator to run on commodity computers instead of dedicated training hardware. We implemented our model and interface on Unity and evaluated our virtual endoscope for compatibility with a deformable colon. Results show that the model performs in real-time, the dynamic elongation of the scope is stable, and the typical maneuvers made in total colonoscopy can be effectively made with the interface. [Display omitted] • We present an endoscope model and provide data demonstrating relevant aspects. • Compatibility with a deformable colon. • Real-time performance that allows for both visual and force-feedback fidelity. • We present an interface that increases the access and availability of the simulator. [ABSTRACT FROM AUTHOR]

Increasingly complex 3D CAD models are essential during different life-cycle stages of modern engineering projects. Even though these models contain several repeated geometries, instancing information is often not available, resulting in increased requirements for storage, transmission, and rendering. Previous research have successfully applied shape matching techniques to identify repeated geometries and thus reduce memory requirements and improve rendering performance. However, these approaches require consistent vertex topology, prior knowledge about the scene, and/or the laborious creation of labeled datasets. In this paper, we present an unsupervised deep-learning method that overcomes these limitations and is capable of identifying repeated geometries and computing their instancing transformations. The method also guarantees a maximum visual error and preserves intrinsic characteristics of surfaces. Results on real-world 3D CAD models demonstrate the effectiveness of our approach: the datasets are reduced by up to 83.93% in size. Our approach achieves better results than previous work that does not rely on supervised learning. The proposed method is applicable to any kind of 3D scene and geometry. [Display omitted] • Unsupervised deep learning method for 3D shape registration. • Does not require any previous knowledge of the 3D geometries. • Does not require a labeled dataset for any supervised training. • Guarantees an upper bound on any visual errors. • Generalizes for any 3D scene and geometry. [ABSTRACT FROM AUTHOR]

Accurate registration of three-dimensional (3D) craniofacial data is fundamental work for craniofacial reconstruction and analysis. The complex topology and low-quality 3D models make craniofacial registration challenging in the iterative optimization process. In this paper, we proposed a craniofacial registration network (CR-Net) that can automatically learn the registration parameters of the non-rigid thin plate spline (TPS) transformation from the training data sets and perform the required geometric transformations to align craniofacial point clouds. The proposed CR-Net employs an improved point cloud encoder architecture, a specially designed attention mechanism that can perceive the geometric structure of the point cloud. In order to align the source and target data, Wasserstein distance loss is introduced to combined with Chamfer loss and Gaussian Mixture Models (GMM) loss as an unsupervised loss function dedicated to improves registration accuracy. After efficient training, the network can automatically generate the transformation parameters for registration, transforming the reference craniofacial data to the target craniofacial data without manual calibration of feature points or performing an iterative optimization process. Experimental results show that our method has high registration accuracy and is robust to low-quality models. [Display omitted] • A neural network for robust craniofacial point cloud registration. • Geometric attention mechanism to perceive the geometric structure of point clouds. • Introducing Wasserstein distance loss constrain unsupervised training. [ABSTRACT FROM AUTHOR]

Multi-View Stereo (MVS) is a crucial technique for reconstructing the geometric structure of a scene, given the known camera parameters. Previous deep learning-based MVS methods have mainly focused on improving the reconstruction quality but overlooked the running efficiency during the actual algorithm deployment. For example, deformable convolutions have been introduced to improve the accuracy of the reconstruction results further, however, its inability for parallel optimization caused low inference speed. In this paper, we propose AdaptMVSNet which is device-friendly and reconstruction-efficient, while preserving the original results. To this end, adaptive convolution is introduced to significantly improve the efficiency in speed and metrics compared to current methods. In addition, an attention fusion module is proposed to blend features from adaptive convolution and the feature pyramid network. Our experiments demonstrate that our proposed approach achieves state-of-the-art performance and is almost 2 × faster than the recent fastest MVS method. We will release our source code. [Display omitted] [ABSTRACT FROM AUTHOR]

Raw point clouds captured by sensing devices are often contaminated with noise, which perturbs the fidelity of the original geometric information. Point cloud denoising is therefore an inseparable post-processing step, aiming to remove the noise in the point clouds. Existing point cloud denoising approaches are typically trained on datasets that have uniform point distributions and densities, making them unsuitable for effectively denoising point clouds with severe noise or irregular point distributions. In this paper, we introduce a novel random screening-based feature aggregation method for point cloud denoising. Our key insight is that merging features of dense and sparse points assists with enhancing the quality of point cloud denoising results. In specific, our approach involves randomly screening the features of local point patches and fusing richer geometric information of denser points into sparser point representations. Comprehensive experiments demonstrate that our method achieves state-of-the-art performance in the point cloud denoising task on both synthetic and real-world datasets. [Display omitted] • Random screening-based point feature aggregation is useful for point cloud denoising. • Merging features of dense points into sparser points boosts point cloud denoising. • Our pipeline demonstrates robustness to noise and saves processing time. • Our technique achieves impressive results on severe noise and irregular points. [ABSTRACT FROM AUTHOR]

3D scene stylization aims to generate impressive stylized images from arbitrary novel views based on the stylistic reference. Existing image-driven 3D scene stylization methods require a specific style reference to be given, and lack the ability to produce diverse stylization results by combining style information from different aspects. In this paper, we propose a text-driven 3D scene stylization method based on semantic contrast learning, which takes Neural Radiance Fields (NeRF) as the 3D scene representation and generates diverse 3D stylized scenes by leveraging the semantic capabilities of the Contrastive Language-Image Pre-Training (CLIP) model. For comprehensively exploiting the semantic knowledge to generate finely stylized results, we design a CLIP-based semantic contrast estimation loss, which can avoid the global stylistic inconsistency caused by the NeRF ray sampling method and avoid the tendency to stylize neutral descriptions due to the semantic averaging of the CLIP space. In addition, to reduce the memory burden arising from NeRF ray sampling, we propose a novel ray sampling method with gradient accumulation to optimize the NeRF rendering process. The experimental results indicate that our method generates high-quality and plausible results with cross-view consistency. Moreover, our method enables the creation of new styles that match the target text by combining multiple domains. The code will be available at. • The text-driven 3D implicit stylization method can intuitively and diversely stylize 3D scenes. • The stylization transfer direction of the CLIP semantic space can be controlled by using the contrast learning method, which can more accurately stylize the 3D scene. • Gradient accumulation for NeRF field ray sampling ensures that the stylization losses are calculated with full-size rendered images, and prevent the memory overload caused by NeRF intensive sampling. • The target stylized semantics are fine-tuned on a specialized art database by using a nearest-neighbor semantic similarity searcher, which can generate multi-domain stylized scenes based on a specialized art database. [Display omitted] [ABSTRACT FROM AUTHOR]

Images taken under poor lighting conditions generally present problems such as low brightness, noise, and color distortion, so dealing with low-light images is challenging. To solve this problem, current mainstream methods focus on designing special deep networks. For example, many methods are based on Retinex-based models. However, the solution of the Retinex model is an ill-conditioned problem in mathematics, so these methods are difficult to obtain the expected result. In this paper, following the new paradigm of embedding prior knowledge in deep learning, we design a simple yet surprisingly effective neural network, called Multi-Resolution Edge-aware Lighting Enhancement Network (MELENet). Since resolution and acutance are two of the key factors related to the visual quality of natural images, MELENet with a dual-branch structure aims to incorporate multi-resolution edge information into deep learning for guiding lighting enhancement. Additionally, by adopting special loss functions, the model suppresses noise and prevents color distortion while obtaining sound lighting enhancement effects. Experiments have verified that such a simple solution exhibits outstanding performance compared to existing mainstream lighting enhancement methods. We also directly apply the proposed model to image de-hazing and obtain state-of-the-art results. [Display omitted] • The proposed algorithm focuses on embedding prior knowledge into the network to assist the low-light image enhancement. • An effective loss function is designed to balance the color distortion and noise suppression. • Our model can be directly applied to other image processing tasks such as image de-hazing and gain competitive results. [ABSTRACT FROM AUTHOR]

Shadow detection is a common computer vision task that identifies the location of shadows. It has been used as a key primitive for many high-level tasks such as object segmentation and video tracking. Due to the complexity and variety of shadows, it is challenging to design a one-size-fits-all shadow detection algorithm in real-world scenarios. Instead, it is often necessary in practice to pick the best shadow detection algorithm for a particular vision application. While shadow detection algorithms can be assessed by comparing the detected shadow with the ground truth, such ground truth is often unavailable in real-world vision systems consistently generating new data. In this paper, we propose a no-reference shadow detection quality assessment network (NSDQA-Net) to rank shadow detection algorithms without needing the ground truth. For a given image, the proposed assessment network first learns a reference map that compensates the false positives and false negatives of multiple input shadow detection algorithms. The reference map, as well as the original image and the input shadow detection results, are then used to regress the quality scores of the shadow detection methods. Our framework supports the assessment of one, two, or multiple algorithms in a single run, making it easier for deployment in practical scenarios. Experiments results show that the proposed network achieves optimal accuracy in no-reference shadow detection assessment. Relying on it for adaptive algorithm selection also improves the performance of shadow detection using a fixed algorithm on benchmark datasets. [Display omitted] • We identify a task of no-reference quality assessment in shadow detection. • We design a novel no-reference shadow detection quality assessment network (NSDQA-Net). • We experimentally demonstrate the state-of-the-art performance of the proposed network. [ABSTRACT FROM AUTHOR]

Chinese painting poetry is an extraordinary art form, which not only describes the painting contexts but also grasps the sentiment of the painters. In this paper, we propose an automatic poetry generation method Poetry4painting , which enhances the poetry diversity for large-size ancient paintings. The basic framework is based on multiple modern sentences, that are first captioned from the ancient painting and then used to generate a poem using CNN and LSTM. To solve the repeatability issue of this framework, four kinds of data augmentation are employed during online processing, including quantity, shape, surrounding, and object augmentation. In offline training, data augmentation is also used to create an image caption dataset with over 1500 painting images and 7500 captions. Through ablation studies, evaluations of poetry qualities and diversities, and comparisons with other methods, we demonstrate the validity of the proposed method. [Display omitted] • A poetry generation method for ancient paintings with high diversity and conformity. • An enlarged dataset for painting poetry generation that has about 8000 annotations. • A mass of experiments and evaluations demonstrate the validity of the proposed method. [ABSTRACT FROM AUTHOR]

3D object retrieval is an important yet challenging task that has drawn more and more attention in recent years. While existing approaches have made strides in addressing this issue, they are often limited to restricted settings such as image and sketch queries, which are often unfriendly interactions for common users. In order to overcome these limitations, this paper presents a novel SHREC challenge track focusing on text-based fine-grained retrieval of 3D animal models. Unlike previous SHREC challenge tracks, the proposed task is considerably more challenging, requiring participants to develop innovative approaches to tackle the problem of text-based retrieval. Despite the increased difficulty, we believe this task can potentially drive useful applications in practice and facilitate more intuitive interactions with 3D objects. Five groups participated in our competition, submitting a total of 114 runs. While the results obtained in our competition are satisfactory, we note that the challenges presented by this task are far from fully solved. As such, we provide insights into potential areas for future research and improvements. We believe we can help push the boundaries of 3D object retrieval and facilitate more user-friendly interactions via vision-language technologies. • ANIMAR dataset is created with 711 3D animal models and 150 text queries. • SHREC challenge track focusing on Text-based 3D ANIMAl model fine-grained Retrieval. • Five groups participated in our competition, submitting a total of 114 runs. • We push boundaries of 3D object retrieval and facilitate user-friendly interactions. [ABSTRACT FROM AUTHOR]

Sand painting is a significant form of artistic expression. It is necessary to convert real photographs into sand painting style automatically. However, current methods for creating artwork mostly concentrate on oil painting, watercolor painting, and other fields, with little involvement in sand painting. In this paper, we propose a sand painting conversion algorithm that aims to create a unique art style while preserving details. Our Sand Painting Conversion (SPC) method consists of two primary processes: coarse waving for the whole image and detail preservation for specific parts. The former process involves parameter optimization to render the entire image and obtain preliminary sand painting results. The latter process comprises three parts: detail painting based on sand reduction, refined waving for inhomogeneous areas, and edge detail processing based on multi-technique selection, in order to achieve detail preservation from a source image to the sand painting style. Our SPC can generate a series of realistic sand painting works without any user intervention. Comparative experiments and user feedback have revealed the effectiveness and superiority of our algorithm in converting natural images into high-quality and realistic sand painting style images. [Display omitted] • Convert a regular image into a sand painting with details for the first time. • Change the inherent practice of learning textures in traditional stroke painting. • Improve the network structure and obtain better stroke colors. • Consider different techniques in sand painting. [ABSTRACT FROM AUTHOR]

Facial information plays an important role in human communication, e.g., rich and nuanced facial expressions effectively convey emotions. Traditional data such as 2D facial images and videos are susceptible to perturbation from lighting and occlusion, while 3D static data such as mesh models lack the temporal information which is necessary to describe facial dynamics. Therefore, researches on 4D facial data (3D facial models together with time as the fourth dimension) have received considerable attention in recent years. 4D data can simultaneously reflect the complex facial temporal and space information. To fully explore these characteristics, we present a systematic overview of the 4D facial research in this paper. We give a review in terms of historical development of 4D facial datasets, acquisition process of these datasets, related algorithms and applications, and discussion of outstanding issues. We also analyze the 4D facial research works by summarizing and comparing them; in particular, we present the results on three kinds of tasks conducted on the 4D facial datasets, i.e., expression-related tasks (AU detection, expression recognition and retrieval), generation tasks (3D facial reconstruction, facial expression and facial animation generation), and other tasks (facial registration, facial recognition and facial disease diagnosis). Finally, we summarize the future research directions. [Display omitted] • We provide a comprehensive overview of a 4D facial dataset. • We classified and elaborated on the different facial studies conducted using 4D facial datasets. • We summarize the future research direction. [ABSTRACT FROM AUTHOR]

Visual querying of location-based data assists users in expressing query requirements, investigating query results and making inferences. However, directly accessing data records exposes individual location information and may cause privacy issues. Conventional aggregation-based methods can preserve location-relevant privacy but may lead to the loss of detailed information and failure of analysis. Visualization aids users in gaining a deeper comprehension of the query process and the variation of information concerning privacy-preservation. In this paper, we present a privacy-aware visual query approach for location-based data. We propose a graph-based privacy-preserving scheme to protect location privacy in the visualization, and two visual metaphors to enhance understandings of information-variation in the privacy-preserving process. We design and implement a visual interface that supports a progressive process of query conditions specification and query results exploration. Experiments on real-world urban datasets demonstrate that our approach is capable of making a fair balance between location privacy and data analysis. [Display omitted] • Directly query location data may lead to privacy issues. • Anonymizing data preserves data privacy but limits data utility. • Graph-based anonymization sanitizes data privacy effectively and dynamically. • Interactive visual query interface enables a fair balance between privacy and utility. [ABSTRACT FROM AUTHOR]

Vision and language representation learning has been demonstrated to be an effective means of enhancing multimodal task performance. However, fashion-specific studies have predominantly focused on object-level features, which might neglect to capture region-level visual features and fail to represent the fine-grained correlations between words in fashion descriptions. To address these issues, we propose a novel framework to achieve a fine-grained vision and language representation in the fashion domain. Specifically, we construct a knowledge-dependency graph structure from fashion descriptions and then aggregate it with word-level embedding, which can strengthen the fashion semantic knowledge and obtain fine-grained textual representations. Moreover, we fine-tune a region-aware fashion segmentation network to capture region-level visual features, and then introduce local vision and language contrastive learning for pulling closer the fine-grained textual representations to the region-level visual features in the same garment. Extensive experiments on downstream tasks, including cross-modal retrieval, category/subcategory recognition, and text-guided image retrieval, demonstrate the superiority of our method over state-of-the-art methods. [Display omitted] • This paper proposes a novel framework to achieve a fine-grained vision and language representation in the fashion domain. • Specifically, we construct a knowledge-dependency graph structure from fashion descriptions and then aggregate it with word-level embedding, which can strengthen the fashion semantic knowledge and obtain fine-grained textual representations. • Moreover, we fine-tune a region-aware fashion segmentation network to capture region-level visual features, and then introduce local vision and language contrastive learning for pulling closer the fine-grained textual representations to the region-level visual features in the same garment. • Extensive experiments on downstream tasks, including cross-modal retrieval, category/subcategory recognition, and text-guided image retrieval, demonstrate the superiority of our method over state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The lens flare phenomenon is often an undesired artifact of the imaging process; however, it has become an important artistic tool in photography and cinematography as well as a highly impactful component for increasing the level of realism for computer-generated images. In this paper, we present a novel method for efficiently simulating the lens flares of optical camera systems in highly interactive environments. Recreating this effect in a physically correct way necessitates the use of ray tracing, which we made much more computationally efficient by using a tiled approach to rasterize the ghosts of the lens flare. One of the main drawbacks of the current state-of-the-art method is the huge pixel overdraw resulting from the large number of ghosts being rasterized individually onto the output image. The problem is made even worse when dense ray grids are utilized for improving the quality of the simulation. We overcome these limitations by collecting all the ray-traced ghost data into screen-aligned tiles and accumulating the per-pixel contributions in a single pass. We demonstrate that our tiled approach significantly outperforms the previous algorithm, scales much better with the number of flares rendered, and facilitates the efficient simulation of lens flares in real-time applications, while maintaining the physical correctness. [Display omitted] • A novel, tile-based lens flare ghost rasterization method. • An efficient, two-level strategy for building the per-tile buffers. • An adaptive method for dynamically merging lens flare primitives. • Our tiled method substantially reduces the ghost rasterization costs. • Our method scales much better with camera complexity and the number of light sources. [ABSTRACT FROM AUTHOR]

Reconstructing 3D human mesh from monocular images has been extensively studied. However, the existing non-parametric reconstruction methods are inefficient when modeling vertex relationship concerning human information due to they generally adopt an uniform template mesh. To this end, this paper proposes a novel hypergraph-based human mesh hierarchical representation that enables the expression of vertices at body, parts, and vertices perspectives, corresponding to global, local and individual, respectively. And then a novel HyperGraph Attention-based human mesh reconstruction network (HGaMRNet) is put forward in turn, which mainly consists of two modules and can efficiently capture human information from different granularities of the mesh. Specifically, the first module, Body2Parts, decouples a body into local parts, and leverages Mix-Attention (MAT) based feature encoder to learn visual cues and semantic information of the parts for capturing complex human kinematic relationships from monocular images. The second module, Part2Vertices, transfers part features to the corresponding vertices through an adaptive incidence matrix, and utilizes a HyperGraph Attention network to update the vertex features. This is conductive to learning the fine-grained morphological information of a human mesh. All in one, supported by the hypergraph-based hierarchical representation of the human mesh, HGaMRNet balances the effects of neighbor vertex from different levels properly and eventually promotes the reconstruction accuracy of human mesh. Experiments conducted on both Human3.6M and 3DPW datasets show that HGaMRNet outperforms most of the existing image-based human mesh reconstruction methods. [Display omitted] • We propose a novel hypergraph-based human mesh hierarchical representation • We introduce a HyperGraph Attention-based human mesh reconstruction network. [ABSTRACT FROM AUTHOR]

In this paper, we address the problem of capturing both the shape and the pose of a human character using a single depth sensor. Some previous works proposed to fit a parametric generic human template into the depth image, while others developed deep learning (DL) approaches to find the correspondence between depth pixels and vertices of the template. We designed a hybrid approach, combining the advantages of both methods, and conducted extensive experiments on the SURREAL Varol et al. (2017), DFAUST datasets Bogo etal (2017) and a subset of AMASS Mahmood et al. (2019). Results show that this hybrid approach enables us to enhance pose and shape estimation compared to using DL or model fitting separately. We also evaluated the ability of the DL-based dense correspondence method to segment also the background — not only the body parts. We also evaluated 4 different methods to perform the model fitting based on a dense correspondence, where the number of available 3D points differs from the number of corresponding template vertices. These two results enabled us to better understand how to combine DL and model fitting, and the potential limits of this approach to deal with real-depth images. Future works could explore the potential of taking temporal information into account, which has proven to increase the accuracy of pose and shape reconstruction based on a unique depth or RGB image. [Display omitted] • Dense correspondence and model fitting enhance 3D shape and pose reconstruction. • Bodypart and background color embedding enables character segmentation in depth map. • Averaging 3D candidate points before model fitting offers the best reconstruction. [ABSTRACT FROM AUTHOR]

Relief patterns represent a surface characteristic that can be seen as the 3D counterpart of the texture concept in 2D images. Such characteristic is well distinct from the 3D object shape but represents a good information source to recognize the object itself. The majority of state-of-the-art techniques for 2D images rely on convolution-based filtering so, the idea of extending such techniques to the mesh manifold domain is quite intriguing as much as challenging. In this paper, we propose a novel approach based on Graph Neural Networks for 3D mesh relief pattern classification. To this end, we designed a bi-level architecture that learns on data structures computed thanks to a mesh resampling algorithm that allows us to represent local surface patches uniformly, while keeping a consistent points order. The local mesh structures are represented by SpiderPatches , that aim to capture local features of the 3D mesh surface, providing a fine-grained rich representation of the relief patterns; global structures are instead captured by MeshGraphs , whose nodes are SpiderPatches, representing the mesh at a macroscopic level. We tested our architecture using SpiderPatches and MeshGraphs on the original meshes of the SHREC'17 and SHREC'20 relief patterns track datasets, showing superior performance to that reported in the literature using a comparable experimental setting. [Display omitted] • A new and original method for relief patterns classification on mesh manifold. • A bi-level graph attention network architecture. • A SpiderPatch surface structure for local mesh description. • A MeshGraph structure for mesh description at higher level of detail. [ABSTRACT FROM AUTHOR]

This paper explores the potential of self-supervised learning as an alternative to supervised learning in the context of geometry-based 3D object retrieval. With the ongoing digitalization of many industries, an exponentially increasing number of 3D objects are processed by retrieval systems. In order to support new shapes, modern deep learning-based retrieval systems require retraining. The dominant paradigm for optimizing neural networks in this field is supervised classification training. Supervised learning requires time-consuming and expensive data annotation. Moreover, training neural networks for classification introduces a bias towards the classes in the training data, which is undesirable for retrieval systems encountering unseen object types in the wild. Through extensive experiments, we make a direct comparison between supervised and self-supervised learning on four datasets from three different domains (household, manufacturing and medical). For object classes seen during training, self-supervised and supervised learning are competitive. For unseen classes, self-supervised learning outperforms supervised learning in many cases. We conclude that self-supervised learning provides a powerful tool for circumventing labeling costs and providing more robust retrieval systems. [Display omitted] • Self-supervised learning is on par with supervised learning for 3D object retrieval. • Self-supervised object retrieval outperforms supervised learning out-of-distribution. • Object alignment is highly impactful on the performance of retrieval systems. • Non-contrastive siamese networks are a practical option in 3D deep learning. [ABSTRACT FROM AUTHOR]

Point cloud saliency detection is an important technique that support downstream tasks in 3D graphics and vision, like 3D model simplification, compression, reconstruction and viewpoint selection. Existing approaches often rely on hand-crafted features and are only applicable to specific datasets. In this paper, we propose a novel weakly supervised classification network, called C2SPoint, which directly performs saliency detection on the point clouds. Unlike previous methods that require per-point saliency annotations, C2SPoint only requires category labels of the point clouds during training. The network consists of two branches: a Classification branch and a Saliency branch. The former branch is composed of two Adaptive Set Abstraction layers for feature extraction and a Saliency Transform layer for learning saliency knowledge from the classification network. The latter branch introduces a multi-scale point-cluster similarity matrix for propagating the cluster saliency to each point within it, resulting in the prediction of point-level saliency. Experimental results demonstrate the effectiveness of our method in point cloud saliency detection, with improvements of 2% in both AUC and NSS compared to state-of-the-art methods. • We propose C2SPoint, a novel neural network for 3D point cloud saliency detection. • It leverages only point cloud category labels without the need for point-wise saliency annotations. • A multi-scale point-cluster similarity matrix is introduced to predict point-level saliency from cluster saliency. • The experimental results demonstrate that the proposed method outperforms the state-of-the-art algorithms. [Display omitted] [ABSTRACT FROM AUTHOR]

Image-based 3D model retrieval aims to search for 3D models according to 2D image queries, which provides a convenient way for the management of large 3D model datasets. Most of the related works put the emphasis on bridging the modality gap between 2D images and 3D models, which faces a lot of challenges due to the huge domain discrepancy. In this paper, we explicitly model and eliminate the domain-specific features of 2D images and 3D models. To alleviate the negative effect of complex background of natural images, we adopt semantic focus loss to constrain networks to learn the most semantically relevant feature representations for both 2D images and 3D models. We conduct extensive experiments on two cross-domain 3D model retrieval datasets, MI3DOR and MI3DOR-2, to show the effectiveness of the proposed method. [Display omitted] • Eliminate the domain-specific features in 2D natural images and 3D model projections. • Alleviate the interference of irrelevant semantic information in 2D images. • Achieve superior performance on two public cross-domain 3D model datasets. [ABSTRACT FROM AUTHOR]

In this paper, we advocate a novel method for analyzing the functionality of parts in 3D objects. In contrast to prior research, our method no longer characterizes the functionality of an object part using its single type of qualities (or attributes), e.g., geometry or interactions, nor by weighing the significance of various qualities. Instead, we consider the latent space of part functions as a semantic feature space comprehensively defined by part qualities. To learn such a space by parameterizing and encoding semantic features from multi-channel, we begin by learning multi-modal latent space using shapes, textures, and interaction scenes. Next, the latent space of part functions is generated by embedding and interweaving these multi-modal spaces into a space with a higher dimension. We devise loss functions to direct the embedding and interweaving of multi-modal spaces while preserving their manifolds. Consequently, the learned functionality latent space can capture the similarities between semantic features related to functionality and encode them into high-level functional representations. We assess this innovative approach on diverse categories of textured 3D shapes. Extensive experiments have exhibited our method's parametric and encoding capability towards functionality-centric shape analysis and synthesis, including shape functionality analysis, functionally-similar shape retrieval, and functionality-aware modeling, all of which are of the essence to new graphics techniques and applications. [Display omitted] • A novel method for functionality analysis using multi-modal latent space embedding. • Encoding different function cues into high-level functionality representations. • New functionality-centric shape analysis and synthesis solutions. [ABSTRACT FROM AUTHOR]

Disk B-spline curve provides an alternative shape modeling tool to standard control point based modeling techniques by extending control points to control disks, resulting in an easy and straightforward way of modeling regions or shapes with adjustable thickness. Due to this feature, disk B-spline curves are frequently applied in artistic shape modeling applications, such as brushstroke representation, calligraphy, and animation, but they are also suitable for engineering purposes. However, the boundary curves of the described shapes tend to have unintentional self-intersections or cusps, negatively affecting the usability and artistic value of the resulting shape and texture. In this paper, we introduce an iterative algorithm to give a solution to this problematic issue with the help of an approximating circle skinning method, while preserving the advantageous properties of the disk B-spline curves. Our method also results in smoother texturing of the shape around the more sharply curved sections. [Display omitted] • Exact computation of cusps and self-intersections of envelope curves. • An iterative method to approximate disk B-spline shapes avoiding self-intersections. • Creating contoured or textured artistic freeform shapes with skinning curves. [ABSTRACT FROM AUTHOR]

In recent years, 3D imaging technologies have advanced tremendously, allowing more faithful representations of the physical world. More specifically, imaging technology advancements have driven the production of plenoptic devices that can capture and display visual contents. These devices represent the visual data using an approximation of the plenoptic illumination function, which can describe visible objects in any position and from any point-of-view of the 3D space. Depending on the capturing device, this approximation can correspond to holograms, light fields, or Point Clouds (PCs) imaging formats. Among these formats, PCs have become very popular for a wide range of applications, such as immersive virtual reality scenarios. As a consequence, in the last couple of years, there has been a great effort to develop novel acquisition, representation, compression, and transmission solutions for PC contents in the research community. In particular, the development of objective quality assessment methods that are able to predict the perceptual quality of PCs has attracted a lot of attention. In this paper, we present an effective framework for assessing the quality of PCs, which is based on descriptors that extract geometry-aware texture information of PC contents. In this framework, the statistics of the extracted information are used to model the PC visual quality. We also present the research and experiments carried to evaluate the most appropriate distance metrics and regression methods to be used together with the proposed descriptors. Experimental results show that the proposed framework exhibit good and robust performance when compared with several state-of-the-art Point Cloud Quality Assessment (PCQA) methods. A C++ implementation of the metrics described in this paper can be found at https://gitlab.com/gpds-unb/pc_metric. [Display omitted] • We provide an framework to develop quality assessment metrics for point clouds. • We propose a set of descriptors to describe point cloud visual data. • The descriptors' statistics model the perceptual quality of point clouds. • Several point cloud quality databases were used to compare with the state-of-the-art. • Experimental results indicate that the proposed framework overcomes state-of-the-art. [ABSTRACT FROM AUTHOR]

Low-light image enhancement is crucial in applications such as traffic safety and medical imaging. Besides having characteristics like low luminance and poor visibility, low-light images are inevitably affected by noise. Noise not only covers image details, introduces artifacts, and decreases image quality, but also interferes with downstream computer vision tasks like object detection, image segmentation, and object tracking. Previous methods in image enhancement often overlook noise handling or fail to accurately suppress noise in adaptive denoising processes, resulting in more severe noise artifacts in the enhanced images. To effectively suppress noise, this paper proposes a Channel Self-Attention Based Low-Light Image Enhancement Network (CAENet), which leverages Transformers and CNNs to model long-range and short-range pixel dependencies, extract global and local features, and construct a Noise Suppression Transformer Block that adaptively suppresses noise regions guided by signal-to-noise ratio priors and attention maps. After adaptive noise suppression, the resulting images exhibit fewer noise artifacts and improved details. The experimental results show that the network in this paper outperforms other state-of-the-art methods overall on five representative paired datasets as well as six unpaired datasets, improving the image quality while effectively suppressing the noise. [Display omitted] • To address the problem of diverse noise types and complex interference levels in low-light image enhancement, we propose an end-to-end network CAENet. • Since the severity of noise varies in regions with different luminance, we designed the NSTB based on channel self-attention. • Aiming at the different degree of interference of noise to the whole and details of the image in different sized regions, we design global and local branches. [ABSTRACT FROM AUTHOR]

Motion matching has become a widely adopted technique for generating high-quality interactive animation systems in video games. However, its current implementations suffer from significant computational and memory resource overheads, limiting its scalability in the context of modern video game performance profiles. Our method significantly reduces the computational complexity of approaches to motion synthesis, such as "Learned Motion Matching", while simultaneously improving the compactness of the data that can be stored and the robustness of pose output. As a result, our method enables the efficient execution of motion matching that significantly outperforms other implementations, by 8.5× times in CPU execution cost and at 80% of the memory requirements of "Learned Motion Matching", on contemporary video game hardware, thereby enhancing its practical applicability and scalability in the gaming industry and unlocking the ability to apply on large numbers of animated in-game characters. In this paper, we expand upon our published paper "Learning Robust and Scalable Motion with Lipschitz Continuity and Sparse Mixture of Experts", where we successfully proposed a novel method for learning motion matching that combines a Sparse Mixture of Experts model architecture and a Lipschitz-continuous latent space for representation of poses. We present further details on our method, with extensions to our approach to expert utilization within our neural networks. [Display omitted] • Applies the Sparse Mixture of Experts architecture to animation synthesis for the first time. • Significantly reduce theoretical and practical compute requirements, 8.5 times that of prior art. • Lipschitz-continuous encoding neural networks producing a linearly interpolatable manifold. • SMoE Networks express a higher density of animation data than Feed-Forward Neural Networks. [ABSTRACT FROM AUTHOR]

In this paper, we present a new volumetric ambient occlusion algorithm called Contextual Ambient Occlusion (CAO) that supports real-time clipping. The algorithm produces ambient occlusion images of exactly the same quality as Local Ambient Occlusion (LAO) while enabling real-time modification to the shape used to clip the volume. The main idea of the algorithm is that clipping only affects the ambient value of a small number of voxels, so by identifying these voxels and recalculating the ambient factor only for them, it is possible to significantly increase the rendering performance (by 2-5x) without decreasing the quality of the rendered image. Due to its fast performance, the algorithm is suitable for interactive environments where clipping changes could occur every frame. Additionally, the algorithm does not have any stereoscopic inconsistency, which makes it suitable for mixed reality environments. This paper is an extended version of the "Contextual Ambient Occlusion" article presented during the 2023 Graphics Interface conference, and includes, among other additions, the source code of the algorithm. [Display omitted] • Clipping only affects the ambient value of a small number of voxels. • It is more efficient to perform recalculation only on voxels affected by clipping. • Contextual Ambient Occlusion achieves same quality as Local Ambient Occlusion. • Contextual Ambient Occlusion is suitable for interactive environments. [ABSTRACT FROM AUTHOR]

Folding a thin sheet material such as paper along curves creates a developable surface composed of ruled surface patches. When using such surfaces in design, designers often repeat a process of folding along curves drawn on a sheet and checking the folded shape. Although several methods for constructing such shapes on a computer have been proposed, it is still difficult to check the folded shapes instantly from the crease patterns. In this paper, we propose a simple method that approximately realizes a simulation of curved folds with a triangular mesh from its crease pattern. The proposed method first approximates curves in a crease pattern with polylines and then generates a triangular mesh. In order to construct the discretized developable surface, the edges in the mesh are rearranged so that they align with the estimated rulings. The proposed method is characterized by its simplicity and is implemented on an existing origami simulator that runs in a web browser. [Display omitted] • A method to simulate curved folds with an estimated ruling arrangement is proposed. • The results are similar to actual folded paper with various crease patterns. • The proposed method has advantages in mesh size, simulation cost and robustness. [ABSTRACT FROM AUTHOR]

[Display omitted] • SMI disseminates new mathematical theories and computational techniques for modeling shapes. • SMI2022 received 24 paper submissions and 10 full papers were accepted for publication. • SMI 2022 organized a panel session called Implicit Surface History Event. • SMI2022 hosted the Fabrication and Sculpting Event (SCULPT). [ABSTRACT FROM AUTHOR]

• Embodiment studies should expect potential changes in participants behaviour while doing a task after a threat was introduced. • Threat introduction and repetition do not seem to impact the subjective measure of the Sense of Embodiment (user responses to questionnaires). • Threat introduction and repetition do not seem to impact the objective measure of the Sense of Embodiment (behavioural responses to threat towards the virtual body). • Individual differences between users, such as users' neuroticism trait, gender or game experience can influence their threat responses and also their Sense of Embodiment. [Display omitted] This paper is an extended version of a previous paper [1] published at ICAT-EGVE 2020. We explored the potential impact of threat occurrence and repeatability on users' Sense of Embodiment (SoE) and threat response. To that aim, we conducted an experiment in which participants were embodied in a virtual avatar, and performed a task in which a threat towards the virtual body was introduced a first time, then repeated several times through the experiment (in total 5 times). The SoE of participants as well as their subjective response to the threat were assessed through subjective questionnaires before the introduction of the threat, after a first introduction of the threat and at the end of the experiment. A control group did the same experiment with no threat introduced during the task. The main findings of our experiment are that the introduction of a threat does not alter users' SoE but might change their behaviour while performing a task after the threat occurrence. In addition, threat repetitions did not show any effect on users' subjective SoE, or subjective and objective responses to threat. Taken together, our results suggest that embodiment studies should expect potential change in participants behaviour while doing a task after a threat was introduced, but that threat introduction and repetition do not seem to impact the subjective measure of the SoE (subective ratings) nor the objective measure of the SoE (physical reaction to threat). [ABSTRACT FROM AUTHOR]

• The focus of this review paper is immersive technology that combines high-performance CAD model processing methods, engaging display technologies, and human-centered interaction modalities, in applications for smart manufacturing. • We reviewed and summarized a collection of publications gathered from proceedings and journals in multiple disciplines, including visual computing, graphics, virtual reality, human-computer interaction, manufacturing, computer-aided design, and industrial systems. • We also looked into particular areas of manufacturing that immersive technology has been integrated into, including product design, prototyping, fabrication, training, and logistics. [Display omitted] With the expanded digitalization of manufacturing and product development process, research into the use of immersive technology in smart manufacturing has increased. The use of immersive technology is theorized to increase the productivity of all steps in the product development process, from the start of the concept generation phase to assembling the final product. Many aspects of immersive technology are considered, including techniques for CAD model conversion and rendering, types of VR/AR displays, interaction modalities, as well as its integration with different areas of product development. The purpose of this survey paper is to investigate the potential applications of immersive technology and advantages and potential drawbacks that should be considered when integrating the technology into the workplace. The potential application is broad, and the possibilities are continuing to expand as the technology used becomes more advanced and more affordable for commercial business to implement on a large scale. The technology is currently being utilized in the concept generation and in the design or engineering of new products. Additionally, the immersive technology have great potential to increase the productivity of assembly line workers and of the factory layout/functionality, and could provide a more hands-on form of training, which leads to the conclusion that immersive technology is the step to the future in terms of smart product development strategies to implement for employers. [ABSTRACT FROM AUTHOR]

• The paper presents the first experiment on perception of perceptibility of Facial Action Units under different activation levels, which allows us to list the importance of blendshape expressions. Our main contribution is our perceptual models for perceptibility of facial action units and we provide our GitHub repository with the data and models in R-Code, allowing others to build on our data investigating a larger range of faces, viewpoints and facial action units, or to try different non-linear models such as neural networks. [Display omitted] Blendshape facial rigs are used extensively in the industry for facial animation of virtual humans. However, storing and manipulating large numbers of facial meshes (blendshapes) is costly in terms of memory and computation for gaming applications. Blendshape rigs are comprised of sets of semantically-meaningful expressions, which govern how expressive the character will be, often based on Action Units from the Facial Action Coding System (FACS). However, the relative perceptual importance of blendshapes has not yet been investigated. Research in Psychology and Neuroscience has shown that our brains process faces differently than other objects so we postulate that the perception of facial expressions will be feature-dependent rather than based purely on the amount of movement required to make the expression. Therefore, we believe that perception of blendshape visibility will not be reliably predicted by numerical calculations of the difference between the expression and the neutral mesh. In this paper, we explore the noticeability of blendshapes under different activation levels, and present new perceptually-based models to predict perceptual importance of blendshapes. The models predict visibility based on commonly-used geometry and image-based metrics. [ABSTRACT FROM AUTHOR]

• The goal of this paper is to create a sketch dataset that is suitable for developing deep learning-based solutions to the problem of search and retrieval in 3D CAD models. • A sketch dataset of query images, called 'CADSketchNet' has been built using the available CAD datasets. For the 58,696 CAD models of the MCB Dataset, we obtain computer-generated sketches. We call this as Dataset-A. We obtain hand-drawn sketch data for all the 801 CAD models from ESB dataset. This is the Dataset-B. • For obtaining Dataset-A, many computer sketch generation methods, including the current state-of-the art methods are experimented. We justify the proposed weighted canny technique based on the comparison. • An analysis of the quality of the hand-drawn sketches in Dataset-B is also performed. • Various deep learning architectures have been tested on the developed CADSketchNet dataset, and a detailed analysis of their performance have been reported. • Among the many experiments conducted, it is shown that a Siamese Network approach yielded the best results as compared to other features/networks. The class-wise results of the approach on both Datasets - A & B. • We also justify the need for a dedicated dataset for CAD over the regular 3D graphical shapes by testing the existing methods on the developed CADSketchNet. [Display omitted] Ongoing advancements in the fields of 3D modelling and digital archiving have led to an outburst in the amount of data stored digitally. Consequently, several retrieval systems have been developed depending on the type of data stored in these databases. However, unlike text data or images, performing a search for 3D models is non-trivial. Among 3D models, retrieving 3D Engineering/CAD models or mechanical components is even more challenging due to the presence of holes, volumetric features, presence of sharp edges etc., which make CAD a domain unto itself. The research work presented in this paper aims at developing a dataset suitable for building a retrieval system for 3D CAD models based on deep learning. 3D CAD models from the available CAD databases are collected, and a dataset of computer-generated sketch data, termed 'CADSketchNet', has been prepared. Additionally, hand-drawn sketches of the components are also added to CADSketchNet. Using the sketch images from this dataset, the paper also aims at evaluating the performance of various retrieval system or a search engine for 3D CAD models that accepts a sketch image as the input query. Many experimental models are constructed and tested on CADSketchNet. These experiments, along with the model architecture, choice of similarity metrics are reported along with the search results. [ABSTRACT FROM AUTHOR]

Folding planar sheets to make 3D shapes from is an ancient practice with many new applications, ranging from personal fabrication of customized items to design of surgical instruments for minimally invasive surgery in self-folding machines. Given a polyhedral mesh, unfolding is an operation of cutting and flattening the mesh. The flattened polyhedral nets are then cut out of planar materials and folded back to 3D. Unfolding a polyhedral mesh into planar nets usually require segmentation. Either used as a preprocessing step to simplify the mesh and provide semantics or as the result of unfolding to avoid overlapping, the segmentation and the unfolding operations are decoupled. Consequently, segmented components may not be unfoldable and unfolded nets usually provide no semantic meaning and make folding difficult. In this paper, we propose a strategy that tightly couples unfolding and segmentation. We show that the proposed method produces unfoldable segmentation that resembles carefully designed paper craft. The key idea that enables this capability is an algorithm that learns from failed unfoldings. [ABSTRACT FROM AUTHOR]

Neural material representations are becoming a popular way to represent materials for rendering. They are more expressive than analytic models and occupy less memory than tabulated BTFs. However, existing neural materials are immutable, meaning that their output for a certain query of UVs, camera, and light vector is fixed once they are trained. While this is practical when there is no need to edit the material, it can become very limiting when the fragment of the material used for training is too small or not tileable, which frequently happens when the material has been captured with a gonioreflectometer. In this paper, we propose a novel neural material representation which jointly tackles the problems of BTF compression, tiling, and extrapolation. At test time, our method uses a guidance image as input to condition the neural BTF to the structural features of this input image. Then, the neural BTF can be queried as a regular BTF using UVs, camera, and light vectors. Every component in our framework is purposefully designed to maximize BTF encoding quality at minimal parameter count and computational complexity, achieving competitive compression rates compared with previous work. We demonstrate the results of our method on a variety of synthetic and captured materials, showing its generality and capacity to learn to represent many optical properties. • Neural BTF representations allow for efficient reflectance encoding. • Autoencoder models trained with data augmentation allow for material propagation. • Neural reflectance propagation enables the creation of tileable BTF. [ABSTRACT FROM AUTHOR]

In this paper, we present an adaptive structure to represent a signed distance field through trilinear or tricubic interpolation of values, and derivatives, that allows for fast querying of the field. We also provide a method to decide when to subdivide a node to achieve a provided threshold error. Both the numerical error control, and the values needed to build the interpolants, require the evaluation of the input field. Still, both are designed to minimize the total number of evaluations. C 0 continuity is guaranteed for both the trilinear and tricubic version of the algorithm. Furthermore, we describe how to preserve C 1 continuity between nodes of different levels when using a tricubic interpolant, and provide a proof that this property is maintained. Finally, we illustrate the usage of our approach in several applications, including direct rendering using sphere marching. • Computation of Signed Distance Field(SDF) adaptative grid from triangle meshes. • Method for storing approximations of distance fields for fast query operations. • Adaptive structure built regarding a target error. • The structure guarantees C0 and C1 continuity. [ABSTRACT FROM AUTHOR]

Implicit Neural Representations are powerful tools for representing 3D shapes. They encode an implicit field in the parameters of a Neural Network, leveraging the power of auto-differentiation for optimizing the implicit function and avoiding the need for a manually crafted function. So far, Implicit Neural Representations have been mainly designed to extract or render object surfaces and methods primarily focus on improving the implicit function near the surface. In this paper we argue that implicit fields are useful for other shape analysis tasks, in particular skeleton (medial axis) extraction. Indeed, a medial axis is defined through distances to the surface, which can be provided by an implicit neural representation, making it robust to noise and missing data. However this requires the implicit field to be reliable away from the surface, something most representations are not optimized for. To achieve this, inspired by variational image denoising techniques, we propose to add a Total Variation term, to regularize the implicit field. We further design a skeleton sampling method working directly on the GPU, and link the extracted points using a coverage formulation. We show that our resulting neural skeleton is more robust to sample defects such as noise or missing data compared to other medial axis extraction methods. [Display omitted] • A TV regularization term yielding a smoother signed distance function away from the surface. • A skeleton sampling method on the GPU. • A complete pipeline to extract a medial complex from an input point cloud. [ABSTRACT FROM AUTHOR]

Alpine skiing beginners are easily injured if they fail make the right decisions when faced with unexpected situations. Excellent decision-making requires extensive training, while alpine skiing training is limited by time and space. To address these issues, this paper combines VR and machine learning techniques to develop a high fidelity VR skiing decision-making training system based on HCPS framework. The aim is to train beginner skiers in decision-making by creating automated and repeatable virtual scenarios. Firstly, physical world (ski resort environment, and user motions) was mapped into the VR world using the HCPS framework. Afterwards, the user's decision motions were made based on the scenarios provided, while being identified through an improved SVM (94.5% accuracy). The motion recognition results were fed into the expert system to assess the feasibility of the decision-making. Finally, the feasibility of the system was assessed with a combination of subjective and objective approaches. The results showed that the VR system significantly improved the participants' skiing decision-making ability and speed compared to traditional 2D video training. The high fidelity environment reduced participants' motion sickness symptoms and resulted in a better sensory experience. In addition, participants were more focused in the high fidelity environment. Overall, these results demonstrated the construct validity of the virtual ski decision-making training system. This study changes the traditional training method to create an immersive training environment, and provides an effective avenue for the development of sports training. [Display omitted] • Designs processes and algorithmic models for virtual ski decision systems are presented. • Interaction between user and system via IMU and EEG signals. • The feasibility of the system is assessed through a combination of subjective and objective methods. • Voting classifier achieves higher action recognition rate. • VR training system improves decision-making skills and speed. [ABSTRACT FROM AUTHOR]

Scene understanding has made tremendous progress over the past few years, as data acquisition systems are now providing an increasing amount of data of various modalities (point cloud, depth, RGB...). However, this improvement comes at a large cost on computation resources and data annotation requirements. To analyze geometric information and images jointly, many approaches rely on both a 2D loss and 3D loss, requiring not only 2D per pixel-labels but also 3D per-point labels. However, obtaining a 3D groundtruth is challenging, time-consuming and error-prone. In this paper, we show that image segmentation can benefit from 3D geometric information without requiring a 3D groundtruth, by training the geometric feature extraction and the 2D segmentation network jointly, in an end-to-end fashion, using only the 2D segmentation loss. Our method starts by extracting a map of 3D features directly from a provided point cloud by using a lightweight 3D neural network. The 3D feature map, merged with the RGB image, is then used as an input to a classical image segmentation network. Our method can be applied to many 2D segmentation networks, improving significantly their performance with only a marginal network weight increase and light input dataset requirements, since no 3D groundtruth is required. • Geometry. • Computer vision. • 2D segmentation. • Neural network. • multi-modalities. [ABSTRACT FROM AUTHOR]

To represent smooth geometric shapes by coarse polygonal meshes, visible edges often follow special families of curves on a surface to achieve visually pleasing results. Important examples of such families are principal curvature lines, asymptotic lines or geodesics. In a surprisingly big amount of use-cases, these curves form an orthogonal net. While the condition of orthogonality between smooth curves on a surface is straightforward, the discrete counterpart, namely orthogonal quad meshes, is not. In this paper, we study the definition of discrete orthogonality based on equal diagonal lengths in every quadrilateral. We embed this definition in the theory of discrete differential geometry and highlight its benefits for practical applications. We demonstrate the versatility of this approach by combining discrete orthogonality with other classical constraints known from discrete differential geometry. Orthogonal multi-nets, i.e. meshes where discrete orthogonality holds on any parameter rectangle, receive an in-depth analysis. [Display omitted] • Discrete orthogonality based on rhombic mesh pairings. • Non-planar orthogonal multi-nets expanding Ivory's Theorem. • Compatibility with other classical nets leading to a wide range of applications. • Well suited for efficient optimization. [ABSTRACT FROM AUTHOR]

Reconstructing the shape of hand-held objects from single-view color images is a long-standing problem in computer vision and computer graphics. The task is complicated by the ill-posed nature of single-view reconstruction, as well as potential occlusions due to both the hand and the object. Previous works mostly handled the problem by utilizing known object templates as priors to reduce the complexity. In contrast, our paper proposes a novel approach without knowing the object templates beforehand but by exploiting prior knowledge of contacts in hand-object interactions to train an attention-based network that can perform precise hand-held object reconstructions with only a single forward pass in inference. The network we propose encodes visual features together with contact features using a multi-head attention module as a way to condition the training of a neural field representation. This neural field representation outputs a Signed Distance Field representing the reconstructed object and extensive experiments on three well-known datasets demonstrate that our method achieves superior reconstruction results even under severe occlusion compared to the state-of-the-art techniques. • Conditioning on contact, we reconstruct the hand-held object from single-view images. • We use an end-to-end attention-based network to better encode contact priors. • We achieve state-of-the-art results on Obman, HO3D, and MOW datasets. [ABSTRACT FROM AUTHOR]

Simulation performance is crucial for real-time interactive cutting simulation of deformable objects, especially when haptic devices are used. In this paper, changes are made to a previous software framework by moving deformation equation solving and calculation of positions and normals of surface mesh vertices from CPU to GPU. A new normal calculation method based on deformation gradients is also proposed. Although these changes increase deformation speed, they actually decrease cutting speed due to increased amount of GPU data structures that need to be updated after cutting. Therefore novel CPU–GPU synchronization mechanisms are proposed to counter the negative impact of these changes to cutting speed. Simulation tests show that our synchronization mechanisms almost make up for the cutting speed loss, and the overall simulation speed is increased by 71.3% to 352.4% in non-cutting periods and 20.8% to 64.6% in cutting periods over the previous software framework. • GPU-based deformation equation solving. • GPU-based calculation of surface mesh vertices' positions and normals. • Using deformation gradients to calculate surface mesh vertices' normals. • CPU–GPU synchronization mechanisms for GPU data structure update after cutting. • 71.3% ∼ 352.4% faster in non-cutting periods, 20.8% ∼ 64.6% faster in cutting periods. [Display omitted] [ABSTRACT FROM AUTHOR]