Search

Due to the widespread applications of curves on n‐dimensional spheres, fitting curves on n‐dimensional spheres has received increasing attention in recent years. However, due to the non‐Euclidean nature of spheres, curve fitting methods on n‐dimensional spheres often struggle to balance fitting accuracy and curve fairness. In this paper, we propose a new fitting framework, GLS‐PIA, for parameterized point sets on n‐dimensional spheres to address the challenge. Meanwhile, we provide the proof of the method. Firstly, we propose a progressive iterative approximation method based on geodesic least squares which can directly optimize the geodesic least squares loss on the n‐sphere, improving the accuracy of the fitting. Additionally, we use an error allocation method based on contribution coefficients to ensure the fairness of the fitting curve. Secondly, we propose an adaptive knot placement method based on geodesic difference to estimate a more reasonable distribution of control points in the parameter domain, placing more control points in areas with greater detail. This enables B‐spline curves to capture more details with a limited number of control points. Experimental results demonstrate that our framework achieves outstanding performance, especially in handling imbalanced data points. (In this paper, “sphere” refers to n‐sphere (n ≥ 2) unless otherwise specified.) [ABSTRACT FROM AUTHOR]

Recognizing the detailed information embedded in rasterized floorplans is at the research forefront in the community of computer graphics and vision. With the advent of deep neural networks, automatic floorplan recognition has made tremendous breakthroughs. However, co‐recognizing both the structures and semantics of floorplans through one neural network remains a significant challenge. In this paper, we introduce a novel framework Raster‐to‐Graph, which automatically achieves structural and semantic recognition of floorplans. We represent vectorized floorplans as structural graphs embedded with floorplan semantics, thus transforming the floorplan recognition task into a structural graph prediction problem. We design an autoregressive prediction framework using the neural network architecture of the visual attention Transformer, iteratively predicting the wall junctions and wall segments of floorplans in the order of graph traversal. Additionally, we propose a large‐scale floorplan dataset containing over 10,000 real‐world residential floorplans. Our autoregressive framework can automatically recognize the structures and semantics of floorplans. Extensive experiments demonstrate the effectiveness of our framework, showing significant improvements on all metrics. Qualitative and quantitative evaluations indicate that our framework outperforms existing state‐of‐the‐art methods. Code and dataset for this paper are available at: https://github.com/HSZVIS/Raster-to-Graph. [ABSTRACT FROM AUTHOR]

As virtual worlds continue to rise in popularity, so do the expectations of users for the content of virtual scenes. Virtual worlds must be large in scope and offer enough freedom of movement to keep the audience occupied at all times. For content creators, it is difficult to keep up by manually producing the surrounding content. Therefore, the application of procedural modelling techniques is required. Virtual worlds often mimic the real world, which is composed of organized and connected outdoor and indoor layouts. It is expected that all content is present on the virtual scene and that a user can navigate streets, enter buildings, and interact with furniture within a single virtual world. While there are many procedural methods for generating different layout types, they mostly focus only on one layout type, whereas complete scene generation is greatly underrepresented. This paper aims to identify the coverage of layout types by different methods because similar issues exist for the generation of content of different layout types. When creating a new method for layout generation, it is important to know if the results of existing methods can be appended to other methods. This paper presents a survey of existing procedural modelling methods, which were organized into five categories based on the core approach: pure subdivision, grammar‐based, data‐driven, optimization, and simulation. Information about the covered layout types, the possibility of user interaction during the generation process, and the input and output shape types of the generated content is provided for each surveyed method. The input and output shape types of the generated content can be useful to identify which methods can continue the generation by using the output of other methods as their input. It was concluded that all surveyed methods work for only a few different layout types simultaneously. Moreover, only 35% of the surveyed methods offer interaction with the user after completing the initial process of space generation. Most existing approaches do not perform transformations of shape types. A significant number of methods use the irregular shape type as input and generate the same shape type as the output, which is sufficient for coverage of all layout types when generating a complete virtual world. [ABSTRACT FROM AUTHOR]

In machine learning and computer graphics, a fundamental task is the approximation of a probability density function through a well‐dispersed collection of samples. Providing a formal metric for measuring the distance between probability measures on general spaces, Optimal Transport (OT) emerges as a pivotal theoretical framework within this context. However, the associated computational burden is prohibitive in most real‐world scenarios. Leveraging the simple structure of OT in 1D, Sliced Optimal Transport (SOT) has appeared as an efficient alternative to generate samples in Euclidean spaces. This paper pushes the boundaries of SOT utilization in computational geometry problems by extending its application to sample densities residing on more diverse mathematical domains, including the spherical space 핊d, the hyperbolic plane ℍd, and the real projective plane ℙd. Moreover, it ensures the quality of these samples by achieving a blue noise characteristic, regardless of the dimensionality involved. The robustness of our approach is highlighted through its application to various geometry processing tasks, such as the intrinsic blue noise sampling of meshes, as well as the sampling of directions and rotations. These applications collectively underscore the efficacy of our methodology. [ABSTRACT FROM AUTHOR]

Generating realistic animated virtual humans is a problem that has been extensively studied with many applications in different types of virtual environments. However, the creation process of such realistic animations is challenging, especially because of the number and variety of influencing factors, that should then be identified and evaluated. In this paper, we attempt to provide a clearer understanding of how the multiple factors that have been studied in the literature impact the level of realism of animated virtual humans, by providing a survey of studies assessing their realism. This includes a review of features that have been manipulated to increase the realism of virtual humans, as well as evaluation approaches that have been developed. As the challenges of evaluating animated virtual humans in a way that agrees with human perception are still active research problems, this survey further identifies important open problems and directions for future research. [ABSTRACT FROM AUTHOR]

Point‐based radiance field rendering has demonstrated impressive results for novel view synthesis, offering a compelling blend of rendering quality and computational efficiency. However, also latest approaches in this domain are not without their shortcomings. 3D Gaussian Splatting [KKLD23] struggles when tasked with rendering highly detailed scenes, due to blurring and cloudy artifacts. On the other hand, ADOP [RFS22] can accommodate crisper images, but the neural reconstruction network decreases performance, it grapples with temporal instability and it is unable to effectively address large gaps in the point cloud. In this paper, we present TRIPS (Trilinear Point Splatting), an approach that combines ideas from both Gaussian Splatting and ADOP. The fundamental concept behind our novel technique involves rasterizing points into a screen‐space image pyramid, with the selection of the pyramid layer determined by the projected point size. This approach allows rendering arbitrarily large points using a single trilinear write. A lightweight neural network is then used to reconstruct a hole‐free image including detail beyond splat resolution. Importantly, our render pipeline is entirely differentiable, allowing for automatic optimization of both point sizes and positions.Our evaluation demonstrate that TRIPS surpasses existing state‐of‐the‐art methods in terms of rendering quality while maintaining a real‐time frame rate of 60 frames per second on readily available hardware. This performance extends to challenging scenarios, such as scenes featuring intricate geometry, expansive landscapes, and auto‐exposed footage. The project page is located at: https://lfranke.github.io/trips [ABSTRACT FROM AUTHOR]

Most real‐time rendering models for atmospheric effects have been designed and optimized for Earth's atmosphere. Some authors have proposed approaches for rendering other atmospheres, but these methods still use approximations that are only valid on Earth. For instance, the iconic blue glow of Martian sunsets can not be represented properly as the complex interference effects of light scattered at dust particles can not be captured by these approximations. In this paper, we present an approach for generalizing an existing model to make it capable of rendering extraterrestrial atmospheres. This is done by replacing the approximations with a physical model based on Mie Theory. We use the particle‐size distribution, the particle‐density distribution as well as the wavelength‐dependent refractive index of atmospheric particles as input. To demonstrate the feasibility of this idea, we extend the model by Bruneton et al. [BN08] and implement it into CosmoScout VR, an open‐source visualization of our Solar System. In a first step, we use Mie Theory to precompute the scattering behaviour of a particle mixture. Then, multi‐scattering is simulated, and finally the precomputation results are used for real‐time rendering. We demonstrate that this not only improves the visualization of the Martian atmosphere, but also creates more realistic results for our own atmosphere. [ABSTRACT FROM AUTHOR]

The field of visual computing is rapidly advancing due to the emergence of generative artificial intelligence (AI), which unlocks unprecedented capabilities for the generation, editing, and reconstruction of images, videos, and 3D scenes. In these domains, diffusion models are the generative AI architecture of choice. Within the last year alone, the literature on diffusion‐based tools and applications has seen exponential growth and relevant papers are published across the computer graphics, computer vision, and AI communities with new works appearing daily on arXiv. This rapid growth of the field makes it difficult to keep up with all recent developments. The goal of this state‐of‐the‐art report (STAR) is to introduce the basic mathematical concepts of diffusion models, implementation details and design choices of the popular Stable Diffusion model, as well as overview important aspects of these generative AI tools, including personalization, conditioning, inversion, among others. Moreover, we give a comprehensive overview of the rapidly growing literature on diffusion‐based generation and editing, categorized by the type of generated medium, including 2D images, videos, 3D objects, locomotion, and 4D scenes. Finally, we discuss available datasets, metrics, open challenges, and social implications. This STAR provides an intuitive starting point to explore this exciting topic for researchers, artists, and practitioners alike. [ABSTRACT FROM AUTHOR]

Collaboration in extended reality (XR) environments presents complex challenges that revolve around how users perceive the presence, intentions, and actions of their collaborators. This paper delves into the intricate realm of group awareness, focusing specifically on workspace awareness and the innovative visual cues designed to enhance user comprehension. The research begins by identifying a spectrum of collaborative situations drawn from an analysis of XR prototypes in the existing literature. Then, we describe and introduce a novel classification for workspace awareness, along with an exploration of visual cues recently employed in research endeavors. Lastly, we present the key findings and shine a spotlight on promising yet unexplored topics. This work not only serves as a reference for experienced researchers seeking to inform the design of their own collaborative XR applications but also extends a welcoming hand to newcomers in this dynamic field. [ABSTRACT FROM AUTHOR]

Full‐body avatar presence is important for immersive social and environmental interactions in digital reality. However, current devices only provide three six degrees of freedom (DOF) poses from the headset and two controllers (i.e. three‐point trackers). Because it is a highly under‐constrained problem, inferring full‐body pose from these inputs is challenging, especially when supporting the full range of body proportions and use cases represented by the general population. In this paper, we propose a deep learning framework, DivaTrack, which outperforms existing methods when applied to diverse body sizes and activities. We augment the sparse three‐point inputs with linear accelerations from Inertial Measurement Units (IMU) to improve foot contact prediction. We then condition the otherwise ambiguous lower‐body pose with the predictions of foot contact and upper‐body pose in a two‐stage model. We further stabilize the inferred full‐body pose in a wide range of configurations by learning to blend predictions that are computed in two reference frames, each of which is designed for different types of motions. We demonstrate the effectiveness of our design on a large dataset that captures 22 subjects performing challenging locomotion for three‐point tracking, including lunges, hula‐hooping, and sitting. As shown in a live demo using the Meta VR headset and Xsens IMUs, our method runs in real‐time while accurately tracking a user's motion when they perform a diverse set of movements. [ABSTRACT FROM AUTHOR]

We present a computational framework for modeling land parcels and streets. In the real world, parcels and streets are highly coupled with each other since a street network connects all the parcels in a certain area. However, existing works model parcels and streets separately to simplify the problem, resulting in urban layouts with irregular parcels and/or suboptimal streets. In this paper, we propose a hierarchical approach to co‐generate parcels and streets from a user‐specified polygonal land shape, guided by a set of fundamental urban design requirements. At each hierarchical level, new parcels are generated based on binary splitting of existing parcels, and new streets are subsequently generated by leveraging efficient graph search tools to ensure that each new parcel has a street access. At the end, we optimize the geometry of the generated parcels and streets to further improve their geometric quality. Our computational framework outputs an urban layout with a desired number of regular parcels that are reachable via a connected street network, for which users are allowed to control the modeling process both locally and globally. Quantitative comparisons with state‐of‐the‐art approaches show that our framework is able to generate parcels and streets that are superior in some aspects. [ABSTRACT FROM AUTHOR]

Explosive volcanic eruptions rank among the most terrifying natural phenomena, and are thus frequently depicted in films, games, and other media, usually with a bespoke once‐off solution. In this paper, we introduce the first general‐purpose model for bi‐directional interaction between the atmosphere and a volcano plume. In line with recent interactive volcano models, we approximate the plume dynamics with Lagrangian disks and spheres and the atmosphere with sparse layers of 2D Eulerian grids, enabling us to focus on the transfer of physical quantities such as temperature, ash, moisture, and wind velocity between these sub‐models. We subsequently generate volumetric animations by noise‐based procedural upsampling keyed to aspects of advection, convection, moisture, and ash content to generate a fully‐realized volcanic skyscape. Our model captures most of the visually salient features emerging from volcano‐sky interaction, such as windswept plumes, enmeshed cap, bell and skirt clouds, shockwave effects, ash rain, and sheathes of lightning visible in the dark. [ABSTRACT FROM AUTHOR]

Designing effective camera trajectories in virtual 3D environments is a challenging task even for experienced animators. Despite an elaborate film grammar, forged through years of experience, that enables the specification of camera motions through cinematographic properties (framing, shots sizes, angles, motions), there are endless possibilities in deciding how to place and move cameras with characters. Dealing with these possibilities is part of the complexity of the problem. While numerous techniques have been proposed in the literature (optimization‐based solving, encoding of empirical rules, learning from real examples,…), the results either lack variety or ease of control.In this paper, we propose a cinematographic camera diffusion model using a transformer‐based architecture to handle temporality and exploit the stochasticity of diffusion models to generate diverse and qualitative trajectories conditioned by high‐level textual descriptions. We extend the work by integrating keyframing constraints and the ability to blend naturally between motions using latent interpolation, in a way to augment the degree of control of the designers. We demonstrate the strengths of this text‐to‐camera motion approach through qualitative and quantitative experiments and gather feedback from professional artists. The code and data are available at https://github.com/jianghd1996/Camera-control. [ABSTRACT FROM AUTHOR]

Terrain generation methods have long been divided between procedural and physically‐based. Procedural methods build upon the fast evaluation of a mathematical function but suffer from a lack of geological consistency, while physically‐based simulation enforces this consistency at the cost of thousands of iterations unraveling the history of the landscape. In particular, the simulation of the competition between tectonic uplift and fluvial erosion expressed by the stream power law raised recent interest in computer graphics as this allows the generation and control of consistent large‐scale mountain ranges, albeit at the cost of a lengthy simulation. In this paper, we explore the analytical solutions of the stream power law and propose a method that is both physically‐based and procedural, allowing fast and consistent large‐scale terrain generation. In our approach, time is no longer the stopping criterion of an iterative process but acts as the parameter of a mathematical function, a slider that controls the aging of the input terrain from a subtle erosion to the complete replacement by a fully formed mountain range. While analytical solutions have been proposed by the geomorphology community for the 1D case, extending them to a 2D heightmap proves challenging. We propose an efficient implementation of the analytical solutions with a multigrid accelerated iterative process and solutions to incorporate landslides and hillslope processes – two erosion factors that complement the stream power law. [ABSTRACT FROM AUTHOR]

High‐fidelity digital human modeling has become crucial in various applications, including gaming, visual effects and virtual reality. Despite the significant impact of eyelashes on facial aesthetics, their reconstruction and modeling have been largely unexplored. In this paper, we introduce the first data‐driven generative model of eyelashes based on semantic features. This model is derived from real data by introducing a new 3D eyelash reconstruction method based on multi‐view images. The reconstructed data is made available which constitutes the first dataset of 3D eyelashes ever published. Through an innovative extraction process, we determine the features of any set of eyelashes, and present detailed descriptive statistics of human eyelashes shapes. The proposed eyelashes model, which exclusively relies on semantic parameters, effectively captures the appearance of a set of eyelashes. Results show that the proposed model enables interactive, intuitive and realistic eyelashes modeling for non‐experts, enriching avatar creation and synthetic data generation pipelines. [ABSTRACT FROM AUTHOR]

Mesh texture synthesis is a key component in the automatic generation of 3D content. Existing learning‐based methods have drawbacks—either by disregarding the shape manifold during texture generation or by requiring a large number of different views to mitigate occlusion‐related inconsistencies. In this paper, we present a novel surface‐aware approach for mesh texture synthesis that overcomes these drawbacks by leveraging the pre‐trained weights of 2D Convolutional Neural Networks (CNNs) with the same architecture, but with convolutions designed for 3D meshes. Our proposed network keeps track of the oriented patches surrounding each texel, enabling seamless texture synthesis and retaining local similarity to classical 2D convolutions with square kernels. Our approach allows us to synthesize textures that account for the geometric content of mesh surfaces, eliminating discontinuities and achieving comparable quality to 2D image synthesis algorithms. We compare our approach with state‐of‐the‐art methods where, through qualitative and quantitative evaluations, we demonstrate that our approach is more effective for a variety of meshes and styles, while also producing visually appealing and consistent textures on meshes. [ABSTRACT FROM AUTHOR]

Many animals, plants or gems exhibit iridescent material appearance in nature. These are due to specific geometric structures at scales comparable to visible wavelengths, yielding so‐called structural colors. The most vivid examples are due to photonic crystals, where a same structure is repeated in one, two or three dimensions, augmenting the magnitude and complexity of interference effects. In this paper, we study the appearance of 1D photonic crystals (repetitive pairs of thin films), also called Bragg mirrors. Previous work has considered the effect of multiple thin films using the classical transfer matrix approach, which increases in complexity when the number of repetitions increases. Our first contribution is to introduce a more efficient closed‐form reflectance formula [Yeh88] for Bragg mirror reflectance to the Graphics community, as well as an approximation that lends itself to efficient spectral integration for RGB rendering. We then explore the appearance of stacks made of rough Bragg layers. Here our contribution is to show that they may lead to a ballistic transmission, significantly speeding up position‐free rendering and leading to an efficient single‐reflection BRDF model. [ABSTRACT FROM AUTHOR]

Mesh denoising is a fundamental yet challenging task. Most of the existing data‐driven methods only consider the zero‐order information (vertex location) and first‐order information (face normal). However, higher‐order geometric information (such as curvature) is more descriptive for the shape of the mesh. Therefore, in order to impose such high‐order information, this paper proposes a novel Curvature‐Driven Multi‐Stream Graph Convolutional Neural Network (CDMS‐Net) architecture. CDMS‐Net has three streams, including curvature stream, face normal stream and vertex stream, where the curvature stream focuses on the high‐order Gaussian curvature information. Moreover, CDMS‐Net proposes a novel block based on residual dense connections, which is used as the core component to extract geometric features from meshes. This innovative design improves the performance of feature‐preserving denoising. The plug‐and‐play modular design makes CDMS‐Net easy to be implemented. Multiple sets of ablation study are carried out to verify the rationality of the CDMS‐Net. Our method establishes new state‐of‐the‐art mesh denoising results on publicly available datasets. [ABSTRACT FROM AUTHOR]

In this paper, we study rendering of end‐to‐end compressed triangle meshes using modern GPU techniques, in particular, mesh shaders. Our approach allows us to keep unstructured triangle meshes in GPU memory in compressed form and decompress them in shader code just in time for rasterization. Typical previous approaches use a compressed mesh format only for persistent storage and streaming, but must decompress it into GPU memory before submitting it to rendering. In contrast, our approach uses an identical compressed format in both storage and GPU memory. Hence, our compression method effectively reduces the in‐memory requirements of huge triangular meshes and avoids any waiting times on streaming geometry induced by the need for a decompression stage on the CPU. End‐to‐end compression also means that scenes with more geometric detail than previously possible can be made fully resident in GPU memory. Our approach is based on a novel decomposition of meshes into meshlets, i.e. disjoint primitive groups that are compressed individually. Decompression using a mesh shader allows de facto random access on the primitive level, which is important for applications such as selective streaming and fine‐grained visibility computation. We compare our approach to multiple commonly used compressed meshlet formats in terms of required memory and rendering times. The results imply that our approach reduces the required CPU–GPU memory bandwidth, a frequent bottleneck in out‐of‐core rendering. [ABSTRACT FROM AUTHOR]

In this paper, we present a method for creating Escher‐like spherical patterns with regular polyhedron symmetries. Using the generators of the symmetry groups associated with regular polyhedra, we first provide fast algorithms to construct spherical tilings. Then, to obtain Escher‐like patterns, we specify texturing techniques to decorate the resulting tilings. Moreover, we present a strategy to create a novel dynamic effect of Escher‐like kaleidoscopes in which the motifs have a complete body. The method has the advantages of simple implementation, fast calculation, good graphics, and artistic effects, which can be used to create rich elegant spherical patterns. [ABSTRACT FROM AUTHOR]

This paper investigates super‐resolution to reduce the number of pixels to render and thus speed up Monte Carlo rendering algorithms. While great progress has been made to super‐resolution technologies, it is essentially an ill‐posed problem and cannot recover high‐frequency details in renderings. To address this problem, we exploit high‐resolution auxiliary features to guide super‐resolution of low‐resolution renderings. These high‐resolution auxiliary features can be quickly rendered by a rendering engine and at the same time provide valuable high‐frequency details to assist super‐resolution. To this end, we develop a cross‐modality transformer network that consists of an auxiliary feature branch and a low‐resolution rendering branch. These two branches are designed to fuse high‐resolution auxiliary features with the corresponding low‐resolution rendering. Furthermore, we design Residual Densely Connected Swin Transformer groups to learn to extract representative features to enable high‐quality super‐resolution. Our experiments show that our auxiliary features‐guided super‐resolution method outperforms both super‐resolution methods and Monte Carlo denoising methods in producing high‐quality renderings. [ABSTRACT FROM AUTHOR]

The real world consists of mass phenomena, such as iridescence on thin film and metal oxide layers, that is only explicable by wave optics. Existing research can reproduce such effects with simple point lights or low‐frequency environmental lighting. However, it remains a difficult task to efficiently rendering these effects when near‐field, high‐frequency area lights are involved. This paper presents a high‐fidelity, real‐time rendering algorithm for the iridescence effect under polygonal lights. We introduce a novel set of spherical functions, Monomial‐Gaussians, to accurately fit iridescent materials' reflectance. With a precomputed lookup table, the Monomial‐Gaussians are easily integrated over spherical polygons in linear time. Importance sampling of Monomial‐Gaussians is also supported to efficiently reduce Monte‐Carlo error. Our approach produces accurate renderings of the iridescence effect while still preserving high frame rates. [ABSTRACT FROM AUTHOR]