Search

Real‐time global illumination is a highly desirable yet challenging task in computer graphics. Existing works well solving this problem are mostly based on some kind of precomputed data (caches), while the final results depend significantly on the quality of the caches. In this paper, we propose a learning‐based pipeline that can reproduce a wide range of complex light transport phenomena, including high‐frequency glossy interreflection, at any viewpoint in real time (> 90 frames per‐second), using information from imperfect caches stored at the barycentre of every triangle in a 3D scene. These caches are generated at a precomputation stage by a physically‐based offline renderer at a low sampling rate (e.g., 32 samples per‐pixel) and a low image resolution (e.g., 64×16). At runtime, a deep radiance reconstruction method based on a dedicated neural network is then involved to reconstruct a high‐quality radiance map of full global illumination at any viewpoint from these imperfect caches, without introducing noise and aliasing artifacts. To further improve the reconstruction accuracy, a new feature fusion strategy is designed in the network to better exploit useful contents from cheap G‐buffers generated at runtime. The proposed framework ensures high‐quality rendering of images for moderate‐sized scenes with full global illumination effects, at the cost of reasonable precomputation time. We demonstrate the effectiveness and efficiency of the proposed pipeline by comparing it with alternative strategies, including real‐time path tracing and precomputed radiance transfer. [ABSTRACT FROM AUTHOR]

Existing stroke‐based painting synthesis methods usually fail to achieve good results with limited strokes because these methods use semantically irrelevant metrics to calculate the similarity between the painting and photo domains. Hence, it is hard to see meaningful semantical information from the painting. This paper proposes a painting synthesis method that uses a CLIP (Contrastive‐Language‐Image‐Pretraining) model to build a semantically‐aware metric so that the cross‐domain semantic similarity is explicitly involved. To ensure the convergence of the objective function, we design a new strategy called decremental optimization. Specifically, we define painting as a set of strokes and use a neural renderer to obtain a rasterized painting by optimizing the stroke control parameters through a CLIP‐based loss. The optimization process is initialized with an excessive number of brush strokes, and the number of strokes is then gradually reduced to generate paintings of varying levels of abstraction. Experiments show that our method can obtain vivid paintings, and the results are better than the comparison stroke‐based painting synthesis methods when the number of strokes is limited. [ABSTRACT FROM AUTHOR]

Single image superresolution (SISR) has achieved substantial progress based on deep learning. Many SISR methods acquire pairs of low‐resolution (LR) images from their corresponding high‐resolution (HR) counterparts. Being unsupervised, this kind of method also demands large‐scale training data. However, these paired images and a large amount of training data are difficult to obtain. Recently, several internal, learning‐based methods have been introduced to address this issue. Although requiring a large quantity of training data pairs is solved, the ability to improve the image resolution is limited if only the information of the LR image itself is applied. Therefore, we further expand this kind of approach by using similar HR reference images as prior knowledge to assist the single input image. In this paper, we proposed zero‐shot single image superresolution with a reference image (Ref‐ZSSR). First, we use an unconditional generative model to learn the internal distribution of the HR reference image. Second, a dual‐path architecture that contains a downsampler and an upsampler is introduced to learn the mapping between the input image and its downscaled image. Finally, we combine the reference image learning module and dual‐path architecture module to train a new generative model that can generate a superresolution (SR) image with the details of the HR reference image. Such a design encourages a simple and accurate way to transfer relevant textures from the reference high‐definition (HD) image to LR image. Compared with using only the image itself, the HD feature of the reference image improves the SR performance. In the experiment, we show that the proposed method outperforms previous image‐specific network and internal learning‐based methods. [ABSTRACT FROM AUTHOR]

Recent methods on image denoising have achieved remarkable progress, benefiting mostly from supervised learning on massive noisy/clean image pairs and unsupervised learning on external noisy images. However, due to the domain gap between the training and testing images, these methods typically have limited applicability on unseen images. Although several attempts have been made to avoid the domain gap issue by learning denoising from singe noisy image itself, they are less effective in handling real‐world noise because of assuming the noise corruptions are independent and zero mean. In this paper, we go step further beyond prior work by presenting a novel unsupervised image denoising framework trained from single noisy image without making any explicit assumptions on the noise statistics. Our approach is built upon the deep image prior (DIP), which enables diverse image restoration tasks. However, as is, the denoising performance of DIP will significantly deteriorate on nonzero‐mean noise and is sensitive to the number of iterations. To overcome this problem, we propose to utilize multi‐scale deep image prior by imposing DIP across different image scales under the constraint of a scale consistency. Experiments on synthetic and real datasets demonstrate that our method performs favorably against the state‐of‐the‐art methods for image denoising. [ABSTRACT FROM AUTHOR]

Evaluating shading functions on geometry surfaces dominates the rendering computation. A high‐quality but time‐consuming estimate is usually achieved with a dense sampling rate for pixels or sub‐pixels. In this paper, we leverage sparsely sampled points on vertices of dynamically‐generated subdivision surfaces to approximate the ground‐truth shading signal by piecewise linear reconstruction. To control the introduced interpolation error at runtime, we analytically derive an L∞ error bound and compute the optimal subdivision surfaces based on a user‐specified error threshold. We apply our analysis on multiple shading functions including Lambertian, Blinn‐Phong, Microfacet BRDF and also extend it to handle textures, yielding easy‐to‐compute formulas. To validate our derivation, we design a forward multirate shading algorithm powered by hardware tessellator that moves shading computation at pixels to the vertices of subdivision triangles on the fly. We show our approach significantly reduces the sampling rates on various test cases, reaching a speedup ratio of 134% ~ 283% compared to dense per‐pixel shading in current graphics hardware. [ABSTRACT FROM AUTHOR]

In this paper we propose a novel and efficient rasterization‐based approach for direct rendering of isosurfaces. Our method exploits the capabilities of task and mesh shader pipelines to identify subvolumes containing potentially visible isosurface geometry, and to efficiently extract primitives which are consumed on the fly by the rasterizer. As a result, our approach requires little preprocessing and negligible additional memory. Direct isosurface rasterization is competitive in terms of rendering performance when compared with ray‐marching‐based approaches, and significantly outperforms them for increasing resolution in most situations. Since our approach is entirely rasterization based, it affords straightforward integration into existing rendering pipelines, while allowing the use of modern graphics hardware features, such as multi‐view stereo for efficient rendering of stereoscopic image pairs for geometry‐bound applications. Direct isosurface rasterization is suitable for applications where isosurface geometry is highly variable, such as interactive analysis scenarios for static and dynamic data sets that require frequent isovalue adjustment. [ABSTRACT FROM AUTHOR]

Memory performance is a crucial bottleneck in many GPGPU applications, making optimizations for hardware and software mandatory. While hardware vendors already use highly efficient caching architectures, software engineers usually have to organize their data accordingly in order to efficiently make use of these, requiring deep knowledge of the actual hardware. In this paper we present a novel technique for fine‐grained memory profiling that simulates the whole pipeline of memory flow and finally accumulates profiling values in a way that the user retains information about the potential region in the GPU program by showing these values separately for each allocation. Our memory simulator turns out to outperform state‐of‐the‐art memory models of NVIDIA architectures by a magnitude of 2.4 for the L1 cache and 1.3 for the L2 cache, in terms of accuracy. Additionally, we find our technique of fine grained memory profiling a useful tool for memory optimizations, which we successfully show in case of ray tracing and machine learning applications. [ABSTRACT FROM AUTHOR]

While motion compensation greatly improves video deblurring quality, separately performing motion compensation and video deblurring demands huge computational overhead. This paper proposes a real‐time video deblurring framework consisting of a lightweight multi‐task unit that supports both video deblurring and motion compensation in an efficient way. The multi‐task unit is specifically designed to handle large portions of the two tasks using a single shared network and consists of a multi‐task detail network and simple networks for deblurring and motion compensation. The multi‐task unit minimizes the cost of incorporating motion compensation into video deblurring and enables real‐time deblurring. Moreover, by stacking multiple multi‐task units, our framework provides flexible control between the cost and deblurring quality. We experimentally validate the state‐of‐the‐art deblurring quality of our approach, which runs at a much faster speed compared to previous methods and show practical real‐time performance (30.99dB@30fps measured on the DVD dataset). [ABSTRACT FROM AUTHOR]

Implicit neural representations, such as MLP, can well recover the topology of watertight object. However, MLP fails to recover geometric details of watertight object and complicated topology due to dealing with point cloud in a point‐wise manner. In this paper, we propose a point cloud transformer called local offset point cloud transformer (LOPCT) as a feature fusion module. Before using MLP to learn the implicit function, the input point cloud is first fed into the local offset transformer, which adaptively learns the dependency of the local point cloud and obtains the enhanced features of each point. The feature‐enhanced point cloud is then fed into the MLP to recover the geometric details and sharp features of watertight object and complex topology. Extensive reconstruction experiments of watertight object and complex topology demonstrate that our method achieves comparable or better results than others in terms of recovering sharp features and geometric details. In addition, experiments on watertight objects demonstrate the robustness of our method in terms of average result. [ABSTRACT FROM AUTHOR]

Point cloud generation aims to synthesize point clouds that do not exist in supervised dataset. Generating a point cloud with certain semantic labels remains an under‐explored problem. This paper proposes a formulation called DiffusionPointLabel, which completes point‐label pair generation based on a DDPM generative model (Denoising Diffusion Probabilistic Model). Specifically, we use a point cloud diffusion generative model and aggregate the intermediate features of the generator. On top of this, we propose Feature Interpreter that transforms intermediate features into semantic labels. Furthermore, we employ an uncertainty measure to filter unqualified point‐label pairs for a better quality of generated point cloud dataset. Coupling these two designs enables us to automatically generate annotated point clouds, especially when supervised point‐labels pairs are scarce. Our method extends the application of point cloud generation models and surpasses state‐of‐the‐art models. [ABSTRACT FROM AUTHOR]

With the rapid development of data‐driven techniques, data has played an essential role in various computer vision tasks. Many realistic and synthetic datasets have been proposed to address different problems. However, there are lots of unresolved challenges: (1) the creation of dataset is usually a tedious process with manual annotations, (2) most datasets are only designed for a single specific task, (3) the modification or randomization of the 3D scene is difficult, and (4) the release of commercial 3D data may encounter copyright issue. This paper presents MINERVAS, a Massive INterior EnviRonments VirtuAl Synthesis system, to facilitate the 3D scene modification and the 2D image synthesis for various vision tasks. In particular, we design a programmable pipeline with Domain‐Specific Language, allowing users to select scenes from the commercial indoor scene database, synthesize scenes for different tasks with customized rules, and render various types of imagery data, such as color images, geometric structures, semantic labels. Our system eases the difficulty of customizing massive scenes for different tasks and relieves users from manipulating fine‐grained scene configurations by providing user‐controllable randomness using multilevel samplers. Most importantly, it empowers users to access commercial scene databases with millions of indoor scenes and protects the copyright of core data assets, e.g., 3D CAD models. We demonstrate the validity and flexibility of our system by using our synthesized data to improve the performance on different kinds of computer vision tasks. The project page is at https://coohom.github.io/MINERVAS. [ABSTRACT FROM AUTHOR]

Existing methods for skeleton extraction have limitations in terms of the amount of memory space available, as the model must be allocated to the random access memory. This challenges the treatment of out‐of‐core models. Although applying out‐of‐core simplification methods to the model can fit in memory, this would induce distortion of the model surface, and so causing the skeleton to be off‐centered or changing the topological structure. In this paper, we propose an efficient out‐of‐core method for extracting skeletons from large volumetric models. The method takes a volumetric model as input and first computes an out‐of‐core distance transform. With the distance transform, we generate a medial mesh to capture the prominent features for skeleton extraction, which significantly reduces the data size and facilitates the process of large models. At last, we contract the medial mesh in an out‐of‐core fashion to generate the skeleton. Experimental results show that our method can efficiently extract high‐quality curve skeletons from large volumetric models with small memory usage. [ABSTRACT FROM AUTHOR]

Temporal networks can naturally model real‐world complex phenomena such as contact networks, information dissemination and physical proximity. However, nodes and edges bear real‐time coordinates, making it difficult to organize them into discrete timeslices, without a loss of temporal information due to projection. Event‐based dynamic graph drawing rejects the notion of a timeslice and allows each node and edge to retain its own real‐valued time coordinate. While existing work has demonstrated clear advantages for this approach, they come at a running time cost. We investigate the problem of accelerating event‐based layout to make it more competitive with existing layout techniques. In this paper, we describe the design, implementation and experimental evaluation of MultiDynNoS, the first multi‐level event‐based graph layout algorithm. We consider three operators for coarsening and placement, inspired by Walshaw, GRIP and FM3, which we couple with an event‐based graph drawing algorithm. We also propose two extensions to the core algorithm: AutoTau and Bend Transfer. We perform two experiments: first, we compare MultiDynNoS variants to existing state‐of‐the‐art dynamic graph layout approaches; second, we investigate the impact of each of the proposed algorithm extensions. MultiDynNoS proves to be competitive with existing approaches, and the proposed extensions achieve their design goals and contribute in opening new research directions. [ABSTRACT FROM AUTHOR]

Despite the potential of neural scene representations to effectively compress 3D scalar fields at high reconstruction quality, the computational complexity of the training and data reconstruction step using scene representation networks limits their use in practical applications. In this paper, we analyse whether scene representation networks can be modified to reduce these limitations and whether such architectures can also be used for temporal reconstruction tasks. We propose a novel design of scene representation networks using GPU tensor cores to integrate the reconstruction seamlessly into on‐chip raytracing kernels, and compare the quality and performance of this network to alternative network‐ and non‐network‐based compression schemes. The results indicate competitive quality of our design at high compression rates, and significantly faster decoding times and lower memory consumption during data reconstruction. We investigate how density gradients can be computed using the network and show an extension where density, gradient and curvature are predicted jointly. As an alternative to spatial super‐resolution approaches for time‐varying fields, we propose a solution that builds upon latent‐space interpolation to enable random access reconstruction at arbitrary granularity. We summarize our findings in the form of an assessment of the strengths and limitations of scene representation networks for compression domain volume rendering, and outline future research directions. Source code: https://github.com/shamanDevel/fV‐SRN [ABSTRACT FROM AUTHOR]

Monoscopic panorama provides the display of omnidirectional contents surrounding the viewer. An increasingly popular way to reconstruct a panorama is to stitch a collection of fisheye images. However, such non‐planar views may result in problems such as distortions and boundary irregularities. In most cases, the computational expense for stitching non‐planar images is also too high to satisfy real‐time applications. In this paper, a novel monoscopic panorama reconstruction pipeline that produces better quad‐fisheye image stitching results for omnidirectional environment viewing is proposed. The main idea is to apply mesh deformation for image alignment. To optimize inter‐lens parallaxes, unwarped images are firstly cropped and reshuffled to facilitate the circular environment scene composition by the seamless ring‐connection of the panorama borders. Several mesh constraints are then adopted to ensure a high alignment accuracy. After alignment, the boundary of the result is rectified to be rectangular to prevent gapping artefacts. We further extend our approach to video stitching. The temporal smoothness model is added to prevent unexpected artefacts in the panoramic videos. To support interactive applications, our stitching algorithm is programmed using CUDA. The camera motion and average gradient per video frame are further calculated to accelerate for synchronous real‐life panoramic scene reconstruction and visualization. Experimental results demonstrate that our method has advantages in respects of alignment accuracy, adaptability and image quality of the stitching result. [ABSTRACT FROM AUTHOR]

This paper presents a fast method to compute large‐scale problems in real‐time finite element simulations in the presence of contact and friction. The approach uses a precondition‐based contact resolution that performs a Cholesky decomposition at low frequency. On exploiting the sparsity in assembled matrices, we propose a reduced and parallel computation scheme to address the expensive computation of the Schur‐complement arisen by detailed mesh and accurate contact response. An efficient GPU‐based solver is developed to parallelise the computation, making it possible to provide real‐time simulations in the presence of coupled constraints for contact and friction response. In addition, the pre‐conditioner is updated at low frequency, implying reuse of the factorised system. To benefit a further speedup, we propose a strategy to share the resolution information between consecutive time steps. We evaluate the performance of our method in different contact applications and compare it with typical approaches on CPU and GPU. [ABSTRACT FROM AUTHOR]

In this paper, we introduce harmonics virtual lights (HVL), to model indirect light sources for interactive global illumination of dynamic 3D scenes. Virtual point lights (VPL) are an efficient approach to define indirect light sources and to evaluate the resulting indirect lighting. Nonetheless, VPL suffer from disturbing artefacts, especially with high‐frequency materials. Virtual spherical lights (VSL) avoid these artefacts by considering spheres instead of points but estimates the lighting integral using Monte‐Carlo which results to noise in the final image. We define HVL as an extension of VSL in a spherical harmonics (SH) framework, defining a closed form of the lighting integral evaluation. We propose an efficient SH projection of spherical lights contribution faster than existing methods. Computing the outgoing luminance requires O(n)$\mathcal {O}(n)$ operations when using materials with circular symmetric lobes, and O(n2)$\mathcal {O}(n^2)$ operations for the general case, where n is the number of SH bands. HVL can be used with either parametric or measured BRDF without extra cost and offers control over rendering time and image quality, by either decreasing or increasing the band limit used for SH projection. Our approach is particularly well‐designed to render medium‐frequency one‐bounce global illumination with arbitrary BRDF at an interactive frame rate. [ABSTRACT FROM AUTHOR]

Many recent works have reconstructed distinctive 3D face shapes by aggregating shape parameters of the same identity and separating those of different people based on parametric models (e.g. 3D morphable models (3DMMs)). However, despite the high accuracy in the face recognition task using these shape parameters, the visual discrimination of face shapes reconstructed from those parameters remains unsatisfactory. Previous works have not answered the following research question: Do discriminative shape parameters guarantee visual discrimination in represented 3D face shapes? This paper analyses the relationship between shape parameters and reconstructed shape geometry, and proposes a novel shape identity‐aware regularization (SIR) loss for shape parameters, aiming at increasing discriminability in both the shape parameter and shape geometry domains. Moreover, to cope with the lack of training data containing both landmark and identity annotations, we propose a network structure and an associated training strategy to leverage mixed data containing either identity or landmark labels. In addition, since face recognition accuracy does not mean the recognizability of reconstructed face shapes from the shape parameters, we propose the SIR metric to measure the discriminability of face shapes. We compare our method with existing methods in terms of the reconstruction error, visual discriminability, and face recognition accuracy of the shape parameters and SIR metric. Experimental results show that our method outperforms the state‐of‐the‐art methods. The code will be released at https://github.com/a686432/SIR. [ABSTRACT FROM AUTHOR]

Spatially varying bi‐directional reflectance distribution functions (SVBRDFs) are crucial for designers to incorporate new materials in virtual scenes, making them look more realistic. Reconstruction of SVBRDFs is a long‐standing problem. Existing methods either rely on an extensive acquisition system or require huge datasets, which are non‐trivial to acquire. We aim to recover SVBRDFs from a single image, without any datasets. A single image contains incomplete information about the SVBRDF, making the reconstruction task highly ill‐posed. It is also difficult to separate between the changes in colour that are caused by the material and those caused by the illumination, without the prior knowledge learned from the dataset. In this paper, we use an unsupervised generative adversarial neural network (GAN) to recover SVBRDFs maps with a single image as input. To better separate the effects due to illumination from the effects due to the material, we add the hypothesis that the material is stationary and introduce a new loss function based on Fourier coefficients to enforce this stationarity. For efficiency, we train the network in two stages: reusing a trained model to initialize the SVBRDFs and fine‐tune it based on the input image. Our method generates high‐quality SVBRDFs maps from a single input photograph, and provides more vivid rendering results compared to the previous work. The two‐stage training boosts runtime performance, making it eight times faster than the previous work. [ABSTRACT FROM AUTHOR]

Efficient approximation of incident radiance functions from a set of samples is still an open problem in physically based rendering. Indeed, most of the computing power required to synthesize a photo‐realistic image is devoted to collecting samples of the incident radiance function, which are necessary to provide an estimate of the rendering equation solution. Due to the large number of samples required to reach a high‐quality estimate, this process is usually tedious and can take up to several days. In this paper, we focus on the problem of approximation of incident radiance functions on the S2$\mathbb {S}^2$ sphere. To this end, we resort to a Gaussian Process (GP), a highly flexible function modelling tool, which has received little attention in rendering. We make an extensive analysis of the application of GPs to incident radiance functions, addressing crucial issues such as robust hyperparameter learning, or selecting the covariance function which better suits incident radiance functions. Our analysis is both theoretical and experimental. Furthermore, it provides a seamless connection between the original spherical domain and the spectral domain, on which we build to derive a method for fast computation and rotation of spherical harmonics coefficients. [ABSTRACT FROM AUTHOR]

This paper presents a novel and practical screen‐space liquid rendering for particle‐based fluids for real‐time applications. Our rendering pipeline performs particle filtering only in a narrow‐band around the boundary particles to provide a smooth liquid surface with volumetric rendering effects. We also introduce a novel boundary detection method allowing the user to select particle layers from the liquid interface. The proposed approach is simple, fast, memory‐efficient, easy to code and it can be adapted straightforwardly in the standard screen‐space rendering methods, even in GPU architectures. We show through a set of experiments how the prior screen‐space techniques can be benefited and improved by our approach. [ABSTRACT FROM AUTHOR]

Image triangulation aims to generate an optimal partition with triangular elements to represent the given image. One bottleneck in ensuring approximation quality between the original image and a piecewise approximation over the triangulation is the inaccurate alignment of straight edges to the curved features. In this paper, we propose a novel variational method called curved optimal triangulation, where not all edges are straight segments, but may also be quadratic Bézier curves. The energy function is defined as the total approximation error determined by vertex locations, connectivity and bending of edges. The gradient formulas of this function are derived explicitly in closed form to optimize the energy function efficiently. We test our method on several models to demonstrate its efficacy and ability in preserving features. We also explore its applications in the automatic generation of stylization and Lowpoly images. With the same number of vertices, our curved optimal triangulation method generates more accurate and visually pleasing results compared with previous methods that only use straight segments. [ABSTRACT FROM AUTHOR]

Specifically, I'll present deep learning methods for body pose estimation, hand pose estimation, fiducial landmark detection, and eye gaze estimation and redirection. By matching the paper research topics with the expertise and preferences of committee members, the program co-chairs assigned one primary and two secondary reviewers from the Program Committee to each paper. The 29th Pacific Conference on Computer Graphics and Applications Online Event Wellington, New Zealand October 18-21, 2021 B General Co-Chairs b Taehyun Rhee, Victoria University of Wellington Shi-Min Hu, Tsinghua University Holly Rushmeier, Yale University B Program Co-Chairs b Fang-Lue Zhang, Victoria University of Wellington Elmar Eisemann, Delft University of Technology Karan Singh, University of Toronto B Conference Paper Co-Chairs b Sung-Hee Lee, KAIST Stefanie Zollmann, University of Otago Makoto Okabe, Shizuoka University Burkhard Wünsche, The University of Auckland B Local Co-Chairs b Alex Doronin, Victoria University of Wellington Neil Dodgson, Victoria University of Wellington Andrew Chalmers, Victoria University of Wellington Sponsors GLO:BNX/01oct21:cgf14422-gra-0001.jpg PHOTO (COLOR): . gl Preface The 28th and 29th Pacific Conference on Computer Graphics and Applications (Pacific Graphics 20+21) are jointly hosted in Wellington, New Zealand. [Extracted from the article]

We survey the state of the art of spatial interfaces for 3D visualization. Interaction techniques are crucial to data visualization processes and the visualization research community has been calling for more research on interaction for years. Yet, research papers focusing on interaction techniques, in particular for 3D visualization purposes, are not always published in visualization venues, sometimes making it challenging to synthesize the latest interaction and visualization results. We therefore introduce a taxonomy of interaction technique for 3D visualization. The taxonomy is organized along two axes: the primary source of input on the one hand and the visualization task they support on the other hand. Surveying the state of the art allows us to highlight specific challenges and missed opportunities for research in 3D visualization. In particular, we call for additional research in: (1) controlling 3D visualization widgets to help scientists better understand their data, (2) 3D interaction techniques for dissemination, which are under‐explored yet show great promise for helping museum and science centers in their mission to share recent knowledge, and (3) developing new measures that move beyond traditional time and errors metrics for evaluating visualizations that include spatial interaction. [ABSTRACT FROM AUTHOR]

Modeling 3D objects on existing software usually requires a heavy amount of interactions, especially for users who lack basic knowledge of 3D geometry. Sketch‐based modeling is a solution to ease the modelling procedure and thus has been researched for decades. However, modelling a man‐made shape with complex structures remains challenging. Existing methods adopt advanced deep learning techniques to map holistic sketches to 3D shapes. They are still bottlenecked to deal with complicated topologies. In this paper, we decouple the task of sketch2shape into a part generation module and a part assembling module, where deep learning methods are leveraged for the implementation of both modules. By changing the focus from holistic shapes to individual parts, it eases the learning process of the shape generator and guarantees high‐quality outputs. With the learned automated part assembler, users only need a little manual tuning to obtain a desired layout. Extensive experiments and user studies demonstrate the usefulness of our proposed system. [ABSTRACT FROM AUTHOR]

This paper introduces a precise and easy to use method for defining the microstructure of heterogeneous solids. This method is based on the concept of Heterogeneous Composite Bézier Hyperpatch, and allows to accurately represent the primary material proportions, as well as the size and shape of the material phases. The solid microstructure is modelled using two functions: a material distribution function (to compute the portion of the solid volume occupied by each primary material), and a modified Perlin noise function that determines the shape and size of each primary material phase. With this method, the position and orientation of the solid in the modeling space R3 does not affect the portion of its volume that is occupied by each primary material, nor the shape and size of the phases. However, the solid microstructure is coherently and automatically modified when the shape of the solid is edited. Regarding continuity, this method allows to define to which extent continuity (both in shape and material distribution) has to be preserved at the junction of the cells that compose the solid. This makes modeling geometrically complex figures very easy. [ABSTRACT FROM AUTHOR]

Visually appealing and vivid simulations of deformable solids represent an important aspect of physically based computer animation. For the temporal discretization, it is customary in computer animation to use first‐order accurate integration methods, such as Backward Euler, due to their simplicity and robustness. Although there is notable research on second‐order methods, their use is not widespread. Many of these well‐known methods have significant drawbacks such as severe numerical damping or scene‐dependent time step restrictions to ensure stability. In this paper, we discuss the most relevant requirements on such methods in computer animation and motivate the interest beyond first‐order accuracy. Keeping these requirements in mind, we investigate several promising methods from the families of diagonally implicit Runge‐Kutta (DIRK) and Rosenbrock methods which currently do not appear to have considerable popularity in this field. We show that the usage of such methods improves the visual quality of physical animations. In addition, we demonstrate that they allow distinctly more control over damping at lower computational cost than classical methods. As part of our theoretical contribution, we review aspects of simulations that are often considered more intricate with higher‐order methods, such as contact handling. To this end, we derive an implicit linearized contact model based on a predictor‐corrector approach that leads to consistent behavior with higher‐order integrators as predictors. Our contact model is well suited for the simulation of stiff, nonlinear materials with the integration methods presented in this paper and more common methods such as Backward Euler alike. [ABSTRACT FROM AUTHOR]

Focuses on the ninth European Association for Computer Graphics Workshop on Visualization in Scientific Computing, held at the Heinrich Fabri Institute in Blaubeuren, Germany from April 20 to 22, 1998. Highlights of the event; Topics covered by the sessions; Details on the papers presented during the sessions.

Virtual point lights (VPLs) are well established for real-time global illumination. However, this method suffers from spiky artifacts and flickering caused by singularities of VPLs, highly glossy materials, high-frequency textures, and discontinuous geometries. To avoid these artifacts, this paper introduces a virtual spherical Gaussian light (VSGL) which roughly represents a set of VPLs. For a VSGL, the total radiant intensity and positional distribution of VPLs are approximated using spherical Gaussians and a Gaussian distribution, respectively. Since this approximation can be computed using summations of VPL parameters, VSGLs can be dynamically generated using mipmapped reflective shadow maps. Our VSGL generation is simple and independent from any scene geometries. In addition, reflected radiance for a VSGL is calculated using an analytic formula. Hence, we are able to render one-bounce glossy interreflections at real-time frame rates with smaller artifacts. [ABSTRACT FROM AUTHOR]

Although geometry-aware filtering and upsampling have often been used for interactive or real-time rendering, they are unsuitable for glossy surfaces because shading results strongly depend on the bidirectional reflectance distribution functions. This paper proposes a novel weighting function of cross bilateral filtering and upsampling to measure the similarity of specular lobes. The difficulty is that a specular lobe is represented with a distribution function in directional space, whereas conventional cross bilateral filtering evaluates similarities using the distance between two points in a Euclidean space. Therefore, this paper first generalizes cross bilateral filtering for the similarity of distribution functions in a non-Euclidean space. Then, the weighting function is specialized for specular lobes. Our key insight is that the weighting function of bilateral filtering can be represented with the product integral of two distribution functions corresponding to two pixels. In addition, we propose spherical Gaussianbased approximations to calculate this weighting function analytically. Our weighting function detects the edges of glossiness, and adapts to all-frequency materials using only a camera position and G-buffer. These features are not only suitable for path tracing, but also deferred shading and non-ray tracing-based methods such as voxel cone tracing. [ABSTRACT FROM AUTHOR]

The article introduces a special issue of "Computer Graphics Forum" that contains 56 conference papers presented at the 30th Annual Conference of the Eurographics Association in München, Germany, from March 30-April 3, 2009. It includes a list of reviewers, an index of authors, and a table of contents for the issue. It also lists the organizers and sponsors of the conference. A preface describes in detail the process by which the conference papers were selected, and presents a list of acknowledgments.

This issue marks the start of a new volume (volume 39) of I Computer Graphics Forum i (CGF). We are very glad about this development and thank all CGF authors, the reviewers and associate editors, the paper chairs of CGF-associated conferences, and our colleagues at Wiley for strengthening this journal. Victor Ostromoukhov, Université de Lyon 1, CNRS-LIRIS. [Extracted from the article]

Detecting and removing raindrops from an image while keeping the high quality of image details has attracted tremendous studies, but remains a challenging task due to the inhomogeneity of the degraded region and the complexity of the degraded intensity. In this paper, we get rid of the dependence of deep learning on image‐to‐image translation and propose a separation‐restoration‐fusion network for raindrops removal. Our key idea is to recover regions of different damage levels individually, so that each region achieves the optimal recovery result, and finally fuse the recovered areas. In the region restoration module, to complete the restoration of a specific area, we propose a multi‐scale feature fusion global information aggregation attention network to achieve global to local information aggregation. Besides, we also design an inside and outside dense connection dilated network, to ensure the fusion of the separated regions and the fine restoration of the image. The qualitatively and quantitatively evaluations are conducted to evaluate our method with the latest existing methods. The result demonstrates that our method outperforms state‐of‐the‐art methods by a large margin on the benchmark datasets in extensive experiments. [ABSTRACT FROM AUTHOR]

Gradient‐domain rendering can highly improve the convergence of light transport simulation using the smoothness in image space. These methods generate image gradients and solve an image reconstruction problem with rendered image and the gradient images. Recently, a previous work proposed a gradient‐domain volumetric photon density estimation for homogeneous participating media. However, the image reconstruction relies on traditional L1 reconstruction, which leads to obvious artifacts when only a few rendering passes are performed. Deep learning based reconstruction methods have been exploited for surface rendering, but they are not suitable for volume density estimation. In this paper, we propose an unsupervised neural network for image reconstruction of gradient‐domain volumetric photon density estimation, more specifically for volumetric photon mapping, using a variant of GradNet with an encoded shift connection and a separated auxiliary feature branch, which includes volume based auxiliary features such as transmittance and photon density. Our network smooths the images on global scale and preserves the high frequency details on a small scale. We demonstrate that our network produces a higher quality result, compared to previous work. Although we only considered volumetric photon mapping, it's straightforward to extend our method for other forms, like beam radiance estimation. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel formulation extending convolutional neural networks (CNN) to arbitrary two‐dimensional manifolds using orthogonal basis functions called Zernike polynomials. In many areas, geometric features play a key role in understanding scientific trends and phenomena, where accurate numerical quantification of geometric features is critical. Recently, CNNs have demonstrated a substantial improvement in extracting and codifying geometric features. However, the progress is mostly centred around computer vision and its applications where an inherent grid‐like data representation is naturally present. In contrast, many geometry processing problems deal with curved surfaces and the application of CNNs is not trivial due to the lack of canonical grid‐like representation, the absence of globally consistent orientation and the incompatible local discretizations. In this paper, we show that the Zernike polynomials allow rigourous yet practical mathematical generalization of CNNs to arbitrary surfaces. We prove that the convolution of two functions can be represented as a simple dot product between Zernike coefficients and the rotation of a convolution kernel is essentially a set of 2 × 2 rotation matrices applied to the coefficients. The key contribution of this work is in such a computationally efficient but rigorous generalization of the major CNN building blocks. [ABSTRACT FROM AUTHOR]

In contrast to 3D model that can be freely observed, p ortrait bas‐relief projects slightly from the background and is limited by fixed viewpoint. In this paper, we propose a novel method to reconstruct the underlying 3D shape from a single 2.5D bas‐relief, providing observers wider viewing perspectives. Our target is to make the reconstructed portrait has natural depth ordering and similar appearance to the input. To achieve this, we first use a 3D template face to fit the portrait. Then, we optimize the face shape by normal transfer and Poisson surface reconstruction. The hair and body regions are finally reconstructed and combined with the 3D face. From the resulting 3D shape, one can generate new reliefs with varying poses and thickness, freeing the input one from fixed view. A number of experimental results verify the effectiveness of our method. [ABSTRACT FROM AUTHOR]

In physics‐based liquid simulation for graphics applications, pressure projection consumes a significant amount of computational time and is frequently the bottleneck of the computational efficiency. How to rapidly apply the pressure projection and at the same time how to accurately capture the liquid geometry are always among the most popular topics in the current research trend in liquid simulations. In this paper, we incorporate an artificial neural network into the simulation pipeline for handling the tricky projection step for liquid animation. Compared with the previous neural‐network‐based works for gas flows, this paper advocates new advances in the composition of representative features as well as the loss functions in order to facilitate fluid simulation with free‐surface boundary. Specifically, we choose both the velocity and the level‐set function as the additional representation of the fluid states, which allows not only the motion but also the boundary position to be considered in the neural network solver. Meanwhile, we use the divergence error in the loss function to further emulate the lifelike behaviours of liquid. With these arrangements, our method could greatly accelerate the pressure projection step in liquid simulation, while maintaining fairly convincing visual results. Additionally, our neutral network performs well when being applied to new scene synthesis even with varied boundaries or scales. [ABSTRACT FROM AUTHOR]

Wang tiles, also known as Wang dominoes, are a jigsaw puzzle system with matching edges. Due to their compactness and expressiveness in representing variations, they have become a popular tool in the procedural synthesis of textures, height fields, 3D printing and representing other large and non‐repetitive data. Multi‐scale tiles created from low‐level tiles allow for a higher tiling efficiency, although they face the problem of combinatorial explosion. In this paper, we propose a generation method for multi‐scale Wang tiles that aims at minimizing the amount of needed tiles while still resembling a tiling appearance similar to low‐level tiles. Based on a set of representative multi‐scale Wang tiles, we use a dynamic generation algorithm for this purpose. Our method can be used for rapid texture synthesis and image halftoning. Respecting physical constraints, our tiles are connected, lightweight, independent of the fabrication scale, able to tile larger areas with image contents and contribute to "mass customization". [ABSTRACT FROM AUTHOR]

Despite the ubiquitous use of materials maps in modern rendering pipelines, their editing and control remains a challenge. In this paper, we present an example‐based material control method to augment input material maps based on user‐provided material photos. We train a tileable version of MaterialGAN and leverage its material prior to guide the appearance transfer, optimizing its latent space using differentiable rendering. Our method transfers the micro and meso‐structure textures of user provided target(s) photographs, while preserving the structure and quality of the input material. We show our methods can control existing material maps, increasing realism or generating new, visually appealing materials. [ABSTRACT FROM AUTHOR]

The development of scattering models and rendering algorithms for human hair remains an important area of research in computer graphics. Virtually all available models for scattering off hair or fur fibers are based on separable lobes, which bring practical advantages in importance sampling, but do not represent physically‐plausible microgeometry. In this paper, we contribute the first microfacet‐based hair scattering model. Based on a rough cylinder geometry with tilted cuticle scales, our far‐field model is non‐separable by nature, yet allows accurate importance sampling. Additional benefits include support for elliptical hair cross‐sections and an analytical solution for the reflected lobe using the GGX distribution. We show that our model captures glint‐like forward scattering features in the R lobe that have been observed before but not properly explained. [ABSTRACT FROM AUTHOR]

The process of forming, expressing, and updating beliefs from data plays a critical role in data‐driven decision making. Effectively eliciting those beliefs has potential for high impact across a broad set of applications, including increased engagement with data and visualizations, personalizing visualizations, and understanding users' visual reasoning processes, which can inform improved data analysis and decision making strategies (e.g., via bias mitigation). Recently, belief‐driven visualizations have been used to elicit and visualize readers' beliefs in a visualization alongside data in narrative media and data journalism platforms such as the New York Times and FiveThirtyEight. However, there is little research on different aspects that constitute designing an effective belief‐driven visualization. In this paper, we synthesize a design space for belief‐driven visualizations based on formative and summative interviews with designers and visualization experts. The design space includes 7 main design considerations, beginning with an assumed data set, then structured according to: from who, why, when, what, and how the belief is elicited, and the possible feedback about the belief that may be provided to the visualization viewer. The design space covers considerations such as the type of data parameter with optional uncertainty being elicited, interaction techniques, and visual feedback, among others. Finally, we describe how more than 24 existing belief‐driven visualizations from popular news media outlets span the design space and discuss trends and opportunities within this space. [ABSTRACT FROM AUTHOR]

Researchers collect large amounts of user interaction data with the goal of mapping user's workflows and behaviors to their high‐level motivations, intuitions, and goals. Although the visual analytics community has proposed numerous taxonomies to facilitate this mapping process, no formal methods exist for systematically applying these existing theories to user interaction logs. This paper seeks to bridge the gap between visualization task taxonomies and interaction log data by making the taxonomies more actionable for interaction log analysis. To achieve this, we leverage structural parallels between how people express themselves through interactions and language by reformulating existing theories as regular grammars. We represent interactions as terminals within a regular grammar, similar to the role of individual words in a language, and patterns of interactions or non‐terminals as regular expressions over these terminals to capture common language patterns. To demonstrate our approach, we generate regular grammars for seven existing visualization taxonomies and develop code to apply them to three public interaction log datasets. In analyzing these regular grammars, we find that the taxonomies at the low‐level (i.e., terminals) show mixed results in expressing multiple interaction log datasets, and taxonomies at the high‐level (i.e., regular expressions) have limited expressiveness, due to primarily two challenges: inconsistencies in interaction log dataset granularity and structure, and under‐expressiveness of certain terminals. Based on our findings, we suggest new research directions for the visualization community to augment existing taxonomies, develop new ones, and build better interaction log recording processes to facilitate the data‐driven development of user behavior taxonomies. [ABSTRACT FROM AUTHOR]

In this paper we present a reconstruction technique for the reduction of unsteady flow data based on neural representations of time‐varying vector fields. Our approach is motivated by the large amount of data typically generated in numerical simulations, and in turn the types of data that domain scientists can generate in situ that are compact, yet useful, for post hoc analysis. One type of data commonly acquired during simulation are samples of the flow map, where a single sample is the result of integrating the underlying vector field for a specified time duration. In our work, we treat a collection of flow map samples for a single dataset as a meaningful, compact, and yet incomplete, representation of unsteady flow, and our central objective is to find a representation that enables us to best recover arbitrary flow map samples. To this end, we introduce a technique for learning implicit neural representations of time‐varying vector fields that are specifically optimized to reproduce flow map samples sparsely covering the spatiotemporal domain of the data. We show that, despite aggressive data reduction, our optimization problem — learning a function‐space neural network to reproduce flow map samples under a fixed integration scheme — leads to representations that demonstrate strong generalization, both in the field itself, and using the field to approximate the flow map. Through quantitative and qualitative analysis across different datasets we show that our approach is an improvement across a variety of data reduction methods, and across a variety of measures ranging from improved vector fields, flow maps, and features derived from the flow map. [ABSTRACT FROM AUTHOR]

We study tangible touch tablets combined with Augmented Reality Head‐Mounted Displays (AR‐HMDs) to perform spatial 3D selections. We are primarily interested in the exploration of 3D unstructured datasets such as cloud points or volumetric datasets. AR‐HMDs immerse users by showing datasets stereoscopically, and tablets provide a set of 2D exploration tools. Because AR‐HMDs merge the visualization, interaction, and the users' physical spaces, users can also use the tablets as tangible objects in their 3D space. Nonetheless, the tablets' touch displays provide their own visualization and interaction spaces, separated from those of the AR‐HMD. This raises several research questions compared to traditional setups. In this paper, we theorize, discuss, and study different available mappings for manual spatial selections using a tangible tablet within an AR‐HMD space. We then study the use of this tablet within a 3D AR environment, compared to its use with a 2D external screen. [ABSTRACT FROM AUTHOR]

Danmu (Danmaku) is a unique social media service in online videos, especially popular in Japan and China, for viewers to write comments while watching videos. The danmu comments are overlaid on the video screen and synchronized to the associated video time, indicating viewers' thoughts of the video clip. This paper introduces an interactive visualization system to analyze danmu comments and associated viewer behaviors in a collection of videos and enable detailed exploration of one video on demand. The watching behaviors of viewers are identified by comparing video time and post time of viewers' danmu. The system supports analyzing danmu content and viewers' behaviors against both video time and post time to gain insights into viewers' online participation and perceived experience. Our evaluations, including usage scenarios and user interviews, demonstrate the effectiveness and usability of our system. [ABSTRACT FROM AUTHOR]

Edit distances between merge trees of scalar fields have many applications in scientific visualization, such as ensemble analysis, feature tracking or symmetry detection. In this paper, we propose branch mappings, a novel approach to the construction of edit mappings for merge trees. Classic edit mappings match nodes or edges of two trees onto each other, and therefore have to either rely on branch decompositions of both trees or have to use auxiliary node properties to determine a matching. In contrast, branch mappings employ branch properties instead of node similarity information, and are independent of predetermined branch decompositions. Especially for topological features, which are typically based on branch properties, this allows a more intuitive distance measure which is also less susceptible to instabilities from small‐scale perturbations. For trees with 풪(n) nodes, we describe an 풪(n4) algorithm for computing optimal branch mappings, which is faster than the only other branch decomposition‐independent method in the literature by more than a linear factor. Furthermore, we compare the results of our method on synthetic and real‐world examples to demonstrate its practicality and utility. [ABSTRACT FROM AUTHOR]