Search

Learning from 3D point-cloud data has rapidly gained momentum, motivated by the success of deep learning on images and the increased availability of 3D~data. In this paper, we aim to construct anisotropic convolution layers that work directly on the surface derived from a point cloud. This is challenging because of the lack of a global coordinate system for tangential directions on surfaces. We introduce DeltaConv, a convolution layer that combines geometric operators from vector calculus to enable the construction of anisotropic filters on point clouds. Because these operators are defined on scalar- and vector-fields, we separate the network into a scalar- and a vector-stream, which are connected by the operators. The vector stream enables the network to explicitly represent, evaluate, and process directional information. Our convolutions are robust and simple to implement and match or improve on state-of-the-art approaches on several benchmarks, while also speeding up training and inference., Comment: 8 pages, 5 figures, 7 tables; ACM Transactions on Graphics 41, 4, Article 105 (SIGGRAPH 2022)

Video-streaming services usually feature post-processing effects to replace the background. However, these often yield inconsistent lighting. Machine-learning-based relighting methods can address this problem, but, at real-time rates, are restricted to a low resolution and can result in an unrealistic skin appearance. Physically-based rendering techniques require complex skin models that can only be acquired using specialised equipment. Our method is lightweight and uses only a standard smartphone. By correcting imperfections during capture, we extract a convincing physically-based skin model. In combination with suitable acceleration techniques, we achieve real-time rates on commodity hardware.

We present a method for the real-time simulation of deformable objects that combines the robustness, generality, and high performance of Projective Dynamics with the efficiency and scalability offered by model reduction techniques. The method decouples the cost for time integration from the mesh resolution and can simulate large meshes in real-time. The proposed hyper-reduction of Projective Dynamics combines a novel fast approximation method for constraint projections and a scalable construction of sparse subspace bases. The resulting system achieves real-time rates for large sub-spaces enabling rich dynamics and can resolve general user interactions, collision constraints, external forces and changes to the materials. The construction of the hyper-reduced system does not require user-interaction and refrains from using training data or modal analysis, which results in a fast preprocessing stage.

Monte-Carlo rendering algorithms have traditionally a high computational cost, because they rely on tracing up to billions of light paths through a scene to physically simulate light transport. Traditional path reusing amortizes the cost of path sampling over multiple pixels, but introduces visually unpleasant correlation artifacts and cannot handle scenes with specular light transport. We present gradient-domain path reusing , a novel unbiased Monte-Carlo rendering technique, which merges the concept of path reusing with the recently introduced idea of gradient-domain rendering. Since correlation is a key element in gradient sampling, it is a natural fit to be performed together with path reusing and we show that the typical artifacts of path reusing are significantly reduced by exploiting the gradient domain. Further, by employing the tools for shifting paths that were designed in the context of gradient-domain rendering over the last years, we can generalize path reusing to support arbitrary scenes including specular light transport. Our method is unbiased and currently the fastest converging unidirectional rendering technique outperforming conventional and gradient-domain path tracing by up to almost an order of magnitude.

Performance capture systems are used to acquire high-quality animated 3D surfaces, usually in form of a dense 3D triangle mesh. Extracting a more compact yet faithful representation is often desirable, but existing solutions for animated sequences are surface based, which leads to a limited approximation power in the case of extreme simplification. We introduce animated sphere-meshes, which are meshes indexing a set of animated spheres. Our solution is the first to output an animated volumetric structure to approximate animated 3D surfaces and optimizes for the sphere approximation, connectivity, and temporal coherence. As a result, our algorithm produces a multiresolution structure from which a level of simplification can be selected in real time, preserving a faithful approximation of the input, even at the coarsest levels. We demonstrate the use of animated sphere-meshes for low-cost approximate collision detection. Additionally, we propose a skinning decomposition, which automatically rigs the input mesh to the chosen level of detail. The resulting set of weights are smooth, compress the animation, and enable easy edits.

Binocular disparity is one of the most important depth cues used by the human visual system. Recently developed stereo-perception models allow us to successfully manipulate disparity in order to improve viewing comfort, depth discrimination as well as stereo content compression and display. Nonetheless, all existing models neglect the substantial influence of luminance on stereo perception. Our work is the first to account for the interplay of luminance contrast (magnitude/frequency) and disparity and our model predicts the human response to complex stereo-luminance images. Besides improving existing disparity-model applications (e.g., difference metrics or compression), our approach offers new possibilities, such as joint luminance contrast and disparity manipulation or the optimization of auto-stereoscopic content. We validate our results in a user study, which also reveals the advantage of considering luminance contrast and its significant impact on disparity manipulation techniques., National Science Foundation (U.S.) (CGV-1111415)

We present a GPU-based real-time rendering method that simulates high-quality depth-of-field effects, similar in quality to multiview accumulation methods. Most real-time approaches have difficulties to obtain good approximations of visibility and view-dependent shading due to the use of a single view image. Our method also avoids the multiple rendering of a scene, but can approximate different views by relying on a layered image-based scene representation. We present several performance and quality improvements, such as early culling, approximate cone tracing, and jittered sampling. Our method achieves artifact-free results for complex scenes and reasonable depth-of-field blur in real time.