Search

The edge information plays a key role in the restoration of a depth map. Most conventional methods assume that the color image and depth map are consistent in edge areas. However, complex texture regions in the color image do not match exactly with edges in the depth map. In this paper, firstly, we point out that in most cases the consistency between normal map and depth map is much higher than that between RGB-D pairs. Then we propose a dual normal-depth regularization term to guide the restoration of depth map, which constrains the edge consistency between normal map and depth map back and forth. Moreover, considering the bimodal characteristic of weight distribution that exists in depth discontinuous areas, a reweighted graph Laplacian regularizer is proposed to promote this bimodal characteristic. And this regularization is incorporated into a unified optimization framework to effectively protect the piece-wise smoothness(PWS) characteristics of depth map. By treating depth image as graph signal, the weight between two nodes is adapted according to its content. The proposed method is tested for both noise-free and noisy cases, and is compared against the state-of-the-art methods on both synthesis and real captured datasets. Extensive experimental results demonstrate the superior performance of our method compared with most state-of-the-art works in terms of both objective and subjective quality evaluations. Specifically, our method is more effective on edge areas and more robust to noises.

In this paper, G-majorization inequalities are proven for the gradients of two Gateaux differentiable G-increasing functions defined on a real linear space endowed with the structure of an Eaton tr...

We present a novel meso-scale model for computing anisotropic and asymmetric friction for contacts in rigid body simulations that is based on surface facet orientations. The main idea behind our approach is to compute a direction dependent friction coefficient that is determined by an object's roughness. Specifically, where the friction is dependent on asperity interlocking, but at a scale where surface roughness is also a visual characteristic of the surface. A GPU rendering pipeline is employed to rasterize surfaces using a shallow depth orthographic projection at each contact point in order to sample facet normal information from both surfaces, which we then combine to produce direction dependent friction coefficients that can be directly used in typical LCP contact solvers, such as the projected Gauss-Seidel method. We demonstrate our approach with a variety of rough textures, where the roughness is both visible in the rendering and in the motion produced by the physical simulation.

Creating fine garment details requires significant efforts and huge computational resources. In contrast, a coarse shape may be easy to acquire in many scenarios (e.g., via low-resolution physically-based simulation, linear blend skinning driven by skeletal motion, portable scanners). In this paper, we show how to enhance, in a data-driven manner, rich yet plausible details starting from a coarse garment geometry. Once the parameterization of the garment is given, we formulate the task as a style transfer problem over the space of associated normal maps. In order to facilitate generalization across garment types and character motions, we introduce a patch-based formulation, that produces high-resolution details by matching a Gram matrix based style loss, to hallucinate geometric details (i.e., wrinkle density and shape). We extensively evaluate our method on a variety of production scenarios and show that our method is simple, light-weight, efficient, and generalizes across underlying garment types, sewing patterns, and body motion.

This paper discusses the applications of unmanned aerial vehicle (UAV) for slope mapping and also its important parameters including perimeter, area and also volume of certain selected area. With the development of modern technology, the utilization of UAV to gather data for slope mapping becoming easier as it is quick, reliable, precise, cost-effective and also easily to operate. Modern UAV able to take high quality image which essential for the effectiveness and nature of normal mapping output such as Digital Surface Model (DSM) and Digital Orthophoto. This photo captured by UAV will later transfer to commercial software to generate full map of study area. With the help of established software, the measurement of selected study areas can be determined easily which can be considered as the main interest in this study. In addition, another outcome of this study is, this modern method of mapping will be compare to traditional method of mapping which proven to be more effective in term of low costing, low time consuming, can gather huge amount of data within short period of time, low man power needed and almost no potential risk of hazardous effect to man.

Studs are small screws welded on a car’s body. Measuring the key point positions of the studs within production cycle time, also known as online measurement, is crucial for the quality management of the cars. However, the existing stud measuring systems can only measure one stud in a camera position, which is not efficient enough for online measurement. Accurately measuring multiple studs in one camera position is still a challenge. In this article, we propose a stud online measuring system (SOMS), which can measure multiple studs with high accuracy based on photometric stereo and deep learning techniques. To achieve accurate measurements, the SOMS uses normal maps, which are obtained by photometric stereo. Conventional photometric-stereo-based inspection methods reconstruct the surface from the normal maps; however, the reconstruction is inaccurate when the object has a large depth variation range. Instead of reconstruction, the SMOS directly uses a convolutional neural network (CNN) to localize the key points of studs from the normal maps. To adapt CNN to normal map input, we proposed a fusion block which extracts the most significant features among three dimensions of the normal maps. A test based on a collection of different types of studs was conducted to evaluate the advantage of the SOMS. Our system outperformed benchmarks in a series of experiments. Because the SOMS was developed without any prior knowledge on the geometry of the stud, it also has the potential for other optical challenging geometric features’ online measurements.

International audience; Rendering glinty details from specular microstructure enhances the level of realism, but previous methods require heavy storage for the high-resolution height field or normal map and associated acceleration structures. In this article, we aim at dynamically generating theoretically infinite microstructure, preventing obvious tiling artifacts, while achieving constant storage cost. Unlike traditional texture synthesis, our method supports arbitrary point and range queries, and is essentially generating the microstructure implicitly. Our method fits the widely used microfacet rendering framework with multiple importance sampling (MIS), replacing the commonly used microfacet normal distribution functions (NDFs) like ground glass distribution (GGX) by a detailed local solution, with a small amount of runtime performance overhead.

Rapid simulation of grinding surface topography at different machining conditions can assist users to determine appropriate grinding parameters. Most simulation methods reported so far are time-consuming and can only simulate the surface topography on a micro-scale, which are certainly not effective and intuitive enough. Hence, in this work, a novel GPU-based prediction and simulation method of the grinding surface topography is proposed. With the method, an artificial neural network (ANN) model is first developed to model the grinding surface roughness and fractal dimension. Then, the Cook-Torrance illumination model is introduced to render the color and glossiness of the grinding surface, and a normal map generation method is proposed to render the rugged detail features of grinding surface topography. Finally, a GPU-based prediction and simulation algorithm is developed. A performance evaluation experiment of the ANN model and a comparative experiment between the simulated and actual grinding surface topography are conducted. The experimental results show that the proposed method can achieve fast prediction and photorealistic simulation of the grinding surface topography for the whole part surface, which provides a more intuitive and effective way to evaluate the surface quality under given processing parameters.

In this paper, we give refinements for the G-majorization triangle inequality (x+y)↓≺Gx↓+y↓ with vectors x,y of an Eaton triple and normal map (⋅)↓. For instance, we establish refinements for the m...

Albeit remarkable progress has been made to improve the accuracy and completeness of multi-view stereo (MVS), existing methods still suffer from either sparse reconstructions of low-textured surfaces or heavy computational burden. In this paper, we propose a Confidence-based Large-scale Dense Multi-view Stereo (CLD-MVS) method for high resolution imagery. Firstly, we formulate MVS as a multi-view depth estimation problem, and employ a normal-aware efficient PatchMatch stereo to estimate the initial depth and normal map for each reference view. A self-supervised deep learning method is then developed to predict the spatial confidence for multi-view depth maps, which is combined with cross-view consistency to generate the ground control points. Subsequently, a confidence-driven and boundary-aware interpolation scheme using static and dynamic guidance is adopted to synthesize dense depth and normal maps. Finally, a refinement procedure which leverages synthesized depth and normal as prior is conducted to estimate cross-view consistent surface. Experiments show that the proposed CLD-MVS method achieves high geometric completeness while preserving fine-scale details. In particular, it has ranked No. 1 on the ETH3D high-resolution MVS benchmark in terms of $F_{1}$ -score.

In this paper, we present a novel facial albedo and 3D shape recovery method with a local spherical harmonic illumination model. From a face in an image, the proposed method can produce a high-quality 3D shape and albedo using a novel parameterization of local illuminations. Because a facial shape is partially convex, a single spherical harmonics is generally used for the illumination of a face within a constrained illumination environment. However, when a facial image is captured in an unconstrained scene, the illumination is inappropriately estimated due to the presence of shadow and specular reflections. To address this issue, we propose a novel local spherical harmonic illumination model for representing the illumination of a face. Unlike the existing parameterization of local illumination, our local spherical harmonic illumination model utilizes a smooth weight function for the seamless representation of natural illumination. Therefore, the albedo and shape information in an image can be precisely estimated using the first-order spherical harmonics. For accurate estimation of albedo, we also utilize facial albedo statistics to prevent the estimated albedo from becoming biased toward input image. Furthermore, we developed an accurate and reliable 3D shape reconstruction method from a normal map based on tetrahedron-based deformation. Comparing to the Laplacian deformation based method, our method is applicable to any mesh regardless of its structure. Through rigorous experiments, we demonstrate that the proposed local spherical harmonic illumination model is effective in estimating the complex illumination and can recover a high-quality facial albedo and 3D shape.

The geometric extraction and semantic understanding in bird's eye view plays an important role in cyber-physical-social systems (CPSS), because it can help human or intelligent agents (IAs) to perceive larger range of environment. Moreover, due to lack of comprehensive dataset from oblique perspective, fog-end deep learning algorithms for this purpose is still in blank. In this paper, we propose a novel method to generate synthetic large-scale dataset for geometric and semantic urban scene understanding from bird's eye view. There are two main steps involved, one is modeling and the other is rendering, which are processed by CityEngine and UnrealEngine4 respectively. In this way, synthetic aligned multi-model data are obtained efficiently, including spectral images, semantic labels, depth and normal maps. Specifically, terrain elevation, street graph, building style and trees distribution are all randomly generated according realistic situation, a few of handcrafted semantic labels annotated by colors spread throughout the scene, virtual cameras moved according to realistic trajectories of unmanned aerial vehicles (UAVs). For evaluation of practicability of our dataset, we manually labeled tens of aerial images downloaded from internet. And the experiment result show that, in both pure and combined mode, the dataset can improve the performance significantly.

This paper presents a new machine vision framework for the efficient examination and classification of surface textures on medium- and large-sized mold products, such as used for automobiles, TVs, and refrigerators. Existing techniques, which are based on the hands and eyes of skilled workers, are inconsistent and time-consuming. Although there are many types of precise surface inspection and measurement methods, most are difficult to apply at industrial sites or by finishing robots due to problems such as speed, setup limitations, and robustness. This paper proposes two techniques based on image processing that aims to automate surface inspection during an unmanned lapping process that is mainly employed to eliminate milling tool marks. First, both the shape of the reflected light and the intensity of the captured near-field contrast image right after the reflected specular are used to determine the machined surface state, and the presence of tool marks as the line light source scans counter-clockwise. Second, the photometric stereo technique is used to detect surface scratches through the normal map that recovers the surface. The proposed techniques show localized machined patterns and classify them with high accuracy.

International audience; To deal with the increasing demand for complex visual details in virtual worlds, procedural methods for content authoring are an expanding field in Computer Graphics. Focusing on on-the-fly texture generation, we present in this paper a content authoring process based on Locally Controlled Spot Noise. Through the control of both the impulses distribution and the spatially-defined kernel, this process can cover a wide range of appearances. In this context, we introduce a new kernel formulation that provides an efficient anisotropic filtering of the generated texture. Furthermore, our method allows users to interactively create the desired appearance by controlling both albedo and meso-geometry of the underlying surface, tackling on-the-fly normal map generation. Our method can be used as an artist friendly tool to model high-quality surface details with direct control over the final appearance in real-time.

This paper presents a photometric stereo technique that uses area lights for normal recovery and 3D geometry reconstruction of mid-sized objects. The object is illuminated in succession by several off-the-shelf LED area lights and images are captured by at least two DSLR cameras. Compared to point light sources, area lights have the advantage of producing high illuminance, resulting in low image noise and fast shutter speed, which is important if the captured object is not completely static during the acquisition of the images, e.g., when capturing a human face. Area lights are standard photo equipment which makes them cheaper, easier to obtain, and install than specialized many-lights hardware. The normal map of the object is recovered by our photometric stereo approach that uses ray tracing techniques to simulate the light transport in the scene. Furthermore, our approach takes the effects of occlusion and interreflections into account. The normal map is iteratively optimized which in turn is utilized to update the depth information of the object. Our synthetic and real-world experiments show that area lights are applicable for photometric stereo at the cost of an increased computational effort.

We propose a geometry-supporting dual convolutional neural network (GeoDualCNN) for both point cloud normal estimation and denoising. GeoDualCNN fuses the geometry domain knowledge that the underlying surface of a noisy point cloud is piecewisely smooth with the fact that a point normal is properly defined only when local surface smoothness is guaranteed. Centered around this insight, we define the homogeneous neighborhood (HoNe) which stays clear of surface discontinuities, and associate each HoNe with a point whose geometry and normal orientation is mostly consistent with that of HoNe. Thus, we not only obtain initial estimates of the point normals by performing PCA on HoNes, but also for the first time optimize these initial point normals by learning the mapping from two proposed geometric descriptors to the ground-truth point normals. GeoDualCNN consists of two parallel branches that remove noise using the first geometric descriptor (a homogeneous height map, which encodes the point-position information), while preserving surface features using the second geometric descriptor (a homogeneous normal map, which encodes the point-normal information). Such geometry-supporting network architectures enable our model to leverage previous geometry expertise and to benefit from training data. Experiments with noisy point clouds show that GeoDualCNN outperforms the state-of-the-art methods in terms of both noise-robustness and feature preservation.

We propose a novel method for addressing the problem of efficiently generating a highly refined normal map for screen-space fluid rendering. Because the process of filtering the normal map is crucially important to ensure the quality of the final screen-space fluid rendering, we employ a conditional generative adversarial network (cGAN) as a filter that learns a deep normal map representation, thereby refining the low-quality normal map. In particular, we have designed a novel loss function dedicated to refining the normal map information, and we use a specific set of auxiliary features to train the cGAN generator to learn features that are more robust with respect to edge details. Additionally, we constructed a dataset of six different typical scenes to enable effective demonstrations of multitype fluid simulation. Experiments indicated that our generator was able to infer clearer and more detailed features for this dataset than a basic screen-space fluid rendering method. Moreover, in some cases, the results generated by our method were even smoother than those generated by the conventional surface reconstruction method. Our method improves the fluid rendering results via the high-quality normal map while preserving the advantages of the screen-space fluid rendering methods and the traditional surface reconstruction methods, including that of the computation time being independent of the number of simulation particles and the spatial resolution being related only to image resolution.

This work presents Swish, a real-time machine-learning based cloth simulation technique for games. Swish was used to generate realistic cloth deformation and wrinkles for NFL player jerseys in Madden NFL 21. To our knowledge, this is the first neural cloth simulation featured in a shipped game. This technique allows accurate high-resolution simulation for tight clothing, which is a case where traditional real-time cloth simulations often achieve poor results. We represent cloth detail using both mesh deformations and a database of normal maps, and train a simple neural network to predict cloth shape from the pose of a character’s skeleton. We share implementation and performance details that will be useful to other practitioners seeking to introduce machine learning into their real-time character pipelines.

Compared with 2D facial expression recognition (FER) and 3D FER, 2D+3D FER can handle the effects of illumination changes and pose variations. The combination of 2D texture and 3D attribute information can further improve the performance. However, most existing approaches still face two challenges: the selection of proper networks for extracting multimodal features, and the significance of local features in salient regions for expression classification. To address these challenges, we propose an efficient feature fusion network with masks (FFNet-M) for 2D+3D FER. Each 3D scan is rep-resented by three types of attribute maps (i.e., depth map, normal map, and texture image), which are then fed into FFNet-M with different networks to extract both 2D and 3D features. Moreover, we design two masks to make FFNet-M focus on 2D local features while paying attention to 3D local features in salient regions. Experimental results show that our FFNet-M outperforms state-of-the-art methods on BU-3DFE dataset and also achieves a high accuracy on Bosphorus dataset.

We present a semi-automatic method for producing human bas-relief from a single photograph. Given an input photo of one or multiple persons, our method first estimates a 3D skeleton for each person in the image. SMPL models are then fitted to the 3D skeletons to generate a 3D guide model. To align the 3D guide model with the image, we compute a 2D warping field to non-rigidly register the projected contours of the guide model with the body contours in the image. Then the normal map of the 3D guide model is warped by the 2D deformation field to reconstruct an overall base shape. Finally, the base shape is integrated with a fine-scale normal map to produce the final bas-relief. To tackle the complex intra- and inter-body interactions, we design an occlusion relationship resolution method that operates at the level of 3D skeletons with minimal user inputs. To tightly register the model contours to the image contours, we propose a non-rigid point matching algorithm harnessing user-specified sparse correspondences. Experiments demonstrate that our human bas-relief generation method is capable of producing perceptually realistic results on various single-person and multi-person images, on which the state-of-the-art depth and pose estimation methods often fail.

This paper presents a surface normal integration method that solves an inverse problem of local plane fitting. Surface reconstruction from normal maps is essential in photometric shape reconstruction. To this end, we formulate normal integration in the camera coordinates and jointly solve for 3D point positions and local plane displacements. Unlike existing methods that consider the vertical distances between 3D points, we minimize the sum of squared point-to-plane distances. Our method can deal with both orthographic or perspective normal maps with arbitrary boundaries. Compared to existing normal integration methods, our method avoids the checkerboard artifact and performs more robustly against natural boundaries, sharp features, and outliers. We further provide a geometric analysis of the source of artifacts that appear in previous methods based on our plane fitting formulation. Experimental results on analytically computed, synthetic, and real-world surfaces show that our method yields accurate and stable reconstruction for both orthographic and perspective normal maps1.

Background This paper investigates the use of a light stage to capture high‐resolution, 3D facial surface textures and proposes novel methods to use the data for skin condition assessment. Materials and Methods We introduce new methods for analysing 3D surface texture using high‐resolution normal fields and apply these to the detection and assessment of skin conditions in human faces, specifically wrinkles, pores and acne. The use of high‐resolution normal maps as input to our texture measures enables us to investigate the 3D nature of texture, while retaining aspects of some well‐known 2D texture measures. The main contributions are as follows: the introduction of three novel methods for extracting texture descriptors from high‐resolution surface orientation fields; a comparative study of 2D and 3D skin texture analysis techniques; and an extensive data set of high‐resolution 3D facial scans presenting various skin conditions, with human ratings as “ground truth.” Results Our results demonstrate an improvement on state‐of‐the‐art methods for the analysis of pores and comparable results to the state of the art for wrinkles and acne using a considerably more compact model. Conclusions The use of high‐resolution normal maps, captured by a light stage, and the methods described, represent an important new set of tools in the analysis of skin texture.

In this paper, we consider Newton–Kantorovich type methods for solving control-constrained optimal control problems that appear in model predictive control. Conditions for convergence are established for an inexact version of the Newton–Kantorovich method applied to variational inequalities. Based on these results, two groups of algorithms are proposed to solve the optimality system. The first group includes exact and inexact Newton and Newton–Kantorovich implementations of the sequential quadratic programming. In the second group, exact and inexact Newton and Newton–Kantorovich methods are developed for solving a nonsmooth normal map equation equivalent to the optimality system. Numerical simulations featuring examples from the aerospace and automotive domain are presented, which show that inexact Newton–Kantorovich type methods can achieve significant reduction of the computational time.

In this paper, we extend Montel's normal criterion to a family F of holomorphic mappings from a domain D of the complex plane C into a complex projective space Pn(C) of dimension n. Furthermore, La...

In this paper, we explore a new way to accelerate and densify unstructured multi-view stereo (MVS). While many unstructured MVS algorithms have been proposed, we discover that the image-guided resizing can easily and significantly benefit their 3D reconstruction results in both efficiency and completeness. Therefore, we build our framework upon a novel selective joint bilateral upsampling and depth propagation strategy. First, we downsample the input unstructured images into lower resolution ones and perform the MVS calculation to efficiently obtain depth and normal maps from these resized pictures. Then, the proposed algorithm upsamples the normal maps with the guidance of input images, and jointly take them into consideration to recover the low-resolution depth maps into high resolution with geometry details simultaneously enriched. Finally by adaptively fusing the reconstructed depth and normal maps, we construct the final dense 3D scene. Quantitative results validate the efficiency and effectiveness of the proposed method.

Reflectance and shape are two important components in visually perceiving the real world. Inferring the reflectance and shape of an object through cameras is a fundamental research topic in the field of computer vision. While three-dimensional shape recovery is pervasive with varieties of approaches and practical applications, reflectance recovery has only emerged recently. Reflectance recovery is a challenging task that is usually conducted in controlled environments, such as a laboratory environment with a special apparatus. However, it is desirable that the reflectance be recovered in the field with a handy camera so that reflectance can be jointly recovered with the shape. To that end, we present a solution that simultaneously recovers the reflectance and shape (i.e., dense depth and normal maps) of an object under natural illumination with commercially available handy cameras. We employ a light field camera to capture one light field image of the object, and a 360-degree camera to capture the illumination. The proposed method provides positive results in both simulation and real-world experiments.

International audience; Lossy texture compression is increasingly used to reduce GPU memory and bandwidth consumption. However, as raised by recent studies, evaluating the quality of compressed textures is a difficult problem. Indeed using Peak Signal-to-Noise Ratio (PSNR) on texture images, like done in most applications, may not be a correct way to proceed. In particular, there is evidence that masking effects apply when the texture image is mapped on a surface and combined with other textures (e.g., affecting geometry or normal). These masking effects have to be taken into account when compressing a set of texture maps, in order to have a real understanding of the visual impact of the compression artifacts on the rendered images. In this work, we present the first psychophysical experiment investigating the perceptual impact of texture compression on rendered images. We explore the influence of compression bit rate, light direction, and diffuse and normal map content on the visual impact of artifacts. The collected data reveal huge masking effects from normal map to diffuse map artifacts and vice versa, and reveal the weakness of PSNR applied on individual textures for evaluating compression quality. The results allow us to also analyze the performance and failures of image quality metrics for predicting the visibility of these artifacts. We finally provide some recommendations for evaluating the quality of texture compression and show a practical application to approximating the distortion measured on a rendered 3D shape.

We are facing a number of applications for altering portrait photographs using the help of artificial intelligence. Apart from the entertainment purposes, modern computer technologies can also help us get dynamic effects from static images, an example of which is the subject of this study. Dealing with the change of lighting in portraits photographs through real-time rendering, this paper provides a method of obtaining an image with variable light source that affects the facial features generated on the basis of the face tracking data acquired from the existing static photograph. Thus, we get a portrait with altered light, as if such a source was present at the actual moment of photographing. This method aims to improve or even change the visual experience when viewing the image, so that its further application corresponds to the given context.

Realistic rendering with materials that exhibit high-frequency spatial variation remains a challenge, as eliminating spatial and temporal aliasing requires prohibitively high sampling rates. Recent work has made the problem more tractable, however existing methods remain prohibitively expensive when using large environmental lights and/or (correctly filtered) global illumination. We present an appearance model with explicit high-frequency micro-normal variation, and a filtering approach that scales to multi-dimensional shading integrals. By combining a novel and compact half-vector histogram scheme with a directional basis expansion, we accurately compute the integral of filtered high-frequency reflectance over large lights with angularly varying emission. Our approach is scalable, rendering images indistinguishable from ground truth at over 10× the speed of the state-of-the-art and with only 15% the memory footprint. When filtering appearance with global illumination, we outperform the state-of-the-art by ~30×.

We present a method to acquire dynamic properties of facial skin appearance, including dynamic diffuse albedo encoding blood flow, dynamic specular intensity, and per-frame high resolution normal maps for a facial performance sequence. The method reconstructs these maps from a purely passive multi-camera setup, without the need for polarization or requiring temporally multiplexed illumination. Hence, it is very well suited for integration with existing passive systems for facial performance capture. To solve this seemingly underconstrained problem, we demonstrate that albedo dynamics during a facial performance can be modeled as a combination of: (1) a static, high-resolution base albedo map, modeling full skin pigmentation; and (2) a dynamic, one-dimensional component in the CIE L*a*b* color space, which explains changes in hemoglobin concentration due to blood flow. We leverage this albedo subspace and additional constraints on appearance and surface geometry to also estimate specular reflection parameters and resolve high-resolution normal maps with unprecedented detail in a passive capture system. These constraints are built into an inverse rendering framework that minimizes the difference of the rendered face to the captured images, incorporating constraints from multiple views for every texel on the face. The presented method is the first system capable of capturing high-quality dynamic appearance maps at full resolution and video framerates, providing a major step forward in the area of facial appearance acquisition.

We propose a novel method for detecting features on normal maps and describing binary features, called BIFNOM, which is three-dimensionally rotation invariant and detects and matches interest points at high speed regardless of whether a target is textured or textureless and rigid or non-rigid. Conventional methods of detecting features on normal maps can also be applied to textureless targets, in contrast with features on luminance images, however, they cannot deal with three-dimensional rotation between each pair of corresponding interest points due to the definition of orientation, or they have difficulty achieving fast detection and matching due to a heavy-weight descriptor. We addressed these issues by introducing a three dimensional local coordinate system and converting a normal vector to a binary code, and achieved more than 750fps real-time feature detection and matching. Furthermore, we present an extended descriptor and criteria for real-time tracking, and evaluate the performance with both simulation and actual system.

Real-time geometric specular anti-aliasing is required when using a low number of pixel samples and high-frequency specular lobes. Several methods have been proposed for mono-lobe bidirectional reflection distribution functions (BRDFs), but none for multi-lobe BRDFs, e.g., a glinty BRDF. We present the first method for real-time geometric glint anti-aliasing (GGAA). It eliminates most of the inconsistent appearing and disappearing of glints on surfaces with significant curvatures during animations. The technique uses the glinty BRDF of Chermain et al. [2020] and leverages hardware GPU-filtering of textures to filter slope distributions on the fly. We also improve this glinty BRDF by adding a correlation factor of slope. This BRDF parameter allows convergence to normal distribution functions that are not aligned on the surface's axes. Above all, this parameter makes glint rendering compatible with normal map filtering using LEAN mapping. Using GGAA increases the rendering time from 0.6 % to 4.2 % and it requires 1/3 more memory due to MIP mapping of tabulated slope distributions. The results are compared with references using a thousand samples per pixel.

In the current field of face reconstruction, the methods of face reconstruction tend to pay more attention to the optimization of the face model, and seldom study the surface details or even the skin realism, which leads to the greatly reduced realistic of the model and can not achieve good application effect. In this paper, we improve the quality of normal map to lay the foundation for the realistic face reconstruction. Based on the existing face reconstruction, the normal mapping of the object surface is reconstructed by image pre-processing and photometric stereo technology for face photos with the same angle and different illumination directions. This paper improves the photometric stereo algorithm to improve the efficiency and effect of normal mapping.

Given a monocular face image as input, 3D face geometry reconstruction aims to recover a corresponding 3D face mesh. Recently, both optimization-based and learning-based face reconstruction methods have taken advantage of the emerging differentiable renderer and shown promising results. However, the differentiable renderer, mainly based on graphics rules, simplifies the realistic mechanism of the illumination, reflection, \etc, of the real world, thus cannot produce realistic images. This brings a lot of domain-shift noise to the optimization or training process. In this work, we introduce a novel Generative Adversarial Renderer (GAR) and propose to tailor its inverted version to the general fitting pipeline, to tackle the above problem. Specifically, the carefully designed neural renderer takes a face normal map and a latent code representing other factors as inputs and renders a realistic face image. Since the GAR learns to model the complicated real-world image, instead of relying on the simplified graphics rules, it is capable of producing realistic images, which essentially inhibits the domain-shift noise in training and optimization. Equipped with the elaborated GAR, we further proposed a novel approach to predict 3D face parameters, in which we first obtain fine initial parameters via Renderer Inverting and then refine it with gradient-based optimizers. Extensive experiments have been conducted to demonstrate the effectiveness of the proposed generative adversarial renderer and the novel optimization-based face reconstruction framework. Our method achieves state-of-the-art performances on multiple face reconstruction datasets., Comment: cvpr2021 oral

This paper starts with the study of realistic three-dimensional models, from the two aspects of ink art style simulation model and three-dimensional display technology, explores the three-dimensional display model of three-dimensional model ink style, and conducts experiments through the software development platform and auxiliary software. The feasibility of the model is verified. Aiming at the problem of real-time rendering of large-scale 3D scenes in the model, efficient visibility rejection method and a multiresolution fast rendering method were designed to realize the rapid construction and rendering of ink art 3D virtual reality scenes in a big data environment. A two-dimensional cellular automaton is used to simulate a brushstroke model with ink and wash style, and outlines are drawn along the path of the brushstroke to obtain an effect close to the artistic style of ink and wash painting. Set the surface of the model with ink style brushstroke texture patterns, refer to the depth map, normal map, and curvature map information of the model, and simulate the drawing effect of the method by procedural texture mapping. Example verification shows that the rapid visualization analysis model of ink art big data designed in this paper is in line with the prediction requirements of ink art big data three-dimensional display indicators. The fast visibility removal method is used to deal with large-scale three-dimensional ink art in a big data environment. High efficiency is achieved in virtual reality scenes, and the multiresolution fast rendering method better maintains the appearance of the prediction model without major deformation.

Monte Carlo light transport simulations often lack robustness in scenes containing specular or near-specular materials. Widely used uni- and bidirectional sampling strategies tend to find light paths involving such materials with insufficient probability, producing unusable images that are contaminated by significant variance., This article addresses the problem of sampling a light path connecting two given scene points via a single specular reflection or refraction, extending the range of scenes that can be robustly handled by unbiased path sampling techniques. Our technique enables efficient rendering of challenging transport phenomena caused by such paths, such as underwater caustics or caustics involving glossy metallic objects., We derive analytic expressions that predict the total radiance due to a single reflective or refractive triangle with a microfacet BSDF and we show that this reduces to the well known Lambert boundary integral for irradiance. We subsequently show how this can be leveraged to efficiently sample connections on meshes comprised of vast numbers of triangles. Our derivation builds on the theory of off-center microfacets and involves integrals in the space of surface slopes., Our approach straightforwardly applies to the related problem of rendering glints with high-resolution normal maps describing specular microstructure. Our formulation alleviates problems raised by singularities in filtering integrals and enables a generalization of previous work to perfectly specular materials. We also extend previous work to the case of GGX distributions and introduce new techniques to improve accuracy and performance.

Aiming at the problem of time consuming of online transmission and rendering of 3D models, a method is proposed to reduce the model transmission time without reducing the quality of rendering. First, we present a mesh model lightweight method base on the butterfly pattern. The geometric error of the simplified low model is reduced compared to the traditional simplification algorithm. Then, an improved normal map algorithm based on 3Dc is presented. First, the minimum and maximum values of the normal map are counted and interpolated. Then, each block in the map is matched, and the minimum and maximum numbers are recorded. The method is used to interpolate the set of points in the fitting block. Experiments show that the proposed algorithm can guarantee the three-dimensional model has a high sense of realism, but also can greatly shorten the 3D model transmission and rendering time.