Search

Describes a conceptual framework for 3D-2D (or 2D-3D) face recognition.Proposes a novel 3D-2D system for 2D image face recognition from 3D datasets.Proposes a method to build subject-specific 3D gallery models, using 3D+2D data, and a method for model-based, texture representation and relighting.3D-2D recognition surpasses 2D-2D on challenging 2D+3D data with pose and illumination variations, and can approximate 3D-3D, shape-based similarity methods.Representation and normalization using 3D models can compensate for non-frontal poses or different lighting conditions. In this paper, we propose a 3D-2D framework for face recognition that is more practical than 3D-3D, yet more accurate than 2D-2D. For 3D-2D face recognition, the gallery data comprises of 3D shape and 2D texture data and the probes are arbitrary 2D images. A 3D-2D system (UR2D) is presented that is based on a 3D deformable face model that allows registration of 3D and 2D data, face alignment, and normalization of pose and illumination. During enrollment, subject-specific 3D models are constructed using 3D+2D data. For recognition, 2D images are represented in a normalized image space using the gallery 3D models and landmark-based 3D-2D projection estimation. A method for bidirectional relighting is applied for non-linear, local illumination normalization between probe and gallery textures, and a global orientation-based correlation metric is used for pairwise similarity scoring. The generated, personalized, pose- and light- normalized signatures can be used for one-to-one verification or one-to-many identification. Results for 3D-2D face recognition on the UHDB11 3D-2D database with 2D images under large illumination and pose variations support our hypothesis that, in challenging datasets, 3D-2D outperforms 2D-2D and decreases the performance gap against 3D-3D face recognition. Evaluations on FRGC v2.0 3D-2D data with frontal facial images, demonstrate that the method can generalize to databases with different and diverse illumination conditions.

In this paper, we propose a 3D-2D framework for face recognition that is more practical than 3D-3D, yet more accurate than 2D-2D. For 3D-2D face recognition, the gallery data comprises of 3D shape and 2D texture data and the probes are arbitrary 2D images. A 3D-2D system (UR2D) is presented that is based on a 3D deformable face model that allows registration of 3D and 2D data, face alignment, and normalization of pose and illumination. During enrollment, subject-specific 3D models are constructed using 3D+2D data. For recognition, 2D images are represented in a normalized image space using the gallery 3D models and landmark-based 3D-2D projection estimation. A method for bidirectional relighting is applied for non-linear, local illumination normalization between probe and gallery textures, and a global orientation-based correlation metric is used for pairwise similarity scoring. The generated, personalized, pose- and light- normalized signatures can be used for one-to-one verification or one-to-many identification. Results for 3D-2D face recognition on the UHDB11 3D-2D database with 2D images under large illumination and pose variations support our hypothesis that, in challenging datasets, 3D-2D outperforms 2D-2D and decreases the performance gap against 3D-3D face recognition. Evaluations on FRGC v2.0 3D-2D data with frontal facial images, demonstrate that the method can generalize to databases with different and diverse illumination conditions. © 2016

The uncontrolled conditions of real-world biometric applications pose a great challenge to any face recognition approach. The unconstrained acquisition of data from uncooperative subjects may result in facial scans with significant pose variations along the yaw axis. Such pose variations can cause extensive occlusions, resulting in missing data. In this paper, a novel 3D face recognition method is proposed that uses facial symmetry to handle pose variations. It employs an automatic landmark detector that estimates pose and detects occluded areas for each facial scan. Subsequently, an Annotated Face Model is registered and fitted to the scan. During fitting, facial symmetry is used to overcome the challenges of missing data. The result is a pose invariant geometry image. Unlike existing methods that require frontal scans, the proposed method performs comparisons among interpose scans using a wavelet-based biometric signature. It is suitable for real-world applications as it only requires half of the face to be visible to the sensor. The proposed method was evaluated using databases from the University of Notre Dame and the University of Houston that, to the best of our knowledge, include the most challenging pose variations publicly available. The average rank-one recognition rate of the proposed method in these databases was 83.7 percent.

While the retrieval of datasets from human subjects based on demographic characteristics such as gender or race is an ability with wide-ranging application, it remains poorly-studied. In contrast, a large body of work exists in the field of biometrics which has a different goal: the recognition of human subjects. Due to this disparity of interest, existing methods for retrieval based on demographic attributes tend to lag behind the more well-studied algorithms designed purely for face matching. The question this raises is whether a face recognition system could be leveraged to solve these other problems and, if so, how effective it could be. In the current work, we explore the limits of such a system for gender and ethnicity identification given (1) a ground truth of demographically-labeled, textureless 3-D models of human faces and (2) a state-of-the-art face-recognition algorithm. Once trained, our system is capable of classifying the gender and ethnicity of any such model of interest. Experiments are conducted on 4007 facial meshes from the benchmark Face Recognition Grand Challenge v2 dataset.

Quantitative analysis of breast morphometry is critical to breast plastic surgery. Recently, three-dimensional (3D) photography has emerged as a strong new alternative for breast morphometry analysis in comparison to other existing techniques. 3D photography enables the capture of the entire breast surface topology virtually in a single snapshot and without any direct contact with the patient, thus causing minimal discomfort. In this paper, we present a set of computational tools for the quantitative analysis of two key morphological properties of the breast that are of interest to breast plastic surgery based on 3D scans, namely breast shape and volume. The breast shape is modeled using a compact geometric model capable of capturing the global shape of the breast with very few parameters. Specifically, the shape model is deduced by applying a set of five global deformations to a geometric primitive. These deformations, defined using very intuitive parameters, closely model the key shape variables that surgeons inherently use to describe the overall shape of the breast. Patient-specific parameters of the breast shape model are automatically recovered by fitting a generic breast shape model to the 3D scan of the patient's breast using a physics-based deformable model framework. The mean error of fit between the automatically fitted shape model and the actual breast surface for 12 subjects varied between 0.9 and 2.6 mm. These results are very encouraging considering the fact that only 17 parameters are used to determine the shape of the breast. The breast volume is estimated automatically by first localizing the breast on a 3D scan of the patient's torso and then computing the volume enclosed between an interpolated breast-less torso surface and the actual breast. The volume estimated by the proposed method was found to be within the intra-operator variability among five segmentation trials performed manually by an expert on 3D torso scans of three subjects.

As the accuracy of biometrics improves, it is getting increasingly hard to push the limits using a single modality. In this paper, a unified approach that fuses three-dimensional facial and ear data is presented. An annotated deformable model is fitted to the data and a geometry image is extracted. Wavelet coefficients are computed from the geometry image and used as a biometric signature. The method is evaluated using the largest publicly available database and achieves 99.7% rank-one recognition rate. The state-of-the-art accuracy of the multimodal fusion is attributed to the low correlation between the individual differentiability of the two modalities.

As the size of the available collections of 3D objects grows, database transactions become essential for their management with the key operation being retrieval (query). Large collections are also precategorized into classes so that a single class contains objects of the same type (e.g., human faces, cars, four-legged animals). It is shown that general object retrieval methods are inadequate for intraclass retrieval tasks. We advocate that such intraclass problems require a specialized method that can exploit the basic class characteristics in order to achieve higher accuracy. A novel 3D object retrieval method is presented which uses a parameterized annotated model of the shape of the class objects, incorporating its main characteristics. The annotated subdivision-based model is fitted onto objects of the class using a deformable model framework, converted to a geometry image and transformed into the wavelet domain. Object retrieval takes place in the wavelet domain. The method does not require user interaction, achieves high accuracy, is efficient for use with large databases, and is suitable for nonrigid object classes. We apply our method to the face recognition domain, one of the most challenging intraclass retrieval tasks. We used the Face Recognition Grand Challenge v2 database, yielding an average verification rate of 95.2 percent at to 10-3 false accept rate. The latest results of our work can be found at http://www.cbl.uh.edu/UR8D/

This paper presents an efficient algorithm for the voxelization of solid objects represented by boundary surfaces of arbitrary topology. The algorithm is based on the use of depth buffers and is suitable for polygonal meshes and parametric surfaces. The presented method unifies and extends our previous approaches under a single general algorithm; three practical variants are then given that implement it on different generations of graphics hardware in order to achieve high performance. The implementations for older hardware impose certain limitations on the depth complexity of the object.

A 3D landmark detection method for 3D facial scans is presented and thoroughly evaluated. The main contribution of the presented method is the automatic and pose-invariant detection of landmarks on 3D facial scans under large yaw variations (that often result in missing facial data), and its robustness against large facial expressions. Three-dimensional information is exploited by using 3D local shape descriptors to extract candidate landmark points. The shape descriptors include the shape index, a continuous map of principal curvature values of a 3D object's surface, and spin images, local descriptors of the object's 3D point distribution. The candidate landmarks are identified and labeled by matching them with a Facial Landmark Model (FLM) of facial anatomical landmarks. The presented method is extensively evaluated against a variety of 3D facial databases and achieves state-of-the-art accuracy (4.5-6.3 mm mean landmark localization error), considerably outperforming previous methods, even when tested with the most challenging data.

We present a novel approach for computing a compact and highly discriminant biometric signature for 3D face recognition using linear dimensionality reduction techniques. Initially, a geometry-image representation is used to effectively resample the raw 3D data. Subsequently, a wavelet transform is applied and a biometric signature composed of 7,200 wavelet coefficients is extracted. Finally, we apply a second linear dimensionality reduction step to the wavelet coefficients using Linear Discriminant Analysis and compute a compact biometric signature. Although this biometric signature consists of just 57 coefficients, it is highly discriminant. Our approach, UR3D-C, is experimentally validated using four publicly available databases (FRGC v1, FRGC v2, Bosphorus and BU-3DFE). State-of-the-art performance is reported in all of the above databases.

In this chapter, we present advances that aid in overcoming the challenges encountered in 3D face recognition. First, we present a fully automatic 3D face recognition system, UR3D, which has been proven to be robust under variations in expressions. Second, we demonstrate how to handle pose variations. Finally, we demonstrate how the problems related to the cost and unfriendliness of 3D scanners can be mitigated through hybrid systems.

In this paper, we present a new method for bidirectional relighting for 3D-aided 2D face recognition under large pose and illumination changes. During subject enrollment, we build subject-specific 3D annotated models by using the subjects’ raw 3D data and 2D texture. During authentication, the probe 2D images are projected onto a normalized image space using the subject-specific 3D model in the gallery. Then, a bidirectional relighting algorithm and two similarity metrics (a view-dependent complex wavelet structural similarity and a global similarity) are employed to compare the gallery and probe. We tested our algorithms on the UHDB11 and UHDB12 databases that contain 3D data with probe images under large lighting and pose variations. The experimental results show the robustness of our approach in recognizing faces in difficult situations.

In this paper, we present a new method for bidirectional relighting for 3D-aided 2D face recognition under large pose and illumination changes. During subject enrollment, we build subject-specific 3D annotated models by using the subjects' raw 3D data and 2D texture. During authentication, the probe 2D images are projected onto a normalized image space using the subject-specific 3D model in the gallery. Then, a bidirectional relighting algorithm and two similarity metrics (a view-dependent complex wavelet structural similarity and a global similarity) are employed to compare the gallery and probe. We tested our algorithms on the UHDB11 and UHDB12 databases that contain 3D data with probe images under large lighting and pose variations. The experimental results show the robustness of our approach in recognizing faces in difficult situations.

An efficient hardware architecture for the acceleration of an integrated 3D reconstruction method is presented, targeting demanding dynamic Integral Imaging applications. It exploits parallel processing and features minimized memory operations by implementing an extended-access memory scheme. Its reduced data throughput, thanks to optimized data utilization, makes it suitable for single FPGA device implementation. Results reveal that the hardware system outperforms the software method by an order of magnitude. Moreover, its processing rate surpasses the typical rate of a dynamic Integral Imaging acquisition system, thus making a significant step towards real-time 3D video reconstruction.

This work presents a hardware acceleration scheme for a 3D reconstruction method, targeting demanding dynamic Integral Imaging applications. The architecture exploits parallel processing and minimizes memory operations by implementing an efficient metric and an extended-access memory scheme. The employed data reutilization technique reduces overall throughput allowing the use of a single FPGA. Results reveal that the hardware system accelerates the software method by an order of magnitude and its processing rate surpasses the typical rate of a dynamic Integral Imaging acquisition system, making it suitable for robust 3D image and video reconstruction applications.

In this paper, we present an overview of the 3D object retrieval method that we employed in our participation to the generic models track of SHREC 2008 organized by the AIM@SHAPE network of excellence. The proposed methodology is detailed in [2]. Our method is based on a hybrid scheme where 2D features as well as 3D features are extracted from a 3D model which has been previously normalized for rotation using two alternative alignment techniques. The alignment methods that are used are CPCA and NPCA. The 2D features are Fourier coefficients extracted from a set of depth buffers and the 3D features are spherical harmonic coefficients extracted from a spherical function-based representation of a 3D model.

Three-dimensional data are increasingly being used for biometric purposes as they offer resilience to problems common in two-dimensional data. They have been successfully applied to face recognition and more recently to ear recognition. However, real-life biometric applications require algorithms that are both robust and efficient so that they scale well with the size of the databases. A novel ear recognition method is presented that uses a generic annotated ear model to register and fit each ear dataset. Then a compact biometric signature is extracted that retains 3D information. The proposed method is evaluated using the largest publicly available 3D ear database appended with our own database, resulting in a database containing data from multiple 3D sensor types. Using this database it is shown that the proposed method is not only robust, accurate and sensor invariant but also extremely efficient, thus making it suitable for real-life biometric applications.

A method for the reconstruction of 3D shape and texture from integral photography (IP) images is presented. Sharing the same principles with stereoscopic-based object reconstruction, it offers increased robustness to noise and occlusions due to the unique characteristics of IP images. A coarse-to-fine approach is used, employing what we believe to be a novel grid refinement step in order to increase the quality of the reconstructed objects. The proposed method's properties include configurable depth accuracy and direct and seamless triangulation. We evaluate our method using synthetic data from a computer-simulated IP setup as well as real data from a simple yet effective digital IP setup. Experiments show reconstructed objects of high-quality indicating that IP can be a competitive modality for 3D object reconstruction.

In this paper, we present the computational tools and a hardware prototype for 3D face recognition. Full automation is provided through the use of advanced multistage alignment algorithms, resilience to facial expressions by employing a deformable model framework, and invariance to 3D capture devices through suitable preprocessing steps. In addition, scalability in both time and space is achieved by converting 3D facial scans into compact metadata. We present our results on the largest known, and now publicly available, Face Recognition Grand Challenge 3D facial database consisting of several thousand scans. To the best of our knowledge, this is the highest performance reported on the FRGC v2 database for the 3D modality.

Face recognition performance has always been affected by the different facial expressions a subject may display. In this paper, we present an extension to the UR3D face recognition algorithm, which enables us to decrease the discrepancy in its performance for datasets from subjects with and without a neutral facial expression, from 15% to 3%.

From a user’s perspective, face recognition is one of the most desirable biometrics, due to its non-intrusive nature; however, variables such as face expression tend to severely affect recognition rates. We have applied to this problem our previous work on elastically adaptive deformable models to obtain parametric representations of the geometry of selected localized face areas using an annotated face model. We then use wavelet analysis to extract a compact biometric signature, thus allowing us to perform rapid comparisons on either a global or a per area basis. To evaluate the performance of our algorithm, we have conducted experiments using data from the Face Recognition Grand Challenge data corpus, the largest and most established data corpus for face recognition currently available. Our results indicate that our algorithm exhibits high levels of accuracy and robustness, and is not gender biased. In addition, it is minimally affected by facial expressions.

A number of three-dimensional algorithms have been proposed to solve the problem of patching surfaces to rectify and extrapolate missing information due to model problems or bad geometry visibility during data capture. On the other hand, a number of similar yet more simple and robust techniques apply to 2D image data and are used for texture restoration. In this paper we make an attempt to bring these two-dimensional techniques to the 3D domain due to their obvious advantage of simplicity and controllability. Creating a depth image with the help of a voxelisation algorithm will allow us to apply a variety of image repair algorithms in order to mend a 3D object. The use of three variations of the texture synthesis algorithm is investigated. Constrained texture synthesis and its variations using the Haar wavelet and image decomposition methods are also proposed in order to preserve patterns appearing on the object while trying to maintain its geometry intact.

A number of three-dimensional algorithms have been proposed to solve the problem of patching surfaces to rectify and extrapolate missing information due to model problems or bad geometry visibility during data capture. On the other hand, a number of similar yet more simple and robust techniques apply to 2D image data and are used for texture restoration. In this paper we make an attempt to bring these two-dimensional techniques to the 3D domain due to their obvious advantage of simplicity and controllability. Creating a depth image with the help of a voxelisation algorithm will allow us to apply a variety of image repair algorithms in order to mend a 3D object. The use of three variations of the texture synthesis algorithm is investigated. Constrained texture synthesis and its variations using the Haar wavelet and image decomposition methods are also proposed in order to preserve patterns appearing on the object while trying to maintain its geometry intact. © Springer-Verlag Berlin Heidelberg 2006.

It is becoming increasingly important to be able to credential and identify authorized personnel at key points of entry. Such identity management systems commonly employ biometric identifiers. In this paper, we present a novel multimodal facial recognition approach that employs data from both visible spectrum and thermal infrared sensors. Data from multiple cameras is used to construct a three-dimensional mesh representing the face and a facial thermal texture map. An annotated face model with explicit two-dimensional parameterization (UV) is then fitted to this data to construct: 1) a three-channel UV deformation image encoding geometry, and 2) a one-channel UV vasculature image encoding facial vasculature. Recognition is accomplished by comparing: 1) the parametric deformation images, 2) the parametric vasculature images, and 3) the visible spectrum texture maps. The novelty of our work lies in the use of deformation images and physiological information as means for comparison. We have performed extensive tests on the Face Recognition Grand Challenge v1.0 dataset and on our own multimodal database with very encouraging results.

Automatic computation of breast volume data is required in post-mastectomy breast reconstructive surgery, where it is very useful to have an estimate of the volume of the tissue to be extracted in advance of the operation. Such an automated process is essential when conducting studies on large patient databases where manual volume estimation is both tedious and subjective. In this paper, we present a noninvasive automatic breast volume estimation method. It employs 3D scanned data of the torso. First, a surface underneath the breast that resembles a 'breast-less' torso is automatically constructed, and then the volume of the breast is estimated as a function of the difference between the new surface and the actual breast. We have performed a number of experiments on both synthetic and real data with very encouraging results.

In this paper, we present a new 3D face recognition approach. Full automation is provided through the use of advanced multi-stage alignment algorithms, resilience to facial expressions by employing a deformable model framework, and invariance to 3D capture devices through suitable preprocessing steps. In addition, scalability in both time and space is achieved by converting 3D facial scans into compact wavelet metadata. We present results on the largest known, and now publicly-available, Face Recognition Grand Challenge 3D facial database consisting of several thousand scans. To the best of our knowledge, our approach has achieved the highest accuracy on this dataset.

Biometrics-based technologies in the area of identity management are gaining increasing importance, as a means of establishing non-falsifiable credentials for end users. However, in the three-way tug-of-war between convenient, unobtrusive data collection (required for user acceptance), accuracy in results (required for justifying deployment), and speed (required for widespread use in practice), no single biometric to date has managed to hold the middle ground that would allow for its ready adoption. The overall goal of our project is to develop the theoretical framework and computational tools that will lead to the development of a practical, unobtrusive, and accurate face recognition system for convenient and effective access control. This framework encompasses 8D characteristics of the face (3D geometry+2D visible texture+2D infrared texture, over time). In this paper, we present a novel multi-modal facial recognition approach that employs data from both visible spectrum and thermal infrared sensors. From the fitted parametric model we extract two images corresponding to the subject's face and process these images to extract biometric signatures. Specifically, the deformation image is compressed using a wavelet transform and the vasculature graph is extracted from the parametric thermal image.