Search

In this work, we analyse the comparison between using the periocular area instead of the full face area for Presentation Attack Detection (PAD) in the visual spectrum (RGB). The analysis was carried out by evaluating the performance of five Convolutional Neural Networks (CNN) using both facial and periocular iris images for PAD with two different attack instruments. Additionally, we improved the CNN results by integrating the ArcFace loss function instead of the traditional categorical cross-entropy loss, highlighting that the ArcFace function enhances the performance of the models for both regions of interest, facial and iris periocular areas. We conducted Binary and Multiclass comparisons, followed by cross-database validation to assess the generalization capabilities of the trained models. Our study also addresses some of the current challenges in PAD research, such as the limited availability of high-quality face datasets in the desired spectrum (RGB), which impacts the quality of Presentation Attack Instruments (PAI) examples used in training and evaluation. Our goal was to address the challenge of detecting Iris periocular presentation attacks by leveraging the ArcFace function. The results demonstrate the effectiveness of our approach and provide valuable insights for improving PAD systems using periocular areas in the visual spectrum.

Biometric data are uniquely suited for connecting individuals to their digital identities. Deriving cryptographic key exchange from successful biometric authentication therefore gives an additional layer of trust compared to password-authenticated key exchange. However, biometric data are sensitive personal data that need to be protected on a long-term basis. Furthermore, efficient feature extraction and comparison components resulting in high intra-subject tolerance and inter-subject distinguishability, documented with good biometric performance, need to be applied in order to prevent zero-effort impersonation attacks. In this work, we present a novel protocol for Biometric Resilient Authenticated Key Exchange that fulfils the above requirements of biometric information protection compliant with the international ISO/IEC 24745 standard. In our protocol, we present a novel modification of unlinkable fuzzy vault schemes that allows their connection with oblivious pseudo-random functions to achieve resilient protection against offline attacks crucial for the protection of biometric data. Our protocol is independent of the biometric modality and can be implemented based on the security of discrete logarithms as well as lattices. We provide an open-source implementation of both instantiations of our protocol which achieve real-time efficiency with transaction times of less than one second from the image capture to the completed key exchange.

Face Morphing Attacks pose a threat to the security of identity documents, especially with respect to a subsequent access control process, because they allow both involved individuals to use the same document. Several algorithms are currently being developed to detect Morphing Attacks, often requiring large data sets of morphed face images for training. In the present study, face embeddings are used for two different purposes: first, to pre-select images for the subsequent large-scale generation of Morphing Attacks, and second, to detect potential Morphing Attacks. Previous studies have demonstrated the power of embeddings in both use cases. However, we aim to build on these studies by adding the more powerful MagFace model to both use cases, and by performing comprehensive analyses of the role of embeddings in pre-selection and attack detection in terms of the vulnerability of face recognition systems and attack detection algorithms. In particular, we use recent developments to assess the attack potential, but also investigate the influence of morphing algorithms. For the first objective, an algorithm is developed that pairs individuals based on the similarity of their face embeddings. Different state-of-the-art face recognition systems are used to extract embeddings in order to pre-select the face images and different morphing algorithms are used to fuse the face images. The attack potential of the differently generated morphed face images will be quantified to compare the usability of the embeddings for automatically generating a large number of successful Morphing Attacks. For the second objective, we compare the performance of the embeddings of two state-of-the-art face recognition systems with respect to their ability to detect morphed face images. Our results demonstrate that ArcFace and MagFace provide valuable face embeddings for image pre-selection. Various open-source and commercial-off-the-shelf face recognition systems are vulnerable to the generated Morphing Attacks, and their vulnerability increases when image pre-selection is based on embeddings compared to random pairing. In particular, landmark-based closed-source morphing algorithms generate attacks that pose a high risk to any tested face recognition system. Remarkably, more accurate face recognition systems show a higher vulnerability to Morphing Attacks. Among the systems tested, commercial-off-the-shelf systems were the most vulnerable to Morphing Attacks. In addition, MagFace embeddings stand out as a robust alternative for detecting morphed face images compared to the previously used ArcFace embeddings. The results endorse the benefits of face embeddings for more effective image pre-selection for face morphing and for more accurate detection of morphed face images, as demonstrated by extensive analysis of various designed attacks. The MagFace model is a powerful alternative to the often-used ArcFace model in detecting attacks and can increase performance depending on the use case. It also highlights the usability of embeddings to generate large-scale morphed face databases for various purposes, such as training Morphing Attack Detection algorithms as a countermeasure against attacks.

With the rise of deep neural networks, the performance of biometric systems has increased tremendously. Biometric systems for face recognition are now used in everyday life, e.g., border control, crime prevention, or personal device access control. Although the accuracy of face recognition systems is generally high, they are not without flaws. Many biometric systems have been found to exhibit demographic bias, resulting in different demographic groups being not recognized with the same accuracy. This is especially true for facial recognition due to demographic factors, e.g., gender and skin color. While many previous works already reported demographic bias, this work aims to reduce demographic bias for biometric face recognition applications. In this regard, 12 face recognition systems are benchmarked regarding biometric recognition performance as well as demographic differentials, i.e., fairness. Subsequently, multiple fusion techniques are applied with the goal to improve the fairness in contrast to single systems. The experimental results show that it is possible to improve the fairness regarding single demographics, e.g., skin color or gender, while improving fairness for demographic subgroups turns out to be more challenging.

Gender classification on normalized iris images has been previously attempted with varying degrees of success. In these previous studies, it has been shown that occlusion masks may introduce gender information; occlusion masks are used in iris recognition to remove non-iris elements. When, the goal is to classify the gender using exclusively the iris texture, the presence of gender information in the masks may result in apparently higher accuracy, thereby not reflecting the actual gender information present in the iris. However, no measures have been taken to eliminate this information while preserving as much iris information as possible. We propose a novel method to assess the gender information present in the iris more accurately by eliminating gender information in the masks. This consists of pairing iris with similar masks and different gender, generating a paired mask using the OR operator, and applying this mask to the iris. Additionally, we manually fix iris segmentation errors to study their impact on the gender classification. Our results show that occlusion masks can account for 6.92% of the gender classification accuracy on average. Therefore, works aiming to perform gender classification using the iris texture from normalized iris images should eliminate this correlation.

Finger vein recognition is gaining popularity in the field of biometrics, yet the inter-operability of finger vein patterns has received limited attention. This study aims to fill this gap by introducing a cross-device finger vein dataset and evaluating the performance of finger vein recognition across devices using a classical method, a convolutional neural network, and our proposed patch-based convolutional auto-encoder (CAE). The findings emphasise the importance of standardisation of finger vein recognition, similar to that of fingerprints or irises, crucial for achieving inter-operability. Despite the inherent challenges of cross-device recognition, the proposed CAE architecture in this study demonstrates promising results in finger vein recognition, particularly in the context of cross-device comparisons.

Abstract Due to migration, terror‐threats and the viral pandemic, various EU member states have re‐established internal border control or even closed their borders. European Association for Biometrics (EAB), a non‐profit organisation, solicited the views of its members on ways which biometric technologies and services may be used to help with re‐establishing open borders within the Schengen area while at the same time mitigating any adverse effects. From the responses received, this position paper was composed to identify ideas to re‐establish free travel between the member states in the Schengen area. The paper covers the contending needs for security, open borders and fundamental rights as well as legal constraints that any technological solution must consider. A range of specific technologies for direct biometric recognition alongside complementary measures are outlined. The interrelated issues of ethical and societal considerations are also highlighted. Provided a holistic approach is adopted, it may be possible to reach a more optimal trade‐off with regards to open borders while maintaining a high‐level of security and protection of fundamental rights. European Association for Biometrics and its members can play an important role in fostering a shared understanding of security and mobility challenges and their solutions.

Face recognition systems are widely deployed for biometric authentication. Despite this, it is well-known that, without any safeguards, face recognition systems are highly vulnerable to presentation attacks. In response to this security issue, several promising methods for detecting presentation attacks have been proposed which show high performance on existing benchmarks. However, an ongoing challenge is the generalization of presentation attack detection methods to unseen and new attack types. To this end, we propose a new T-shirt Face Presentation Attack (TFPA) database of 1,608 T-shirt attacks using 100 unique presentation attack instruments. In an extensive evaluation, we show that this type of attack can compromise the security of face recognition systems and that some state-of-the-art attack detection mechanisms trained on popular benchmarks fail to robustly generalize to the new attacks. Further, we propose three new methods for detecting T-shirt attack images, one which relies on the statistical differences between depth maps of bona fide images and T-shirt attacks, an anomaly detection approach trained on features only extracted from bona fide RGB images, and a fusion approach which achieves competitive detection performance.

In the last decade, breakthroughs in the field of deep learning have led to the development of powerful presentation attack detection (PAD) algorithms which reported reliable performance across different realistic scenarios. Typically, most of these techniques analyse the full face to detect attack presentations (APs), ignoring that the attributes or artefacts produced in the fabrication of the attacks vary their location on the face depending on the presentation attack instruments (PAI) species, subject and environmental conditions. In addition, they still fail to categorise bona fide subjects who inadvertently occlude their face with accessories such as glasses, scarves or masks to prevent respiratory infections. To mitigate these issues, this paper explores the utility of using different facial regions for PAD. In this context, a new metric, Face Region Utility, is proposed, which indicates the usefulness of a particular test region to spot an attack attempt based on another training region. A thorough evaluation in challenging scenarios on well-known databases shows which face regions can successfully substitute the full face to detect APs in scenarios where pristine subjects use some of the mentioned accessories: up to a 67.73% of detection performance improvement is yielded by applying our proposed analysis when pristine subjects wear masks to prevent respiratory infections.

Iris Recognition (IR) is one of the market’s most reliable and accurate biometric systems. Today, it is challenging to build NearInfraRed (NIR) capturing devices under the premise of hardware price reduction. Commercial NIR sensors are protected from modification. The process of building a new device is not trivial because it is required to start from scratch with the process of capturing images with quality, calibrating operational distances, and building lightweight software such as eyes/iris detectors and segmentation sub-systems. In light of such challenges, this work aims to develop and implement iris recognition software in an embedding system and calibrate NIR in a contactless binocular setup. We evaluate and contrast speed versus performance obtained with two embedded computers and infrared cameras. Further, a lightweight segmenter sub-system called “Unet_xxs” is proposed, which can be used for iris semantic segmentation under restricted memory resources. The evaluations reveal that Unet_xxs reduces the number of parameters by 77% and duplicates the speed of state-of-the-art segmentation models with an EER drop smaller than 1% in Iris Recognition with 8.06 frame per second (fps) and Intersection Over Union (IOU) of 0.8382.

The rise of subscription-based business models in social media is part of a broader trend that can be observed in many industries. Against this background, it is necessary to adapt European consumer law to the new risks of the subscription economy.However, it is not enough to give consumers rights on paper. Nor is it sufficient to inform consumers about their rights in the small print. Effective consumer protection in digital markets requires a user interface design that enables consumers to exercise their rights with a simple click.

Abstract The LivDet‐2020 competition focuses on Presentation Attacks Detection (PAD) algorithms, has still open problems, mainly unknown attack scenarios. It is crucial to enhance PAD methods. This can be achieved by augmenting the number of Presentation Attack Instruments (PAI) and Bona fide (genuine) images used to train such algorithms. Unfortunately, the capture and creation of PAI and even the capture of Bona fide images are sometimes complex to achieve. The generation of synthetic images with Generative Adversarial Networks (GAN) algorithms may help and has shown significant improvements in recent years. This paper presents a benchmark of GAN methods to achieve a novel synthetic PAI from a small set of periocular near‐infrared images. The best PAI was obtained using StyleGAN2, and it was tested using the best PAD algorithm from the LivDet‐2020. The synthetic PAI was able to fool such an algorithm. As a result, all images were classified as Bona fide. A MobileNetV2 was trained using the synthetic PAI as a new class to achieve a more robust PAD. The resulting PAD was able to classify 96.7% of synthetic images as attacks. BPCER10 was 0.24%. Such results demonstrated the need for PAD algorithms to be constantly updated and trained with synthetic images.

Abstract Doppelgängers (or lookalikes) usually yield an increased probability of false matches in a facial recognition system, as opposed to random face image pairs selected for non‐mated comparison trials. In this work, the impact of doppelgängers on the HDA Doppelgänger and Disguised Faces in The Wild databases is assessed using a state‐of‐the‐art face recognition system. It is found that doppelgänger image pairs yield very high similarity scores resulting in a significant increase of false match rates. Further, a doppelgänger detection method is proposed, which distinguishes doppelgängers from mated comparison trials by analysing differences in deep representations obtained from face image pairs. The proposed detection system employs a machine learning‐based classifier, which is trained with generated doppelgänger image pairs utilising face morphing techniques. Experimental evaluations conducted on the HDA Doppelgänger and Look‐Alike Face databases reveal a detection equal error rate of approximately 2.7% for the task of separating mated authentication attempts from doppelgängers.

Abstract The growing scope, scale, and number of biometric deployments around the world emphasise the need for research into technologies facilitating efficient and reliable biometric identification queries. This work presents a method of indexing biometric databases, which relies on signal‐level fusion of facial images (morphing) to create a multi‐stage data structure and retrieval protocol. By successively pre‐filtering the list of potential candidate identities, the proposed method makes it possible to reduce the necessary number of biometric template comparisons to complete a biometric identification transaction. The proposed method is extensively evaluated on publicly available databases using open‐source and commercial off‐the‐shelf recognition systems. The results show that using the proposed method, the computational workload can be reduced down to around 30% while the biometric performance of a baseline exhaustive search‐based retrieval is fully maintained, both in closed‐set and open‐set identification scenarios.

Enhancing the privacy of machine learning (ML) algorithms has become crucial with the presence of different types of attacks on AI applications. Continual learning (CL) is a branch of ML with the aim of learning a set of knowledge sequentially and continuously from a data stream. On the other hand, differential privacy (DP) has been extensively used to enhance the privacy of deep learning (DL) models. However, the task of adding DP to CL would be challenging, because on one hand the DP intrinsically adds some noise that reduce the utility, on the other hand the endless learning procedure of CL is a serious obstacle, resulting in the catastrophic forgetting (CF) of previous samples of ongoing stream. To be able to add DP to CL, we have proposed a methodology by which we cannot only strike a tradeoff between privacy and utility, but also mitigate the CF. The proposed solution presents a set of key features: (1) it guarantees theoretical privacy bounds via enforcing the DP principle; (2) we further incorporate a robust procedure into the proposed DP-CL scheme to hinder the CF; and (3) most importantly, it achieves practical continuous training for a CL process without running out of the available privacy budget. Through extensive empirical evaluation on benchmark datasets and analyses, we validate the efficacy of the proposed solution.

Face inpainting is a challenging task aiming to fill the damaged or masked regions in face images with plausibly synthesized contents. Based on the given information, the reconstructed regions should look realistic and more importantly preserve the demographic and biometric properties of the individual. The aim of this paper is to reconstruct the face based on the periocular region (eyes-to-face). To do this, we proposed a novel GAN-based deep learning model called Eyes-to-Face GAN (E2F-GAN) which includes two main modules: a coarse module and a refinement module. The coarse module along with an edge predictor module attempts to extract all required features from a periocular region and to generate a coarse output which will be refined by a refinement module. Additionally, a dataset of eyes-to-face synthesis has been generated based on the public face dataset called CelebA-HQ for training and testing. Thus, we perform both qualitative and quantitative evaluations on the generated dataset. Experimental results demonstrate that our method outperforms previous learning-based face inpainting methods and generates realistic and semantically plausible images. We also provide the implementation of the proposed approach to support reproducible research via (https://github.com/amiretefaghi/E2F-GAN).

Selfie-based biometrics has great potential for a wide range of applications since, e.g. periocular verification is contactless and is safe to use in pandemics such as COVID-19, when a major portion of a face is covered by a facial mask. Despite its advantages, selfie-based biometrics presents challenges since there is limited control over data acquisition at different distances. Therefore, Super-Resolution (SR) has to be used to increase the quality of the eye images and to keep or improve the recognition performance. We propose an Efficient Single Image Super-Resolution algorithm, which takes into account a trade-off between the efficiency and the size of its filters. To that end, the method implements a loss function based on the Sharpness metric used to evaluate iris images quality. Our method drastically reduces the number of parameters compared to the state-of-the-art: from 2,170,142 to 28,654. Our best results on remote verification systems with no redimensioning reached an EER of 8.89% for FaceNet, 12.14% for VGGFace, and 12.81% for ArcFace. Then, embedding vectors were extracted from SR images, the FaceNet-based system yielded an EER of 8.92% for a resizing of x2, 8.85% for x3, and 9.32% for x4.

In recent years, increasing deployment of face recognition technology in security-critical settings, such as border control or law enforcement, has led to considerable interest in the vulnerability of face recognition systems to attacks utilising legitimate documents, which are issued on the basis of digitally manipulated face images. As automated manipulation and attack detection remains a challenging task, conventional processes with human inspectors performing identity verification remain indispensable. These circumstances merit a closer investigation of human capabilities in detecting manipulated face images, as previous work in this field is sparse and often concentrated only on specific scenarios and biometric characteristics. This work introduces a web-based, remote visual discrimination experiment on the basis of principles adopted from the field of psychophysics and subsequently discusses interdisciplinary opportunities with the aim of examining human proficiency in detecting different types of digitally manipulated face images, specifically face swapping, morphing, and retouching. In addition to analysing appropriate performance measures, a possible metric of detectability is explored. Experimental data of 306 probands indicate that detection performance is widely distributed across the population and detection of certain types of face image manipulations is much more challenging than others.

Single Image Super-Resolution (SISR) using Convolutional Neural Networks (CNNs) for many applications in supervised manner has resulted in significant improvement in state-of-the-art performance. Such supervised models achieve remarkable accuracy; albeit their poor generalization ability for real-world Low-Resolution (LR) images. Supervised training in many SR works involves synthetically generated LR images from its corresponding High-Resolution (HR) images. As the distribution of such LR observation is relatively different from that of real LR image, the supervised training in SISR task results in a degradation when applied on real-world data. SISR has been scaled to real-world data recently by posing the unsupervised problem into a supervised one through learning the distribution of noisy LR observation first, following which supervised training is performed to obtain the SR image. It therefore involves two steps where the accuracy of SR image relies on how closely the LR’s distribution is learnt in the first step. In this work, we overcome such limitation by introducing unsupervised denoising network to transform real noisy LR image to clean image and then pre-trained SR network is utilised to increase the spatial resolution of cleaned LR image to generate SR image. Thus, instead of evaluating the denoised image in LR space to train the denoising network, we inspect the denoised image in SR space which allows to overcome the SR network’s generalization problem. The proposed Unsupervised Denoising framework for Super-Resolution (UDSR) is validated on real-world datasets (NTIRE-2020 Real-World SR Challenge validation and testing dataset (Track-1)) by comparing it with many recent unsupervised SISR methods. The performance of denoising and SR networks is superior in terms of various perceptual indices such as Perceptual Index (PI) and Ma Score in addition to numerous non-references metrics.

Abstract The intention of this position paper is to comment on the joint European Data Protection Supervisor (EDPS)‐Agencia Española de Protección de Datos (aepd) publication ‘14 Misunderstandings with regard to Biometric Identification and Authentication’ that was published in June 2020 and to provide additional input to help with the better understanding of the issues raised in that publication. In particular, it aims to highlight some important missing information in the aforementioned publication. It is hoped that this paper will help with any future revision of the EDPS‐aepd publication, such that it includes a full picture of the current state of the art in biometrics and the availability of standards and privacy enhancing techniques.

Abstract The growth of biometrics‐based authentication in various services raises the need to protect biometric data at the storage level. Specifically, biometric templates need to be protected after features are extracted to avoid the leakage of biometric data and subsequent linkability issues. An approach based on discrete hashing is presented with the assistance of semantic labels to generate discriminative and privacy preserving protected templates in this work. The proposed approach can easily be adopted for a closed‐enrolment set in which enrolment images are known a priori whereas the challenge of learning templates for a single subject remains open. To extend this approach for individual subject, the concept of auxiliary pseudouser enrolment data is introduced, through which a protected template can be generated at the user level. Through the use of a moderately sized multimodal biometric database of 94 subjects, the effectiveness of the proposed approach is illustrated to achieve a robust and secure template protection with irreversibility, unlinkability and renewability. With the set of experiments, the performance of the template protection approach is established and benchmarked against the popular bloom‐filter technique. The proposed approach results in a high genuine match rate (≈100% at a false accept rate of 0.01%) and low equal error rate (EER ≈ 0%) and outperforms traditional approaches while satisfying other requirements of biometric template protection when the closed enrolment set is known. With auxiliary pseudousers, the performance of the proposed approach for user‐level protected template creation results in an EER of 2.5%, indicating very low performance degradation compared with the known enrolment dataset. Along with the set of experimental validation of the proposed approach, a security analysis of the proposed approach is presented to demonstrate the unlinkability of the biometric templates using a state‐of‐art unlinkability metric.

Abstract In recent years, face recognition technologies have shown impressive recognition performance, mainly due to recent developments in deep convolutional neural networks. Notwithstanding those improvements, several challenges which affect the performance of face recognition systems remain. The authors investigate the impact that facial tattoos and paintings have on current face recognition systems. To this end, they first collected an appropriate database containing image‐pairs of individuals with and without facial tattoos or paintings. The assembled database was used to evaluate how facial tattoos and paintings affect the detection, quality estimation, as well as the feature extraction and comparison modules of a face recognition system. The impact on these modules was evaluated using state‐of‐the‐art open‐source and commercial systems. The obtained results show that facial tattoos and paintings affect all the tested modules, especially for images where a large area of the face is covered with tattoos or paintings. The authors’ work is an initial case study and indicates a need to design algorithms which are robust to the visual changes caused by facial tattoos and paintings.

Abstract In past decades, the broad development experienced by biometric systems has unveiled several threats that may decrease their trustworthiness. These are attack presentations that can be easily carried out by a non‐authorised subject to gain access to the biometric system. To mitigate those security concerns, most face presentation attack detection techniques have reported good detection performance when they are evaluated on known presentation attack instruments (PAIs) and under acquisition conditions. In contrast, for more realistic scenarios, existing algorithms face difficulties in detecting unknown PAI species which are only included in the test set. A feature space based on Fisher Vectors computed from compact binarised statistical image features histograms, which allows discovering semantic feature subsets from known samples to enhance the detection of unknown attacks is presented. This representation, which is evaluated for challenging unknown attacks taken from freely available facial databases, reports promising results. A Bona Fide Presentation Classification Error Rate 100 under 17% together with an area under the receiving operating characteristic curve of over 98% can be achieved in the presence of unknown attacks. In addition, by training a limited number of parameters, the method is able to achieve state‐of‐the‐art deep learning‐based approaches for cross‐dataset scenarios.

Abstract Nowadays, fingerprint‐based biometric recognition systems are becoming increasingly popular. However, in spite of their numerous advantages, biometric capture devices are usually exposed to the public and thus vulnerable to presentation attacks (PAs). Therefore, presentation attack detection (PAD) methods are of utmost importance in order to distinguish between bona fide and attack presentations. Owing to the nearly unlimited possibilities to create new presentation attack instruments (PAIs), unknown attacks are a threat to the existing PAD algorithms. This fact motivates research on generalisation capabilities in order to find PAD methods that are resilient to new attacks. In this context, the authors evaluate the generalisability of multiple PAD algorithms on a dataset of 19,711 bona fide and 4339 PA samples, including 45 different PAI species. The PAD data is captured in the short wave infrared domain, and the results discuss the advantages and drawbacks of this PAD technique regarding unknown attacks.

Abstract In spite of the advantages of using fingerprints for subject authentication, several works have shown that fingerprint recognition systems can be easily circumvented by means of artificial fingerprints or presentation attack instruments (PAIs). In order to address that threat, the existing presentation attack detection (PAD) methods have reported a high detection performance when materials used for the fabrication of PAIs and capture devices are known. However, for more complex and realistic scenarios where one of those factors remains unknown, these PAD methods are unable to correctly separate a PAI from a real fingerprint (i.e. bona fide presentation). In this article, a new PAD approach based on the Fisher Vector technique, which combines local and global information of several local feature descriptors in order to improve the PAD generalisation capabilities, was proposed. The experimental results over unknown scenarios taken from LivDet 2011 to LivDet 2017 show that our proposal reduces the top state‐of‐the‐art average classification error rates by up to four times, thereby making it suitable in real applications demanding high security. In addition, the best single configuration achieved the best results in the LivDet 2019 competition, with an overall accuracy of 96.17%.

Abstract Numerous methods have been introduced to reliably detect digital face image manipulations in the past. Of late, the generalisability of these schemes has been questioned in particular with respect to image post‐processing. Image compression represents a post‐processing which is frequently applied in diverse biometric application scenarios. Severe compression might erase digital traces of face image manipulation and hence hamper a reliable detection thereof. In this work, the effects of image compression on the face image manipulation detection are analysed. In particular, a case study on facial retouching detection under the influence of image compression is presented. To this end, ICAO‐compliant subsets of two public face databases are used to automatically create a database containing more than 9000 retouched reference images together with unconstrained probe images. Subsequently, reference images are compressed applying JPEG and JPEG 2000 at compression levels recommended for face image storage in electronic travel documents. Novel detection algorithms utilising texture descriptors and deep face representations are proposed and evaluated in a single image and differential scenario. Results obtained from challenging cross‐database experiments in which the analysed retouching technique is unknown during training yield interesting findings: (1) most competitive detection performance is achieved for differential scenarios employing deep face representations; (2) image compression severely impacts the performance of face image manipulation detection schemes based on texture descriptors while methods utilising deep face representations are found to be highly robust; (3) in some cases, the application of image compression might as well improve detection performance.

Abstract Touchless fingerprint recognition represents a rapidly growing field of research which has been studied for more than a decade. Through a touchless acquisition process, many issues of touch-based systems are circumvented, e.g., the presence of latent fingerprints or distortions caused by pressing fingers on a sensor surface. However, touchless fingerprint recognition systems reveal new challenges. In particular, a reliable detection and focusing of a presented finger as well as an appropriate preprocessing of the acquired finger image represent the most crucial tasks. Also, further issues, e.g., interoperability between touchless and touch-based fingerprints or presentation attack detection, are currently investigated by different research groups. Many works have been proposed so far to put touchless fingerprint recognition into practice. Published approaches range from self identification scenarios with commodity devices, e.g., smartphones, to high performance on-the-move deployments paving the way for new fingerprint recognition application scenarios.This work summarizes the state-of-the-art in the field of touchless 2D fingerprint recognition at each stage of the recognition process. Additionally, technical considerations and trade-offs of the presented methods are discussed along with open issues and challenges. An overview of available research resources completes the work.

This special issue tackles the question of whether and how data shapes private law. The development of new technologies enabled the generation, collection and processing of both personal and non-personal data on an unprecedented scale. The implications of this phenomenon for private law are threefold. One, how does data affect our understanding of technology regulation in private law relationships? Two, how does data affect the way in which private law is applied? Three, what is the role of data in the design of law from a public policy perspective that transcends doctrinal considerations relating to private law?

People differ with respect to their preferences, personalities, cognitive abilities, or attitudes. Yet the way in which private law has evolved in the past centuries sacrifices heterogeneity for the sake of the legal certainty that flows out of generalizations and typifications. Law distinguishes between different groups of individuals such as consumers and professionals, or even between average and vulnerable consumers. These groups are however based on conspicuous features that would justify differential treatment. For instance, determining a profile of the average consumer requires a context, such as a given industry or age group, and reflects specific considerations such as the consumer’s skills in retrieving information about a transaction. Decades of research in psychology and behavioral economics generated a tremendous amount of knowledge about people’s behavior, creating typologies with regard to their personality traits, intertemporal or social preferences as well as cognitive skills. Later use of Big Data analysis showed that these types can be predictive of people’s behavior, as well as informational needs or other specific characteristics. Recently, legal scholars proposed that insights generated by this research on granular legal rules could be embedded in private law by, for instance, introducing different default rules or privacy disclosures depending on people’s personality traits or preferences. This special issue tackles the question of whether and how data shapes private law. The development of new technologies enables the collection and processing of both personal and non-personal data at an unprecedented scale. The implications of this phenomenon for private law are twofold. On the one hand, the use of data in interactions between individuals may require adjustments or reconceptualization of private law rules and principles. On the other hand, data might be also used by legislators to help create new private law rules as well as to develop consumer empowerment tools to balance their position when transacting with businesses. Taking these different perspectives, the papers included in this special issue explore the implications of data for private law. The first article by Antonio Davola addresses the question of how the law deals with the use of data by businesses in their interactions with consumers. Davola analyzes existing private law rules on defective consent and argues that these rules could offer potential protection to consumers when they are targeted by businesses’ personalized commercial practices. He juxtaposes this solution with those provided by consumer law and data protection regulation, as well as competition law. Exploring the second perspective – how data can be used in the development of private law – Fabiana di Porto, Tatjana Grote, Gabriele Volpi, Riccardo Invernizzi demonstrate how data can be relied on in the legislative process. Di Porto et al. propose an automated text analysis method to extract information from contributions submitted by stakeholders in the process of public consultation. Specifically, the authors compare the use and understanding of core terms by various stakeholder groups consulted when developing proposals of Digital Markets Act and Digital Services Act. Further papers in the special issue will be announced soon.

Biometric recognition is a trending technology that uses unique characteristics data to identify or verify/authenticate security applications. Amidst the classically used biometrics, voice and face attributes are the most propitious for prevalent applications in day-to-day life because they are easy to obtain through restrained and user-friendly procedures. The pervasiveness of low-cost audio and face capture sensors in smartphones, laptops, and tablets has made the advantage of voice and face biometrics more exceptional when compared to other biometrics. For many years, acoustic information alone has been a great success in automatic speaker verification applications. Meantime, the last decade or two has also witnessed a remarkable ascent in face recognition technologies. Nonetheless, in adverse unconstrained environments, neither of these techniques achieves optimal performance. Since audio-visual information carries correlated and complementary information, integrating them into one recognition system can increase the system's performance. The vulnerability of biometrics towards presentation attacks and audio-visual data usage for the detection of such attacks is also a hot topic of research. This paper made a comprehensive survey on existing state-of-the-art audio-visual recognition techniques, publicly available databases for benchmarking, and Presentation Attack Detection (PAD) algorithms. Further, a detailed discussion on challenges and open problems is presented in this field of biometrics.

Fingerprint-based biometric systems have experienced a large development in the past. In spite of many advantages, they are still vulnerable to attack presentations (APs). Therefore, the task of determining whether a sample stems from a live subject (i.e., bona fide) or from an artificial replica is a mandatory requirement which has recently received a considerable attention. Nowadays, when the materials for the fabrication of the Presentation Attack Instruments (PAIs) have been used to train the Presentation Attack Detection (PAD) methods, the PAIs can be successfully identified in most cases. However, current PAD methods still face difficulties detecting PAIs built from unknown materials and/or unknown recepies, or acquired using different capture devices. To tackle this issue, we propose a new PAD technique based on three image representation approaches combining local and global information of the fingerprint. By transforming these representations into a common feature space, we can correctly discriminate bona fide from attack presentations in the aforementioned scenarios. The experimental evaluation of our proposal over the LivDet 2011 to 2019 databases, yielded error rates outperforming the top state-of-the-art results by up to 72% in the most challenging scenarios. In addition, the best representation achieved the best results in the LivDet 2019 competition (overall accuracy of 96.17%).

Smartphones have been employed with biometric-based verification systems to provide security in highly sensitive applications. Audio-visual biometrics are getting popular due to their usability, and also it will be challenging to spoof because of their multimodal nature. In this work, we present an audio-visual smartphone dataset captured in five different recent smartphones. This new dataset contains 103 subjects captured in three different sessions considering the different real-world scenarios. Three different languages are acquired in this dataset to include the problem of language dependency of the speaker recognition systems. These unique characteristics of this dataset will pave the way to implement novel state-of-the-art unimodal or audio-visual speaker recognition systems. We also report the performance of the bench-marked biometric verification systems on our dataset. The robustness of biometric algorithms is evaluated towards multiple dependencies like signal noise, device, language and presentation attacks like replay and synthesized signals with extensive experiments. The obtained results raised many concerns about the generalization properties of state-of-the-art biometrics methods in smartphones.

Face Age Progression (FAP) refers to synthesizing face images while simulating ageing effects, thus enabling predicting the future appearance of an individual. The generation of age-progressed face images brings benefits for various applications, ranging from face recognition systems to forensic investigations and digital entertainment. In particular, the recent success achieved with deep generative networks significantly leveraged the quality of age-synthesized face images in terms of visual fidelity, ageing accuracy and identity preservation. However, the high number of contributions in recent years requires systematically structuring new findings and ideas to identify a common taxonomy, accelerate future research and reduce redundancy. Therefore, we present a comparative analysis of recent deep learning based face age progression methods for both adult and child-based face ageing, broken down into three high-level concepts: translation-based, condition-based, and sequence-based FAP. Further, we offer a comprehensive summary of the most common performance evaluation techniques, cross-age datasets, and open challenges to steer future research in the right direction.

This paper proposes a new framework to detect, segment, and estimate the localization of the eyes from a periocular Near-Infra-Red iris image under alcohol consumption. This stage will take part in the final solution to measure the fitness for duty. Fitness systems allow us to determine whether a person is physically or psychologically able to perform their tasks. Our segmentation framework is based on an object detector trained from scratch to detect both eyes from a single image. Then, two efficient networks were used for semantic segmentation; a Criss-Cross attention network and DenseNet10, with only 122,514 and 210,732 parameters, respectively. These networks can find the pupil, iris, and sclera. In the end, the binary output eye mask is used for pupil and iris diameter estimation with high precision. Five state-of-the-art algorithms were used for this purpose. A mixed proposal reached the best results. A second contribution is establishing an alcohol behavior curve to detect the alcohol presence utilizing a stream of images captured from an iris instance. Also, a manually labeled database with more than 20k images was created. Our best method obtains a mean Intersection-over-Union of 94.54% with DenseNet10 with only 210,732 parameters and an error of only 1-pixel on average.

Computationally efficient, accurate, and privacy-preserving data storage and retrieval are among the key challenges faced by practical deployments of biometric identification systems worldwide. In this work, a method of protected indexing of biometric data is presented. By utilising feature-level fusion of intelligently paired templates, a multi-stage search structure is created. During retrieval, the list of potential candidate identities is successively pre-filtered, thereby reducing the number of template comparisons necessary for a biometric identification transaction. Protection of the biometric probe templates, as well as the stored reference templates and the created index is carried out using homomorphic encryption. The proposed method is extensively evaluated in closed-set and open-set identification scenarios on publicly available databases using two state-of-the-art open-source face recognition systems. With respect to a typical baseline algorithm utilising an exhaustive search-based retrieval algorithm, the proposed method enables a reduction of the computational workload associated with a biometric identification transaction by 90%, while simultaneously suffering no degradation of the biometric performance. Furthermore, by facilitating a seamless integration of template protection with open-source homomorphic encryption libraries, the proposed method guarantees unlinkability, irreversibility, and renewability of the protected biometric data.

Face morphing attack detection is a challenging task. Automatic classification methods and manual inspection are realised in automatic border control gates to detect morphing attacks. Understanding how a machine learning system can detect morphed faces and the most relevant facial areas is crucial. Those relevant areas contain texture signals that allow us to separate the bona fide and the morph images. Also, it helps in the manual examination to detect a passport generated with morphed images. This paper explores features extracted from intensity, shape, texture, and proposes a feature selection stage based on the Mutual Information filter to select the most relevant and less redundant features. This selection allows us to reduce the workload and know the exact localisation of such areas to understand the morphing impact and create a robust classifier. The best results were obtained for the method based on Conditional Mutual Information and Shape features using only 500 features for FERET images and 800 features for FRGCv2 images from 1,048 features available. The eyes and nose are identified as the most critical areas to be analysed.

Systems that analyse faces have seen significant improvements in recent years and are today used in numerous application scenarios. However, these systems have been found to be negatively affected by facial alterations such as tattoos. To better understand and mitigate the effect of facial tattoos in facial analysis systems, large datasets of images of individuals with and without tattoos are needed. To this end, we propose a generator for automatically adding realistic tattoos to facial images. Moreover, we demonstrate the feasibility of the generation by using a deep learning-based model for removing tattoos from face images. The experimental results show that it is possible to remove facial tattoos from real images without degrading the quality of the image. Additionally, we show that it is possible to improve face recognition accuracy by using the proposed deep learning-based tattoo removal before extracting and comparing facial features.

Conventional fingerprint sensors that are deployed in real-life applications lack the ability to peer inside a finger beyond the external surface. Subsurface information can provide complimentary biometric characteristics associated with the finger. The subsurface fingerprints can also be employed when the quality of the external/surface fingerprints is affected. One of the most promising technologies for imaging below the surface of an external fingerprint is full-field optical coherent tomography (FF-OCT). However, the FF-OCT can be expensive and cumbersome, despite its proven ability for biometric use. In this paper, we describe the design and implementation of a compact, mobile and cost-effective FF-OCT sensor that is stable and easy to use. The newly designed sensor, being 30 cm × 30 cm × 10 cm in size, comprises of a dedicated silicon camera, stable Michelson interferometer and a bright Near-Infra-Red (NIR) light emitting diode. It enables recording of 1.7 cm × 1.7 cm images of subsurface finger features, such as internal fingerprints and sweat ducts. We show the employability of the newly designed sensor for different applications. Specifically, we validate its usefulness by capturing subsurface fingerprints of 585 subjects leading to 3510 unique fingerprints. The resulting accuracy of 0.74% as Equal Error Rate (EER) indicates the backward compatibility of the proposed sensor with the existing commercial off-the-shelf algorithms. Thanks to the large fingerprint database collected in this work we determined the most useful imaging depth for the fingerprint matching purposes to be around 100 gym. As an additional advantage, the sensor could be readily used in other applications with little or no modification, such as in vivo skin imaging.

The application of facial cosmetics may cause substantial alterations in the facial appearance, which can degrade the performance of facial biometrics systems. Additionally, it was recently demonstrated that makeup can be abused to launch so-called makeup presentation attacks. More precisely, an attacker might apply heavy makeup to obtain the facial appearance of a target subject with the aim of impersonation or to conceal their own identity. We provide a comprehensive survey of works related to the topic of makeup presentation attack detection, along with a critical discussion. Subsequently, we assess the vulnerability of a commercial off-the-shelf and an open-source face recognition system against makeup presentation attacks. Specifically, we focus on makeup presentation attacks with the aim of impersonation employing the publicly available Makeup Induced Face Spoofing (MIFS) and Disguised Faces in the Wild (DFW) databases. It is shown that makeup presentation attacks might seriously impact the security of face recognition systems. Further, we propose different image pair-based, i.e. differential, attack detection schemes which analyse differences in feature representations obtained from potential makeup presentation attacks and corresponding target face images. The proposed detection systems employ various types of feature extractors including texture descriptors, facial landmarks, and deep (face) representations. To distinguish makeup presentation attacks from genuine, i.e. bona fide presentations, machine learning-based classifiers are used. The classifiers are trained with a large number of synthetically generated makeup presentation attacks utilising a generative adversarial network for facial makeup transfer in conjunction with image warping. Experimental evaluations conducted using the MIFS database and a subset of the DFW database reveal that deep face representations achieve competitive detection equal error rates of 0.7% and 1.8%, respectively.

Despite the long history of fingerprint biometrics and its use to authenticate individuals, there are still some unsolved challenges with fingerprint acquisition and presentation attack detection (PAD). Currently available commercial fingerprint capture devices struggle with non-ideal skin conditions, including soft skin in infants. They are also susceptible to presentation attacks, which limits their applicability in unsupervised scenarios such as border control. Optical coherence tomography (OCT) could be a promising solution to these problems. In this work, we propose a digital signal processing chain for segmenting two complementary fingerprints from the same OCT fingertip scan: One fingerprint is captured as usual from the epidermis (“outer fingerprint”), whereas the other is taken from inside the skin, at the junction between the epidermis and the underlying dermis (“inner fingerprint”). The resulting 3D fingerprints are then converted to a conventional 2D grayscale representation from which minutiae points can be extracted using existing methods. Our approach is device-independent and has been proven to work with two different time domain OCT scanners. Using efficient GPGPU computing, it took less than a second to process an entire gigabyte of OCT data. To validate the results, we captured OCT fingerprints of 130 individual fingers and compared them with conventional 2D fingerprints of the same fingers. We found that both the outer and inner OCT fingerprints were backward compatible with conventional 2D fingerprints, with the inner fingerprint generally being less damaged and, therefore, more reliable.

Recently, researchers found that the intended generalizability of (deep) face recognition systems increases their vulnerability against attacks. In particular, the attacks based on morphed face images pose a severe security risk to face recognition systems. In the last few years, the topic of (face) image morphing and automated morphing attack detection has sparked the interest of several research laboratories working in the field of biometrics and many different approaches have been published. In this paper, a conceptual categorization and metrics for an evaluation of such methods are presented, followed by a comprehensive survey of relevant publications. In addition, technical considerations and tradeoffs of the surveyed methods are discussed along with open issues and challenges in the field.

Facial beautification induced by plastic surgery, cosmetics or retouching has the ability to substantially alter the appearance of face images. Such types of beautification can negatively affect the accuracy of face recognition systems. In this work, a conceptual categorisation of beautification is presented, relevant scenarios with respect to face recognition are discussed, and related publications are revisited. Additionally, technical considerations and trade-offs of the surveyed methods are summarized along with open issues and challenges in the field. This survey is targeted to provide a comprehensive point of reference for biometric researchers and practitioners working in the field of face recognition, who aim at tackling challenges caused by facial beautification.

This work presents an automated contactless fingerprint recognition system for smartphones. We provide a comprehensive description of the entire recognition pipeline and discuss important requirements for a fully automated capturing system. In addition, our implementation is made publicly available for research purposes. During a database acquisition, a total number of 1360 contactless and contact-based samples of 29 subjects are captured in two different environmental situations. Experiments on the acquired database show a comparable performance of our contactless scheme and the contact-based baseline scheme under constrained environmental influences. A comparative usability study on both capturing device types indicates that the majority of subjects prefer the contactless capturing method. Based on our experimental results, we analyze the impact of the current COVID-19 pandemic on fingerprint recognition systems. Finally, implementation aspects of contactless fingerprint recognition are summarized.

Within the last few decades, the need for subject authentication has grown steadily, and biometric recognition technology has been established as a reliable alternative to passwords and tokens, offering automatic decisions. However, as unsupervised processes, biometric systems are vulnerable to presentation attacks targeting the capture devices, where presentation attack instruments (PAI) instead of bona fide characteristics are presented. Due to the capture devices being exposed to the public, any person could potentially execute such attacks. In this work, a fingerprint capture device based on thin film transistor (TFT) technology has been modified to additionally acquire the impedances of the presented fingers. Since the conductance of human skin differs from artificial PAIs, those impedance values were used to train a presentation attack detection (PAD) algorithm. Based on a dataset comprising 42 different PAI species, the results showed remarkable performance in detecting most attack presentations with an APCER = 2.89% in a user-friendly scenario specified by a BPCER = 0.2%. However, additional experiments utilising unknown attacks revealed a weakness towards particular PAI species.

The term “plenoptic” comes from the Latin words plenus (“full”) + optic. The plenoptic function is the 7-dimensional function representing the intensity of the light observed from every position and direction in 3-dimensional space. Thanks to the plenoptic function it is thus possible to define the direction of every ray in the light-field vector function. Imaging systems are rapidly evolving with the emergence of light-field-capturing devices. Consequently, existing image-processing techniques need to be revisited to match the richer information provided. This article explores the use of light fields for face analysis. This field of research is very recent but already includes several works reporting promising results. Such works deal with the main steps of face analysis and include but are not limited to: face recognition; face presentation attack detection; facial soft-biometrics classification; and facial landmark detection. This article aims to review the state of the art on light fields for face analysis, identifying future challenges and possible applications.

In a biometric authentication system using protected templates, a pseudonymous identifier is the part of a protected template that can be directly compared. Each compared pair of pseudonymous identifiers results in a decision testing whether both identifiers are derived from the same biometric characteristic. Compared to an unprotected system, most existing biometric template protection methods cause to a certain extent degradation in biometric performance. Fusion is therefore a promising way to enhance the biometric performance in template-protected biometric systems. Compared to feature level fusion and score level fusion, decision level fusion has not only the least fusion complexity, but also the maximum interoperability across different biometric features, template protection and recognition algorithms, templates formats, and comparison score rules. However, performance improvement via decision level fusion is not obvious. It is influenced by both the dependency and the performance gap among the conducted tests for fusion. We investigate in this paper several fusion scenarios (multi-sample, multi-instance, multi-sensor, multi-algorithm, and their combinations) on the binary decision level, and evaluate their biometric performance and fusion efficiency on a multi-sensor fingerprint database with 71,994 samples.