Search

In recent years, the increasing availability of personal data has raised concerns regarding privacy and security. One of the critical processes to address these concerns is data anonymization, which aims to protect individual privacy and prevent the release of sensitive information. This research focuses on the importance of face anonymization. Therefore, we introduce GANonymization, a novel face anonymization framework with facial expression-preserving abilities. Our approach is based on a high-level representation of a face, which is synthesized into an anonymized version based on a generative adversarial network (GAN). The effectiveness of the approach was assessed by evaluating its performance in removing identifiable facial attributes to increase the anonymity of the given individual face. Additionally, the performance of preserving facial expressions was evaluated on several affect recognition datasets and outperformed the state-of-the-art methods in most categories. Finally, our approach was analyzed for its ability to remove various facial traits, such as jewelry, hair color, and multiple others. Here, it demonstrated reliable performance in removing these attributes. Our results suggest that GANonymization is a promising approach for anonymizing faces while preserving facial expressions. [ABSTRACT FROM AUTHOR]

Vision-based driver monitoring, a non-invasive method designed to identify potentially dangerous operations, has attracted increasing attention in recent years. In this study, a head pitch angle detection method was established to evaluate the driver's drowsiness. Rather than employing the front facial landmarks to estimate head pitch angle, the proposed method measure this angel directly from driver's profile face. To meet the requirement of real-time detection, the method applies the YOLOv8 network of single-stage detection and utilizes MobileNetV3 and FasterNet for lightweight improvement. The detector is trained with re-labeled CFP datasets, and real-time speed tests have been performed. Results demonstrate that the non-improved detector can achieve an mAP50 of 97.3% of the keypoints in a single frame, meanwhile realizing the frame rate of 30.41 FPS. After improvement, parameters of the model have been reduced by 21.3% and 40.9% respectively, while the frame rate can be increased to 37.13 FPS and 52.70 FPS, and the mAP50 of keypoints is increased by 0.41% and 0.51%. The results during the in-car experiment have proved that the developed detection method can effectively evaluate the head pitch angle, thus detect the driver's drowsiness. We provide open-access to the annotated data and pre-trained models in this study. [ABSTRACT FROM AUTHOR]

In this paper, based on facial landmark approaches, the possible vulnerability of ensemble algorithms to the FGSM attack has been assessed using three commonly used models: convolutional neural network-based antialiasing (A_CNN), Xc_Deep2-based DeepLab v2, and SqueezeNet (Squ_Net)-based Fire modules. Firstly, the three individual deep learning classifier-based Facial Emotion Recognition (FER) classifications have been developed; the predictions from all three classifiers are then merged using majority voting to develop the HEM_Net-based ensemble model. Following that, an in-depth investigation of their performance in the case of attack-free has been carried out in terms of the Jaccard coefficient, accuracy, precision, recall, F1 score, and specificity. When applied to three benchmark datasets, the ensemble-based method (HEM_Net) significantly outperforms in terms of precision and reliability while also decreasing the dimensionality of the input data, with an accuracy of 99.3%, 87%, and 99% for the Extended Cohn-Kanade (CK+), Real-world Affective Face (RafD), and Japanese female facial expressions (Jaffee) data, respectively. Further, a comprehensive analysis of the drop in performance of every model affected by the FGSM attack is carried out over a range of epsilon values (the perturbation parameter). The results from the experiments show that the advised HEM_Net model accuracy declined drastically by 59.72% for CK + data, 42.53% for RafD images, and 48.49% for the Jaffee dataset when the perturbation increased from A to E (attack levels). This demonstrated that a successful Fast Gradient Sign Method (FGSM) can significantly reduce the prediction performance of all individual classifiers with an increase in attack levels. However, due to the majority voting, the proposed HEM_Net model could improve its robustness against FGSM attacks, indicating that the ensemble can lessen deception by FGSM adversarial instances. This generally holds even as the perturbation level of the FGSM attack increases. [ABSTRACT FROM AUTHOR]

Abstract In this paper, based on facial landmark approaches, the possible vulnerability of ensemble algorithms to the FGSM attack has been assessed using three commonly used models: convolutional neural network-based antialiasing (A_CNN), Xc_Deep2-based DeepLab v2, and SqueezeNet (Squ_Net)-based Fire modules. Firstly, the three individual deep learning classifier-based Facial Emotion Recognition (FER) classifications have been developed; the predictions from all three classifiers are then merged using majority voting to develop the HEM_Net-based ensemble model. Following that, an in-depth investigation of their performance in the case of attack-free has been carried out in terms of the Jaccard coefficient, accuracy, precision, recall, F1 score, and specificity. When applied to three benchmark datasets, the ensemble-based method (HEM_Net) significantly outperforms in terms of precision and reliability while also decreasing the dimensionality of the input data, with an accuracy of 99.3%, 87%, and 99% for the Extended Cohn-Kanade (CK+), Real-world Affective Face (RafD), and Japanese female facial expressions (Jaffee) data, respectively. Further, a comprehensive analysis of the drop in performance of every model affected by the FGSM attack is carried out over a range of epsilon values (the perturbation parameter). The results from the experiments show that the advised HEM_Net model accuracy declined drastically by 59.72% for CK + data, 42.53% for RafD images, and 48.49% for the Jaffee dataset when the perturbation increased from A to E (attack levels). This demonstrated that a successful Fast Gradient Sign Method (FGSM) can significantly reduce the prediction performance of all individual classifiers with an increase in attack levels. However, due to the majority voting, the proposed HEM_Net model could improve its robustness against FGSM attacks, indicating that the ensemble can lessen deception by FGSM adversarial instances. This generally holds even as the perturbation level of the FGSM attack increases.

The investigation of demographic bias in facial analysis applications is a topic of growing interest with achievements in face recognition and gender classification. State-of-the-art convolutional neural networks (CNN) and traditional machine learning models for locating facial landmarks have reached overall performance levels close to human annotation. However, recent studies demonstrated that these models presented performance gaps when applied to different populations, characterizing bias led by demographic features. Nevertheless, few studies have addressed this problem in face alignment and facial landmarks localization methods. In this work, we propose a multi-level face alignment approach settled on CNN models to reduce performance gaps among different populations. We created facial subunit CNN models tied to a facial subunit detector at a higher level. The proposal seeks to improve bad results caused by facial impairment, guided by the following assumptions: facial unit landmarks localization does not require global texture, and combining different facial unit models can improve the final model's variability. We applied the models in a balanced dataset mixing healthy controls and individuals with neurological disorders: amyotrophic lateral sclerosis and post-stroke. With fewer samples for training, our approach significantly reduced face alignment performance differences among those groups as compared to a state-of-the-art CNN model. [ABSTRACT FROM AUTHOR]

Micro-expression recognition (MER) holds significance in uncovering hidden emotions. Most works take image sequences as input and cannot effectively explore ME information because subtle ME-related motions are easily submerged in unrelated information. Instead, the facial landmark is a low-dimensional and compact modality, which achieves lower computational cost and potentially concentrates on ME-related movement features. However, the discriminability of facial landmarks for MER is unclear. Thus, this article investigates the contribution of facial landmarks and proposes a novel framework to efficiently recognize MEs with facial landmarks. First, a geometric two-stream graph network is constructed to aggregate the low-order and high-order geometric movement information from facial landmarks to obtain discriminative ME representation. Second, a self-learning fashion is introduced to automatically model the dynamic relationship between nodes even long-distance nodes. Furthermore, an adaptive action unit loss is proposed to reasonably build a strong correlation between landmarks, facial action units and MEs. Notably, this work provides a novel idea with much higher efficiency to promote MER, only utilizing graph-based geometric features. The experimental results demonstrate that the proposed method achieves competitive performance with a significantly reduced computational cost. Furthermore, facial landmarks significantly contribute to MER and are worth further study for high-efficient ME analysis. [ABSTRACT FROM AUTHOR]

Objectives: To understand attentional preferences for normal and microtia ears. Methods: Eye‐tracking technology was used to characterize gaze preferences. A total of 71 participants viewed images of 5 patients with unilateral microtia. Profile images of patient faces and isolated ears including normal, microtia, and post‐reconstruction microtia ears were shown. Total time of fixation in predefined areas of interest (AOI) was measured. Inferential statistics were used to assess significance of fixation differences between AOIs within and between facial or auricular features. Results: The ear received most visual attention in lateral view of the face (1.91 s, 1.66–2.16 s) [mean, 95% CI], followed by features of the "central triangle"—the eyes (1.26 s, 1.06–1.46), nose (0.48 s, 0.38–0.58), and mouth (0.15 s, 0.15–0.20). In frontal view, microtia ears received less attention following surgical reconstruction (0.74 s vs. 0.4 s, p < 0.001). The concha was the most attended feature for both normal (2.97 s, 2.7–3.23) and reconstructed microtia ears (1.87 s, 1.61–2.13). Scars on reconstructed ears altered the typical visual scanpath. Conclusion: The ear is an attentional gaze landmark of the face. Attention to microtia ears, both pre‐ and post‐reconstruction, differs from gaze patterns of normal ears. The concha was the most attended to subunit of the ear. Attentional gaze may provide an unbiased method to determine what is important in reconstructive surgery. Level of Evidence: NA Laryngoscope, 134:3136–3142, 2024 [ABSTRACT FROM AUTHOR]

Facial anatomy is highly individual in each patient. Anthropometric measurements can be a useful tool to objectively analyze individual facial anatomy to allow for better comparability before and after treatments to ultimately improve standardization of facial procedures, both nonsurgical and surgical. The aim of this study was to provide a comprehensive overview over clinically relevant and feasible facial anthropometric measurements and principles for aesthetic medicine. A literature review was conducted to describe the most important and clinically relevant anthropometric measurements and principles for both the entire face and for three aesthetically relevant facial regions: the periorbital region, the nose, and the perioral region. A multitude of different anthropometric measurements and principles have been described in the literature for both the overall facial appearance and specific facial regions. Certain generally accepted anthropometric principles and proportions need to be respected to achieve aesthetic and harmonious results. For the overall facial appearance, a focus on symmetry, certain proportions, facial angles, and indices has been described. Principles and measurements were also described for the periorbital region, the nose, and the perioral region. Although attractiveness and aesthetic perception are subjective, objective evaluation of facial surface anatomy via anthropometric measurements can improve pre- and postinterventional analysis of the face and help the treating physician to individualize treatments, both nonsurgical and surgical. [ABSTRACT FROM AUTHOR]

Different kinds of occlusion have proven to disrupt the accuracy of face recognition systems, one of them being masks. The problem of masked faces has become even more apparent with the widespread of the COVID-19 virus, with most people wearing masks in public. This brings up the issue of existing face recognition systems been able to accurately recognize people even when part of their face and the major identifiers (such as the nose and mouth) are covered by a facemask. In addition, most of the databases that have been curated in different organizations, countries are majorly of non-masked faces, and masked databases are rarely stored or universally accepted compared with conventional face datasets. Therefore, this paper aim at the development of a Masked Face Recognition System using facial anthropometrics technique (FAT). FAT is the science of calculating the measurements, proportion and dimension of human face and their features. A dataset of faces with individual wearing medical face mask was curated. Using the Facial anthropometry based technique a Masked Face Recognition System developed. This system was implemented using Local Binary Patterns Histogram algorithms for recognition. On testing the developed system trained with unmasked dataset, show a high recognition performance of 94% and 96.8% for masked and non-masked face recognition respectively because of the Facial anthropometry based technique adapted. On deployment, users were been recognized when they are wearing a mask with part of their face covered in real-time. [ABSTRACT FROM AUTHOR]

Consumer feedback is often used for various purposes in many fields. However, traditional paper questionnaires or surveys cannot fully meet the demands for accurately understanding consumers' feelings. Consumers often convey their feelings through their facial expressions, whether consciously or unconsciously. Understanding these feelings can provide very direct and useful feedbacks. Yet, humans may miss those subtle changes because the micro expression is too brief to be captured. Therefore, in this paper. we proposed a deep learning based recognition approach of fucial micro expressions, in which more realistic emotional feedback of users can be extracted. To achieve this goal, we integrated several approaches including: ( 1) using trained face detection model to capture face image from input; (2) training a high accurate 468- point landmark detection model with multiple face dataset. Based on the FACS (Facial Action Coding System) table, we categorized these landmarks into 13 groups of facial regions. These regions with specific emotion labels are used as our target units of AU (Action Unit) detection; (3) training CNN model to detect and analyze AUs from facial landmark data: (4) implying FACS to evaluate the facial expressions and emotions: and (5) using a straightforward GUI plotter to show the digitized emotions. The experiment results show that not only the primaty emotion but also the secondary emotion of users in movies can be detected and evaluated successfully. Therefore, our system has a great potential for obtaining users' feedbacks iii a more accurate and comprehensive manner. [ABSTRACT FROM AUTHOR]

Abstract Cluster analyzes of facial models of autistic patients aim to clarify whether it is possible to diagnose autism on the basis of facial features and further to stratify the autism spectrum disorder. We performed a cluster analysis of sets of 3D scans of ASD patients (116) and controls (157) using Euclidean and geodesic distances in order to recapitulate the published results on the Czech population. In the presented work, we show that the major factor determining the clustering structure and consequently also the correlation of resulting clusters with autism severity degree is body mass index corrected for age (BMIFA). After removing the BMIFA effect from the data in two independent ways, both the cluster structure and autism severity correlations disappeared. Despite the fact that the influence of body mass index (BMI) on facial dimensions was studied many times, this is the first time to our knowledge when BMI was incorporated into the faces clustering study and it thereby casts doubt on previous results. We also performed correlation analysis which showed that the only correction used in the existing clustering studies—dividing the facial distance by the average value within the face—is not eliminating correlation between facial distances and BMIFA within the facial cohort.

Cluster analyzes of facial models of autistic patients aim to clarify whether it is possible to diagnose autism on the basis of facial features and further to stratify the autism spectrum disorder. We performed a cluster analysis of sets of 3D scans of ASD patients (116) and controls (157) using Euclidean and geodesic distances in order to recapitulate the published results on the Czech population. In the presented work, we show that the major factor determining the clustering structure and consequently also the correlation of resulting clusters with autism severity degree is body mass index corrected for age (BMIFA). After removing the BMIFA effect from the data in two independent ways, both the cluster structure and autism severity correlations disappeared. Despite the fact that the influence of body mass index (BMI) on facial dimensions was studied many times, this is the first time to our knowledge when BMI was incorporated into the faces clustering study and it thereby casts doubt on previous results. We also performed correlation analysis which showed that the only correction used in the existing clustering studies—dividing the facial distance by the average value within the face—is not eliminating correlation between facial distances and BMIFA within the facial cohort. [ABSTRACT FROM AUTHOR]

In this article, we address the problem of 4D facial expressions generation. This is usually addressed by animating a neutral 3D face to reach an expression peak, and then get back to the neutral state. In the real world though, people show more complex expressions, and switch from one expression to another. We thus propose a new model that generates transitions between different expressions, and synthesizes long and composed 4D expressions. This involves three sub-problems: (1) modeling the temporal dynamics of expressions, (2) learning transitions between them, and (3) deforming a generic mesh. We propose to encode the temporal evolution of expressions using the motion of a set of 3D landmarks, that we learn to generate by training a manifold-valued GAN (Motion3DGAN). To allow the generation of composed expressions, this model accepts two labels encoding the starting and the ending expressions. The final sequence of meshes is generated by a Sparse2Dense mesh Decoder (S2D-Dec) that maps the landmark displacements to a dense, per-vertex displacement of a known mesh topology. By explicitly working with motion trajectories, the model is totally independent from the identity. Extensive experiments on five public datasets show that our proposed approach brings significant improvements with respect to previous solutions, while retaining good generalization to unseen data. [ABSTRACT FROM AUTHOR]

Lipreading refers to translating the lip motion regarding a video speaker into the corresponding texts. Existing lipreading methods typically describe the lip motion using visual appearance variations. However, merely using the lip visual variations is prone to associating with inaccurate texts due to the similar lip shapes for different words. Also, visual features are hard to generalize to unseen speakers, especially when the training data is limited. In this paper, we leverage both lip visual motion and facial landmarks and propose an effective sentence-level end-to-end approach for lipreading. The facial landmarks are introduced to eliminate the irrelevant visual features which are sensitive to specific lip appearance of individual speakers. This enables the model to adapt to different lip shapes of speakers and generalize to unseen speakers. In specific, the proposed framework consists of two branches corresponding to the visual features and facial landmarks. The visual branch extracts high-level visual features from the lip movement, and the landmark branch learns to extract both spatial and temporal patterns described by the landmarks. The feature embeddings from two streams for each frame are fused to form its latent vector which can be decoded into texts. We employ a sequence-to-sequence model to operate the feature embeddings of all frames as input, and decode them to generate the texts. The proposed method is demonstrated to well generalize to unseen speakers on benchmark data sets. [ABSTRACT FROM AUTHOR]

Abstract Background The profound influence of orthodontic treatments on facial aesthetics has been a topic of increasing interest. This study delves into the intricate interplay between orthodontic treatments, facial feature alterations, and aesthetic perceptions. Methods A total of 73 patients who had undergone orthodontic treatment were included in this study. Facial photographs were taken before and after treatment. Ten orthodontists provided facial aesthetic ratings (FAR) for each patient's frontal, profile, and overall views. 48 facial landmarks were manually placed by the orthodontists and normalized using Generalized Procrustes analysis (GPA). Two types of phenotypes were derived from facial landmarks. Global facial phenotypes were then extracted using principal component analysis (PCA). Additionally, 37 clinical features related to aesthetics and orthodontics were extracted. The association between facial features and changes in FAR after orthodontic treatment was determined using these two types of phenotypes. Results The FAR exhibited a high correlation among orthodontic experts, particularly in the profile view. The FAR increased after orthodontic treatment, especially in profile views. Extraction of premolars and orthognathic surgery were found to result in higher FAR change. For global facial phenotypes, the most noticeable changes in the frontal and profile views associated with FAR occurred in the lip area, characterized by inward retraction of the lips and slight chin protrusion in the profile view, as well as a decrease in lip height in the frontal view. The changes observed in the profile view were statistically more significant than those in the frontal view. These facial changes were consistent with the changes from orthodontic treatment. For clinical features, two profile features, namely pg.sm.hori and pg.n.ls, were found to be associated with FAR following orthodontic treatment. The highest FAR scores were achieved when pg.sm.hori was at 80° and pg.n.ls was at 8°. On the other hand, frontal clinical features had a subtle effect on FAR during orthodontic treatment. Conclusions This study demonstrated that orthodontic treatment improves facial aesthetics, particularly at lip aera in the profile view. Profile clinical features, such as pg.sm.hori and pg.n.ls, are essential in orthodontic treatment which could increase facial aesthetics.

The proper functioning of many real-world applications in biometrics and surveillance depends on the robustness of face recognition systems against pose, and illumination variations. In this work, we employ ensemble systems in conjunction with local descriptors to address pose-invariant face recognition (PIFR). Facial landmarks are detected during the first step with a two fold usage. The landmark locations are employed to perform head pose classification (HPC). HPC allows to select only the visible landmarks for further processing. Then, local descriptors are extracted from the selected landmarks within a face image. We are proposing a novel learned descriptor (LS-SIFT) to overcome the robustness limitations of SIFT against large viewpoint variability during face recognition. Second, the extracted descriptors are used to train the base learners comprising an ensemble system for each subject in a face database (one ensemble per subject, one base learner per landmark). A novel GMM-based base learner model, named Mahalanobis Similarity (MS), is introduced in this work. Finally, face recognition is performed based on the ensemble systems’ outputs from all the subjects in a face database. During the experimental trials, SIFT and LS-SIFT are employed individually for local feature extraction, whereas GMM and MS are used to build the ensemble systems, in an independent manner, for further comparison. The whole PIFR system is tested on CMU-PIE, Multi-PIE, and FERET databases, outperforming most of the state-of-the-art works regarding images with pose angles in the range of $\pm 90^{o}$ .

The field of animated portrait generation, driven by audio cues, has seen remarkable advancements in creating lifelike visuals. Despite these strides, current methodologies struggle with the challenge of balancing stability and naturalism. Audio-driven techniques often encounter difficulties due to the subtlety of audio signals, leading to inconsistencies. In contrast, methods that rely solely on facial landmarks, while more stable, can produce artificial-looking results due to excessive manipulation of key point data. This paper introduces RealMock, an innovative approach that harmonizes audio inputs with facial landmarks during the training phase. RealMock pioneers a dual-driven training regimen, enabling the generation of animated portraits from audio, facial landmarks, or a combination of both. This novel methodology results in a more stable and authentic animation process. Extensive testing against existing algorithms across various public datasets, as well as our proprietary dataset, has highlighted RealMock’s superiority in both quantitative metrics and qualitative assessments. The RealMock framework has broad practical implications, including revolutionizing media production with realistic animated characters and improving online education through engaging avatar-based learning experiences.

This paper discusses the implementation of a blink detection method using 68 facial markers and the eye aspect ratio (EAR) to provide strong protection for access systems. It investigates the importance of 68 facial markers and explores how to use eye landmarks to calculate the eye aspect ratio. Access systems, which should have good security measures and be difficult to decipher, are typically safeguarded by passwords or multi-factor verification, such as passwords combined with facial recognition. However, these methods have inherent weaknesses, including the risk of shoulder surfing with passwords and the potential to be deceived by fake face images with facial recognition. To address these issues, a two-factor authentication method by using password with eye blink recognition is proposed as an effective solution for access control. By incorporating real-time eye blinking action, the system can avoid the use of fake images and prevent shoulder spoofing. To demonstrate the practical application of eye blink detection for enhanced two-factor authentication, a web application called "Eblink" is introduced. Functional tests have been conducted to validate the application's core features.

Facial expression recognition (FER) is a technology that recognizes human emotions based on biometric markers. Over the past decade, FER has been a popular research area, particularly in the computer vision community. With deep learning (DL) development, FER can achieve impressive recognition accuracy. In addition, favorable Internet-of-Things (IoT) advancements generate massive amounts of visual data needed to enable reliable DL-based emotion analysis. However, training DL models can suffer from significant memory consumption and computational costs, complicating many vision tasks. Additionally, the direct use of RGB images during the training and inference stages might raise privacy concerns in various FER applications. On the other hand, adopting large deep networks hampers quick and accurate recognition on resource-constrained end devices such as smartphones. As a viable solution, edge computing can be employed to bring data storage and computation closer to end devices rather than relying on a central cloud server. As a result, it can potentially facilitate the deployment of real-time FER applications since the latency and efficiency problems are well addressed by utilizing the computing resources at the edge. In this paper, we develop an efficient FER framework that integrates DL with edge computing. Our framework relies on facial landmarks to enable privacy-preserving and low-latency FER. Accordingly, various landmark detection models and feature types are studied empirically to investigate their capabilities in capturing the dynamic information of facial expressions in videos. Then, using the extracted landmark-based features, we design lightweight DL models to classify human emotions on IoT devices. Extensive experiments performed on benchmark datasets further validate the efficiency and robustness of our framework. [ABSTRACT FROM AUTHOR]

Background: The profound influence of orthodontic treatments on facial aesthetics has been a topic of increasing interest. This study delves into the intricate interplay between orthodontic treatments, facial feature alterations, and aesthetic perceptions. Methods: A total of 73 patients who had undergone orthodontic treatment were included in this study. Facial photographs were taken before and after treatment. Ten orthodontists provided facial aesthetic ratings (FAR) for each patient's frontal, profile, and overall views. 48 facial landmarks were manually placed by the orthodontists and normalized using Generalized Procrustes analysis (GPA). Two types of phenotypes were derived from facial landmarks. Global facial phenotypes were then extracted using principal component analysis (PCA). Additionally, 37 clinical features related to aesthetics and orthodontics were extracted. The association between facial features and changes in FAR after orthodontic treatment was determined using these two types of phenotypes. Results: The FAR exhibited a high correlation among orthodontic experts, particularly in the profile view. The FAR increased after orthodontic treatment, especially in profile views. Extraction of premolars and orthognathic surgery were found to result in higher FAR change. For global facial phenotypes, the most noticeable changes in the frontal and profile views associated with FAR occurred in the lip area, characterized by inward retraction of the lips and slight chin protrusion in the profile view, as well as a decrease in lip height in the frontal view. The changes observed in the profile view were statistically more significant than those in the frontal view. These facial changes were consistent with the changes from orthodontic treatment. For clinical features, two profile features, namely pg.sm.hori and pg.n.ls, were found to be associated with FAR following orthodontic treatment. The highest FAR scores were achieved when pg.sm.hori was at 80° and pg.n.ls was at 8°. On the other hand, frontal clinical features had a subtle effect on FAR during orthodontic treatment. Conclusions: This study demonstrated that orthodontic treatment improves facial aesthetics, particularly at lip aera in the profile view. Profile clinical features, such as pg.sm.hori and pg.n.ls, are essential in orthodontic treatment which could increase facial aesthetics. [ABSTRACT FROM AUTHOR]

Automated solutions for medical diagnosis based on computer vision form an emerging field of science aiming to enhance diagnosis and early disease detection. The detection and quantification of facial asymmetries enable facial palsy evaluation. In this work, a detailed review of the quantification of facial palsy takes place, covering all methods ranging from traditional manual mathematical modeling to automated computer vision-based methods. Moreover, facial palsy quantification is defined in terms of facial asymmetry indices calculation for different image modalities. The aim is to introduce readers to the concept of mathematical modeling approaches for facial palsy detection and evaluation and present the process of the development of this separate application field over time. Facial landmark extraction, facial datasets, and palsy grading systems are included in this research. As a general conclusion, machine learning methods for the evaluation of facial palsy lead to limited performance due to the use of handcrafted features, combined with the scarcity of the available datasets. Deep learning methods allow the automatic learning of discriminative deep facial features, leading to comparatively higher performance accuracies. Datasets limitations, proposed solutions, and future research directions in the field are also presented. [ABSTRACT FROM AUTHOR]

The popularity of massive open online courses (MOOCs) and other forms of distance learning has increased recently. Schools and institutions are going online to serve their students better. Exam integrity depends on the effectiveness of proctoring remote online exams. Proctoring services powered by computer vision and artificial intelligence have also gained popularity. Such systems should employ methods to guarantee an impartial examination. This research demonstrates how to create a multi-model computer vision system to identify and prevent abnormal student behaviour during exams. The system uses You only look once (YOLO) models and Dlib facial landmarks to recognize faces, objects, eye, hand, and mouth opening movement, gaze sideways, and use a mobile phone. Our approach offered a model that analyzes student behaviour using a deep neural network model learned from our newly produced dataset" StudentBehavioralDS." On the generated dataset, the "Behavioral Detection Model" had a mean Average Precision (mAP) of 0.87, while the "Mouth Opening Detection Model" and "Person and Objects Detection Model" had accuracies of 0.95 and 0.96, respectively. This work demonstrates good detection accuracy. We conclude that using computer vision and deep learning models trained on a private dataset, our idea provides a range of techniques to spot odd student behaviour during online tests.

Facial expression recognition (FER) poses a complex challenge due to diverse factors such as facial morphology variations, lighting conditions, and cultural nuances in emotion representation. To address these hurdles, specific FER algorithms leverage advanced data analysis for inferring emotional states from facial expressions. In this study, we introduce a universal validation methodology assessing any FER algorithm's performance through a web application where subjects respond to emotive images. We present the labelled data database, FeelPix, generated from facial landmark coordinates during FER algorithm validation. FeelPix is available to train and test generic FER algorithms, accurately identifying users' facial expressions. A testing algorithm classifies emotions based on FeelPix data, ensuring its reliability. Designed as a computationally lightweight solution, it finds applications in online systems. Our contribution improves facial expression recognition, enabling the identification and interpretation of emotions associated with facial expressions, offering profound insights into individuals' emotional reactions. This contribution has implications for healthcare, security, human-computer interaction, and entertainment. [ABSTRACT FROM AUTHOR]

Several studies have demonstrated accelerated brain aging in Alzheimer's dementia (AD). Previous studies have also reported that facial asymmetry increases with age. Because obtaining facial images is much easier than obtaining brain images, the aim of this work was to investigate whether AD exhibits accelerated aging patterns in facial asymmetry. We developed new facial asymmetry measures to compare Alzheimer's patients with healthy controls. A three-dimensional camera was used to capture facial images, and 68 facial landmarks were identified using an open-source machine-learning algorithm called OpenFace. A standard image registration method was used to align the three-dimensional original and mirrored facial images. This study used the registration error, representing landmark superimposition asymmetry distances, to examine 29 pairs of landmarks to characterize facial asymmetry. After comparing the facial images of 150 patients with AD with those of 150 age- and sex-matched non-demented controls, we found that the asymmetry of 20 landmarks was significantly different in AD than in the controls (p < 0.05). The AD-linked asymmetry was concentrated in the face edge, eyebrows, eyes, nostrils, and mouth. Facial asymmetry evaluation may thus serve as a tool for the detection of AD. [ABSTRACT FROM AUTHOR]

Objectives: The aim of this study was to analyse changes in facial soft tissue thickness (FSTT) after corrective surgeries for dental malocclusion. The correlation between body mass index (BMI) and sex of patients and their FSTT before undergoing surgery was analysed. Materials and methods: Cone beam computed tomography of seventeen patients that underwent Le Fort I osteotomy in combination with bilateral sagittal split osteotomy were collected. Hard and soft tissue landmarks were selected basing on the interventions. FSTT were computed, and measurements from pre- to post-operative were compared. The relationship between FSTT, sex, and BMI was investigated. Results: Considering the comparison between pre- and post-operative measurements, any significant difference emerged (p >.05). The Pearson's correlation coefficient computed between BMI and the FSTT (pre-operative) showed a correlation in normal-weight patients; the region-specific analysis highlighted a stronger correlation for specific landmarks. Higher median values emerged for women than for men; the subset-based analysis showed that women presented higher values in the malar region, while men presented higher values in the nasal region. Conclusions: The considered surgeries did not affect the FSTT of the patients; differences related to BMI and sex were found. A collection of FSTT mean values was provided for twenty landmarks of pre- and post-operative of female and male subjects. Clinical relevance: This exploratory analysis gave insights on the behaviour of STT after maxillofacial surgeries that can be applied in the development of predictive methodologies for soft tissue displacements and to study modifications in the facial aspect of the patients. [ABSTRACT FROM AUTHOR]

Blendshape models are used in various computer vision applications, such as expression transfer, 3D reconstruction, expression analysis and so on. Blendshape models have received lot of attention in last decades, and extensive research is carried out in the 3D face modeling. Computer-based animations use blendshape models for the transfer of target facial expression to animated character. The major challenge in such animation is developing blendshape models that can represent various facial expressions. To create such large data set of expression, one requires a dedicated expert team and it is a time-consuming process. In this paper, we propose a framework for automatic facial landmark detection and blendshape generation through expression transfer. The proposed work is in two main folds: Initially, facial landmarks are extracted based on geometric information of the given facial mesh. Further, the extracted landmark points and the estimated correspondence between source and target facial models are used to perform deformation transfer. Experiment results show that our method is able to transfer expression by using automatic landmarks and also the smoothness of deformation around facial landmark areas proves that our proposed landmark-based deformation method is as good as the state-of-the-art methods. Our proposed method for automatic facial landmark detection based on geometric information of 3D face model has the ability to detect reliable correspondences, and it is faster and simpler compared with the state-of-the-art automatic deformation transfer method on the facial models. [ABSTRACT FROM AUTHOR]

In recent years, intelligent emotion recognition is active research in computer vision to understand the dynamic communication between machines and humans. As a result, automatic emotion recognition allows the machine to assess and acquire the human emotional state to predict the intents based on the facial expression. Researchers mainly focus on speech features and body motions; identifying affect from facial expressions remains a less explored topic. Hence, this paper proposes a novel approach for intelligent facial emotion recognition using optimal geometrical features from facial landmarks using VGG-19s (FCNN). Here, we utilize Haarcascade to detect the subject face and determine the distance and angle measurements.The entire process is to classify the facial expressions based on extracting relevant features with the normalized angle and distance measures. The experimental analysis shows high accuracy on the MUG dataset of 94.22% and 86.45% on GEMEP datasets, respectively. [ABSTRACT FROM AUTHOR]

The purpose of the present paper is to study the process of face detection and recognition, followed by the ability to detect the drowsiness state of an individual. This experiment has been part of the bachelor thesis and aims to highlight some of the true power of Artificial Intelligence [1]. For a full perspective on the topic, an experiment was held in order to emphasize the flexibility and power of facial detection. It aims to analyze the real-time possible drowsiness of a driver by using a key point facial landmark detection and warn the individual when necessary.

In recent years, pose-invariant face recognition has been mainly approached from a holistic insight. DCNNs (ArcFace, Elastic Face, FaceNet) are used to compute a face image embedding, which is used later to perform face recognition. This paper presents a novel approach to pose-invariant face recognition through the use of ensemble learning and local feature descriptors. The proposed method trains a base learner for each person’s face recognition ensemble system, based on feature vectors (SIFT, GMM, LBP) extracted from image regions surrounding specific facial landmarks. Three different classification models (SVM, Naive Bayes, GMM) are exclusively used as base learners, and the training procedure for each of these models is detailed. The proposed methodology includes a novel face pose descriptor referred to as the Face Angle Vector (FAV) which is utilized by a head pose classification model to determine the pose class of a face image. This model works together with a Base Learner Selection (BLS) block, to determine a set of facial landmarks to extract local feature descriptors, and uses them as the input to their corresponding base learners. Experimental results show a better performance over state-of-the-art methods using the CMU-PIE as the testing dataset, and face poses within ±90°.

Today, facial emotion recognition is widely adopted in many intelligent applications including the driver monitoring system, the smart customer care as well as the e-learning system. In fact, the human emotions can be well represented by facial landmarks which are hard to be detected from images, due to the high number of discrete landmarks, the variation of shapes and poses of the human face in real world. Over decades, many methods have been proposed for facial landmark detection including the shape fitting, the coordinate regression such as ASMNet and AnchorFace. However, their performance is still limited for real-time applications in terms of both accuracy and efficiency. In this paper, we propose a novel method called SHELF which is the first to combine the shape fitting and heatmap regression approaches for landmark detection in human face. The heatmap model aims to generate the landmarks that fit to the common shapes. The method has been evaluated on three datasets 300W-Challenging, WFLW, 300VW-E with 31557 images and achieved a normalized mean error (NME) of 6.67%, 7.34%, 12.55% correspondingly, which overcomes most existing methods. For the first two datasets, the method is also comparable to the state of the art AnchorFace with a NME of 6.19%, 4.62%, respectively. [ABSTRACT FROM AUTHOR]

With the current market's growing need for electric vehicles and technologies in high-end vehicles, distracted driver detection requires the artificial intelligence's attention. In this paper, new strategies for improving the performance of the driver detection methodology are proposed. The proposed approach consists of two sub-systems namely driver activity detection and driver fatigue detection systems. The former one detects the activities of driver. The latter one is based on the facial feature recognition and determines the driver's fatigue level. The proposed model is evaluated on the activity detection and attained the classification accuracy of 99.69%, compared to the 94.32% accuracy in the state-of-the-art comparison. The KNN classifier had the best accuracy for detecting driver fatigue, with a 76.33% success rate. Experimental results reveal the superiority of proposed model over the existing models. The proposed model can be applied in the real-life environment. [ABSTRACT FROM AUTHOR]