Search

Prolonged impairment of protective ocular functions can compromise vision and lead to blindness if uncorrected. Several facial transplants have incorporated periorbital structures with variable eyelid preservation, but objective assessment of post-transplant periorbital function has been limited.Kinematic data were collected from a full-face recipient that included the fist total eyelid transplantation at 5 separate pre-transplant (PRE) and post-transplant time points (T1-T4). Using optical facial tracking, eyelid movements were tracked during involuntary blinking and compared with controls.There was significant improvement in right eye aperture from PRE to T1 (β = 5.54, P 0.001), with no change between T1 and T4. Aperture fluctuated in the left eye, with a temporary decrease between T2 and T3 corresponding with revision brow lift (β = -4.57, P 0.001). Although improved from the pre-transplantation, right and left eye apertures remained significantly smaller than controls at T1 and T4 (P 0.001). Similarly, spatial coupling increased from PRE to T1 (β = 0.63, P 0.001) and remained high at T4, albeit significantly less than controls (P 0.001). Temporal coupling improved from PRE to T2 (β = 2.29, P 0.02) and was sustained at subsequent time points, with no difference relative to controls at T4. Considerable improvement was observed on clinical examination, with full functional status.Application of a novel method for assessing functional eyelid recovery using facial tracking technology to the first total eyelid transplantation in the setting of a full facial transplant shows clear functional improvement after transplantation and suggests revisions can be performed safely to optimize aesthetic outcomes without permanent negative functional impact.

Real time facial animation for virtual 3D characters has important applications such as AR/VR, interactive 3D entertainment, pre-visualization and video conferencing. Yet despite important research breakthroughs in facial tracking and performance capture, there are very few commercial examples of real-time facial animation applications in the consumer market. Mass adoption requires realtime performance on commodity hardware and visually pleasing animation that is robust to real world conditions, without requiring manual calibration. We present an end-to-end deep learning framework for regressing facial animation weights from video that addresses most of these challenges. Our formulation is fast (3.2 ms), utilizes images of real human faces along with millions of synthetic rendered frames to train the network on real-world scenarios, and produces jitter-free visually pleasing animations.

Kreacija i modifikacija 3D modela pomoću programa VRoidStudio i Blender. Glavni zadatak je bio animirati lice 3D modela pomoću praćenja lica. Zadatak je napravljen koristeći detekciju lica pomoću biblioteke Visage|SDK. Pomoću kamere i Visage|SDK smo dobili razne podatke o licu koje smo na 3D model prenijeli pomoću blend shapes ili morph targets. Blend shapes je tehnika deformiranja jednog mesha kako bi dobili niz izvedbi tog mesha. 3D model ima blend shape lica za većinu pojedinačnih pomaka pravog lica. Rezultat je da 3D model oponaša većinu ekspresija snimljenog lica u pravom vremenu. The creation and modification of 3D models using VRoidStudio and Blender. The main goal was to animate a 3D model's face using face tracking. The task was done by using face detection with the Visage|SDK library. With the camera and Visage|SDK, we obtained various face data that we translated to the 3D model using blend shapes or morph targets. Blend shapes is a technique of deforming one mesh to get various similar meshes. The 3D model's face has a blend shape for most real life face movements. The result is that the 3D model mimics most of the expressions of the captured face in real time.

3D facial tracking has become vital to the continued integration of computers, technology, and human society. In recent decades, the integration of technology has increased, and the use of surveillance, conference calls, gaming components, and other similar applications has spurred demand for the ability to recognize the distinctive features of humans. However, in order for these new technologies to function effectively and reach their fullest potential, a great deal of work is still needed. The field of facial mapping and tracking is still in its early developmental stages, necessitating additional research into the best methods of tracking and monitoring specific human faces. To this end, an algorithm has been created that would allow for improvements in this area; however, a video was first required that could be used effectively for the algorithm. Two web cameras running on Raspberry Pi were used to gather the footage necessary for detecting and tracking specific facial features. While certain limitations were identified throughout the process, the algorithm still achieved significant successful tracking results. In spite of this success, further efforts are still needed to effectively explore the proposed algorithm and improve upon these initial results.

Avatar use on video-conference platforms has found dual purpose in recent times as a potential method for ensuring privacy and improving subjective engagement with remote meeting, provided one can also ensure a minimal loss in the quality of social interaction and sense of personal presence. This work focuses on interactions of this sort through real-time motion captured 3D personalized virtual avatars in a 2D video-conferencing context. Our experiments were designed with the intention of exploring previously defined perceptual illusions that occur with avatar-use in Virtual and Augmented Reality settings, outside of the immersive technological domains where they are normally measured. The research described here was aimed at empirically evaluating three separate dimensions of human-avatar interaction. The first was humans-as-avatars, with experimental conditions that were designed to measure changes to subjective perceptions of self-face ownership and self-concept. The second focus was other-perception, with the unique design of the studies outlined below among the first to measure social presence in a video-call between two human-driven avatars. The third emphasis was on the experiential content involved in avatar use, as there were measurements for emotion induction, fatigue and behavior change included in the data collection. The results describe some evidence for face and body ownership, while participants also reported high levels of social presence with the other avatar, indicating that avatar cameras could be a favorable alternative to non-camera feeds in video conferencing. There were also some useful insights gained regarding emotion elicitation in non-video vs. avatar conditions, as well as avatar-induced behavior change.

Facial alignment is an essential task for many higher level facial analysis applications, such as animation, human activity recognition and human - computer interaction. Although the recent availability of big datasets and powerful deep-learning approaches have enabled major improvements on the state of the art accuracy, the performance of current approaches can severely deteriorate when dealing with images in highly unconstrained conditions, which limits the real-life applicability of such models. In this paper, we propose a composite recurrent tracker with internal denoising that jointly address both single image facial alignment and deformable facial tracking in the wild. Specifically, we incorporate multilayer LSTMs to model temporal dependencies with variable length and introduce an internal denoiser which selectively enhances the input images to improve the robustness of our overall model. We achieve this by combining 4 different sub-networks that specialize in each of the key tasks that are required, namely face detection, bounding-box tracking, facial region validation and facial alignment with internal denoising. These blocks are endowed with novel algorithms resulting in a facial tracker that is both accurate, robust to in-the-wild settings and resilient against drifting. We demonstrate this by testing our model on 300-W and Menpo datasets for single image facial alignment, and 300-VW dataset for deformable facial tracking. Comparison against 20 other state of the art methods demonstrates the excellent performance of the proposed approach. This work is partly supported by the Spanish Ministry of Economy and Competitiveness under project grant TIN2017-90124-P, the Ramon y Cajal Programme, the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502) and the donation bahi2018-19 to the CMTech at the UPF.

The accident rate in Saudi Arabia is quite high due to several factors such as speeding, lousy weather, pedestrian unawareness, etc. For the past year, Kingdom of Saudi Arabia suffered from a significant loss of human life and economical due to a traffic accident. Ministry of Interior Riyadh Traffic department reporting that traffic accident has made financial loss about 21 billion riyals a year. The traffic accident has increased rapidly, six injuries per eight incidents are caused by a traffic accident, whereas the global ratio is one injury for every eight accidents. This paper proposed unique solution by using two primary cameras for tracking the driver and pedestrian. The recognition rate of facial tracking have been evaluated through several machine learning algorithm and Tree has achieved the highest recognition rate with 87% followed by KNN with 77%. The result is quite promising with this double tracking: facial tracking to track the drowsiness and awareness while pedestrian tracking brake analysis for the object in front of or behind the vehicle. The future work with advanced sensor such as sonar and high speed camera might improve the accuracy and efficiency of the system.

In recent years, technologies such as Kinect and Emotiv, combined with predetermined stimuli, have been widely used to perform studies in different fields such as neuromarketing, rehabilitation, medicine, among others. However, how facial and neurological activation parameters can be related has been little explored. This research aims to develop a way to synchronize data acquisition on both devices and find possible relationships between the data produced by facial tracking using the Kinect and the encephalographic signals obtained from the Emotiv. For this, an experimental design has been built with a population of 15 test subjects including both genders, who were exposed to a predetermined routine of images, which have been demonstrated and used by different neuroscience research centers since they can induce in humans emotions such as happiness, sadness, disgust or fear. Measurements were carried out in a controlled space, in total calm, away from any noise or interference that may affect data collection. Within the obtained results, it was identified some possible relationships between facial and encephalographic signals, as well as a dependent activation of the brain’s areas with the stimuli presented, in both the time and frequency domain.

This paper presents a real time facial feature tracking system employing an inexpensive imaging device and two servo motors, to control the pan and tilt of the camera, for face tracking to capture vital facial features. Kanade Lucas Tomasi (KLT) method was adopted to track the facial features. The system was tested on various indoor and outdoor environments and the method shows good robustness and accuracy with a success rate of 95.7%. The proposed method has the ability to track from a selection of facial parts in comparison to being confined to track only the face.

Applications utilising 3D Camera technologies for the measurement of health outcomes in the health and wellness sector continues to expand. The Intel® RealSense™ is one of the leading 3D depth sensing cameras currently available on the market and aligns itself for use in many applications, including robotics, automation, and medical systems. One of the most prominent areas is the production of interactive solutions for rehabilitation which includes gait analysis and facial tracking. Advancements in depth camera technology has resulted in a noticeable increase in the integration of these technologies into portable platforms, suggesting significant future potential for pervasive in-clinic and field based health assessment solutions. This paper reviews the Intel RealSense technology’s technical capabilities and discusses its application to clinical research and includes examples where the Intel RealSense camera range has been used for the measurement of health outcomes. This review supports the use of the technology to develop robust, objective movement and mobility-based endpoints to enable accurate tracking of the effects of treatment interventions in clinical trials.

LSA is a useful user study technique, it is well known and used to design and evaluate Ubicomp systems. The LSA technique enables researchers to collect data, analyze it, and obtain quantitative and statistical results. A key advantage of using LSA is that it is performed in the user’s environment. However, analyzing large amounts of data is considered by some researchers to be a burden and time consuming, prone to human error. In this paper we explore the use of computer vision techniques to automate the data analysis and coding when using LSA. We present a system that uses facial tracking, object recognition and composite correlation filters to detect the Attention behavior of a subject. Our results indicate that computer vision can automate the LSA technique and reduce the burden of coding data manually by the researcher. The findings from this study reveal emergent practices of the use of our proposed system to automate the evaluation of Ubicomp environments.

Object Tracking has primarily been characterized as the study of object motion trajectory over constraint subspaces under attempts to mimic human efficiency. However, the trend of monotonically increasing applicability and integrated relevance over distributed commercial frontiers necessitates that scalability be addressed. The present work proposes a system for fast large scale facial tracking over distributed systems beyond individual human capabilities leveraging the computational prowess of large scale processing engines such as Apache Spark. The system is pivoted on an interval based approach for receiving the input feed streams, which is followed by a deep encoder-decoder network for generation of robust environment invariant feature encoding. The system performance is analyzed while functionally varying various pipeline components, to highlight the robustness of the vector representations and near real-time processing performance.

In this paper, we present a system for real-time performance-driven facial animation. With the system, the user can control the facial expression of a digital character by acting out the desired facial action in front of an ordinary camera. First, we create a muscle-based 3D face model. The muscle actuation parameters are used to animate the face model. To increase the reality of facial animation, the orbicularis oris in our face model is divided into the inner part and outer part. We also establish the relationship between jaw rotation and facial surface deformation. Second, a real-time facial tracking method is employed to track the facial features of a performer in the video. Finally, the tracked facial feature points are used to estimate muscle actuation parameters to drive the face model. Experimental results show that our system runs in real time and outputs realistic facial animations. Compared with most existing performance-based facial animation systems, ours does not require facial markers, intrusive lighting, or special scanning equipment, thus it is inexpensive and easy to use.

Humor and other entertaining content, as opposed to demonstrations of product features and "selling," are increasingly used in advertising, such as TV commercials, to attract and keep consumers' attention. This study uses facial tracking to explore how marketers can best use entertainment in ads to increase their effectiveness in increasing intent to purchase. The findings suggest that the optimal amount of entertainment differs by type of entertainment and target group, but not by product category, and confirms that the funniest ads are not necessarily the most effective. [ABSTRACT FROM AUTHOR]

Real-time facial performance capture has recently been gaining popularity in virtual film production, driven by advances in machine learning, which allows for fast inference of facial geometry from video streams. These learning-based approaches are significantly influenced by the quality and amount of labelled training data. Tedious construction of training sets from real imagery can be replaced by rendering a facial animation rig under on-set conditions expected at runtime. We learn a synthetic actor-specific prior by adapting a state-of-the-art facial tracking method. Synthetic training significantly reduces the capture and annotation burden and in theory allows generation of an arbitrary amount of data. But practical realities such as training time and compute resources still limit the size of any training set. We construct better and smaller training sets by investigating which facial image appearances are crucial for tracking accuracy, covering the dimensions of expression, viewpoint and illumination. A reduction of training data in 1-2 orders of magnitude is demonstrated whilst tracking accuracy is retained for challenging on-set footage.

In this paper we propose a novel pointing technique leveraging the user’s body motion to achieve smooth, efficient user experiences on wall-sized displays. Our proposal substantially consists of two parts: a graphical cursor controlled by the user’s hand motions, and mechanisms to assist the cursor manipulation by tracking the user’s face orientation. By interaction design associating the user’s face and hand motions to different aspects of the cursor’s movement, we aimed to bring swiftness to the interaction in large-display environments with necessary precision. A prototype was built to instantiate the concept, and two comparative experiments were conducted to evaluate the effectiveness of the proposal.

La localización de caras es una característica ampliamente utilizada actualmente en diferentes productos software. Además, con la aparición de sensores RGBZ (como la Kinect o la RealSense) se ha añadido la capacidad no sólo detectar dónde hay una cabeza, si no de obtener información tridimensional sobre la misma. En este proyecto se diseña, desarrolla y analiza un software que permita obtener, mediante el uso de las cámaras RGBZ anteriormente mencionadas, la orientación 3D de la cabeza del usuario que esté delante de ellas, es decir, los ángulos que determinan hacia qué dirección está mirando el usuario. Para ello se ha diseñado un algoritmo basado en el método Iterative Closest Point, de forma que por cada imagen capturada por la cámara se detecte qué ángulos presenta la cabeza. También se ha desarrollado una plataforma externa utilizando un servomotor y un microcontrolador Arduino, permitiendo realizar pruebas de los diferentes parámetros del algoritmo para validar sus resultados, mediante una plataforma giratoria sobre la que se puede orientar con precisión una reproducción a escala de una cabeza 3D. La localització de cares es una característica àmpliament utilitzada en diferents productes software actualment. A més, amb l’aparició de sensors RGBZ (com la Kinect o la RealSense) s’ha afegit la capacitat de, no només detectar a on hi ha una cara, si no d’obtenir la informació tridimensional d’aquesta. En aquest projecte es dissenya, desenvolupa i s’analitza un software que permeti obtenir, mitjançant l’ús de les càmeres RGBZ anteriorment nombrades, la orientació del cap de l’usuari que es trobi davant d’elles, és a dir, dels angles que defineixen cap a quina direcció està mirant l’usuari. Per aconseguir-ho s’ha dissenyat un algoritme basat en el mètode Iterative Closest Point, de manera que per cada imatge capturada per la càmera es detecti quins angles presenta el cap. També s’ha desenvolupat una plataforma externa utilitzant un motor i un microcontrolador Arduino, a on es poden realitzar proves dels diferents paràmetres de l’algoritme per validar els resultats mitjançant una plataforma giratòria sobre la qual s’ha col·locat una reproducció a escala d’un cap en tres dimensions que es pot orientar amb precisió. Face localization has become a hugely demanded feature in many different sofware products. In addition, with the appearence of RGBZ sensors (such as the Kinect and the RealSense) the capacity of not only detecting where the face is located but also obtaining the 3D orientation of the face has been added. In this project we aim to design, develop and test a software able to, using the RGBZ sensors, detect the pose of the head of a user in front of the camera, that is, extract the three angles that define the direction of the head. To do that, we developed an algorithm based on the Iterative Closest Point family. For each image provided by the camera, the angles are detected. An external platform was also developed using a servomotor and an Arduino microcontroller, able to perform tests of the different parameters of the algorithm to validate the results using a rotating base that can turn precisely a reproduction of a real-size 3D printed head.

Heart rate (HR) is one of vital biomedical signals for medical diagnosis. Previously, conventional camera is proven to be able to detect small changes in the skin due to the cardiac activity and can be used to measure the HR. However, most of the previous systems operate on near distance mode with a single face patch, thus the feasibility of the remote heart rate for various distances remains vague. This paper tackles this issue by analyzing an optimal framework that capable to works under the mentioned issues. Initially, plausible face landmarks are estimated by employing cascaded of regression mechanism. Next, the region of interest (ROI) was constructed from the landmarks in a face location where non rigid motion is minimal. From the ROI, temporal photoplethysmograph (PPG) signal is calculated based on the average green pixels intensity and environmental illumination is separated using Independent Component Analysis (ICA) filter. Eventually, the PPG signal is further processed using series of temporal filter to exclude frequencies outside the range of interest prior to estimate the HR. As a conclusion, the HR can be detected up to 5 meters range with 94% accuracy using lower part of face region.

We present automated, real-time models built with machine learning algorithms which use videotapes of subjects' faces in conjunction with physiological measurements to predict rated emotion (trained coders' second-by-second assessments of sadness or amusement). Input consisted of videotapes of 41 subjects watching emotionally evocative films along with measures of their cardiovascular activity, somatic activity, and electrodermal responding. We built algorithms based on extracted points from the subjects' faces as well as their physiological responses. Strengths of the current approach are (1) we are assessing real behavior of subjects watching emotional videos instead of actors making facial poses, (2) the training data allow us to predict both emotion type (amusement versus sadness) as well as the intensity level of each emotion, (3) we provide a direct comparison between person-specific, gender-specific, and general models. Results demonstrated good fits for the models overall, with better performance for emotion categories than for emotion intensity, for amusement ratings than sadness ratings, for a full model using both physiological measures and facial tracking than for either cue alone, and for person-specific models than for gender-specific or general models.

We propose a novel system that incorporates physiological monitoring as part of the human-computer interface. The sensing element is a thermal camera that is employed as a computer peripheral. Through bioheat modeling of facial imagery almost the full range of vital signs can be extracted, including localize blood flow, cardiac pulse, and breath rate. This physiological information can then be used to draw inferences about a variety of health symptoms and psychological states. Our research aims to realize the notion of desktop health monitoring and create truly collaborative interactions in which humans and machines are both observing and responding.

Previous work has demonstrated the correlation of increased blood perfusion in the orbital muscles and stress levels for human beings. It has also been suggested that this periorbital perfusion can be quantified through the processing of thermal video. The idea has been based on the fact that skin temperature is heavily modulated by superficial blood flow. Proof of this concept was established for two different types of stress inducing experiments: startle experiments and mock-crime polygraph interrogations. However, the polygraph interrogation scenarios were simplistic and highly constrained. In the present paper, we report results derived from a large and realistic mock-crime interrogation experiment. The interrogation is free flowing and no restrictions have been placed on the subjects. Additionally, we propose a new methodology to compute the mean periorbital temperature signal. The present approach addresses the deficiencies of the earlier methodology and is capable of coping with the challenges posed by the realistic setting. Specifically, it features a tandem CONDENSATION tracker to register the periorbital area in the context of a moving face. It operates on the raw temperature signal and tries to improve the information content by suppressing the noise level instead of amplifying the signal as a whole. Finally, a pattern recognition method classifies stressful (Deceptive) from non-stressful (Non-Deceptive) subjects based on a comparative measure between the entire interrogation signal (baseline) and a critical subsection of it (transient response). The successful classification rate is 87.2% for 39 subjects. This is on par with the success rate achieved by highly trained psycho-physiological experts and opens the way for automating lie detection in realistic settings.

When face and facial part detectors encounter partial or complete occlusion, their output becomes increasingly noisy, making it difficult for such algorithms to tackle complex scenes: parts are only detected intermittently, false alarms become more prevalent, parts are mis-identified, and some parts are consistently missed altogether. In this paper we propose a remedy to this problem: probabilistic fusion of face and facial detectors. The method jointly improves the localization and tracking performance of face and facial parts detectors by modelling the spatial and temporal inter-dependences of these detectors in a probabilistic graphical model. Spatial modelling can be used to rectify mis-identification, filter out false detections and infer the presence of missed detections during partial occlusion. Temporal modelling improves face and facial tracking in scenarios involving completely missed detections or complete occlusion. Experimental results quantify substantial benefits in terms of improved precision, recall and F-measure both for the detection of facial parts and of the face.

A user study was performed to compare two non-touch input methods for mobile gaming: tilt-input and facial tracking. User performance was measured on a mobile game called Star Jelly installed on a Google Nexus 7 HD tablet. The tilt-input method yielded significantly better performance. The mean game-score attained using tilt-input was 665.8. This was 7× higher than the mean of 95.1 for facial tracking. Additionally, participants were more precise with tilt-input with a mean star count of 19.7, compared to a mean of 1.9 using facial tracking. Although tilt-input was superior, participants praised facial tracking as challenging and innovative.