Search

Recent research has demonstrated that neural and behavioral data acquired in response to viewing face images can be used to reconstruct the images themselves. However, the theoretical implications, promises, and challenges of this direction of research remain unclear. We evaluate the potential of this research for elucidating the visual representations underlying face recognition. Specifically, we outline complementary and converging accounts of the visual content, the representational structure, and the neural dynamics of face processing. We illustrate how this research addresses fundamental questions in the study of normal and impaired face recognition, and how image reconstruction provides a powerful framework for uncovering face representations, for unifying multiple types of empirical data, and for facilitating both theoretical and methodological progress. The visual content of human face representations can be derived through image reconstruction at an unprecedented level of detail. The neural underpinnings of psychological face representations can be elucidated via image reconstruction from behavioral and multiple types of neural data. Facial image reconstruction capitalizes on the structure of face space and also serves to validate its representational properties. The recovery of face representations from both perception and memory can help to clarify the relationship between these two components of cognition. Image reconstruction can pinpoint the loss of information and/or misrepresentations in face perception across a broad population, including individuals with deficits in face processing. [ABSTRACT FROM AUTHOR]

Highlights • First report of ERPs to faces and anti-faces of familiar and unfamiliar persons. • P200 decreases with distance-to-norm irrespective of face familiarity. • N250 and late negativity are much larger to familiar faces than their anti-faces. • Unfamiliar faces and their anti-faces show similar N250 and late negativity. • Face-sensitive ERPs dissociate different aspects of representational face space. Abstract The norm-based face space model (nMDFS) predicts that perceived typicality decreases with distance to the norm. Accordingly, an original face and its corresponding anti-face (deviating in exactly opposite direction from the norm) should inherit equivalent levels of typicality. Similarly, varying distance-to-norm (DTN, in absolute values) should have equivalent effects on typicality for both unfamiliar faces and their anti-faces. Here we reasoned that familiar faces would not follow this rule, due to the special nature of their mental representations. We investigated event-related brain potentials (ERPs) to familiar and unfamiliar faces and their anti-faces with different DTN (±0.33, ±0.66, and ±0.99). Across DTN, rated typicality was highest for original familiar faces, whereas all other face stimuli exhibited the expected gradual decrease of typicality with increasing DTN. ERPs showed pronounced DTN effects in the occipitotemporal P200 (190–260 ms), with progressively smaller amplitudes for faces with higher DTN, for all face types. By contrast, prominent effects of familiarity were found for the subsequent N250 (260–400 ms) and a late occipitotemporal negativity (LN, 400–800 ms). A large increase of occipitotemporal negativity was observed for original familiar faces versus their non-recognizable anti-faces. Analogous effects were small or absent for unfamiliar faces versus their anti-faces. These results confirm the N250 and LN as correlates of the processing of the identity of known faces, and show for the first time that the face-sensitive P200 is primarily modulated by DTN of a face in nMDFS. [ABSTRACT FROM AUTHOR]

Face recognition is a computationally challenging task that humans perform effortlessly. Nonetheless, this remarkable ability is better for familiar faces than unfamiliar faces. To account for humans' superior ability to recognize familiar faces, current theories suggest that different features are used for the representation of familiar and unfamiliar faces. In the current study, we applied a reverse engineering approach to reveal which facial features are critical for familiar face recognition. In contrast to current views, we discovered that the same subset of features that are used for matching unfamiliar faces, are also used for matching as well as recognition of familiar faces. We further show that these features are also used by a deep neural network face recognition algorithm. We therefore propose a new framework that assumes similar perceptual representation for all faces and integrates cognition and perception to account for humans' superior recognition of familiar faces. [ABSTRACT FROM AUTHOR]

Face recognition is a challenging classification task that humans perform effortlessly for familiar faces. Recent studies have emphasized the importance of exposure to high variability appearances of the same identity to perform this task. However, these studies did not explicitly measure the perceptual similarity between the learned images and the images presented at test, which may account for the advantage of learning from high variability. Particularly, randomly selected test images are more likely to be perceptually similar to learned high variability images, and dissimilar to learned low variability images. Here we dissociated effects of learning from variability and study-test perceptual similarity, by collecting human similarity ratings for the study and test images. Using these measures, we independently manipulated the variability between the learning images and their perceptual similarity to the test images. Different groups of participants learned face identities from a low or high variability set of images. The learning phase was followed by a face matching test (Experiment 1) or a face recognition task (Experiment 2) that presented novel images of the learned identities that were perceptually dissimilar or similar to the learned images. Results of both experiments show that perceptual similarity between study and test, rather than image variability at learning per se, predicts face recognition. We conclude that learning from high variability improves face recognition for perceptually similar but not for perceptually dissimilar images. These findings may not be specific to faces and should be similarly evaluated for other domains. [ABSTRACT FROM AUTHOR]

In this paper, we constrain faces to points on a manifold within the parameter space of a linear statistical model. The manifold is the subspace of faces which have maximally likely distinctiveness and different points correspond to unique identities. We provide a detailed empirical validation for the chosen manifold. We show how the Log and Exponential maps for a hyperspherical manifold can be used to replace linear operations such as warping and averaging with operations on this manifold. Finally, we use the manifold to develop a new method for fitting a statistical face shape model to data, which is both robust (avoids overfitting) and overcomes model dominance (is not susceptible to local minima close to the mean face). We provide experimental results for fitting a dense 3D morphable face model to data using two different objective functions (one underconstrained and one with many local minima). Our method outperforms generic nonlinear optimisers based on the BFGS Quasi-Newton method and the Levenberg–Marquardt algorithm when fitting using the Basel Face Model. [ABSTRACT FROM AUTHOR]

Human adults implicitly learn the prototype and the principal components of the variability distinguishing faces (Gao & Wilson, 2014). Here we measured the implicit learning effect in adults and 9-year-olds, and with a modified child-friendly procedure, in 7-year-olds. All age groups showed the implicit learning effect by falsely recognizing the average (the prototype effect) and the principal component faces as having been seen (the PCA learning effect). The PCA learning effect, but not the prototype effect increased between 9 years of age and adulthood and at both ages was the better predictor of memory for the actually studied faces. In contrast, for the 7-year-olds, the better predictor of face memory was the prototype effect. The pattern suggests that there may be a developmental change between ages 7 and 9 in the mechanism underlying memory for faces. We provide the first evidence that children as young as age 7 can extract the most important dimensions of variation represented by principal components among individual faces, a key ability that grows stronger with age and comes to underlie memory for faces. [ABSTRACT FROM AUTHOR]

Abstract: There is evidence that humans implicitly learn an average or prototype of previously studied faces, as the unseen face prototype is falsely recognized as having been learned (Solso & McCarthy, 1981). Here we investigated the extent and nature of face prototype formation where observers’ memory was tested after they studied synthetic faces defined purely in geometric terms in a multidimensional face space. We found a strong prototype effect: The basic results showed that the unseen prototype averaged from the studied faces was falsely identified as learned at a rate of 86.3%, whereas individual studied faces were identified correctly 66.3% of the time and the distractors were incorrectly identified as having been learned only 32.4% of the time. This prototype learning lasted at least 1week. Face prototype learning occurred even when the studied faces were further from the unseen prototype than the median variation in the population. Prototype memory formation was evident in addition to memory formation of studied face exemplars as demonstrated in our models. Additional studies showed that the prototype effect can be generalized across viewpoints, and head shape and internal features separately contribute to prototype formation. Thus, implicit face prototype extraction in a multidimensional space is a very general aspect of geometric face learning. [Copyright &y& Elsevier]

Abstract: Previous neural research on face perception has mainly focused on the distinction between faces and non-face stimuli. However, the brain mechanisms for differentiating one face from another are not well understood. In the present study, using scalp-recorded event-related potentials (ERPs), we investigated the brain responses to faces that varied in identity strength as a result of morphing individual faces to an “average” face in steps of 10%. Participants performed a face identification task. Behavioral results showed categorical boundaries of face identification at 30% and 70%. Face identity strength related to initial brain responses occurring shortly after 200ms in the ventral P2 and the N250 components: stronger identity strength was associated with a smaller P2 and a larger N250. In contrast, the brain responses within 200ms, as reflected by the P1, the N170, and the dorsal P2 component, were not affected by face identity strength. Consistent with recent imaging studies and animal research, our results provide the ERP evidence for brain responses to variations in face identity strength relative to an “average” face. Furthermore, with the high temporal resolution of ERPs, our results help to clarify the timing of neural events that are associated with the different stages involved in recognizing individual faces, thus providing a timeline for the classical face recognition model in the brain. [Copyright &y& Elsevier]

Abstract: Previous research has suggested that an object’s category is retrieved as soon as it is detected (). Here we examined whether face views and identities are likewise treated as categories. We measured behavioural performance on three tasks: face detection, recognition of face view within identity, and within-view face identification, by using the method of constant stimuli combined with a two-alternative forced-choice (2AFC) paradigm. Stimulus duration was varied between 13ms and 133ms in order to estimate the time required for 75%-correct discrimination in each task. The results showed, respectively, 24- and 31-ms shorter threshold durations for face detection than for viewpoint recognition and face identification, while similar threshold durations for viewpoint recognition and face identification. We demonstrated that face view and identity are retrieved after face detection, and importantly, the view-based categorical analysis takes almost as long as the face identification process. Thus, additional processing is essential for viewpoint and identity extraction as opposed to face detection. [Copyright &y& Elsevier]

Recent evidence has shown that face space represents facial identity information using two-pool opponent coding. Here we ask whether the shape of the monotonic neural response functions underlying such coding is linear (i.e. face space codes all equal-sized physical changes with equal sensitivity) or nonlinear (e.g. face space shows greater coding sensitivity around the average face). Using adaptation aftereffects and pairwise discrimination tasks, our results for face attributes of eye height and mouth height demonstrate linear shape; including for bizarre faces far outside the normal range. We discuss how linear coding explains some results in the previous literature, including failures to find that adaptation enhances face discrimination, and suggest possible reasons why face space can maintain detailed coding of values far outside the normal range. We also discuss specific nonlinear coding models needed to explain other findings, and conclude face space appears to use a mixture of linear and nonlinear representations. [Copyright &y& Elsevier]

Abstract: The effects of learning on the geometry of face space were investigated by measuring thresholds for discrimination and recognition of synthetic faces. This was based on a novel experimental technique that permitted measurement of psychometric functions for face recognition. Two major results were obtained. First, thresholds for face recognition were significantly better than thresholds for discrimination among novel faces. Second, rapid discrimination in the neighborhood of learned faces was better than discrimination near novel faces. Control experiments showed that this discrimination improvement occurred only with learned faces, and it could not be explained by generalized discrimination learning. Thus, face learning selectively alters or distorts face space in the vicinity of learned faces. This alteration may be due to an improvement in the signal/noise ratio as a result of face learning. [Copyright &y& Elsevier]

Abstract: Recent evidence demonstrates that adapting to a face will systematically bias the perception of faces that lie along the same identity trajectory in geometric face space but not faces that lie along different identity trajectories [Leopold, D.A., O’Toole, A. J., Vetter, T., & Blanz, V. (2001). Prototype-referenced shape encoding revealed by high-level aftereffects. Nature Neuroscience, 4(1), 89–94]. We explored this configural aftereffect using synthetic face stimuli developed to measure face-specific processing. Adapting to synthetic “anti-faces” resulted in an identity-specific aftereffect that was characterized by a marked decrease in the slope of the psychometric functions. Adaptation transferred across different face sizes, but not different face viewpoints nor faces constructed about a non-mean face. Performance was captured by a model where responses were modulated through a divisive gain control and an additive constant reflecting a shift in the origin of perceived face space. Together, these results suggest that face adaptation reflects activity from mechanisms common to various processing stages along the visual pathway. [Copyright &y& Elsevier]

Abstract: The neural principles of the encoding of face spaces in visual cortex are still unclear and multiple competing theories have been proposed. Based on new electrophysiological data from macaque area IT we test two models realizing example-based and norm-referenced encoding. Comparing the experimentally measured tuning properties with predictions from the two models we find a better agreement for the norm-referenced encoding model. This suggests that a majority of IT neurons might represent deviations from a norm face, which is determined by an average over the distribution of typically occurring faces. [Copyright &y& Elsevier]

To simplify the study of visual face processing, we introduce a novel class of synthetic face stimuli based upon 37 measurements (head shape, feature locations, etc.) extracted from individual face photographs in both frontal and 20° side views. Synthetic faces are bandpass filtered optimally for face perception and include both line and edge information. Pilot experiments establish that subjects are extremely accurate in matching a synthetic face with the original grayscale photograph, even across views. To determine the perceptual metric of face space, we introduce face cubes in which the geometric differences between any faces in a four-dimensional face subspace can be precisely determined. Experiments on face discrimination using face cubes establish the metric of synthetic face space as locally Euclidean, with discrimination thresholds representing 4–6% total geometric variation (as a percent of mean head radius) between faces. Discrimination thresholds are lowest for face cubes constructed around the average face, thus indicating that the mean face for each gender represents a natural origin for face space. Finally, synthetic faces exhibit a pronounced inversion effect for 20° side views and a characteristic “Thatcher effect” for inverted front views. Synthetic faces and face cubes thus provide a useful new quantitative approach to the study of face perception and face space. [Copyright &y& Elsevier]