Search

Understanding facial emotions is fundamental to interact in social environments and modify behavior accordingly. Neurodegenerative processes can progressively transform affective responses and affect social competence. This exploratory study examined the neurocognitive correlates of face recognition, in individuals with two mild cognitive impairment (MCI) etiologies (prodromal to dementia - MCI, or consequent to Parkinson's disease - PD-MCI). Performance on the identification and memorization of neutral and emotional facial expressions was assessed in 31 individuals with MCI, 26 with PD-MCI, and 30 healthy controls (HC). Individuals with MCI exhibited selective impairment in recognizing faces expressing fear, along with difficulties in remembering both neutral and emotional faces. Conversely, individuals with PD-MCI showed no differences compared with the HC in either emotion recognition or memory. In MCI, no significant association emerged between the memory for facial expressions and cognitive difficulties. In PD-MCI, regression analyses showed significant associations with higher-level cognitive functions in the emotional memory task, suggesting the presence of compensatory mechanisms. In a subset of participants, voxel-based morphometry revealed that the performance on emotional tasks correlated with regional changes in gray matter volume. The performance in the matching of negative expressions was predicted by volumetric changes in brain areas engaged in face and emotional processing, in particular increased volume in thalamic nuclei and atrophy in the right parietal cortex. Future studies should leverage on neuroimaging data to determine whether differences in emotional recognition are mediated by pathology-specific atrophic patterns., (© 2024. The Psychonomic Society, Inc.)

Survival prompts organisms to prepare adaptive behavior in response to environmental and social threat. However, what are the specific features of the appearance of a conspecific that trigger such adaptive behaviors? For social species, the prime candidates for triggering defense systems are the visual features of the face and the body. We propose a novel approach for studying the ability of the brain to gather survival-relevant information from seeing conspecific body features. Specifically, we propose that behaviorally relevant information from bodies and body expressions is coded at the levels of midlevel features in the brain. These levels are relatively independent from higher-order cognitive and conscious perception of bodies and emotions. Instead, our approach is embedded in an ethological framework and mobilizes computational models for feature discovery., Competing Interests: Declaration of interests The authors declare no conflicts of interest., (Copyright © 2021 The Authors. Published by Elsevier Ltd.. All rights reserved.)

Recent behavioural studies have provided evidence that virtual reality (VR) experiences have an impact on socio-affective processes, and a number of findings now underscore the potential of VR for therapeutic interventions. An interesting recent result is that when male offenders experience a violent situation as a female victim of domestic violence in VR, their sensitivity for recognition of fearful facial expressions improves. A timely question now concerns the underlying brain mechanisms of these behavioural effects as these are still largely unknown. The current study used fMRI to measure the impact of a VR intervention in which participants experienced a violent aggression from the specific vantage point of the victim. We compared brain processes related to facial and bodily emotion perception before and after the VR experience. Our results show that the virtual abuse experience led to an enhancement of Default Mode Network (DMN) activity, specifically associated with changes in the processing of ambiguous emotional stimuli. In contrast, DMN activity was decreased when observing fully fearful expressions. Finally, we observed increased variability in brain activity for male versus female facial expressions. Taken together, these results suggest that the first-person perspective of a virtual violent situation impacts emotion recognition through modifications in DMN activity. Our study contributes to a better understanding of the brain mechanisms associated with the behavioural effects of VR interventions in the context of a violent confrontation with the male participant embodied as a female victim. Furthermore, this research also consolidates the use of VR embodied perspective-taking interventions for addressing socio-affective impairments., Competing Interests: Declaration of competing interest The authors declare no conflict of interest., (Copyright © 2020 The Author(s). Published by Elsevier Ltd.. All rights reserved.)

Emotions are expressed by the face, the voice and the whole body. Research on the face and the voice has not only demonstrated that emotions are perceived categorically, but that this perception can be manipulated. The purpose of this study was to investigate, via two separate experiments using adaptation and multisensory techniques, whether the perception of body emotion expressions also shows categorical effects and plasticity. We used an approach developed for studies investigating both face and voice emotion perception and created novel morphed affective body stimuli, which varied in small incremental steps between emotions. Participants were instructed to perform an emotion categorisation of these morphed bodies after adaptation to bodies conveying different expressions (Experiment 1), or while simultaneously hearing affective voices (Experiment 2). We show that not only is body expression perceived categorically, but that both adaptation to affective body expressions and concurrent presentation of vocal affective information can shift the categorical boundary between body expressions, specifically for the angry body expressions. Overall, our findings provide significant new insights into emotional body categorisation, which may prove important in gaining a deeper understanding of body expression perception in everyday social situations.

Introduction: Neuromuscular diseases may be of neuropsychological interest insofar as they may affect representations based on embodied cognition theories. Previous studies have shown impaired ability to recognize facial emotions and an association between facial emotion recognition and visuospatial abilities in myotonic dystrophy type 1 (DM1) patients. Here we examined the ability of both DM1 and DM2 patients to recognize emotions expressed by body postures and its relation, and their association with cognitive performance., Methods: Participants included 34 DM1 patients, 8 DM2 patients, and 24 healthy control subjects. Emotional recognition ability was assessed through two computerized matching tasks (face and bodies). A neuropsychological battery was used to measure cognition in three domains and global cognition. We used univariate and adjusted linear regression models to investigate the association between cognition and emotion recognition performance., Results: DM patients (combined DM1 and DM2) performed worse on emotional facial expression (p = .006) and body posture (p = .004) accuracy measures than healthy controls. In linear regression models, DM patients' facial expression accuracy was associated with executive function (p = .013) and visuospatial (p < .001) z-scores. Body posture accuracy was associated with visuospatial (p = .001) and memory (p = .012) z-scores. There were no associations among controls or between cognition and reaction time., Discussion: These findings suggest that impaired emotional recognition among DM patients is also extended to emotions conveyed by body postures. Consistent with embodied cognition theories, people affected in their body and its movement may have impaired sensorimotor representation in ways that have yet to be fully understood., Competing Interests: Declaration of Competing Interest Nothing to declare., (Copyright © 2020 Elsevier Ltd. All rights reserved.)

Computer-generated (CG) faces are an important visual interface for human-computer interaction in social contexts. Here we investigated whether the human brain processes emotion and gaze similarly in real and carefully matched CG faces. Real faces evoked greater responses in the fusiform face area than CG faces, particularly for fearful expressions. Emotional (angry and fearful) facial expressions evoked similar activations in the amygdala in real and CG faces. Direct as compared with averted gaze elicited greater fMRI responses in the amygdala regardless of facial expression but only for real and not for CG faces. We observed an interaction effect between gaze and emotion (i.e., the shared signal effect) in the right posterior temporal sulcus and other regions, but not in the amygdala, and we found no evidence for different shared signal effects in real and CG faces. Taken together, the present findings highlight similarities (emotional processing in the amygdala) and differences (overall processing in the fusiform face area, gaze processing in the amygdala) in the neural processing of real and CG faces., (Copyright © 2019 The Authors. Published by Elsevier Inc. All rights reserved.)

In the natural world, faces are not isolated objects but are rather encountered in the context of the whole body. Previous work has studied the perception of combined faces and bodies using behavioural and electrophysiological measurements, but the neural correlates of emotional face-body perception still remain unexplored. Here, we combined happy and fearful faces and bodies to investigate the influence of body expressions on the neural processing of the face, the effect of emotional ambiguity between the two and the role of the amygdala in this process. Our functional magnetic resonance imaging analyses showed that the activity in motor, prefrontal and visual areas increases when facial expressions are presented together with bodies rather than in isolation, consistent with the notion that seeing body expressions triggers both emotional and action-related processes. In contrast, psychophysiological interaction analyses revealed that amygdala modulatory activity increases after the presentation of isolated faces when compared to combined faces and bodies. Furthermore, a facial expression combined with a congruent body enhanced both cortical activity and amygdala functional connectivity when compared to an incongruent face-body compound. Finally, the results showed that emotional body postures influence the processing of facial expressions, especially when the emotion conveyed by the body implies danger., (© The Author (2017). Published by Oxford University Press.)

The eye-region conveys important emotional information that we spontaneously attend to. Socially submissive individuals avoid other's gaze which is regarded as avoidance of others' emotional face expressions. But this interpretation ignores the fact that there are other sources of emotional information besides the face. Here we investigate whether gaze-aversion is associated with increased attention to emotional signals from the hands. We used eye-tracking to compare eye-fixations of pre-selected high and low socially anxious students when labeling bodily expressions (Experiment 1) with (non)-matching facial expressions (Experiment 2) and passively viewed (Experiment 3). High compared to low socially anxious individuals attended more to hand-regions. Our findings demonstrate that socially anxious individuals do attend to emotions, albeit to different signals than the eyes and the face. Our findings call for a closer investigation of alternative viewing patterns explaining gaze-avoidance and underscore that other signals besides the eyes and face must be considered to reach conclusions about social anxiety., (Copyright © 2016. Published by Elsevier B.V.)

The amygdala is a complex structure that plays its role in perception and threat-related behaviour by activity of its specific nuclei and their separate networks. In the present functional magnetic resonance imaging study, we investigated the role of the basolateral amygdala in face and context processing. Five individuals with focal basolateral amygdala damage and 12 matched controls viewed fearful or neutral faces in a threatening or neutral context. We tested the hypothesis that basolateral amygdala damage modifies the relation between face and threatening context, triggering threat-related activation in the dorsal stream. The findings supported this hypothesis. First, activation was increased in the right precentral gyrus for threatening versus neutral scenes in the basolateral amygdala damage group compared with the control group. Second, activity in the bilateral middle frontal gyrus, and left anterior inferior parietal lobule was enhanced for neutral faces presented in a threatening versus neutral scene in the group with basolateral amygdala damage compared with controls. These findings provide the first evidence for the neural consequences of basolateral amygdala damage during the processing of complex emotional situations., (© 2016 The Author(s).)

Emotional signals are perceived whether or not we are aware of it. The evidence so far mostly came from studies with facial expressions. Here, we investigated whether the pattern of non-conscious face expression perception is found for whole body expressions. Continuous flash suppression (CFS) was used to measure the time for neutral, fearful, and angry facial or bodily expressions to break from suppression. We observed different suppression time patterns for emotions depending on whether the stimuli were faces or bodies. The suppression time for anger was shortest for bodily expressions, but longest for the facial expressions. This pattern indicates different processing and detection mechanisms for faces and bodies outside awareness, and suggests that awareness mechanisms associated with dorsal structures might play a role in becoming conscious of angry bodily expressions.

Most face processing studies in humans show stronger activation in the right compared to the left hemisphere. Evidence is largely based on studies with static stimuli focusing on the fusiform face area (FFA). Hence, the pattern of lateralization for dynamic faces is less clear. Furthermore, it is unclear whether this property is common to human and non-human primates due to predisposing processing strategies in the right hemisphere or that alternatively left sided specialization for language in humans could be the driving force behind this phenomenon. We aimed to address both issues by studying lateralization for dynamic facial expressions in monkeys and humans. Therefore, we conducted an event-related fMRI experiment in three macaques and twenty right handed humans. We presented human and monkey dynamic facial expressions (chewing and fear) as well as scrambled versions to both species. We studied lateralization in independently defined face-responsive and face-selective regions by calculating a weighted lateralization index (LIwm) using a bootstrapping method. In order to examine if lateralization in humans is related to language, we performed a separate fMRI experiment in ten human volunteers including a 'speech' expression (one syllable non-word) and its scrambled version. Both within face-responsive and selective regions, we found consistent lateralization for dynamic faces (chewing and fear) versus scrambled versions in the right human posterior superior temporal sulcus (pSTS), but not in FFA nor in ventral temporal cortex. Conversely, in monkeys no consistent pattern of lateralization for dynamic facial expressions was observed. Finally, LIwms based on the contrast between different types of dynamic facial expressions (relative to scrambled versions) revealed left-sided lateralization in human pSTS for speech-related expressions compared to chewing and emotional expressions. To conclude, we found consistent laterality effects in human posterior STS but not in visual cortex of monkeys. Based on our results, it is tempting to speculate that lateralization for dynamic face processing in humans may be driven by left-hemispheric language specialization which may not have been present yet in the common ancestor of human and macaque monkeys., (Copyright © 2014 Elsevier Inc. All rights reserved.)

It has been shown that dominant individuals sustain eye-contact when non-consciously confronted with angry faces, suggesting reflexive mechanisms underlying dominance behaviors. However, dominance and submission can be conveyed and provoked by means of not only facial but also bodily features. So far few studies have investigated the interplay of body postures with personality traits and behavior, despite the biological relevance and ecological validity of these postures. Here we investigate whether non-conscious exposure to bodily expressions of anger evokes reflex-like dominance behavior. In an interactive eye-tracking experiment thirty-two participants completed three social dominance tasks with angry, happy and neutral facial, bodily and face and body compound expressions that were masked from consciousness. We confirmed our predictions of slower gaze-aversion from both non-conscious bodily and compound expressions of anger compared to happiness in high dominant individuals. Results from a follow-up experiment suggest that the dominance behavior triggered by exposure to bodily anger occurs with basic detection of the category, but not recognition of the emotional content. Together these results suggest that dominant staring behavior is reflexively driven by non-conscious perception of the emotional content and triggered by not only facial but also bodily expression of anger.

Facial expression perception can be influenced by the natural visual context in which the face is perceived. We performed an fMRI experiment presenting participants with fearful or neutral faces against threatening or neutral background scenes. Triangles and scrambled scenes served as control stimuli. The results showed that the valence of the background influences face selective activity in the right anterior parahippocampal place area (PPA) and subgenual anterior cingulate cortex (sgACC) with higher activation for neutral backgrounds compared to threatening backgrounds (controlled for isolated background effects) and that this effect correlated with trait empathy in the sgACC. In addition, the left fusiform gyrus (FG) responds to the affective congruence between face and background scene. The results show that valence of the background modulates face processing and support the hypothesis that empathic processing in sgACC is inhibited when affective information is present in the background. In addition, the findings reveal a pattern of complex scene perception showing a gradient of functional specialization along the posterior-anterior axis: from sensitivity to the affective content of scenes (extrastriate body area: EBA and posterior PPA), over scene emotion-face emotion interaction (left FG) via category-scene interaction (anterior PPA) to scene-category-personality interaction (sgACC)., (© The Author (2013). Published by Oxford University Press. For Permissions, please email: journals.permissions@oup.com.)

Studies of cortical blindness have suggested that some residual visual function may persist without perceptual awareness, a condition known as blindsight. To investigate electrophysiological evidence of unconscious processing of emotional stimuli, we examined the event-related oscillations (EROs) in a 62year-old male patient (TN) with affective blindsight during random stimulation of three facial expressions (fearful, happy and neutral). Spectral power analysis in response to the different emotions revealed significant differences between fearful and happy faces over the right frontal regions at 7-8Hz (low α), and between emotional and neutral faces over the left frontal sites at 12-13Hz (low β) in a time period between 100-400ms after visual stimulus onset. These results demonstrate that emotional face processing occurs very early in time in the absence of any functional striate cortex, and further reveals the existence of specific oscillatory frequencies that reflect unconscious processing of facial expressions in affective blindsight., (© 2013.)

Face perception and emotion recognition have been extensively studied in the past decade; however, the relation between them is still poorly understood. A traditional view is that successful emotional categorization requires categorization of the stimulus as a 'face', at least at the basic level. Here we tested whether emotional information could still be recognized accurately without explicit categorization of a stimulus as a face. For this purpose we created a stimulus set in which facial stimuli expressing a range of happy-to-fear emotions were morphed into another object category (shoe). Interestingly, participants categorized emotions with great accuracy in stimuli that contained so little face information that they were explicitly categorized as shoes. Hence, our results show that accurate emotion categorization can take place in stimuli that contain surprisingly little face information. This finding raises interesting questions about the extent to which processes leading to emotion recognition and categorical face perception might be separable.

Emotional facial expressions play an important role in social communication across primates. Despite major progress made in our understanding of categorical information processing such as for objects and faces, little is known, however, about how the primate brain evolved to process emotional cues. In this study, we used functional magnetic resonance imaging (fMRI) to compare the processing of emotional facial expressions between monkeys and humans. We used a 2×2×2 factorial design with species (human and monkey), expression (fear and chewing) and configuration (intact versus scrambled) as factors. At the whole brain level, neural responses to conspecific emotional expressions were anatomically confined to the superior temporal sulcus (STS) in humans. Within the human STS, we found functional subdivisions with a face-selective right posterior STS area that also responded to emotional expressions of other species and a more anterior area in the right middle STS that responded specifically to human emotions. Hence, we argue that the latter region does not show a mere emotion-dependent modulation of activity but is primarily driven by human emotional facial expressions. Conversely, in monkeys, emotional responses appeared in earlier visual cortex and outside face-selective regions in inferior temporal cortex that responded also to multiple visual categories. Within monkey IT, we also found areas that were more responsive to conspecific than to non-conspecific emotional expressions but these responses were not as specific as in human middle STS. Overall, our results indicate that human STS may have developed unique properties to deal with social cues such as emotional expressions., (Copyright © 2012 Elsevier Inc. All rights reserved.)

This study addresses two central and controversial issues in developmental prosopagnosia (DP), configuration- versus feature-based face processing and the influence of affective information from either facial or bodily expressions on face recognition. A sample of 10 DPs and 10 controls were tested with a previously developed face and object recognition and memory battery (Facial Expressive Action Stimulus Test, FEAST), a task measuring the influence of emotional faces and bodies on face identity matching (Face-Body Compound task), and an emotionally expressive face memory task (Emotional Face Memory task, FaMe-E). We show that DPs were impaired in upright, but not inverted, face matching but they performed at the level of controls on part-to-whole matching. Second, DPs showed impaired memory for both neutral and emotional faces and scored within the normal range on the Face-Body Compound task. Third, configural perception but not feature-based processing was significantly associated with memory performance. Taken together the results indicate that DPs have a deficit in configural processing at the perception stage that may underlie the memory impairment.

The spontaneous tendency to synchronize our facial expressions with those of others is often termed emotional contagion. It is unclear, however, whether emotional contagion depends on visual awareness of the eliciting stimulus and which processes underlie the unfolding of expressive reactions in the observer. It has been suggested either that emotional contagion is driven by motor imitation (i.e., mimicry), or that it is one observable aspect of the emotional state arising when we see the corresponding emotion in others. Emotional contagion reactions to different classes of consciously seen and "unseen" stimuli were compared by presenting pictures of facial or bodily expressions either to the intact or blind visual field of two patients with unilateral destruction of the visual cortex and ensuing phenomenal blindness. Facial reactions were recorded using electromyography, and arousal responses were measured with pupil dilatation. Passive exposure to unseen expressions evoked faster facial reactions and higher arousal compared with seen stimuli, therefore indicating that emotional contagion occurs also when the triggering stimulus cannot be consciously perceived because of cortical blindness. Furthermore, stimuli that are very different in their visual characteristics, such as facial and bodily gestures, induced highly similar expressive responses. This shows that the patients did not simply imitate the motor pattern observed in the stimuli, but resonated to their affective meaning. Emotional contagion thus represents an instance of truly affective reactions that may be mediated by visual pathways of old evolutionary origin bypassing cortical vision while still providing a cornerstone for emotion communication and affect sharing.

Most studies investigating speeded orientation towards threat have used manual responses. By measuring orienting behaviour using eye movements a more direct and ecologically valid measure of attention can be made. Here, we used a forced-choice saccadic and manual localization task to investigate the speed of discrimination for fearful and neutral body and face images. Fearful/neutral body or face pairs were bilaterally presented for either 20 or 500 ms. Results showed faster saccadic orienting to fearful body and face emotions compared with neutral only at the shortest presentation time (20 ms). For manual responses, faster discrimination of fearful bodies and faces was observed only at the longest duration (500 ms). More errors were made when localizing neutral targets, suggesting that fearful bodies and faces may have captured attention automatically. Results were not attributable to low-level image properties as no threat bias, in terms of reaction time or accuracy, was observed for inverted presentation. Taken together, the results suggest faster localization of threat conveyed both by the face and the body within the oculomotor system. In addition, enhanced detection of fearful body postures suggests that we can readily recognize threat-related information conveyed by body postures in the absence of any face cues.

Many studies provide support for the role of the fusiform gyrus in face recognition and its sensitivity to emotional expressions. Recently, category-specific representation was also observed for neutral human bodies in the middle temporal/middle occipital gyrus (extrastriate body area) but it is not clear whether this area is also sensitive to emotional bodily expressions. Besides these areas, other regions that process the affective information carried by the face and the body may be common and/or specific to the face or the body. To clarify these issues we performed a systematic comparison of how the whole brain processes faces and bodies and how their affective information is represented. Participants categorized emotional facial and bodily expressions while brain activity was measured using functional magnetic resonance imaging. Our results show that, first, the amygdala and the fusiform gyrus are sensitive to recognition of facial and bodily fear signals. Secondly, the extrastriate body area-area V5/MT is specifically involved in processing bodies without being sensitive to the emotion displayed. Thirdly, other important areas such as the superior temporal sulcus, the parietal lobe and subcortical structures represent selectively facial and bodily expressions. Finally, some face/body differences in activation are a function of the emotion expressed.

Facial expression and direction of gaze are two important sources of social information, and what message each conveys may ultimately depend on how the respective information interacts in the eye of the perceiver. Direct gaze signals an interaction with the observer but averted gaze amounts to "pointing with the eyes", and in combination with a fearful facial expression may signal the presence of environmental danger. We used fMRI to examine how gaze direction influences brain processing of facial expression of fear. The combination of fearful faces and averted gazes activated areas related to gaze shifting (STS, IPS) and fear-processing (amygdala, hypothalamus, pallidum). Additional modulation of activation was observed in motion detection areas, in premotor areas and in the somatosensory cortex, bilaterally. Our results indicate that the direction of gaze prompts a process whereby the brain combines the meaning of the facial expression with the information provided by gaze direction, and in the process computes the behavioral implications for the observer.

Many people experience transient difficulties in recognizing faces but only a small number of them cannot recognize their family members when meeting them unexpectedly. Such face blindness is associated with serious problems in everyday life. A better understanding of the neuro-functional basis of impaired face recognition may be achieved by a careful comparison with an equally unique object category and by a adding a more realistic setting involving neutral faces as well facial expressions. We used event-related functional magnetic resonance imaging (fMRI) to investigate the neuro-functional basis of perceiving faces and bodies in three developmental prosopagnosics (DP) and matched healthy controls. Our approach involved materials consisting of neutral faces and bodies as well as faces and bodies expressing fear or happiness. The first main result is that the presence of emotional information has a different effect in the patient vs. the control group in the fusiform face area (FFA). Neutral faces trigger lower activation in the DP group, compared to the control group, while activation for facial expressions is the same in both groups. The second main result is that compared to controls, DPs have increased activation for bodies in the inferior occipital gyrus (IOG) and for neutral faces in the extrastriate body area (EBA), indicating that body and face sensitive processes are less categorically segregated in DP. Taken together our study shows the importance of using naturalistic emotional stimuli for a better understanding of developmental face deficits.

In daily life, we perceive a person's facial reaction as part of the natural environment surrounding it. Because most studies have investigated how facial expressions are recognized by using isolated faces, it is unclear what role the context plays. Although it has been observed that the N170 for facial expressions is modulated by the emotional context, it was not clear whether individuals use context information on this stage of processing to discriminate between facial expressions. The aim of the present study was to investigate how the early stages of face processing are affected by emotional scenes when explicit categorizations of fearful and happy facial expressions are made. Emotion effects were found for the N170, with larger amplitudes for faces in fearful scenes as compared to faces in happy and neutral scenes. Critically, N170 amplitudes were significantly increased for fearful faces in fearful scenes as compared to fearful faces in happy scenes and expressed in left-occipito-temporal scalp topography differences. Our results show that the information provided by the facial expression is combined with the scene context during the early stages of face processing.

Recognition of facial expressions has traditionally been investigated by presenting facial expressions without any context information. However, we rarely encounter an isolated facial expression; usually, we perceive a person's facial reaction as part of the surrounding context. In the present study, we addressed the question of whether emotional scenes influence the explicit recognition of facial expressions. In three experiments, participants were required to categorize facial expressions (disgust, fear, happiness) that were shown against backgrounds of natural scenes with either a congruent or an incongruent emotional significance. A significant interaction was found between facial expressions and the emotional content of the scenes, showing a response advantage for facial expressions accompanied by congruent scenes. This advantage was robust against increasing task load. Taken together, the results show that the surrounding scene is an important factor in recognizing facial expressions.

Three experiments investigated whether emotional information influences perceptual dominance during binocular rivalry. In Experiment 1, rival emotional and neutral faces in the background were coupled with grating stimuli in the foreground. Results showed that gratings paired with emotional faces dominated over those paired with neutral faces. In Experiment 2, emotional and neutral faces were presented dichoptically, without being paired with other stimuli. Dominance of emotional faces was observed. Fusion and low-level image differences were ruled out by examining dominance periods of upright and inverted emotional and neutral faces presented as face-house pairs (Experiment 3). Here, face stimuli dominated over house stimuli only for upright face conditions. In addition, upright emotional faces were perceived for significantly longer durations than upright neutral faces. The results provide further support for the influence of emotional meaning on binocular rivalry.

The emotional valence of facial expressions can be reliably discriminated even in the absence of conscious visual experience by patients with lesions to the primary visual cortex (affective blindsight). Prior studies in one such patient (GY) also showed that this non-conscious perception can influence conscious recognition of normally seen emotional faces. Here we report a similar online interaction across hemispheres between conscious and non-conscious perception of emotions in normal observers. Fearful and happy facial expressions were presented either unilaterally (to the left or right visual field) or simultaneously to both visual fields. In bilateral displays, conscious perception of one face in a pair was prevented by backward masking after 20 ms, while the opposite expression remained normally visible. The results showed a bidirectional influence of non-conscious fear processing over conscious recognition of happy as well as fearful expressions. Consciously perceived fearful faces were more readily recognized when they were paired with invisible emotionally congruent fearful expressions in the opposite field, as compared to the single presentation of the same unmasked faces. On the other hand, recognition of unmasked happy faces was delayed by the simultaneous presence of a masked fearful face. No such effect was reported for masked happy expressions. These findings show that non-conscious processing of fear may modulate ongoing conscious evaluation of facial expressions via neural interhemispheric summation even in the intact brain.

Background: Despite extensive research, it is still debated whether impairments in social skills of individuals with pervasive developmental disorder (PDD) are related to specific deficits in the early processing of emotional information. We aimed to test both automatic processing of facial affect as well as the integration of auditory and visual emotion cues in individuals with PDD., Methods: In a group of high-functioning adult individuals with PDD and an age- and IQ-matched control group, we measured facial electromyography (EMG) following presentation of visual emotion stimuli (facial expressions) as well as the presentation of audiovisual emotion pairs (faces plus voices). This emotionally driven EMG activity is considered to be a direct correlate of automatic affect processing that is not under intentional control., Results: Our data clearly indicate that among individuals with PDD facial EMG activity is heightened in response to happy and fearful faces, and intact in response to audiovisual affective information., Conclusions: This study provides evidence for enhanced sensitivity to facial cues at the level of reflex-like emotional responses in individuals with PDD. Furthermore, the findings argue against impairments in crossmodal affect processing at this level of perception. However, given how little comparative work has been done in the area of multisensory perception, there is certainly need for further exploration.

The most familiar emotional signals consist of faces, voices, and whole-body expressions, but so far research on emotions expressed by the whole body is sparse. The authors investigated recognition of whole-body expressions of emotion in three experiments. In the first experiment, participants performed a body expression-matching task. Results indicate good recognition of all emotions, with fear being the hardest to recognize. In the second experiment, two alternative forced choice categorizations of the facial expression of a compound face-body stimulus were strongly influenced by the bodily expression. This effect was a function of the ambiguity of the facial expression. In the third experiment, recognition of emotional tone of voice was similarly influenced by task irrelevant emotional body expressions. Taken together, the findings illustrate the importance of emotional whole-body expressions in communication either when viewed on their own or, as is often the case in realistic circumstances, in combination with facial expressions and emotional voices., (((c) 2007 APA, all rights reserved))

Observing facial expressions automatically prompts imitation, as can be seen with facial electromyography. To investigate whether this reaction is driven by automatic mimicry or by recognition of the emotion displayed we recorded electromyograph responses to presentations of facial expressions, face-voice combinations and bodily expressions, which resulted from happy and fearful stimuli. We observed emotion-specific facial muscle activity (zygomaticus for happiness, corrugator for fear) for all three stimulus categories. This indicates that spontaneous facial expression is more akin to an emotional reaction than to facial mimicry and imitation of the seen face stimulus. We suggest that seeing a facial expression, an emotional body expression or hearing an emotional tone of voice all activate the affect program corresponding to the emotion displayed.

Hemispheric asymmetry in emotional perception has been traditionally studied for basic emotions and very little is known about laterality for more complex social emotions. Here, we used the "redundant target paradigm" to investigate interhemispheric asymmetry and cooperation for two social emotions in healthy subjects. Facial expressions of flirtatiousness or arrogance were briefly presented either unilaterally in the left (LVF) or right visual field (RVF), or simultaneously to both visual fields (BVF) while participants responded to the target expression (flirtatious or arrogant, counterbalanced between blocks). In bilateral conditions the faces could show the same emotion (congruent condition) or two different expressions (incongruent condition). No difference between unilateral presentations was found, suggesting that the perception of social emotions is not hemispherically lateralized. Responses were faster and more accurate in bilateral displays with two emotionally congruent but physically different faces (i.e., a male and a female expressing the same emotion) than in unilateral conditions. This "redundant target effect" was consistent with a neural summation model, thereby showing that interhemispheric cooperation may occur for social emotions despite major perceptual differences between faces posing the same expression.

Electrophysiological and hemodynamic correlates of processing isolated faces have been investigated extensively over the last decade. A question not addressed thus far is whether the visual scene, which normally surrounds a face or a facial expression, has an influence on how the face is processed. Here we investigated this issue by presenting faces in natural contexts and measuring whether the emotional content of the scene influences processing of a facial expression. Event-related potentials were recorded to faces (fearful/neutral) embedded in scene contexts (fearful/neutral) while participants performed an orientation-decision task (face upright or inverted). Two additional experiments were run, one to examine the effects of context that occur without a face and the other to evaluate the effects of faces isolated from contexts. Faces without any context showed the largest N170 amplitudes. The presence of a face in a fearful context enhances the N170 amplitude over a face in neutral contexts, an effect that is strongest for fearful faces on left occipito-temporal sites. This N170 effect, and the corresponding topographic distribution, was not found for contexts-only, indicating that the increased N170 amplitude results from the combination of face and fearful context. These findings suggest that the context in which a face appears may influence how it is encoded.

The present study used the redundant target paradigm on healthy subjects to investigate functional hemispheric asymmetries and interhemispheric cooperation in the perception of emotions from faces. In Experiment 1 participants responded to checkerboards presented either unilaterally to the left (LVF) or right visual half field (RVF), or simultaneously to both hemifields (BVF), while performing a pointing task for the control of eye movements. As previously reported (Miniussi et al. in J Cogn Neurosci 10:216-230, 1998), redundant stimulation led to shorter latencies for stimulus detection (bilateral gain or redundant target effect, RTE) that exceeded the limit for a probabilistic interpretation, thereby validating the pointing procedure and supporting interhemispheric cooperation. In Experiment 2 the same pointing procedure was used in a go/no-go task requiring subjects to respond when seeing a target emotional expression (happy or fearful, counterbalanced between blocks). Faster reaction times to unilateral LVF than RVF emotions, regardless of valence, indicate that the perception of positive and negative emotional faces is lateralized toward the right hemisphere. Simultaneous presentation of two congruent emotional faces, either happy or fearful, produced an RTE that cannot be explained by probability summation and suggests interhemispheric cooperation and neural summation. No such effect was present with BVF incongruent facial expressions. In Experiment 3 we studied whether the RTE for emotional faces depends on the physical identity between BVF stimuli, and we set a second BVF congruent condition in which there was only emotional but not physical or gender identity between stimuli (i.e. two different faces expressing the same emotion). The RTE and interhemispheric cooperation were present also in this second BVF congruent condition. This shows that emotional congruency is the sufficient condition for the RTE to take place in the intact brain and that the cerebral hemispheres can interact in spite of physical differences between stimuli.

Humans optimize behavior by deriving context-based expectations. Contextual data that are important for survival are extracted rapidly, using coarse information, adaptive decision strategies, and dedicated neural infrastructure. In the field of object perception, the influence of a surrounding context has been a major research theme, and it has generated a large literature. That visual context, as typically provided by natural scenes, facilitates object recognition as has been convincingly demonstrated (Bar, M. (2004) Nat. Rev. Neurosci., 5: 617-629). Just like objects, faces are generally encountered as part of a natural scene. Thus far, the facial expression literature has neglected such context and treats facial expressions as if they stand on their own. This constitutes a major gap in our knowledge. Facial expressions tend to appear in a context of head and body orientations, body movements, posture changes, and other object-related actions with a similar or at least a closely related meaning. For instance, one would expect a frightened face when confronted to an external danger to be at least accompanied by withdrawal movements of head and shoulders. Furthermore, some cues provided by the environment or the context in which a facial expression appears may have a direct relation with the emotion displayed by the face. The brain may even fill in the natural scene context typically associated with the facial expression. Recognition of the facial expression may also profit from processing the vocal emotion as well as the emotional body language that normally accompany it. Here we review the emerging evidence on how the immediate visual and auditory contexts influence the recognition of facial expressions.

In our natural world, a face is usually encountered not as an isolated object but as an integrated part of a whole body. The face and the body both normally contribute in conveying the emotional state of the individual. Here we show that observers judging a facial expression are strongly influenced by emotional body language. Photographs of fearful and angry faces and bodies were used to create face-body compound images, with either matched or mismatched emotional expressions. When face and body convey conflicting emotional information, judgment of facial expression is hampered and becomes biased toward the emotion expressed by the body. Electrical brain activity was recorded from the scalp while subjects attended to the face and judged its emotional expression. An enhancement of the occipital P1 component as early as 115 ms after presentation onset points to the existence of a rapid neural mechanism sensitive to the degree of agreement between simultaneously presented facial and bodily emotional expressions, even when the latter are unattended.

Using positron emission tomography we explored brain regions activated during the perception of face expressions, emotional voices and combined audio-visual pairs. A convergence region situated in the left lateral temporal cortex was more activated by bimodal stimuli than by either visual only or auditory only stimuli. Separate analyses for the emotions happiness and fear revealed supplementary convergence areas situated mainly anteriorly in the left hemisphere for happy pairings and in the right hemisphere for fear pairings indicating different neuro-anatomical substrates for multisensory integration of positive versus negative emotions. Activation in the right extended amygdala was obtained for fearful faces and fearful audio-visual pairs but not for fearful voices only. These results suggest that during the multisensory perception of emotion, affective information from face and voice converge in heteromodal regions of the human brain.

In their natural environment, organisms receive information through multiple sensory channels and these inputs from different sensory systems are routinely combined into integrated percepts. Previously, we reported that in a population of schizophrenics, deficits in audiovisual integration were observed for complex stimuli (auditory and visual syllables), but not for more simple ones (beeps and light flashes). Here, we investigated multisensory integration of emotional information in a group of schizophrenic patients. In Experiment 1, we found a reduced effect of an emotional voice on the categorization of a facial expression. In Experiment 2, the reverse test situation was presented, and, here, we observed an exaggerated effect of a face expression on the categorization of an emotional voice. Results are discussed in the light of current models of multisensory integration and their relevance for schizophrenia.

Brain-damaged patients experience difficulties in recognizing a face (prosopagnosics), but they can still recognize its expression. The dissociation between these two face-related skills has served as a keystone of models of face processing. We now report that the presence of a facial expression can influence face identification. For normal viewers, the presence of a facial expression influences performance negatively, whereas for prosopagnosic patients, it improves performance dramatically. Accordingly, although prosopagnosic patients show a failure to process the facial configuration in the interest of face identification, that ability returns when the face shows an emotional expression. Accompanying brain-imaging results indicate activation in brain areas (amygdala, superior temporal sulcus, parietal cortex) outside the occipitotemporal areas normally activated for face identification and lesioned in these patients. This finding suggests a modulatory role of these areas in face identification that is independent of occipitotemporal face areas.

Two experiments with upright and inverted face and object images were carried out to investigate whether face processing in autism is more feature-based than in individuals with typical development. Participants were 17 high-ability adolescents with autistic disorder (16-24 years), 24 typically developing children (9-10 years) and 16 adults (18-33 years). In Experiment 1, a normal inversion effect was found for the adolescents with autism in a standard face recognition paradigm with reduced memory demands, except for a subgroup with low social intelligence who were not better in recognizing upright relative to inverted photographs of faces. In Experiment 2, the group with autism did not show the composite effect like the adult group did: they recognized face halves as well in aligned composite faces as in non-aligned composite faces. The results on the inversion task suggest that most adolescents with autism form a normal configuration-based face representation, but the absence of the composite effect indicates that they are less prone to use the contextual information of the face in a visual-search task.

Long latency auditory brain potentials were recorded while subjects listened to bi-syllabic words spoken with an emotional expression and concurrently viewed congruent or incongruent facial expressions. Analysis of the auditory waveforms suggests the existence of a positive deflection around 240 ms post-stimulus with a clear posterior topography (the P2b component). This potential is subsequent upon the modality-specific auditory N1-P2 components and precedes the amodal N2-P3 complex. Congruent face-voice trials elicited an earlier P2b component than incongruent trials suggesting that auditory processing is delayed in the presence of an incongruent facial context. These electrophysiological results are consistent with previous behavioural studies showing an acceleration of reaction times for rating voice expressions that are part of congruent bimodal stimulus pairs. A source localisation analysis performed on the scalp EEG during the time-window corresponding to the P2b component disclosed a single dipole solution in the anterior cingulate cortex.

Multisensory integration is a powerful mechanism for increasing adaptive responses, as illustrated by binding of fear expressed in a face with fear present in a voice. To understand the role of awareness in intersensory integration of affective information we studied multisensory integration under conditions of conscious and nonconscious processing of the visual component of an audiovisual stimulus pair. Auditory-event-related potentials were measured in two patients (GY and DB) who were unable to perceive visual stimuli consciously because of striate cortex damage. To explore the role of conscious vision of audiovisual pairing, we also compared audiovisual integration in either naturalistic pairings (a facial expression paired with an emotional voice) or semantic pairings (an emotional picture paired with the same voice). We studied the hypothesis that semantic pairings, unlike naturalistic pairings, might require mediation by intact visual cortex and possibly by feedback to primary cortex from higher cognitive processes. Our results indicate that presenting incongruent visual affective information together with the voice translates as an amplitude decrease of auditory-event-related potentials. This effect obtains for both naturalistic and semantic pairings in the intact field, but is restricted to the naturalistic pairings in the blind field.

There are many ways to assess face perception skills. In this study, we describe a novel task battery FEAST (Facial Expressive Action Stimulus Test) developed to test recognition of identity and expressions of human faces as well as stimulus control categories. The FEAST consists of a neutral and emotional face memory task, a face and shoe identity matching task, a face and house part-to-whole matching task, and a human and animal facial expression matching task. The identity and part-to-whole matching tasks contain both upright and inverted conditions. The results provide reference data of a healthy sample of controls in two age groups for future users of the FEAST. [ABSTRACT FROM AUTHOR]

Faces provide information about multiple characteristics like personal identity and emotion. Classical models of face perception postulate separate sub-systems for identity and expression recognition but recent studies have documented emotional contextual influences on recognition of faces. The present study reports three experiments where participants were presented realistic face-body compounds in a 2 category (face and body) × 2 emotion (neutral and fearful) factorial design. The task always consisted of two-alternative forced choice facial identity matching. The results show that during simultaneous face identity matching, the task irrelevant bodily expressions influence processing of facial identity, under conditions of unlimited viewing (Experiment 1) as well as during brief (750 ms) presentation (Experiment 2). In addition, delayed (5000 ms) face identity matching of rapidly (150 ms) presented face-body compounds, was also influenced by the body expression (Experiment 3). The results indicate that face identity perception mechanisms interact with processing of bodily and facial expressions. [ABSTRACT FROM AUTHOR]

We receive emotional signals from different sources, including the face, the whole body, and the natural scene. Previous research has shown the importance of context provided by the whole body and the scene on the recognition of facial expressions. This study measured physiological responses to face-body-scene combinations. Participants freely viewed emotionally congruent and incongruent face-body and body-scene pairs whilst eye fixations, pupil-size, and electromyography (EMG) responses were recorded. Participants attended more to angry and fearful vs. happy or neutral cues, independent of the source and relatively independent from whether the face body and body scene combinations were emotionally congruent or not. Moreover, angry faces combined with angry bodies and angry bodies viewed in aggressive social scenes elicited greatest pupil dilation. Participants' face expressions matched the valence of the stimuli but when face-body compounds were shown, the observed facial expression influenced EMG responses more than the posture. Together, our results show that the perception of emotional signals from faces, bodies and scenes depends on the natural context, but when threatening cues are presented, these threats attract attention, induce arousal, and evoke congruent facial reactions. [ABSTRACT FROM AUTHOR]

Abstract: Emotional influences on memory for events have long been documented yet surprisingly little is known about how emotional signals conveyed by contextual cues influence memory for face identity. This study investigated how positively and negatively valenced contextual emotion cues conveyed by body expressions or background scenes influence face memory. The results provide evidence of emotional context influence on face recognition memory and show that faces encoded in emotional (either fearful or happy) contexts (either the body or background scene) are less well recognized than faces encoded in neutral contexts and this effect is larger for body context than for scene context. The findings are compatible with the hypothesis that emotional signals in visual scenes trigger orienting responses which may lead to a less elaborate processing of featural details like the identity of a face, in turn resulting in a decreased facial recognition memory. [Copyright &y& Elsevier]

Most studies investigating emotion recognition in schizophrenia have focused on facial expressions and neglected bodily and vocal expressions. Furthermore, little is known about affective multisensory integration in schizophrenia. In the first experiment, the authors investigated recognition of static, face-blurred, whole-body expressions (instrumental, angry, fearful, and sad) with a two-alternative, forced-choice, simultaneous matching task in a sample of schizophrenia patients, nonschizophrenic psychotic patients, and matched controls. In the second experiment, dynamic, face-blurred, whole-body expressions (fearful and happy) were presented simultaneously with either congruent or incongruent human or animal vocalizations to schizophrenia patients and controls. Participants were instructed to categorize the emotion expressed by the body and to ignore the auditory information. The results of Experiment 1 show an emotion recognition impairment in the schizophrenia group and to a lesser extent in the nonschizophrenic psychosis group, and this for all four expressions. The findings of Experiment 2 show that schizophrenia patients are more influenced by the auditory information than controls, but only when the auditory information consists of human vocalizations. This shows that schizophrenia patients are impaired in recognizing whole-body expressions, and they show abnormal affective multisensory integration of bimodal stimuli originating from the same source. [ABSTRACT FROM AUTHOR]

Previous studies have shown that recognition of facial expressions is influenced by the affective information provided by the surrounding scene. The goal of this study was to investigate whether similar effects could be obtained for bodily expressions. Images of emotional body postures were briefly presented as part of social scenes showing either neutral or emotional group actions. In Experiment 1, fearful and happy bodies were presented in fearful, happy, neutral and scrambled contexts. In Experiment 2, we compared happy with angry body expressions. In Experiment 3 and 4, we blurred the facial expressions of all people in the scene. This way, we were able to ascribe possible scene effects to the presence of body expressions visible in the scene and we were able to measure the contribution of facial expressions to the body expression recognition. In all experiments, we observed an effect of social scene context. Bodily expressions were better recognized when the actions in the scenes expressed an emotion congruent with the bodily expression of the target figure. The specific influence of facial expressions in the scene was dependent on the emotional expression but did not necessarily increase the congruency effect. Taken together, the results show that the social context influences our recognition of a person’s bodily expression. [ABSTRACT FROM AUTHOR]