Search

Motion information has been argued to be central to the subjective segmentation of observed actions. Concerning object-directed actions, object-associated action information might as well inform efficient action segmentation and prediction. The present study compared the segmentation and neural processing of object manipulations and equivalent dough ball manipulations to elucidate the effect of object-action associations. Behavioral data corroborated that objective relational changes in the form of (un-)touchings of objects, hand, and ground represent meaningful anchor points in subjective action segmentation rendering them objective marks of meaningful event boundaries. As expected, segmentation behavior became even more systematic for the weakly informative dough. fMRI data were modeled by critical subjective, and computer-vision-derived objective event boundaries. Whole-brain as well as planned ROI analyses showed that object information had significant effects on how the brain processes these boundaries. This was especially pronounced at untouchings, that is, events that announced the beginning of the upcoming action and might be the point where competing predictions are aligned with perceptual input to update the current action model. As expected, weak object-action associations at untouching events were accompanied by increased biological motion processing, whereas strong object-action associations came with an increased contextual associative information processing, as indicated by increased parahippocampal activity. Interestingly, anterior inferior parietal lobule activity increased for weak object-action associations at untouching events, presumably because of an unrestricted number of candidate actions for dough manipulation. Our findings offer new insights into the significance of objects for the segmentation of action., (© 2024 Massachusetts Institute of Technology. Published under a Creative Commons Attribution 4.0 International (CC BY 4.0) license.)

How does the brain represent information about motion events in relation to agentive and physical forces? In this study, we investigated the neural activity patterns associated with observing animated actions of agents (e.g., an agent hitting a chair) in comparison to similar movements of inanimate objects that were either shaped solely by the physics of the scene (e.g., gravity causing an object to fall down a hill and hit a chair) or initiated by agents (e.g., a visible agent causing an object to hit a chair). Using an fMRI-based multivariate pattern analysis (MVPA), this design allowed testing where in the brain the neural activity patterns associated with motion events change as a function of, or are invariant to, agentive versus physical forces behind them. A total of 29 human participants (nine male) participated in the study. Cross-decoding revealed a shared neural representation of animate and inanimate motion events that is invariant to agentive or physical forces in regions spanning frontoparietal and posterior temporal cortices. In contrast, the right lateral occipitotemporal cortex showed a higher sensitivity to agentive events, while the left dorsal premotor cortex was more sensitive to information about inanimate object events that were solely shaped by the physics of the scene., (Copyright © 2024 the authors.)

Adaptive behavior such as social interaction requires our brain to predict unfolding external dynamics. While theories assume such dynamic prediction, empirical evidence is limited to static snapshots and indirect consequences of predictions. We present a dynamic extension to representational similarity analysis that uses temporally variable models to capture neural representations of unfolding events. We applied this approach to source-reconstructed magnetoencephalography (MEG) data of healthy human subjects and demonstrate both lagged and predictive neural representations of observed actions. Predictive representations exhibit a hierarchical pattern, such that high-level abstract stimulus features are predicted earlier in time, while low-level visual features are predicted closer in time to the actual sensory input. By quantifying the temporal forecast window of the brain, this approach allows investigating predictive processing of our dynamic world. It can be applied to other naturalistic stimuli (e.g., film, soundscapes, music, motor planning/execution, social interaction) and any biosignal with high temporal resolution., (© 2023. The Author(s).)

Observing others' actions recruits frontoparietal and posterior temporal brain regions - also called the action observation network. It is typically assumed that these regions support recognizing actions of animate entities (e.g., person jumping over a box). However, objects can also participate in events with rich meaning and structure (e.g., ball bouncing over a box). So far, it has not been clarified which brain regions encode information specific to goal-directed actions or more general information that also defines object events. Here, we show a shared neural code for visually presented actions and object events throughout the action observation network. We argue that this neural representation captures the structure and physics of events regardless of animacy. We find that lateral occipitotemporal cortex encodes information about events that is also invariant to stimulus modality. Our results shed light onto the representational profiles of posterior temporal and frontoparietal cortices, and their roles in encoding event information., (© 2023. The Author(s).)

Episodic memories are not static but can change on the basis of new experiences, potentially allowing us to make valid predictions in the face of an ever-changing environment. Recent research has identified prediction errors during memory retrieval as a possible trigger for such changes. In this study, we used modified episodic cues to investigate whether different types of mnemonic prediction errors modulate brain activity and subsequent memory performance. Participants encoded episodes that consisted of short toy stories. During a subsequent fMRI session, participants were presented videos showing the original episodes, or slightly modified versions thereof. In modified videos, either the order of two subsequent action steps was changed or an object was exchanged for another. Content modifications recruited parietal, temporo-occipital, and parahippocampal areas reflecting the processing of the new object information. In contrast, structure modifications elicited activation in right dorsal premotor, posterior temporal, and parietal areas, reflecting the processing of new sequence information. In a post-fMRI memory test, the participants' tendency to accept modified episodes as originally encoded increased significantly when they had been presented modified versions already during the fMRI session. After experiencing modifications, especially those of the episodes' structure, the recognition of originally encoded episodes was impaired as well. Our study sheds light onto the neural processing of different types of episodic prediction errors and their influence on subsequent memory recall., (© 2022 Massachusetts Institute of Technology. Published under a Creative Commons Attribution 4.0 International (CC BY 4.0) license.)

The ventral visual stream is conceived as a pathway for object recognition. However, we also recognize the actions an object can be involved in. Here, we show that action recognition critically depends on a pathway in lateral occipitotemporal cortex, partially overlapping and topographically aligned with object representations that are precursors for action recognition. By contrast, object features that are more relevant for object recognition, such as color and texture, are typically found in ventral occipitotemporal cortex. We argue that occipitotemporal cortex contains similarly organized lateral and ventral 'what' pathways for action and object recognition, respectively. This account explains a number of observed phenomena, such as the duplication of object domains and the specific representational profiles in lateral and ventral cortex., Competing Interests: Declaration of interests The authors declare no conflicts of interest., (Copyright © 2021 The Authors. Published by Elsevier Ltd.. All rights reserved.)

Intuitively, we assume that we remember episodes better when we actively participated in them and were not mere observers. Independently of this, we can recall episodes from either the first-person perspective (1pp) or the third-person perspective (3pp). In this functional magnetic resonance imaging (fMRI) study, we tested whether agency and perspective modulate neural activity during memory retrieval and subsequently enhance memory performance. Subjects encoded a set of different episodes by either imitating or only observing videos that showed short toy stories. A week later, we conducted fMRI and cued episodic retrieval by presenting the original videos, or slightly modified versions thereof, from 1pp or from 3pp. The hippocampal formation was sensitive to self-performed vs. only observed actions only when there was an episodic mismatch. In a post-fMRI memory test a history of self-performance did not improve behavioral memory performance. However, modified videos were often (falsely) accepted as showing truly experienced episodes when: (i) they were already presented in this modified version during fMRI or (ii) they were presented in their original form during fMRI but from 3pp. While the overall effect of modification was strong, the effects of perspective and agency were more subtle. Together, our findings demonstrate that self-performance and self-perspective modulate the strength of a memory trace in different ways. Even when memory performance remains the same for different agentive states, the brain is capable of detecting mismatching information. Re-experiencing the latter impairs memory performance as well as retrieving encoded episodes from 3pp., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2022 Jainta, Siestrup, El-Sourani, Trempler, Wurm, Werning, Cheng and Schubotz.)

Recognizing the actions of others depends on segmentation into meaningful events. After decades of research in this area, it remains still unclear how humans do this and which brain areas support underlying processes. Here we show that a computer vision-based model of touching and untouching events can predict human behavior in segmenting object manipulation actions with high accuracy. Using this computational model and functional Magnetic Resonance Imaging (fMRI), we pinpoint the neural networks underlying this segmentation behavior during an implicit action observation task. Segmentation was announced by a strong increase of visual activity at touching events followed by the engagement of frontal, hippocampal and insula regions, signaling updating expectation at subsequent untouching events. Brain activity and behavior show that touching-untouching motifs are critical features for identifying the key elements of actions including object manipulations., (Copyright © 2021. Published by Elsevier Inc.)

Extracting the number of objects in perceived scenes is a fundamental cognitive ability. Number processing is proposed to rely on two consecutive stages: an early object location map that captures individuated objects in a location-specific way and a subsequent location-invariant representation that captures numerosity at an abstract level. However, it is unclear whether this framework applies to small numerosities that can be individuated at once ("subitized"). Here, we reanalyzed data from two electroencephalography (EEG) experiments using multivariate pattern decoding to identify location-specific and location-invariant stages of numerosity processing in the subitizing range. In these experiments, one to three targets were presented in the left or right hemifield, which allowed for decoding target numerosity within each hemifield separately (location specific) or across hemifields (location invariant). Experiment 1 indicated the presence of a location-specific stage (180-200 ms after stimulus), followed by a location-invariant stage (300 ms after stimulus). A time-by-channel searchlight analysis revealed that the early location-specific stage is most evident at occipital channels, whereas the late location-invariant stage is most evident at parietal channels. Experiment 2 showed that both location-specific and location-invariant components are engaged only during tasks that explicitly require numerosity processing, ruling out automatic, and passive recording of numerosity. These results suggest that numerosity coding in subitizing is strongly grounded on an attention-based, location-specific stage. This stage overlaps with the subsequent activation of a location-invariant stage, where a full representation of numerosity is finalized. Taken together, our findings provide clear evidence for a temporal and spatial segregation of location-specific and location-invariant numerosity coding of small object numerosities., (© 2021 The Authors. European Journal of Neuroscience published by Federation of European Neuroscience Societies and John Wiley & Sons Ltd.)

While prediction errors have been established to instigate learning through model adaptation, recent studies have stressed the role of model-compliant events in predictive processing. Specifically, probabilistic information at critical points in time (so-called checkpoints) has been suggested to be sampled in order to evaluate the internal model, particularly in uncertain contexts. This way, initial model-based expectations are iteratively reaffirmed under uncertainty, even in the absence of prediction errors. Using electroencephalography (EEG), the present study aimed to investigate the interplay of such global uncertainty information and local adjustment cues prompting on-line adjustments of expectations. Within a stream of single digits, participants were to detect ordered sequences (i.e., 3-4-5-6-7) that had a regular length of five digits and were occasionally extended to seven digits. Over time, these extensions were either rare (low irreducible uncertainty) or frequent (high uncertainty) and could be unexpected or indicated by incidental colour cues. Accounting for cue information, an N400 component was revealed as the correlate of locally unexpected (vs expected) outcomes, reflecting effortful integration of incongruous information. As for model-compliant information, multivariate pattern decoding within the P3b time frame demonstrated effective exploitation of local (adjustment cues vs non-informative analogues) and global information (high vs low uncertainty regular endings) sampled from probabilistic events. Finally, superior fit of a global model (disregarding local adjustments) compared to a local model (including local adjustments) in a representational similarity analysis underscored the precedence of global reference frames in hierarchical predictive processing. Overall, results suggest that just like error-induced model adaptation, model evaluation is not limited to either local or global information. Following the hierarchical organisation of predictive processing, model evaluation too can occur at several levels of the processing hierarchy., Competing Interests: The authors have declared that no competing interests exist.

The processing of congruent stimuli, such as an object or action in its typical location, is usually associated with reduced neural activity, probably due to facilitated recognition. However, in some situations, congruency increases neural activity-for example, when objects next to observed actions are likely versus unlikely to be involved in forthcoming action steps. Here, we investigated using fMRI whether the processing of contextual cues during action perception is driven by their (in)congruency and, thus, informative value to make sense of an observed scene. Specifically, we tested whether both highly congruent contextual objects (COs), which strongly indicate a future action step, and highly incongruent COs, which require updating predictions about possible forthcoming action steps, provide more anticipatory information about the action course than moderately congruent COs. In line with our hypothesis that especially the inferior frontal gyrus (IFG) subserves the integration of the additional information into the predictive model of the action, we found highly congruent and incongruent COs to increase bilateral activity in action observation nodes, that is, the IFG, the occipitotemporal cortex, and the intraparietal sulcus. Intriguingly, BA 47 was significantly stronger engaged for incongruent COs reflecting the updating of prediction in response to conflicting information. Our findings imply that the IFG reflects the informative impact of COs on observed actions by using contextual information to supply and update the currently operating predictive model. In the case of an incongruent CO, this model has to be reconsidered and extended toward a new overarching action goal.

Object identification and enumeration rely on the ability to distinguish, or individuate, objects from the background. Does multiple object individuation operate only over bounded, separable objects or does it operate equally over connected features within a single object? While previous fMRI experiments suggest that connectedness affects the processing and enumeration of objects, recent behavioral and EEG studies demonstrated that parallel individuation occurs over both object parts and distinct objects. However, it is unclear whether individuation of object parts and distinct objects relies on a common or independent neural mechanisms. Using fMRI-based multivariate pattern analyses, we here demonstrate that activity patterns in inferior and superior intraparietal sulci (IPS) encode numerosity independently of whether the individuated items are connected parts of a single object or distinct objects. Lateral occipital cortex is more sensitive to perceptual aspects of the two stimulus types and the targets of the stimuli, suggesting a dissociation between ventral and dorsal areas in representing perceptual object properties and more general information about numerosity, respectively. Our results suggest that objecthood is not a necessary prerequisite for parallel individuation in IPS. Rather, our results point toward a common individuation mechanism that selects targets over a flexible object hierarchy, independently of whether the targets are distinct separable objects or parts of a single object., (Copyright © 2019 Elsevier Ltd. All rights reserved.)

Neuroimaging studies suggest that areas in the lateral occipitotemporal cortex (LOTC) play an important role in the perception of social actions. However, it is unclear what precisely about social actions these areas represent: perceptual features that may be indicative of social actions - such as the presence of persons in a scene, their orientation toward each other, and in particular the directedness of action movements toward persons or other targets - or more abstract representations that capture whether an action is meant to be social. In two fMRI experiments, we used representational similarity analysis (RSA) to test whether LOTC is sensitive to perceptual action components important for social interpretation and/or more general representations of sociality (Experiment 1) and implied person-directedness (Experiment 2). We found that LOTC is sensitive to perceptual action components (person presence, person orientation, and action directedness toward different types of recipients). By contrast, more general levels of sociality and implied person-directedness were not captured by LOTC. Our findings suggest that regions in LOTC provide the perceptual basis for social action interpretation but challenge accounts that posit specialization at more general levels sensitive to social actions and sociality as such. We propose that the interpretation of an action - in terms of sociality or other intentional aspects - arises from the interaction of multiple areas in processing relevant action components in a situation-dependent manner., (Copyright © 2019 Elsevier Inc. All rights reserved.)

Both temporal and frontoparietal brain areas are associated with the representation of knowledge about the world, in particular about actions. However, what these brain regions represent and precisely how they differ remains unknown. Here, we reveal distinct functional profiles of lateral temporal and frontoparietal cortex using fMRI-based MVPA. Frontoparietal areas encode representations of observed actions and corresponding written sentences in an overlapping way, but these representations do not generalize across stimulus type. By contrast, only left lateral posterior temporal cortex (LPTC) encodes action representations that generalize across observed action scenes and written descriptions. The representational organization of stimulus-general action information in LPTC can be predicted from models that describe basic agent-patient relations (object- and person-directedness) and the general semantic similarity between actions. Our results suggest that LPTC encodes general, conceptual aspects of actions whereas frontoparietal representations appear to be tied to specific stimulus types.

Action recognition involves not only the readout of body movements and involved objects but also the integration of contextual information, e.g. the environment in which an action takes place. Notably, inferring superordinate goals and generating predictions about forthcoming action steps should benefit from screening the actor's immediate environment, in particular objects located in the actor's peripersonal space and thus potentially used in following action steps. Critically, if such contextual objects (COs) afford actions that are semantically related to the observed action, they may trigger or facilitate the inference of goals and the prediction of following actions. This fMRI study investigated the neural mechanisms underlying the integration of COs in semantic and spatial relation to observed actions. Specifically, we tested the hypothesis that the inferior frontal gyrus (IFG) subserves this integration. Participants observed action videos in which COs and observed actions had common overarching goals or not (goal affinity) and varied in their location relative to the actor. High goal affinity increased bilateral activity in action observation network nodes, i.e. the occipitotemporal cortex and the intraparietal sulcus, but also in the precuneus and middle frontal gyri. This finding suggests that the semantic relation between COs and actions is considered during action observation and triggers (rather than facilitates) processes beyond those usually involved in action observation. Moreover, COs with high goal affinity located close to the actor's dominant hand additionally engaged bilateral IFG, corroborating the view that IFG is critically involved in the integration of action steps under a common overarching goal., (Copyright © 2017 Elsevier Inc. All rights reserved.)

Recognizing and understanding the actions of others is usually coupled with perceiving someone else's body movements from a third person perspective (3pp) whereas we perceive our own actions from a first person perspective (1pp). From a neural viewpoint, a recent finding is that perceiving actions from a 3pp as compared to a 1pp activates the temporoparietal junction, a brain region associated with visuospatial transformation and perspective taking but also with mental state inference and Theory of Mind (ToM). The present fMRI study characterizes the response profile of TPJ to elucidate its role in action observation. Participants observed naturalistic and pixelized object-directed actions from a 3pp and 1pp. Critically, in the pixelized condition the action goal could only be inferred from the movement kinematics. Both left and right TPJ revealed an interaction: Neural activity in TPJ was enhanced for 3pp vs. 1pp actions in the naturalistic but not pixelized condition. This finding contradicts theories proposing that TPJ is generally involved in transforming the action into the observer's perspective to match perceived body movements with visuomotor representations in the observer's motor system, which would be particularly required when actions can only be inferred from movement kinematics. Instead, our results support the theory that perceptual 3pp-selective cues trigger ToM-related processes such as detection of other agents and reasoning about an action's underlying mental states., (Copyright © 2017 Elsevier Inc. All rights reserved.)

How do humans recognize humans among other creatures? Recent studies suggest that a preference for conspecifics may emerge already in perceptual processing, in regions such as the right posterior superior temporal sulcus (pSTS), implicated in visual perception of biological motion. In the current functional MRI study, participants viewed point-light displays of human and nonhuman creatures moving in their typical bipedal (man and chicken) or quadrupedal mode (crawling-baby and cat). Stronger activity for man and chicken versus baby and cat was found in the right pSTS responsive to biological motion. The novel effect of pedalism suggests that, if right pSTS contributes to recognizing of conspecifics, it does so by detecting perceptual features (e.g. bipedal motion) that reliably correlate with their appearance. A searchlight multivariate pattern analysis could decode humans and nonhumans across pedalism in the left pSTS and bilateral posterior cingulate cortex. This result implies a categorical human-nonhuman distinction, independent from within-category physical/perceptual variation. Thus, recognizing conspecifics involves visual classification based on perceptual features that most frequently co-occur with humans, such as bipedalism, and retrieval of information that determines category membership above and beyond visual appearance. The current findings show that these processes are at work in separate brain networks.

Specific spatial environments are often indicative of where certain actions may take place: In kitchens we prepare food, and in bathrooms we engage in personal hygiene, but not vice versa. In action recognition, contextual cues may constrain an observer's expectations toward actions that are more strongly associated with a particular context than others. Such cues should become particularly helpful when the action itself is difficult to recognize. However, to date only easily identifiable actions were investigated, and the effects of context on recognition were rather interfering than facilitatory. To test whether context also facilitates action recognition, we measured recognition performance of hardly identifiable actions that took place in compatible, incompatible, and neutral contextual settings. Action information was degraded by pixelizing the area of the object manipulation while the room in which the action took place remained fully visible. We found significantly higher accuracy for actions that took place in compatible compared to incompatible and neutral settings, indicating facilitation. Additionally, action recognition was slower in incompatible settings than in compatible and neutral settings, indicating interference. Together, our findings demonstrate that contextual information is effectively exploited during action observation, in particular when visual information about the action itself is sparse. Differential effects on speed and accuracy suggest that contexts modulate action recognition at different levels of processing. Our findings emphasize the importance of contextual information in comprehensive, ecologically valid models of action recognition.

Actions are recognized faster and with higher accuracy when they take place in their typical environments. It is unclear, however, when contextual cues from the environment become effectively exploited during childhood and whether contextual integration interacts with other factors such as children's perceptual or motor experience with an action. In the present experiment, we asked 4- to 8-year-olds (n = 159) to recognize pantomime actions that took place in compatible, incompatible, or neutral contextual settings. In each age cohort, children recognized more actions taking place in compatible compared to incompatible and neutral contexts. This result demonstrates robust facilitation effects of context on action recognition independent of age. Additionally, we found an interaction of context effects with action familiarity: Context effects were strongest when the children were less familiar with the actions, suggesting that contextual settings are particularly beneficial for action recognition when experience with an action is sparse. (PsycINFO Database Record, ((c) 2017 APA, all rights reserved).)

How neural specificity for distinct conceptual knowledge categories arises is central for understanding the organization of semantic memory in the human brain. Although there is a large body of research on the neural processing of distinct object categories, the organization of action categories remains largely unknown. In particular, it is unknown whether different action categories follow a specific topographical organization on the cortical surface analogously to the category-specific organization of object knowledge. Here, we tested whether the neural representation of action knowledge is organized in terms of nonsocial versus social and object-unrelated versus object-related actions (sociality and transitivity, respectively, hereafter). We hypothesized a major distinction of sociality and transitivity along dorsal and ventral lateral occipitotemporal cortex (LOTC), respectively. Using fMRI-based multivoxel pattern analysis, we identified neural representations of action information associated with sociality and transitivity in bilateral LOTC. Representational similarity analysis revealed a dissociation between dorsal and ventral LOTC. We found that action representations in dorsal LOTC are segregated along features of sociality, whereas action representations in ventral LOTC are segregated along features of transitivity. In addition, representations of sociality and transitivity features were found more anteriorly in LOTC than representations of specific subtypes of actions, suggesting a posterior-anterior gradient from concrete to abstract action features. These findings elucidate how the neural representations of perceptually and conceptually diverse actions are organized in distinct subsystems in the LOTC., Significance Statement: The lateral occipitotemporal cortex (LOTC) is critically involved in the recognition of objects and actions, but our knowledge about the underlying organizing principles is limited. Here, we discovered a dorsal-ventral distinction of actions in LOTC: dorsal LOTC represents actions based on sociality (how much an action is directed to another person) in proximity to person knowledge. In contrast, ventral LOTC represents actions based on transitivity (how much an action involves the interaction with inanimate objects) in proximity to tools/artifacts in ventral LOTC, suggesting a mutually dependent organization of actions and objects. In addition, we found a posterior-to-anterior organization of the LOTC for concrete and abstract representations, respectively. Our findings provide important insights about the organization of actions in LOTC., (Copyright © 2017 the authors 0270-6474/17/370562-14$15.00/0.)

Action understanding requires a many-to-one mapping of perceived input onto abstract representations that generalize across concrete features. It is debated whether such abstract action concepts are encoded in ventral premotor cortex (PMv; motor hypothesis) or, alternatively, are represented in lateral occipitotemporal cortex (LOTC; cognitive hypothesis). We used fMRI-based multivoxel pattern analysis to decode observed actions at concrete and abstract, object-independent levels of representation. Participants observed videos of 2 actions involving 2 different objects, using either an explicit or implicit task with respect to conceptual action processing. We decoded concrete action representations by training and testing a classifier to discriminate between actions within each object category. To identify abstract action representations, we trained the classifier to discriminate actions in one object and tested the classifier on actions performed on the other object, and vice versa. Region-of-interest and searchlight analyses revealed decoding in LOTC at both concrete and abstract levels during both tasks, whereas decoding in PMv was restricted to the concrete level during the explicit task. In right inferior parietal cortex, decoding was significant for the abstract level during the explicit task. Our findings are incompatible with the motor hypothesis, but support the cognitive hypothesis of action understanding., (© The Author 2015. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.)

During movement planning, brain activity within parietofrontal networks encodes information about upcoming actions that can be driven either externally (e.g., by a sensory cue) or internally (i.e., by a choice/decision). Here we used multivariate pattern analysis (MVPA) of fMRI data to distinguish between areas that represent (1) abstract movement plans that generalize across the way in which these were driven, (2) internally driven movement plans, or (3) externally driven movement plans. In a delayed-movement paradigm, human volunteers were asked to plan and execute three types of nonvisually guided right-handed reaching movements toward a central target object: using a precision grip, a power grip, or touching the object without hand preshaping. On separate blocks of trials, movements were either instructed via color cues (Instructed condition), or chosen by the participant (Free-Choice condition). Using ROI-based and whole-brain searchlight-based MVPA, we found abstract representations of planned movements that generalize across the way these movements are selected (internally vs externally driven) in parietal cortex, dorsal premotor cortex, and primary motor cortex contralateral to the acting hand. In addition, we revealed representations specific for internally driven movement plans in contralateral ventral premotor cortex, dorsolateral prefrontal cortex, supramarginal gyrus, and in ipsilateral posterior parietotemporal regions, suggesting that these regions are recruited during movement selection. Finally, we observed representations of externally driven movement plans in bilateral supplementary motor cortex and a similar trend in presupplementary motor cortex, suggesting a role in stimulus-response mapping., Significance Statement: The way the human brain prepares the body for action constitutes an essential part of our ability to interact with our environment. Previous studies demonstrated that patterns of neuronal activity encode upcoming movements. Here we used multivariate pattern analysis of human fMRI data to distinguish between brain regions containing movement plans for instructed (externally driven) movements, areas involved in movement selection (internally driven), and areas containing abstract movement plans that are invariant to the way these were generated (i.e., that generalize across externally and internally driven movement plans). Our findings extend our understanding of the neural basis of movement planning and have the potential to contribute to the development of brain-controlled neural prosthetic devices., (Copyright © 2015 the authors 0270-6474/15/3514160-12$15.00/0.)

Actions performed by others are mostly not observed in isolation, but embedded in sequences of actions tied together by an overarching goal. Therefore, preceding actions can modulate the observer's expectations in relation to the currently perceived action. Ventrolateral prefrontal cortex (vlPFC), and inferior frontal gyrus (IFG) in particular, is suggested to subserve the integration of episodic as well as semantic information and memory, including action scripts. The present fMRI study investigated if activation in IFG varies with the effort to integrate expected and observed action, even when not required by the task. During an fMRI session, participants were instructed to attend to short videos of single actions and to deliver a judgment about the actor's current goal. We manipulated the strength of goal expectation induced by the preceding action, implementing the parameter "goal-relatedness" between the preceding and the currently observed action. Moreover, since objects point to the probability of certain actions, we also manipulated whether the current and the preceding action shared at least one object or not. We found an interaction between the two factors goal-relatedness and shared object: IFG activation increased the weaker the goal-relatedness between the preceding and the current action was, but only when they shared at least one object. Here, integration of successive action steps was triggered by the re-appearing (shared) object but hampered by a weak goal-relatedness between the actually observed manipulation. These findings foster the recently emerging view that IFG is enhanced by goal-related conflicts during action observation.

Brain regions that mediate action understanding must contain representations that are action specific and at the same time tolerate a wide range of perceptual variance. Whereas progress has been made in understanding such generalization mechanisms in the object domain, the neural mechanisms to conceptualize actions remain unknown. In particular, there is ongoing dissent between motor-centric and cognitive accounts whether premotor cortex or brain regions in closer relation to perceptual systems, i.e., lateral occipitotemporal cortex, contain neural populations with such mapping properties. To date, it is unclear to which degree action-specific representations in these brain regions generalize from concrete action instantiations to abstract action concepts. However, such information would be crucial to differentiate between motor and cognitive theories. Using ROI-based and searchlight-based fMRI multivoxel pattern decoding, we sought brain regions in human cortex that manage the balancing act between specificity and generality. We investigated a concrete level that distinguishes actions based on perceptual features (e.g., opening vs closing a specific bottle), an intermediate level that generalizes across movement kinematics and specific objects involved in the action (e.g., opening different bottles with cork or screw cap), and an abstract level that additionally generalizes across object category (e.g., opening bottles or boxes). We demonstrate that the inferior parietal and occipitotemporal cortex code actions at abstract levels whereas the premotor cortex codes actions at the concrete level only. Hence, occipitotemporal, but not premotor, regions fulfill the necessary criteria for action understanding. This result is compatible with cognitive theories but strongly undermines motor theories of action understanding., (Copyright © 2015 the authors 0270-6474/15/357727-09$15.00/0.)

Actions are usually made of several action steps gearing towards an overarching goal. During observation of such action episodes the overarching action goal becomes more and more clear and upcoming action steps can be predicted with increasing precision. To tap this process, the present fMRI study investigated the dynamic changes of neural activity during the observation of distinct action steps that cohere by an overarching goal. Our hypotheses specifically addressed the role of the inferior frontal gyrus (IFG), a region assumed to be a key hub for integration functions during action processing, as well as the role of regions involved in action perception (often referred to as action observation network or AON) that should benefit from the predictability of forthcoming action steps. Participants watched separate action steps that formed a coherent action goal or not (factor goal coherence) and were performed by a single actor or not (factor actor coherence). Independent of actor coherence, neural activity in IFG and occipitotemporal cortex decreased as a function of goal predictability during the unfolding of goal-coherent episodes. In addition, we identified a network (precuneus, dorsolateral prefrontal and orbitofrontal cortex, angular gyrus, and middle temporal gyrus) that showed increased activity for goal coherence. We conclude that IFG fosters the integration of action steps to build overarching goals. Identifying the unifying goal of an action episode allows anticipation, and thus efficient processing, of forthcoming action steps. To this end, past action steps of the action episode are buffered and recollected with recourse to episodic memory., (Copyright © 2014 Elsevier B.V. All rights reserved.)

Objects are reminiscent of actions often performed with them:knife and apple remind us on peeling the apple or cutting it. Mnemonic representations of object-related actions (action codes) evoked by the sight of an object may constrain and hence facilitate recognition of unrolling actions. The present fMRI study investigated if and how action codes influence brain activation during action observation. The average number of action codes (NAC) of 51 sets of objects was rated by a group of n = 24 participants. In an fMRI study, different volunteers were asked to recognize actions performed with the same objects presented in short videos. To disentangle areas reflecting the storage of action codes from those exploiting them, we showed object-compatible and object-incompatible (pantomime) actions. Areas storing action codes were considered to positively co-vary with NAC in both object-compatible and object-incompatible action; due to its role in tool-related tasks, we here hypothesized left anterior inferior parietal cortex (aIPL). In contrast, areas exploiting action codes were expected to show this correlation only in object-compatible but not incompatible action, as only object-compatible actions match one of the active action codes. For this interaction, we hypothesized ventrolateral premotor cortex (PMv) to join aIPL due to its role in biasing competition in IPL. We found left anterior intraparietal sulcus (IPS) and left posterior middle temporal gyrus (pMTG) to co-vary with NAC. In addition to these areas, action codes increased activity in object-compatible action in bilateral PMv, right IPS, and lateral occipital cortex (LO). Findings suggest that during action observation, the brain derives possible actions from perceived objects, and uses this information to shape action recognition. In particular, the number of expectable actions quantifies the activity level at PMv, IPL, and pMTG, but only PMv reflects their biased competition while observed action unfolds.

Actions observed in everyday life normally consist of one person performing sequences of goal-directed actions. The present fMRI study tested the hypotheses that observers are influenced by the actor's identity, even when this information is task-irrelevant, and that this information shapes their expectation on subsequent actions of the same actor. Participants watched short video clips of action steps that either pertained to a common action with an overarching goal or not, and were performed by either one or by varying actors (2 × 2 design). Independent of goal coherence, actor coherence elicited activation in dorsolateral and ventromedial frontal cortex, together pointing to a spontaneous attempt to integrate all actions performed by one actor. Interestingly, watching an actor performing unrelated actions elicited additional activation in left inferior frontal gyrus, suggesting a search in semantic memory in an attempt to construct an overarching goal that can reconcile the disparate action steps with a coherent intention. Post-experimental surveys indicate that these processes occur mostly unconsciously. Findings strongly suggest a spontaneous expectation bias toward actor-related episodes in action observers, and hence to the immense impact of actor information on action observation., (Copyright © 2013 Wiley Periodicals, Inc.)

To recognize an action, an observer exploits information about the applied manipulation, the involved objects, and the context where the action occurs. Context, object, and manipulation information are hence expected to be tightly coupled in a triadic relationship (the COM triad hereafter). The current fMRI study investigated the hemodynamic signatures of reciprocal modulation in the COM triad. Participants watched short video clips of pantomime actions, that is, actions performed with inappropriate objects, taking place at compatible or incompatible contexts. The usage of pantomime actions enabled the disentanglement of the neural substrates of context-manipulation (CM) and context-object (CO) associations. There were trials in which (1) both manipulation and objects, (2) only manipulation, (3) only objects, or (4) neither manipulation nor objects were compatible with the context. CM compatibility effects were found in an action-related network comprising ventral premotor cortex, SMA, left anterior intraparietal sulcus, and bilateral occipito-temporal cortex. Conversely, CO compatibility effects were found bilaterally in lateral occipital complex. These effects interacted in subregions of the lateral occipital complex. An overlap of CM and CO effects was observed in the occipito-temporal cortex and the dorsal attention network, that is, superior frontal sulcus/dorsal premotor cortex and superior parietal lobe. Results indicate that contextual information is integrated into the analysis of actions. Manipulation and object information is linked by contextual associations as a function of co-occurrence in specific contexts. Activation of either CM or CO associations shifts attention to either action- or object-related relevant information.

Most every day actions take place in domestic rooms that are specific for certain classes of actions. Contextual information derived from domestic settings may therefore influence the efficiency of action recognition. The present studies investigated whether action recognition is modulated by compatibility of the context an action is embedded in. To this end, subjects watched video clips of actions performed in compatible, incompatible, and neutral contexts. Recognition was significantly slower when actions took place in an incompatible as compared to a compatible or a neutral context (Experiment 1). Functional MRI revealed increased activation for incompatible context in Brodmann Areas (BA) 44, 45, and 47 of the left ventrolateral prefrontal cortex (vlPFC; Experiment 2). Results suggest that contextual information - even when task-irrelevant - informs a high processing level of action analysis. In particular, the functional profiles assigned to these prefrontal regions suggest that contextual information activates associated action representations as a function of (in-)compatibility. Thus, incompatibility effects may reflect the attempt to resolve the conflict between action and context by embedding the presented action step into an overarching action that is again compatible with the provided context., (Copyright © 2011 Elsevier Inc. All rights reserved.)

Influential concepts in neuroscientific research cast the brain a predictive machine that revises its predictions when they are violated by sensory input. This relates to the predictive coding account of perception, but also to learning. Learning from prediction errors has been suggested for take place in the hippocampal memory system as well as in the basal ganglia. The present fMRI study used an action-observation paradigm to investigate the contributions of the hippocampus, caudate nucleus and midbrain dopaminergic system to different types of learning: learning in the absence of prediction errors, learning from prediction errors, and responding to the accumulation of prediction errors in unpredictable stimulus configurations. We conducted analyses of the regions of interests' BOLD response towards these different types of learning, implementing a bootstrapping procedure to correct for false positives. We found both, caudate nucleus and the hippocampus to be activated by perceptual prediction errors. The hippocampal responses seemed to relate to the associative mismatch between a stored representation and current sensory input. Moreover, its response was significantly influenced by the average information, or Shannon entropy of the stimulus material. In accordance with earlier results, the habenula was activated by perceptual prediction errors. Lastly, we found that the substantia nigra was activated by the novelty of sensory input. In sum, we established that the midbrain dopaminergic system, the hippocampus, and the caudate nucleus were to different degrees significantly involved in the three different types of learning: acquisition of new information, learning from prediction errors and responding to unpredictable stimulus developments. We relate learning from perceptual prediction errors to the concept of predictive coding and related information theoretic accounts.

Action observation engages higher motor areas, possibly reflecting an internal simulation. However, actions considered odd or unusual were found to trigger additional activity in the so-called theory of mind (ToM) network, pointing to deliberations on the actor's mental states. In this functional magnetic resonance imaging study, the hypothesis was tested that an allocentric perspective on a normal action, and even more so the sight of the actor's face, suffices to evoke ToM activity. Subjects observed short videos of object manipulation filmed from either the egocentric or the allocentric perspective, the latter including the actor's face in half of the trials. On the basis of a regions of interest analysis using ToM coordinates, we found increased neural activity in several regions of the ToM network. First, perceiving actions from an allocentric compared with the egocentric perspective enhanced activity in the left temporoparietal junction (TPJ). Second, the presence of the actor's face enhanced activation in the TPJ bilaterally, the medial prefrontal cortex (mPFC) and posterior cingulate cortex (PCC). Finally, the mPFC and PCC showed increased responses when the actor changed with respect to the preceding trial. These findings were further corroborated by zmap findings for the latter two contrasts. Together, findings indicate that observation of normal everyday actions can engage ToM areas and that an allocentric perspective, seeing the actor's face and seeing a face switch, are effective triggers., (Copyright © 2011 Wiley Periodicals, Inc.)