Search

Video location prediction for handwritten digits presents unique challenges in computer vision due to the complex spatiotemporal dependencies and the need to maintain digit legibility across predicted frames, while existing deep learning-based video prediction models have shown promise, they often struggle with preserving local details and typically achieve clear predictions for only a limited number of frames. In this paper, we present a novel video location prediction model based on Convolutional Gated Recurrent Units (ConvGRU) that specifically addresses these challenges in the context of handwritten digit sequences. Our approach introduces three key innovations. Firstly, we introduce a specialized decoupling model using modified Generative Adversarial Networks (GANs) that effectively separates background and foreground information, significantly improving prediction accuracy. Secondly, we introduce an enhanced ConvGRU architecture that replaces traditional linear operations with convolutional operations in the gating mechanism, substantially reducing spatiotemporal information loss. Finally, we introduce an optimized parameter-tuning strategy that ensures continuous feature transmission while maintaining computational efficiency. Extensive experiments on both the MNIST dataset and custom mobile datasets demonstrate the effectiveness of our approach. Our model achieves a structural similarity index of 0.913 between predicted and actual sequences, surpassing current state-of-the-art methods by 1.2%. Furthermore, we demonstrate superior performance in long-term prediction stability, with consistent accuracy maintained across extended sequences. Notably, our model reduces training time by 9.5% compared to existing approaches while maintaining higher prediction accuracy. These results establish new benchmarks for handwritten digit video prediction and provide practical solutions for real-world applications in digital education, document processing, and real-time handwriting recognition systems. [ABSTRACT FROM AUTHOR]

Robot pick-and-place for unknown objects is still a very challenging research topic. This paper proposes a multi-modal learning method for robot one-shot imitation of pick-and-place tasks. This method aims to enhance the generality of industrial robots while reducing the amount of data and training costs the one-shot imitation method relies on. The method first categorizes human demonstration videos into different tasks, and these tasks are classified into six types to symbolize as many types of pick-and-place tasks as possible. Second, the method generates multi-modal prompts and finally predicts the action of the robot and completes the symbolic pick-and-place task in industrial production. A carefully curated dataset is created to complement the method. The dataset consists of human demonstration videos and instance images focused on real-world scenes and industrial tasks, which fosters adaptable and efficient learning. Experimental results demonstrate favorable success rates and loss results both in simulation environments and real-world experiments, confirming its effectiveness and practicality. [ABSTRACT FROM AUTHOR]

Most of the earlier proposed action prediction methods, such as PoseGAN, C2GAN, etc. are limited to unidirectional prediction and not intended to perform the desired class of action. In this paper, we propose an action-guided Cycle Generative Adversarial Network (AGCGAN), which is a bi-directional video prediction model to anticipate future frames from current visual frames and vice versa. The proposed CycleGAN architecture consists of two generators and two discriminators along with two additional keypoint detectors to make keypoint loss measurements. The adversarial and cycle losses perform the appearance modelling whereas the keypoint loss provides the necessary motion correction. We evaluated the effectiveness of our proposed method on the UT-Interaction dataset. Furthermore, we also tested our model on the standard Market-1501 dataset against the state-of-the-art schemes. Results show that our proposed method provides comparable visual quality with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Accumulating evidence suggests that individuals with autism spectrum disorder (ASD) show impairments in using contextual priors to predict others' actions and make intention inference. Yet less is known about whether and how children with ASD acquire contextual priors during action observation and how contextual priors relate to their action prediction and intention inference. To form proper contextual priors, individuals need to observe the social scenes in a reliable manner and focus on socially relevant information. By employing a data‐driven scan path method and areas of interest (AOI)‐based analysis, the current study investigated how contextual priors would relate to action prediction and intention understanding in 4‐to‐9‐year‐old children with ASD (N = 56) and typically developing (TD) children (N = 50) during free viewing of dynamic social scenes with different intentions. Results showed that children with ASD exhibited higher intra‐subject variability when scanning social scenes and reduced attention to socially relevant areas. Moreover, children with high‐level action prediction and intention understanding showed lower intra‐subject variability and increased attention to socially relevant areas. These findings suggest that altered fixation patterns might restrain children with ASD from acquiring proper contextual priors, which has cascading downstream effects on their action prediction and intention understanding. Lay Summary: Skilled social interactions require individuals to infer others' intentions and predict others' actions based on prior experiences, which is impaired in children with ASD. The present study investigated whether and how children with ASD acquire prior knowledge and how their prior knowledge would influence action prediction and intention understanding during viewing dynamic social scenes. We found that children with ASD viewed social scenes in an unpredictable way and showed decreased attention to socially relevant areas as compared with typically developing children. Moreover, children with high‐level action prediction and intention understanding observed the social scenes in a more predictable way and exhibited increased attention to socially relevant areas. These findings suggest that children with ASD may acquire unreliable prior knowledge, which has adverse effects on their action prediction and intention understanding. [ABSTRACT FROM AUTHOR]

Video location prediction for handwritten digits presents unique challenges in computer vision due to the complex spatiotemporal dependencies and the need to maintain digit legibility across predicted frames, while existing deep learning-based video prediction models have shown promise, they often struggle with preserving local details and typically achieve clear predictions for only a limited number of frames. In this paper, we present a novel video location prediction model based on Convolutional Gated Recurrent Units (ConvGRU) that specifically addresses these challenges in the context of handwritten digit sequences. Our approach introduces three key innovations. Firstly, we introduce a specialized decoupling model using modified Generative Adversarial Networks (GANs) that effectively separates background and foreground information, significantly improving prediction accuracy. Secondly, we introduce an enhanced ConvGRU architecture that replaces traditional linear operations with convolutional operations in the gating mechanism, substantially reducing spatiotemporal information loss. Finally, we introduce an optimized parameter-tuning strategy that ensures continuous feature transmission while maintaining computational efficiency. Extensive experiments on both the MNIST dataset and custom mobile datasets demonstrate the effectiveness of our approach. Our model achieves a structural similarity index of 0.913 between predicted and actual sequences, surpassing current state-of-the-art methods by 1.2%. Furthermore, we demonstrate superior performance in long-term prediction stability, with consistent accuracy maintained across extended sequences. Notably, our model reduces training time by 9.5% compared to existing approaches while maintaining higher prediction accuracy. These results establish new benchmarks for handwritten digit video prediction and provide practical solutions for real-world applications in digital education, document processing, and real-time handwriting recognition systems.

Developmental science has experienced a vivid debate on whether young children prioritize goals over means in their prediction of others' actions. Influential developmental theories highlight the role of goal objects for action understanding. Yet, recent infant studies report evidence for the opposite. The empirical evidence is therefore inconclusive. The current study advanced this debate by assessing preschool children's verbal predictions of others' actions. In five experiments (N = 302), we investigated whether preschool children and adults predict agents to move towards their previous goal (that is, show goal‐related predictions) or predict agents to move along the same movement path that they pursued before. While Experiments 1a, 1b and 1c presented young children and adults with animated agents, Experiments 2a and 2b presented participants with human grasping action. An integrative analysis across experiments revealed that children were more likely to predict the agent to move along the same movement path, Z = ‐4.574, p ≤ 0.0001 (r = 0.304). That is, preschool children were more likely to predict that agents would move along the same trajectory even though this action would lead to a new goal object. Thus, our findings suggest that young children's action prediction relies on the detection of spatial and movement information. Overall, we discuss our findings in terms of theoretical frameworks that conceive of action understanding as an umbrella term that comprises different forms and facets in which humans understand others' actions. Research Highlights: We investigated whether preschool children predict agents to move towards their previous goal or to move along the same movement path that they pursued before.Unlike adults, preschool children predicted that agents would move along the same trajectory even though this action would lead to a new goal.Adults' goal‐based predictions were affected from contextual details, whereas children systematically made path‐based predictions.Young children's action prediction relies on the detection of spatial and movement information. [ABSTRACT FROM AUTHOR]

Humans' capacity to predict actions and to socially categorize individuals is at the basis of social cognition. Such capacities emerge in early infancy. By 6months of age, infants predict others' reaching actions considering others' epistemic state. At a similar age, infants are biased to attend to and interact with more familiar individuals, considering adult-like social categories such as the language people speak. We report that these two core processes are interrelated early on in infancy. In a belief-based action prediction task, 6-month-old infants (males and females) presented with a native speaker generated online predictions about the agent's actions, as revealed by the activation of participants' sensorimotor areas before the agent's movement. However, infants who were presented with a foreign speaker did not recruit their motor system before the agent's action. The eyetracker analysis provided further evidence that linguistic group familiarity influences how infants predict others' actions, as only infants presented with a native speaker modified their attention to the stimuli as a function of the agent's forthcoming behavior. The current findings suggest that infants' emerging capacity to predict others' actions is modulated by social cues, such as others' linguistic group. A facilitation to predict and encode the actions of native speakers relative to foreign speakers may explain, in part, why infants preferentially attend to, imitate, and learn from the actions of native speakers. [ABSTRACT FROM AUTHOR]

In this paper, we propose a frame-part-activated deep reinforcement learning (FPA-DRL) for action prediction. Most existing methods for action prediction utilize the evolution of whole frames to model actions, which cannot avoid the noise of the current action, especially in the early prediction. Moreover, the loss of structural information of human body diminishes the capacity of features to describe actions. To address this, we design a FPA-DRL to exploit the structure of the human body by extracting skeleton proposals and reduce the redundancy of frames under a deep reinforcement learning framework. Specifically, we extract features from different parts of the human body individually, activate the action-related parts in features and the action-related frames in videos to enhance the representation. Our method not only exploits the structure information of the human body, but also considers the attention frame for expressing actions. We evaluate our method on three popular action prediction datasets: UT-Interaction, BIT-Interaction and UCF101. Our experimental results demonstrate that our method achieves the very competitive performance with state-of-the-arts. • We design the part-activated module to enhance the action-related parts of features. • We design the frame-activated module to reduce the redundancy of frames. • We achieved very competitive results of state-of-the-arts on three datasets. [ABSTRACT FROM AUTHOR]

In ball sports, such as volleyball or soccer, the impact sound of the ball is used to define the shot length and predict the final ball position on the field. Several studies thoroughly investigated the role of sound intensity in predicting the ball's landing point, neglecting the equally important contribution of the sound frequency spectrum component. Here, we investigated the contribution of the sound frequency spectrum in defining a volleyball serve's length (i.e., distance). We recorded the sound of short and long volleyball serves (ecological sound) and changed their frequency spectrum in pink noise, preserving their intensity (noise sound). We then presented both sounds (ecological and noise) to expert volleyball players and measured their performance in short and long-shot classifications. The role of auditory-motor experience in sound classification was also explored by testing soccer players (experience on ball shots but not sport-specific auditory-motor experience) and non-athletes (no auditory-motor experience in sports). We found that hearing ecological sound leads to better sound discrimination performance than noise sound. Concurrently, we found an effect of the specific auditory-motor experience, with volleyball players outperforming non-athletes and soccer players. Interestingly, experts in different sports domains (soccer players) mainly used sound intensity for shot classification. This underlines the importance of specific auditory-motor experience in encoding specific sound frequency spectrum for the sound of action classification. [ABSTRACT FROM AUTHOR]

In the fast-changing video surveillance area, there exists a critical need for new ways in which the huge and complex data availed from CCTV systems can be correctly analyzed. Most already-existing methods of person detection and video analysis are not accurate, particularly in high dynamic scenes like crowded city centers, hospitals, and industries. These methods generally have a host of problems: poor adaptability to occlusions and lighting changes, and low precision for continuous action prediction. This paper provides an integrated model system for CCTV video analysis in a way that basically improves person detection, action prediction, and video synthesis. Concretely, this integrates some of the state-of-the-art methods: Cascade R-CNN for person detection, Dynamic Mask Graph Network(DMGN) for semantic segmentation, Hierarchical Deep Dyna-Q Network(HDDQN) for action prediction, Multimodal Attention Fusion Network(MAFN) with fuzzy logic for attention refinement, and Generative Transformer-based Video Synthesis Model(GT-VSM) for video generation. Attention mechanisms enabled in Cascade R-CNN can avoid the common pitfalls among occlusions and scale variations effectively for performing better in mAP scores. Bringing in temporal consistency in DMGN raises the accuracy in segmentation. In addition, HDDQN will predict complex actions accurately. MAFN fuses the multimodal data for enhancing context awareness. GT-VSM synthesizes video sequences of high fidelity. The system, though proposed, has several key improvements over existing models in terms of better detection accuracy, action prediction, and quality of video synthesis, which are all empirically validated in real-world surveillance scenarios. This work allows chances to improve safety, security, and behavior analysis across very different domains by enabling a real-time video surveillance tool that is based on scalability, efficiency, and accuracy levels.

Previous studies have shown that contextual information may aid in guessing the intention underlying others' actions in conditions of perceptual ambiguity. Here, we aimed to evaluate the temporal deployment of contextual influence on action prediction with increasing availability of kinematic information during the observation of ongoing actions. We used action videos depicting an actor grasping an object placed on a container to perform individual or interpersonal actions featuring different kinematic profiles. Crucially, the container could be of different colors. First, in a familiarization phase, the probability of co-occurrence between each action kinematics and color cues was implicitly manipulated to 80% and 20%, thus generating contextual priors. Then, in a testing phase, participants were asked to predict action outcome when the same action videos were occluded at five different timeframes of the entire movement, ranging from when the actor was still to when the grasp of the object was fully accomplished. In this phase, all possible action–contextual cues' associations were equally presented. The results showed that for all occlusion intervals, action prediction was more facilitated when action kinematics deployed in high- than low-probability contextual scenarios. Importantly, contextual priors shaped action prediction even in the latest occlusion intervals, where the kinematic cues clearly unveiled an action outcome that was previously associated with low-probability scenarios. These residual contextual effects were stronger in individuals with higher subclinical autistic traits. Our findings highlight the relative contribution of kinematic and contextual information to action understanding and provide evidence in favor of their continuous integration during action observation. [ABSTRACT FROM AUTHOR]

In recent years, with the rapid advancements in Natural Language Processing (NLP) technologies, large models have become widespread. Traditional reinforcement learning algorithms have also started experimenting with language models to optimize training. However, they still fundamentally rely on the Markov Decision Process (MDP) for reinforcement learning, and do not fully exploit the advantages of language models for dealing with long sequences of problems. The Decision Transformer (DT) introduced in 2021 is the initial effort to completely transform the reinforcement learning problem into a challenge within the NLP domain. It attempts to use text generation techniques to create reinforcement learning trajectories, addressing the issue of finding optimal trajectories. However, the article places the training trajectory data of reinforcement learning directly into a basic language model for training. Its aim is to predict the entire trajectory, encompassing state and reward information. This approach deviates from the reinforcement learning training objective of finding the optimal action. Furthermore, it generates redundant information in the output, impacting the final training effectiveness of the agent. This paper proposes a more reasonable network model structure, the Action-Translator Transformer (ATT), to predict only the next action of the agent. This makes the language model more interpretable for the reinforcement learning problem. We test our model in simulated gaming scenarios and compare it with current mainstream methods in the offline reinforcement learning field. Based on the presented experimental results, our model demonstrates superior performance. We hope that introducing this model will inspire new ideas and solutions for combining language models and reinforcement learning, providing fresh perspectives for offline reinforcement learning research. [ABSTRACT FROM AUTHOR]

Early action prediction (EAP) aims to recognize human actions from a part of action execution in ongoing videos, which is an important task for many practical applications. Most prior works treat partial or full videos as a whole, ignoring rich action knowledge hidden in videos, i.e., semantic consistencies among different partial videos. In contrast, we partition original partial or full videos to form a new series of partial videos and mine the Action-Semantic Consistent Knowledge (ASCK) among these new partial videos evolving in arbitrary progress levels. Moreover, a novel Rich Action-semantic Consistent Knowledge network (RACK) under the teacher-student framework is proposed for EAP. Firstly, we use a two-stream pre-trained model to extract features of videos. Secondly, we treat the RGB or flow features of the partial videos as nodes and their action semantic consistencies as edges. Next, we build a bi-directional semantic graph for the teacher network and a single-directional semantic graph for the student network to model rich ASCK among partial videos. The MSE and MMD losses are incorporated as our distillation loss to enrich the ASCK of partial videos from the teacher to the student network. Finally, we obtain the final prediction by summering the logits of different subnetworks and applying a softmax layer. Extensive experiments and ablative studies have been conducted, demonstrating the effectiveness of modeling rich ASCK for EAP. With the proposed RACK, we have achieved state-of-the-art performance on three benchmarks. The code is available at https://github.com/lily2lab/RACK.git. [ABSTRACT FROM AUTHOR]

The cerebellum causally supports social processing by generating internal models of social events based on statistical learning of behavioral regularities. However, whether the cerebellum is only involved in forming or also in using internal models for the prediction of forthcoming actions is still unclear. We used cerebellar transcranial Direct Current Stimulation (ctDCS) to modulate the performance of healthy adults in using previously learned expectations in an action prediction task. In a first learning phase of this task, participants were exposed to different levels of associations between specific actions and contextual elements, to induce the formation of either strongly or moderately informative expectations. In a following testing phase, which assessed the use of these expectations for predicting ambiguous (i.e. temporally occluded) actions, we delivered ctDCS. Results showed that anodic, compared to sham, ctDCS boosted the prediction of actions embedded in moderately, but not strongly, informative contexts. Since ctDCS was delivered during the testing phase, that is after expectations were established, our findings suggest that the cerebellum is causally involved in using internal models (and not just in generating them). This encourages the exploration of the clinical effects of ctDCS to compensate poor use of predictive internal models for social perception. [ABSTRACT FROM AUTHOR]

Self‐initiated movements are accompanied by an efference copy, a motor command sent from motor regions to the sensory cortices, containing a prediction of the movement's sensory outcome. Previous studies have proposed pre‐motor event‐related potentials (ERPs), including the readiness potential (RP) and its lateralized sub‐component (LRP), as potential neural markers of action feedback prediction. However, it is not known how specific these neural markers are for voluntary (active) movements as compared to involuntary (passive) movements, which produce much of the same sensory feedback (tactile, proprioceptive) but are not accompanied by an efference copy. The goal of the current study was to investigate how active and passive movements are distinguishable from premotor electroencephalography (EEG), and to examine if this change of neural activity differs when participants engage in tasks that differ in their expectation of sensory outcomes. Participants made active (self‐initiated) or passive (finger moved by device) finger movements that led to either visual or auditory stimuli (100 ms delay), or to no immediate contingency effects (control). We investigated the time window before the movement onset by measuring pre‐movement ERPs time‐locked to the button press. For RP, we observed an interaction between task and movement. This was driven by movement differences in the visual and auditory but not the control conditions. LRP conversely only showed a main effect of movement. We then used multivariate pattern analysis to decode movements (active vs. passive). The results revealed ramping decoding for all tasks from around −800 ms onwards up to an accuracy of approximately 85% at the movement. Importantly, similar to RP, we observed lower decoding accuracies for the control condition than the visual and auditory conditions, but only shortly (from −200 ms) before the button press. We also decoded visual vs. auditory conditions. Here, task is decodable for both active and passive conditions, but the active condition showed increased decoding shortly before the button press. Taken together, our results provide robust evidence that pre‐movement EEG activity may represent action‐feedback prediction in which information about the subsequent sensory outcome is encoded. [ABSTRACT FROM AUTHOR]

Tennis has gained global popularity, prompting a surge in interest towards 3D video-based tennis motion recognition. Early action recognition, which predates activity completion, is a critical classification task to preempt adverse outcomes. Prior research emphasizes effective feature extraction and modeling for swift, accurate classification, despite limited data availability. To establish a robust foundation, this study introduces an anticipatory action prediction module preceding the recognition component. The module forecasts subsequent motions based on observed ones, using an LSTM-GAN structure to mitigate motion blurring and generate predictions. This paper presents an innovative framework that leverages deep learning, particularly dilated neural networks, for real-time spatio-temporal tennis analysis on standard hardware, aiming to enhance player performance insights and action prediction through TensorFlow. The dilated RNN and CNN are integrated into the recognition module for comprehensive spatiotemporal feature modeling. To foster synergy between the prediction and recognition modules, a hard class mining mechanism is devised to enhance the learning capabilities of challenging class samples. As a result, the LSTM architecture combined with GAN provides an excellent 92.1 Precision, 91.2 Recall, 94.5 F-1 score and 95.0 Accuracy in action recognition and prediction of tennis sport, which is significantly higher than classical models i.e. GAN, Conv3DJ, Co-occurrence LSTM, and GAN + L1 + Mining. [ABSTRACT FROM AUTHOR]

Background: Converging evidence points to impairments of the predictive function exerted by the cerebellum as one of the causes of the social cognition deficits observed in patients with cerebellar disorders. Objective: We tested the neurorestorative effects of cerebellar transcranial direct current stimulation (ctDCS) on the use of contextual expectations to interpret actions occurring in ambiguous sensory sceneries in a sample of adolescents and young adults with congenital, non-progressive cerebellar malformation (CM). Methods: We administered an action prediction task in which, in an implicit-learning phase, the probability of co-occurrence between actions and contextual elements was manipulated to form either strongly or moderately informative expectations. Subsequently, in a testing phase, we probed the use of these contextual expectations for predicting ambiguous (i.e., temporally occluded) actions. In a sham-controlled, within-subject design, participants received anodic or sham ctDCS during the task. Results: Anodic ctDCS, compared to sham, improved patients’ ability to use contextual expectations to predict the unfolding of actions embedded in moderately, but not strongly, informative contexts. Conclusions: These findings corroborate the role of the cerebellum in using previously learned contextual associations to predict social events and document the efficacy of ctDCS to boost social prediction in patients with congenital cerebellar malformation. The study encourages the further exploration of ctDCS as a neurorestorative tool for the neurorehabilitation of social cognition abilities in neurological, neuropsychiatric, and neurodevelopmental disorders featured by macro- or micro-structural alterations of the cerebellum.

This study examined the ability of children with autism spectrum disorder (ASD) to generate joint intention‐based action prediction in a joint action task. Children were presented with a series of videos in which two actors either played with blocks based on joint intention (social condition) or played with blocks independently (nonsocial condition). In the familiarization stage, two actors demonstrated how they played with blocks three times. In the test stage, one actor left the scene, and another actor grasped a block and asked where she should place it. Children's gaze behavior was assessed by an eye tracker. After watching videos, children were asked to answer two questions: an action prediction question and an intention understanding question. The results showed that in the implicit eye movement task, children with ASD and typically developing (TD) children exhibited location‐based anticipatory gaze under both conditions. However, in terms of explicit behavioral responses, TD children showed higher accuracy in response to action prediction questions and intention understanding questions than children with ASD in the social condition, while no significant group difference was found in the nonsocial condition. These results indicate that children with ASD have difficulty understanding joint intention and that their action prediction is primarily driven by bottom‐up sensory inputs. Lay Summary: Impaired social interactions are a hallmark of autism spectrum disorder. A possible explanation is that children with ASD have difficulty predicting others' actions based on shared intention in social interactions. We investigated whether children with ASD could predict others' actions based on joint intention in a joint action task by tracking their eye movements and asking them questions. We found that children with ASD had difficulty understanding joint intention but showed intact understanding of personal intentions, which provides unique evidence that impaired social interactions in children with ASD might be due to diminished joint intention understanding. [ABSTRACT FROM AUTHOR]

The future of human–robot collaboration relies on people's ability to understand and predict robots' actions. The machine-like appearance of robots, as well as contextual information, may influence people's ability to anticipate the behaviour of robots. We conducted six separate experiments to investigate how spatial cues and task instructions modulate people's ability to understand what a robot is doing. Participants observed goal-directed and non-goal directed gaze shifts made by human and robot agents, as well as directional cues displayed by a triangle. We report that biasing an observer's attention, by showing just one object an agent can interact with, can improve people's ability to understand what humanoid robots will do. Crucially, this cue had no impact on people's ability to predict the upcoming behaviour of the triangle. Moreover, task instructions that focus on the visual and motor consequences of the observed gaze were found to influence mentalising abilities. We suggest that the human-like shape of an agent and its physical capabilities facilitate the prediction of an upcoming action. The reported findings expand current models of gaze perception and may have important implications for human–human and human–robot collaboration. [ABSTRACT FROM AUTHOR]

Several studies have shown that infants anticipate human goal-directed actions, but not robot's ones. However, the studies focusing on the robot goal-directed actions have mainly analyzed the effect of mechanical arms on infant's attention. To date, the prediction of goal-directed actions in infants has not yet been studied when the agent is a humanoid robot. Given this lack of evidence in infancy research, the present study aims at analyzing infants' action anticipation of both a human's and a humanoid robot's goal-directed action. Data were acquired on thirty 17-month-old infants, watching four video clips, where either a human or a humanoid robot performed a goal-directed action, i.e. reaching a target. Infants looking behavior was measured through the eye-tracking technique. The results showed that infants anticipated the goal-directed action of both the human and the robot and there were no differences in the anticipatory gaze behavior between the two agents. Furthermore, the findings indicated different attentional patterns for the human and the robot, showing a greater attention paid to the robot's face than the human's face. Overall, the results suggest that 17-month-old infants may infer also humanoid robot' underlying action goals. [ABSTRACT FROM AUTHOR]

The cortical learning algorithm (CLA) is a time series prediction algorithm. Memory elements called columns and cells discretely represent data with their state combinations, whereas linking elements called synapses change their state combinations. For tasks requiring to take actions, the action-prediction CLA (ACLA) has an advantage to complement missing state values with their predictions. However, an increase in the number of missing state values (i) generates excess synapses negatively affect the action predictions and (ii) decreases the stability of data representation and makes the output of action values difficult. This paper proposes an adaptive ACLA using (i) adaptive synapse adjustment and (ii) adaptive action-separated decoding in an uncertain environment, missing multiple input state values probabilistically. (i) The proposed adaptive synapse adjustment suppresses unnecessary synapses. (ii) The proposed adaptive action-separated decoding adaptively outputs an action prediction separately for each action value. Experimental results using uncertain two- and three-dimensional mountain car tasks show that the proposed adaptive ACLA achieves a more robust action prediction performance than the conventional ACLA, DDPG, and the three LSTM-assisted reinforcement learning algorithms of DDPG, TD3, and SAC, even though the number of missing state values and their frequencies increase. These results implicate that the proposed adaptive ACLA is a way to making decisions for the future, even in cases where information surrounding the situation partially lacked. [ABSTRACT FROM AUTHOR]

In sports, understanding others' actions represents a fundamental skill that allows players to predict the outcome of teammates' and opponents' actions and counteract them properly. While it is well known that motor expertise sets better premises for predicting the result of an observed sports action, it remains untested whether this principle applies to a team where players cover different positions that imply different motor repertoires. To test this hypothesis, we selected rugby as a paradigmatic example in which only one or two players out of 22 train and perform placed kicks. We administered a placed kick outcome prediction task to three groups of participants, namely, rugby kickers, rugby non-kickers, and controls, thus spanning over different combinations of motor expertise and visual experience. Kickers outperformed both their non-kicking teammates and controls in overall prediction accuracy. We documented how the viewpoint of observation, the expertise of the observed kicker, and the position of the kick on the court influenced the prediction performance across the three groups. Finally, we revealed that within rugby players, the degree of motor expertise (but not the visual experience) causally affects accuracy, and such a result stands even after accounting for the level of visual experience. These findings extend the role of motor expertise in decoding and predicting others' behaviors to sports teammates, among which every member is equipped with a position-specific motor repertoire, advocating for new motor training procedures combining the gestures to-be-performed with those to-be-faced. [ABSTRACT FROM AUTHOR]

• Adults but not 3- to10-year-olds based their visual anticipations on kinematic cues. • Anticipations are based on perceptual mechanism rather than motor simulation. • Kinematic cues are used for verbal prediction before being used for visual anticipation of action. Expectations about how others' actions unfold in the future are crucial for our everyday social interactions. The current study examined the development of the use of kinematic cues for action anticipation and prediction in 3-year-olds, 4-year-olds, 10-year-olds, and adults in two experiments. Participants observed a hand repeatedly reaching for either a close or far object. The motor kinematics of the hand varied depending on whether the hand reached for the close or far object. We assessed whether participants would use kinematic cues to visually anticipate (Experiment 1; N =98) and verbally predict (Experiment 2; N =80) which object the hand was going to grasp. We found that only adults, but not 3- to 10-year-olds, based their visual anticipations on kinematic cues (Experiment 1). This speaks against claims that action anticipations are based on simulating others' motor processes and instead provides evidence that anticipations are based on perceptual mechanisms. Interestingly, 10-year-olds used kinematic cues to correctly verbally predict the target object, and 4-year-olds learned to do so over the trials (Experiment 2). Thus, kinematic cues are used earlier in life for explicit action predictions than for visual action anticipations. This adds to a recent debate on whether or not an implicit understanding of others' actions precedes their ability to verbally reason about the same actions. [ABSTRACT FROM AUTHOR]

In sports, understanding others’ actions represents a fundamental skill that allows players to predict the outcome of teammates’ and opponents’ actions and counteract them properly. While it is well known that motor expertise sets better premises for predicting the result of an observed sports action, it remains untested whether this principle applies to a team where players cover different positions that imply different motor repertoires. To test this hypothesis, we selected rugby as a paradigmatic example in which only one or two players out of 22 train and perform placed kicks. We administered a placed kick outcome prediction task to three groups of participants, namely, rugby kickers, rugby non-kickers, and controls, thus spanning over different combinations of motor expertise and visual experience. Kickers outperformed both their non-kicking teammates and controls in overall prediction accuracy. We documented how the viewpoint of observation, the expertise of the observed kicker, and the position of the kick on the court influenced the prediction performance across the three groups. Finally, we revealed that within rugby players, the degree of motor expertise (but not the visual experience) causally affects accuracy, and such a result stands even after accounting for the level of visual experience. These findings extend the role of motor expertise in decoding and predicting others’ behaviors to sports teammates, among which every member is equipped with a position-specific motor repertoire, advocating for new motor training procedures combining the gestures to-be-performed with those to-be-faced.

Response process data collected from human–computer interactive items contain detailed information about respondents' behavioural patterns and cognitive processes. Such data are valuable sources for analysing respondents' problem‐solving strategies. However, the irregular data format and the complex structure make standard statistical tools difficult to apply. This article develops a computationally efficient method for exploratory analysis of such process data. The new approach segments a lengthy individual process into a sequence of short subprocesses to achieve complexity reduction, easy clustering and meaningful interpretation. Each subprocess is considered a subtask. The segmentation is based on sequential action predictability using a parsimonious predictive model combined with the Shannon entropy. Simulation studies are conducted to assess the performance of the new method. We use a case study of PIAAC 2012 to demonstrate how exploratory analysis for process data can be carried out with the new approach. [ABSTRACT FROM AUTHOR]

Predicting the unfolding of others' actions (action prediction) is crucial for successfully navigating the social world and interacting efficiently. Age‐related changes in this domain have remained largely unexplored, especially for predictions regarding simple gestures and independent of contextual information or motor expertise. Here, we evaluated whether healthy aging impacts the neurophysiological processes recruited to anticipate, from the observation of implied‐motion postures, the correct conclusion of simple grasping and pointing actions. A color‐discrimination task served as a control condition to assess the specificity of the age‐related effects. Older adults showed reduced efficiency in performance that was yet not specific to the action prediction task. Nevertheless, fMRI results revealed task‐specific age‐related differences: while both groups showed stronger recruitment of the lateral occipito‐temporal cortex bilaterally during the action prediction than the control task, the younger participants additionally showed a higher bilateral engagement of parietal regions. Importantly, in both groups, the recruitment of visuo‐motor processes in the right posterior parietal cortex was a predictor of good performance. These results support the hypothesis of decreased involvement of sensorimotor processes in cognitive tasks when processing action‐ and body‐related stimuli in healthy aging. These results have implications for social interaction, which requires the fast reading of others' gestures. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Shanghai Jiaotong University (Science) is the property of Springer Nature and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Computer-Aided Design & Computer Graphics / Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao is the property of Gai Kan Bian Wei Hui and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Humans take a teleological stance when observing others' actions, interpreting them as intentional and goal directed. In predictive processing accounts of social perception, this teleological stance would be mediated by a perceptual prediction of an ideal energy-efficient reference trajectory with which a rational actor would achieve their goals within the current environmental constraints. Hudson and colleagues (2018 Proc. R. Soc. B 285, 20180638. (doi:10.1098/rspb.2018.0638)) tested this hypothesis in a series of experiments in which participants reported the perceived disappearance points of hands reaching for objects. They found that these judgements were biased towards the expected efficient reference trajectories. Observed straight reaches were reported higher when an obstacle needed to be overcome than if the path was clear. By contrast, unnecessarily high reaches over empty space were perceptually flattened. Moreover, these perceptual biases increased the more the environmental constraints and expected action trajectories were explicitly processed. These findings provide an important advance to our understanding of the mechanisms underlying social perception. The current replication tests the robustness of these findings and whether they uphold in an online setting.

It has been widely assumed that internal forward models use efference copies to create predictions about the sensory consequences of our own actions. While these predictions have frequently been associated with a reduced blood oxygen level dependent (BOLD) response in sensory cortices, the timing and duration of the hemodynamic response for the processing of video feedback of self‐generated (active) versus externally generated (passive) movements is poorly understood. In the present study, we tested the hypothesis that predictive mechanisms for self‐generated actions lead to early and shorter neural processing compared with externally generated movements. We investigated active and passive movements using a custom‐made fMRI‐compatible movement device. Visual video feedback of the active and passive movements was presented in real time or with variable delays. Participants had to judge whether the feedback was delayed. Timing and duration of BOLD impulse response was calculated using a first (temporal derivative [TD]) and second‐order (dispersion derivative [DD]) Taylor approximation. Our reanalysis confirmed our previous finding of reduced BOLD response for active compared to passive movements. Moreover, we found positive effects of the TD and DD in the supplementary motor area, cerebellum, visual cortices, and subcortical structures, indicating earlier and shorter hemodynamic responses for active compared to passive movements. Furthermore, earlier activation in the putamen for active compared to passive conditions was associated with reduced delay detection performance. These findings indicate that efference copy‐based predictive mechanisms enable earlier processing of action feedback, which might have reduced the ability to detect short delays between action and feedback. [ABSTRACT FROM AUTHOR]

Recent advances in social neuroscience have highlighted the critical role of the cerebellum and especially the posterior cerebellar Crus in social mentalizing (i.e., theory of mind). Research in the past 5 years has provided growing evidence supporting the view that the posterior cerebellum builds internal action models of our social interactions to predict how other people's actions will be executed, and what our most likely responses to these actions will be. This paper presents an overview of a series of fMRI experiments on novel tasks involving a combination of (a) the learning or generation of chronological sequences of social actions either in an explicit or implicit manner, which (b) require social mentalizing on another person's mental state such as goals, beliefs, and implied traits. Together, the results strongly confirm the central role of the posterior cerebellar Crus in identifying and automatizing action sequencing during social mentalizing, and in predicting future action sequences based on social mentalizing inferences about others. This research program has important implications: It provides for the first time (a) fruitful starting points for diagnosing and investigating social sequencing dysfunctions in a variety of mental disorders which have also been related to cerebellar dysfunctions, (b) provides the necessary tools for testing whether non-invasive neurostimulation targeting the posterior cerebellum has a causal effect on social functioning, and (c) whether these stimulation techniques and training programs guided by novel cerebellar social sequencing insights, can be exploited to increase posterior cerebellar plasticity in order to alleviate social impairments in mental disorders. [ABSTRACT FROM AUTHOR]

Humans have a flexible and accurate ability to coordinate their movement in time with external rhythms. However, it remains unclear when and how, during their development, human children acquire the ability to adjust tempo and control the timing of their movement toward others. A previous study suggested that such self-regulation of coordination develops at around 18 and 30 months after birth. In this study, we investigated the performance of 24-month-old children and compared their data with those of 18- and 30-month-olds provided in our previous study. In the joint-drumming task, each child was enticed to drum under four conditions [partner: mother or robot; speed: 400 or 600 ms inter-stimulus-interval (ISI)]. The most pivotal test condition was the 600 ms ISI speed condition (slower than children’s spontaneous motor tempo in these age groups). We found that from the age of 24 months, children try to slow down their drumming tempo toward the 600 ms ISI speed condition, regardless of the drumming partner. On the other hand, significant timing control toward the onset of the 600 ms ISI condition was observed from the age of 30 months. This implies that both motor and cognitive mechanisms are required for flexible tempo adjustment and accurate synchronization and that these develop gradually among 18-, 24-, and 30-month-olds. [ABSTRACT FROM AUTHOR]

In this paper, we propose an ambiguousness-aware state evolution (AASE) method which represents the uncertainty of the input sequence and evolves the subsequent skeletons to generate a reasonable full-length sequence for action prediction. Unlike most existing methods that enforce partial sequences with the labels of full-length videos and ignore the semantic information of the subsequent action, we develop an evolution method by predicting the instructional actions and generating the reasonable candidate subsequent actions, so that the ambiguity of the full sequence’s label supervising for the partial actions can be effectively alleviated. Our method generates the rational subsequent actions under the instructional action class to complement the partially observed action sequence. We design two criteria for a rational generation: 1) the instruction of subsequent action keeps the semantic consistency with the observed sequence; 2) the generation sequence is satisfied with the distribution of the sequence of real data. Moreover, we design an uncertainty module to decide the instructional action class for the generation network. AASE predicts instructional actions with uncertainty learning and evolves different instructional actions by generating the subsequent skeletons, which find the most probable action to represent the partially observed action by learning the way of perceiving the tendency of the ongoing action. We conduct experiments on seven widely used action datasets: NTU-60, NTU-120, UCF101, UT-Interaction, BIT, PKU-MMD and HMDB51, and our experimental results clearly demonstrate that our method achieves very competitive performance with state-of-the-art. [ABSTRACT FROM AUTHOR]

This letter addresses the challenging problem of action prediction with partially observed sequences of skeletons. Towards this goal, we propose a novel two-stage knowledge distillation framework, which transfers prior knowledge to assist the early prediction of ongoing actions. In the first stage, the action prediction model (also referred to as the student) learns from a couple of teachers to adaptively distill action knowledge at different progress levels for partial sequences. Then the learned student acts as a teacher in the next stage, with the objective of optimizing a better action prediction model in a self-training manner. We design an adaptive self-training strategy from the perspective of undermining the supervision from the annotated labels, since this hard supervision is actually too strict for partial sequences without enough discriminative information. Finally, the action prediction models trained in the two stages jointly constitute a two-stream architecture for action prediction. Extensive experiments on the large-scale NTU RGB+D dataset validate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Action prediction involves observing and predicting the actions of others and plays an important role in social cognition and interacting with others. It is thought to use simulation, whereby the observers use their own motor system to predict the observed actions. As individuals diagnosed with autism are characterized by difficulties understanding the actions of others and motor coordination issues, it is possible that action prediction ability is altered in this population. This study compared action prediction ability between 20 autistic and 22 non‐autistic adults using an occlusion paradigm. Participants watched different videos of a female actor carrying out everyday actions. During each video, the action was transiently occluded by a gray rectangle for 1000 ms. During occlusions, the video was allowed to continue as normal or was moved forward (i.e., appearing to continue too far ahead) or moved backwards (i.e., appearing to continue too far behind). Participants were asked to indicate after each occlusion whether the action continued with the correct timing or was too far ahead/behind. Autistic individuals were less accurate than non‐autistic individuals, particularly when the video was too far behind. A trend analysis suggested that autistic participants were more likely to judge too far behind occlusions as being in time. These preliminary results suggest that prediction ability may be altered in autistic adults, potentially due to slower simulation or a delayed onset of these processes. Lay Summary: When we observe other people performing everyday actions, we use their movements to help us understand and predict what they are doing. In this study, we found that autistic compared to non‐autistic adults were slightly less accurate at predicting other people's actions. These findings help to unpick the different ways that social understanding is affected in autism. [ABSTRACT FROM AUTHOR]

Unsuccessful replication attempts of paradigms assessing children's implicit tracking of false beliefs have instigated the debate on whether or not children have an implicit understanding of false beliefs before the age of four. A novel multi‐trial anticipatory looking false belief paradigm yielded evidence of implicit false belief reasoning in 3‐ to 4‐year‐old children using a combined score of two false belief conditions (Grosse Wiesmann, C., Friederici, A. D., Singer, T., & Steinbeis, N. [2017]. Developmental Science, 20(5), e12445). The present study is a large‐scale replication attempt of this paradigm. The task was administered three times to the same sample of N = 185 children at 2, 3, and 4 years of age. Using the original stimuli, we did not replicate the original finding of above‐chance belief‐congruent looking in a combined score of two false belief conditions in either of the three age groups. Interestingly, the overall pattern of results was comparable to the original study. Post‐hoc analyses revealed, however, that children performed above chance in one false belief condition (FB1) and below chance in the other false belief condition (FB2), thus yielding mixed evidence of children's false belief‐based action predictions. Similar to the original study, participants' performance did not change with age and was not related to children's general language skills. This study demonstrates the importance of large‐scaled replications and adds to the growing number of research questioning the validity and reliability of anticipatory looking false belief paradigms as a robust measure of children's implicit tracking of beliefs. [ABSTRACT FROM AUTHOR]

The ability to act together with others to achieve common goals is crucial in life, yet there is no full consensus on the underlying cognitive skills. While influential theoretical accounts suggest that interaction requires sophisticated insights into others' minds, alternative views propose that high-level social skills might not be necessary because interactions are grounded on sensorimotor predictive mechanisms. At present, empirical evidence is insufficient to decide between the two. This study addressed this issue and explored the association between performance at joint action tasks and cognitive abilities in three domains—action prediction, perspective-taking, and theory of mind—in healthy adults (N = 58). We found that, while perspective-taking played a role in reading the behaviour of others independently of the social context, action prediction abilities specifically influenced the agents' performance in an interactive task but not in a control (social but non-interactive) task. In our study, performance at a theory of mind test did not play any role, as confirmed by Bayesian analyses. The results suggest that, in adults, sensorimotor predictive mechanisms might play a significant and specific role in supporting interpersonal coordination during motor interactions. We discuss the implications of our findings for the contrasting theoretical views described earlier and propose a way they might be partly reconciled. [ABSTRACT FROM AUTHOR]