Search

T. Uchida, H. Ishiguro and P. F. Dominey, "Improving Quality of Life with a Narrative Robot Companion: II – Creating Group Cohesion via Shared Narrative Experience*," 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Naples, Italy, 2020, pp. 906-913, doi: 10.1109/RO-MAN47096.2020.9223600., The 29th IEEE International Conference on Robot & Human Interactive Communication [31 AUG - 04 SEPT, 2020], One of the most difficult things for social robots is to enter the realm of human social relations. Here, we exploit recent advances in natural language processing (NLP) that provide robots access to human experience, which can allow them to enter into human social relations. When people are put together in arbitrary conditions, as in a home for the elderly, it can be difficult for them to share experience. To address this, we use recent advances in NLP to allow the robot to discover shared narratives between such group members. Our narrative companion extends the human capability to make social narrative links for building group coherence through sharing experience.Requirements are identified for a narrative companion to allow individuals within a group to focus their interactions on shared experiences and interests, to improve group coherence. The system should collect and organize members' experiences, and should discover semantic similarity between different members' experiences in order to create a group narrative. It should then accompany the group in their cohesion-enhancing experience of this narrative. Based on these requirements, and extending our previous work, we implement the V2.0 narrative companion on the Pepper robot.The system is validated in a case study where participants provide 5 favorite photographs and short answers to questions. The Narrative Semantic Similarity Analysis System (NarSim) generates a meaningful trajectory of narrative linking people and events depicted in the photographs. With this, Pepper then accompanies the group and prompts group members to enrich the shared narrative, to further enhance the group pleasure and cohesion. Results are presented, and future applications for improved quality of life are discussed.

Recent research has revealed that during continuous perception of movies or stories, humans display cortical activity patterns that reveal hierarchical segmentation of event structure. Thus, sensory areas like auditory cortex display high frequency segmentation related to the stimulus, while semantic areas like posterior middle cortex display a lower frequency segmentation related to transitions between events. These hierarchical levels of segmentation are associated with different time constants for processing. Likewise, when two groups of participants heard the same sentence in a narrative, preceded by different contexts, neural responses for the groups were initially different and then gradually aligned. The time constant for alignment followed the segmentation hierarchy: sensory cortices aligned most quickly, followed by mid-level regions, while some higher-order cortical regions took more than 10 seconds to align. These hierarchical segmentation phenomena can be considered in the context of processing related to comprehension. In a recently described model of discourse comprehension word meanings are modeled by a language model pre-trained on a billion word corpus. During discourse comprehension, word meanings are continuously integrated in a recurrent cortical network. The model demonstrates novel discourse and inference processing, in part because of two fundamental characteristics: real-world event semantics are represented in the word embeddings, and these are integrated in a reservoir network which has an inherent gradient of functional time constants due to the recurrent connections. Here we demonstrate how this model displays hierarchical narrative event segmentation properties beyond the embeddings alone, or their linear integration. The reservoir produces activation patterns that are segmented by a hidden Markov model (HMM) in a manner that is comparable to that of humans. Context construction displays a continuum of time constants across reservoir neuron subsets, while context forgetting has a fixed time constant across these subsets. Importantly, virtual areas formed by subgroups of reservoir neurons with faster time constants segmented with shorter events, while those with longer time constants preferred longer events. This neurocomputational recurrent neural network simulates narrative event processing as revealed by the fMRI event segmentation algorithm provides a novel explanation of the asymmetry in narrative forgetting and construction. The model extends the characterization of online integration processes in discourse to more extended narrative, and demonstrates how reservoir computing provides a useful model of cortical processing of narrative structure., Author summary When we watch movies or listen to stories, our brains are led through a trajectory of activation whose structure reflects that of the event structure of the story. This takes place at multiple timescales across the brain, likely corresponding to different timescales of event representation. While this has been well described in human fMRI, the underlying computations that lead to these activation trajectories has not yet been fully characterized. The current research develops and explores a recurrent network “reservoir” model of cortical computation, whose natural internal dynamics help to provide an explanation of the trajectory of brain states that are observed in different cortical areas in humans. The model is exposed to narratives in the form of word embeddings for words in the narrative transcript. Neural activation in the model reveals event structure at multiple levels of temporal structure. This begins to provide insight into the computations underlying the event structure observed in the human brain during narrative processing.

Computational Intelligence and Artificial Intelligence are both aiming at building machines and softwares capable of intelligent behavior. They are consequently prone to interactions, even if the latter is not necessarily interested in understanding how cognition emerges from the brain substrate. In this chapter, we enumerate, describe and discuss the most important fields of interactions. Some are methodological and are concerned with information representation, processing and learning. At the functional level, the focus is set on major cognitive functions like perception, navigation, decision making and language. Among the salient characteristics of the critical contributions of Computational Neuroscience to the development of intelligent systems, its systemic view of the cerebral functioning is particularly precious to model highly multimodal cognitive functions like decision making and language and to design cognitive architectures for the autonomous behavior of robots.

International audience; An open problem in the cognitive dimensions of navigation concerns how previous exploratory experience is reorganized in order to allow the creation of novel efficient navigation trajectories. This behavior is revealed in the "traveling salesrat problem" (TSP) when rats discover the shortest path linking baited food wells after a few exploratory traversals. We have recently published a model of navigation sequence learning, where sharp wave ripple replay of hippocampal place cells transmit "snippets" of the recent trajectories that the animal has explored to the prefrontal cortex (PFC) (Cazin et al. in PLoS Comput Biol 15:e1006624, 2019). PFC is modeled as a recurrent reservoir network that is able to assemble these snippets into the efficient sequence (trajectory of spatial locations coded by place cell activation). The model of hippocampal replay generates a distribution of snippets as a function of their proximity to a reward, thus implementing a form of spatial credit assignment that solves the TSP task. The integrative PFC reservoir reconstructs the efficient TSP sequence based on exposure to this distribution of snippets that favors paths that are most proximal to rewards. While this demonstrates the theoretical feasibility of the PFC-HIPP interaction, the integration of such a dynamic system into a real-time sensory-motor system remains a challenge. In the current research, we test the hypothesis that the PFC reservoir model can operate in a real-time sensory-motor loop. Thus, the main goal of the paper is to validate the model in simulated and real robot scenarios. Place cell activation encoding the current position of the simulated and physical rat robot feeds the PFC reservoir which generates the successor place cell activation that represents the next step in the reproduced sequence in the readout. This is input to the robot, which advances to the coded location and then generates de novo the current place cell activation. This allows demonstration of the crucial role of embodiment. If the spatial code readout from PFC is played back directly into PFC, error can accumulate, and the system can diverge from desired trajectories. This required a spatial filter to decode the PFC code to a location and then recode a new place cell code for that location. In the robot, the place cell vector output of PFC is used to physically displace the robot and then generate a new place cell coded input to the PFC, replacing part of the software recoding procedure that was required otherwise. We demonstrate how this integrated sensory-motor system can learn simple navigation sequences and then, importantly, how it can synthesize novel efficient sequences based on prior experience, as previously demonstrated (Cazin et al. 2019). This contributes to the understanding of hippocampal replay in novel navigation sequence formation and the important role of embodiment.Prefrontal cortex; Hippocampus; Navigation; Replay; Sharp wave ripple; Reservoir computing; Traveling sales person; Reinforcement learning ;Spatial Cognition; Memory Consolidation; Natural-Language; Reverse Replay; Grid Cells; Model; Reactivation; Cortex; State; Representation

International audience

Whereas common semantic representations for individual words across languages have been identified, a common meaning system at sentence-level based on the grammatical construction of words has not been determined. In this study, fMRI was used to investigate whether an across-language sentence comprehension system exists. Chinese–Japanese bilingual participants (n = 32) were asked to determine whether two consecutive stimuli were related (congruent) or not (incongruent) to the same event. Stimuli were displayed with three different modalities (Chinese written sentences, Japanese written sentences, and photographs). The behavioral results showed no significant difference in accuracy and response times among the three modalities. Multi-voxel pattern analysis (MVPA) of fMRI data was used to classify the semantic relationship (congruent or incongruent) across the stimulus modalities. The classifier was first trained to determine congruency within Chinese sentences, and then tested with Japanese sentences, and vice versa. A whole-brain searchlight analysis revealed significant above-chance classification accuracy across Chinese and Japanese sentences in the supramarginal gyrus (BA 40), extending into the angular gyrus (BA 39), and the opercular (BA 44) and triangular (BA 45) part of the inferior frontal gyrus in the left hemisphere (cluster-level FWE corrected p < 0.05). Significant above-chance classification accuracy was also found across Japanese sentences and photographs in the supramarginal (BA 40) and angular gyrus (BA 39). These results indicate that a common meaning system for sentence processing across languages and modalities exists, and it involves the left inferior parietal gyrus.

Heyes does well to argue that some of the apparently innate human capabilities for cultural learning can be considered in terms of more general-purpose mechanisms. In the application of this to language, she overlooks some of its most interesting properties. I review three, and then illustrate how mindreading can come from general-purpose mechanism via language.

Understanding the neural process underlying the comprehension of visual images and sentences remains a major open challenge in cognitive neuroscience. We previously demonstrated with fMRI and DTI that comprehension of visual images and sentences describing human activities recruits a common semantic system. The current research tests the hypothesis that this common semantic system will display similar neural dynamics during processing in these two modalities. To investigate these neural dynamics we recorded EEG from naïve subjects as they saw simple narratives made up of a first visual image depicting a human event, followed by a second that was either a sequentially coherent narrative follow-up, or not, of the first image. In separate blocks of trials the same protocol was presented using sentences. Analysis of the EEG signal revealed common neural dynamics for semantic processing across image and sentence modalities. Late positive ERPs were observed in response to sequential incoherence for sentences and images, consistent with previous studies that examined coherence in these two modalities separately. Analysis of oscillatory power revealed increased gamma-band activity for sequential coherence, again consistent with previous studies showing gamma increases for coherence and matching in sentence and image processing. Multivariate analysis demonstrated that training a classifier on data from one modality (images or sentences) allowed reliable decoding of the sequential coherence of data from trials in the untrained modality, providing further support for a common underlying semantic system for images and sentences. Processing sequential coherence of successive stimuli is associated with neural dynamics that are common to sentence and visual image modalities and that can be decoded across modalities. These results are discussed in the context of EEG signatures of narrative processing and meaning, and more general neural mechanisms for structure processing.

International audience; Today users can interact with popular virtual assistants such as Siri to accomplish their tasks on a digital environment. In these systems, links between natural language requests and their concrete realizations are specified at the conception phase. A more adaptive approach would be to allow the user to provide natural language instructions or demonstrations when a task is unknown by the assistant. An adaptive solution should allow the virtual assistant to operate a much larger digital environment composed of multiple application domains and providers and better match user needs. We have previously developed robotic systems, inspired by human language developmental studies, that provide such a usage-based adaptive capacity. Here we extend this approach to human interaction with a virtual assistant that can first learn the mapping between verbal commands and basic action semantics of a specific domain. Then, it can learn higher level mapping by combining previously learned procedural knowledge in interaction with the user. The flexibility of the system is demonstrated as the virtual assistant can learn actions in a new domains (Email, Wikipedia,...), and can then learn how email and Wikipedia basic procedures can be combined to form hybrid procedural knowledge.

To explore the effects of practice we scanned participants with fMRI while they were performing four-key unfamiliar and familiar sequences, and compared the associated activities relative to simple control sequences. On the basis of a recent cognitive model of sequential motor behavior (C-SMB), we propose that the observed neural activity would be associated with three functional networks that can operate in parallel and that allow (a) responding to stimuli in a reaction mode, (b) sequence execution using spatial sequence representations in a central-symbolic mode, and (c) sequence execution using motor chunk representations in a chunking mode. On the basis of this model and findings in the literature, we predicted which neural areas would be active during execution of the unfamiliar and familiar keying sequences. The observed neural activities were largely in line with our predictions, and allowed functions to be attributed to the active brain areas that fit the three above functional systems. The results corroborate C-SMB’s assumption that at advanced skill levels the systems executing motor chunks and translating key-specific stimuli are racing to trigger individual responses. They further support recent behavioral indications that spatial sequence representations continue to be used.

In robotics research with language-based interaction, simplifications are made, such that a given event can be described in a unique manner, where there is a direct mapping between event representations and sentences that can describe these events. However, common experience tells us that the same physical event can be described in multiple ways, depending on the perspective of the speaker. The current research develops methods for representing events from multiple perspectives, and for choosing the perspective that will be used for generating a linguistic construal, based on attentional processes in the system. The multiple perspectives are based on the principle that events can be considered in terms of the force driving the event, and the result obtained from the event, based on the theory of Gärdenfors. In addition, within these perspectives a further refinement can be made with respect to the agent, object, and recipient perspectives. We develop a system for generating appropriate construals of meaning, and demonstrate how this can be used in a realistic dialogic interaction between a behaving robot and a human interlocutor.

International audience; Keywords: Diffusion tension imaging-functional interaction Multimodal Cognitive functions-semantic framework-human A B S T R A C T Understanding events requires interplaying cognitive processes arising in neural networks whose organisation and connectivity remain subjects of controversy in humans. In the present study, by combining diffusion tensor imaging and functional interaction analysis, we aim to provide new insights on the organisation of the structural and functional pathways connecting the multiple nodes of the identified semantic system-shared by vision and language (Jouen et al., 2015). We investigated a group of 19 healthy human subjects during experimental tasks of reading sentences or seeing pictures. The structural connectivity was realised by deterministic tractography using an algorithm to extract white matter fibers terminating in the selected regions of interest (ROIs) and the functional connectivity by independent component analysis to measure correlated activities among these ROIs. The major connections link ventral neural stuctures including the parietal and temporal cortices through inferior and middle longitudinal fasciculi, the retrosplenial and parahippocampal cortices through the cingulate bundle, and the temporal and prefrontal structures through the uncinate fasciculus. The imageability score provided when the subject was reading a sentence was significantly correlated with the factor of anisotropy of the left parieto-temporal connections of the middle longitudinal fasciculus. A large part of this ventrally localised structural connectivity corresponds to functional interactions between the main parietal, temporal and frontal nodes. More precisely, the strong coactivation both in the anterior temporal pole and in the region of the temporo-parietal cortex suggests dual and cooperating roles for these areas within the semantic system. These findings are discussed in terms of two semantics-related subsystems responsible for conceptual representation.

Two experiments examine how grammatical verb aspect constrains our understanding of events. According to linguistic theory, an event described in the perfect aspect (John had opened the bottle) should evoke a mental representation of a finished event with focus on the resulting object, whereas an event described in the imperfective aspect (John was opening the bottle) should evoke a representation of the event as ongoing, including all stages of the event, and focusing all entities relevant to the ongoing action (instruments, objects, agents, locations, etc.). To test this idea, participants saw rebus sentences in the perfect and imperfective aspect, presented one word at a time, self-paced. In each sentence, the instrument and the recipient of the action were replaced by pictures (John was using/had used a *corkscrew* to open the *bottle* at the restaurant). Time to process the two images as well as speed and accuracy on sensibility judgments were measured. Although experimental sentences always made sense, half of the object and instrument pictures did not match the temporal constraints of the verb. For instance, in perfect sentences aspect-congruent trials presented an image of the corkscrew closed (no longer in-use) and the wine bottle fully open. The aspect-incongruent yet still sensible versions either replaced the corkscrew with an in-use corkscrew (open, in-hand) or the bottle image with a half-opened bottle. In this case, the participant would still respond "yes", but with longer expected response times. A three-way interaction among Verb Aspect, Sentence Role, and Temporal Match on image processing times showed that participants were faster to process images that matched rather than mismatched the aspect of the verb, especially for resulting objects in perfect sentences. A second experiment replicated and extended the results to confirm that this was not due to the placement of the object in the sentence. These two experiments extend previous research, showing how verb aspect drives not only the temporal structure of event representation, but also the focus on specific roles of the event. More generally, the findings of visual match during online sentence-picture processing are consistent with theories of perceptual simulation.

This article briefly reviews research in cognitive development concerning the nature of the human self. It then reviews research in developmental robotics that has attempted to retrace parts of the developmental trajectory of the self. This should be of interest to developmental psychologists, and researchers in developmental robotics. As a point of departure, a characteristic aspect of human social interaction is cooperation – the process of entering into a joint enterprise to achieve a common goal. Fundamental to this ability is the underlying ability to engage in self-other relations. This suggests that if we intend for robots to cooperate with humans, then to some extent robots must engage in these self-other relations, and hence they must have some aspect of self. Decades of research in human cognitive development indicate that the self is not present from the outset, but rather that it is developed in a usage-based fashion, that is, through engaging with the world, including the physical world and the social world of animate intentional agents. In an effort to characterize the self, Ulric Neisser noted that self is not unitary, and he thus proposed five types of self-knowledge that correspond to five distinct components of self: ecological, interpersonal, conceptual, temporally extended, and private. He emphasized the ecological nature of each of these levels, how they are developed through the engagement of the developing child with the physical and interpersonal worlds. Crucially, development of the self has been shown to rely on the child’s autobiographical memory. From the developmental robotics perspective, this suggests that in principal it would be possible to develop certain aspects of self in a robot cognitive system where the robot is engaged in the physical and social world, equipped with an autobiographical memory system. We review a series of developmental robotics studies that make progress in this enterprise. We conclude with a summary of the properties that are required for the development of these different levels of self, and we identify topics for future research. While this work has been developed in the context of iCub, it is conceptually extensible to any robot.

From our ability to attend to many stimuli occurring in rapid succession to the transformation of memories during a night of sleep, cognition occurs over widely varying time scales spanning milliseconds to days and beyond. Cognitive processing is often influenced by several behavioral variables as well as non-linear interactions between multiple neural systems. This frequently produces unpredictable patterns of behavior and makes understanding the underlying temporal factors influencing cognition a fruitful area of hypothesis development and scientific inquiry. Across two reviews, a perspective, and 12 original research articles covering the domains of learning, memory, attention, cognitive control, and social decision making this research topic sheds new light on the temporal dynamics of cognitive processing.

Primates display a remarkable ability to adapt to novel situations. Determining what is most pertinent in these situations is not always possible based only on the current sensory inputs, and often also depends on recent inputs and behavioral outputs that contribute to internal states. Thus, one can ask how cortical dynamics generate representations of these complex situations. It has been observed that mixed selectivity in cortical neurons contributes to represent diverse situations defined by a combination of the current stimuli, and that mixed selectivity is readily obtained in randomly connected recurrent networks. In this context, these reservoir networks reproduce the highly recurrent nature of local cortical connectivity. Recombining present and past inputs, random recurrent networks from the reservoir computing framework generate mixed selectivity which provides pre-coded representations of an essentially universal set of contexts. These representations can then be selectively amplified through learning to solve the task at hand. We thus explored their representational power and dynamical properties after training a reservoir to perform a complex cognitive task initially developed for monkeys. The reservoir model inherently displayed a dynamic form of mixed selectivity, key to the representation of the behavioral context over time. The pre-coded representation of context was amplified by training a feedback neuron to explicitly represent this context, thereby reproducing the effect of learning and allowing the model to perform more robustly. This second version of the model demonstrates how a hybrid dynamical regime combining spatio-temporal processing of reservoirs, and input driven attracting dynamics generated by the feedback neuron, can be used to solve a complex cognitive task. We compared reservoir activity to neural activity of dorsal anterior cingulate cortex of monkeys which revealed similar network dynamics. We argue that reservoir computing is a pertinent framework to model local cortical dynamics and their contribution to higher cognitive function., Author Summary One of the most noteworthy properties of primate behavior is its diversity and adaptability. Human and non-human primates can learn an astonishing variety of novel behaviors that could not have been directly anticipated by evolution. How then can the nervous system be prewired to anticipate the ability to represent such an open class of behaviors? Recent developments in a branch of recurrent neural networks, referred to as reservoir computing, begins to shed light on this question. The novelty of reservoir computing is that the recurrent connections in the network are fixed, and only the connections from these neurons to the output neurons change with learning. The fixed recurrent connections provide the network with an inherent high dimensional dynamics that creates essentially all possible spatial and temporal combinations of the inputs which can then be selected, by learning, to perform the desired task. This high dimensional mixture of activity inherent to reservoirs has begun to be found in the primate cortex. Here we make direct comparisons between dynamic coding in the cortex and in reservoirs performing the same task, and contribute to the emerging evidence that cortex has significant reservoir properties.

In this study, we investigated the neural substrates involved in visual working memory (WM) and the resulting effects of subthalamic nucleus (STN) stimulation in Parkinson's disease (PD). Cerebral activation revealed by positron emission tomography was compared among Parkinson patients with (PD-ON) or without (PD-OFF) STN stimulation, and a group of control subjects (CT) in two visual WM tasks with spatial (SP) and nonspatial (NSP) components. PD-OFF patients displayed significant reaction time (RT) deficits for both memory tasks. Although there were no significant differences in RT between patients with PD-ON and -OFF stimulation, patients with PD-ON stimulation performed comparably to controls. The memory tasks were executed with normal error rates in PD-ON and -OFF stimulation. In contrast to these behavioral results, whether the corresponding prefrontal activation was differentially affected by deep brain stimulation status in patients depended on whether the WM modality was SP versus NSP. Thus, SP WM was associated with (1) abnormal reduction in dorsolateral prefrontal activity in PD-OFF and -ON stimulation and (2) abnormal overactivation in parieto-temporal cortex in PD-OFF and in limbic circuits in PD-ON stimulation. In NSP WM, normal activation of the ventral prefrontal cortex was restored in PD-ON stimulation. In both visual modalities the posterior cerebral regions including fusiform cortex and cerebellum, displayed abnormally reduced activity in PD. These results indicate that PD induces a prefrontal hypoactivation that STN stimulation can partially restore in a modality selective manner by additional recruitment of limbic structures in SP WM or by recovery of the ventral prefrontal activation in NSP WM.

International audience; Categorical encoding is crucial for mastering large bodies of related sensory-motor experiences, but what is its neural substrate? In an effort to respond to this question, recent single-unit recording studies in the macaque lateral prefrontal cortex (LPFC) have demonstrated two characteristic forms of neural encoding of the sequential structure of the animal's sensory-motor experience. One population of neurons encodes the specific behavioral sequences. A second population of neurons encodes the sequence category (e.g. ABAB, AABB or AAAA) and does not differentiate sequences within the category (Shima, K., Isoda, M., Mushiake, H., Tanji, J., 2007. Categorization of behavioural sequences in the prefrontal cortex. Nature 445, 315-318.). Interestingly these neurons are intermingled in the lateral prefrontal cortex, and not topographically segregated. Thus, LPFC may provide a neurophysiological basis for sensorimotor catego-rization. Here we report on a neural network simulation study that reproduces and explains these results. We model a cortical circuit composed of three layers (infragranular, granular, and supragranular) of 5 x 5 leaky integrator neurons with a sigmoidal output function, and we examine 1000 such circuits running in parallel. Crucially the three layers are interconnected with recurrent connections, thus producing a dynamical system that is inherently sensitive to the spatiotemporal structure of the sequential inputs. The model is presented with 11 four-element sequences following Shima et al. We isolated one subpop-ulation of neurons each of whose activity predicts individual sequences, and a second population that predicts category independent of the specific sequence. We argue that a richly interconnected cortical circuit is capable of internally generating a neural representation of category membership, thus significantly extending the scope of recurrent network computation. In order to demonstrate that these representations can be used to create an explicit categorization capability, we introduced an additional neural structure corresponding to the striatum. We showed that via cortico-striatal plasticity, neurons in the striatum could produce an explicit representation both of the identity of each sequence, and its category membership.

International audience; With this research we investigated the real-time electrophysiological correlates of noun-verb agreement checking during the comprehension of correct passive sentences in French. Event-related potentials were acquired while participants read passive sentences that contained covert (singular, masculine) or overt (plural, feminine) noun-verb agreement. Results show that the processing of overtly or covertly agreeing verbs in passive sentences is associated with an asymmetrical electrophysiological response, reflecting former psycholinguistic evidence of markedness and unmarkedness of certain features. The reading of an overtly marked verb agreeing in number and gender with a feminine plural subject was associated with a left anterior negativity (LAN), whereas covertly marked verbs were associated with a negativity presenting a central-posterior distribution, an N400. These results, confirming the lexical status of features and their immediate but asymmetrical checking during sentence comprehension are discussed in the context of current linguistic and psycholinguistic models of agreement checking. (C) 2007 Elsevier Ltd. All rights reserved.

International audience; To explain the high level of flexibility in primate decision-making, theoretical models often invoke reinforcement-based mechanisms, performance monitoring functions, and core neural features within frontal cortical regions. However, the underlying biological mechanisms remain unknown. In recent models, part of the regulation of behavioral control is based on meta-learning principles, e.g. driving exploratory actions by varying a meta-parameter, the inverse temperature, which regulates the contrast between competing action probabilities. Here we investigate how complementary processes between lateral prefrontal cortex (LPFC) and dorsal anterior cingulate cortex (dACC) implement decision regulation during exploratory and exploitative behaviors. Model-based analyses of unit activity recorded in these two areas in monkeys first revealed that adaptation of the decision function is reflected in a covariation between LPFC neural activity and the control level estimated from the animal's behavior. Second, dACC more prominently encoded a reflection of outcome uncertainty useful for control regulation based on task monitoring. Model-based analyses also revealed higher information integration before feedback in LPFC, and after feedback in dACC. Overall the data support a role of dACC in integrating reinforcement-based information to regulate decision functions in LPFC. Our results thus provide biological evidence on how prefrontal cortical subregions may cooperate to regulate decision-making.

International audience; Language production requires selection of the appropriate sentence structure to accommodate the communication goal of the speaker-the transmission of a particular meaning. Here we consider event meanings , in terms of predicates and thematic roles, and we address the problem that a given event can be described from multiple perspectives, which poses a problem of response selection. We present a model of response selection in sentence production that is inspired by the primate corticostriatal system. The model is implemented in the context of reservoir computing where the reservoir-a recurrent neural network with fixed connections-corresponds to cortex, and the readout corresponds to the striatum. We demonstrate robust learning, and generalization properties of the model, and demonstrate its cross linguistic capabilities in English and Japanese. The results contribute to the argument that the corticostriatal system plays a role in response selection in language production, and to the stance that reservoir computing is a valid potential model of corticostriatal processing.

The current research describes a functional trajectory from sensorimotor sequence learning to the learning of grammatical constructions in language. A brief review of the functional neurophysiology of the cortex and basal ganglia will be provided as background for a neural network model of this system in sensorimotor sequence learning. Sequential behavior is then defined in terms of serial, temporal and abstract structure. The resulting neuro-computational framework is demonstrated to account for observed sequence learning behavior. More interestingly, this framework naturally extends to grammatical constructions as form-to-meaning mappings. Predictions from the neuro-computational model concerning parallels in language and cognitive sequence processing are tested against behavioral and neurophysiological observations in humans, resulting in a refinement of the allocation of model functions to subdivisions of Broca's area. From a functional perspective this analysis will provide insight into the relation between the coding structure in human languages, and constraints derived from the underlying neurophysiological computational mechanisms.

International audience; Decoding neuronal information is important in neuroscience, both as a basic means to understand how neuronal activity is related to cerebral function and as a processing stage in driving neuroprosthetic effectors. Here, we compare the readout performance of six commonly used classifiers at decoding two different variables encoded by the spiking activity of the non-human primate frontal eye fields (FEF): the spatial position of a visual cue, and the instructed orientation of the animal's attention. While the first variable is exogenously driven by the environment, the second variable corresponds to the interpretation of the instruction conveyed by the cue; it is endogenously driven and corresponds to the output of internal cognitive operations performed on the visual attributes of the cue. These two variables were decoded using either a regularized optimal linear estimator in its explicit formulation, an optimal linear artificial neural network estimator, a non-linear artificial neural network estimator, a non-linear naïve Bayesian estimator, a non-linear Reservoir recurrent network classifier or a non-linear Support Vector Machine classifier. Our results suggest that endogenous information such as the orientation of attention can be decoded from the FEF with the same accuracy as exogenous visual information. All classifiers did not behave equally in the face of population size and heterogeneity, the available training and testing trials, the subject's behavior and the temporal structure of the variable of interest. In most situations, the regularized optimal linear estimator and the non-linear Support Vector Machine classifiers outperformed the other tested decoders.

Cooperation1 is at the core of human social life. In this context, two major challenges face research on humanrobot interaction: the first is to understand the underlying structure of cooperation, and the second is to build, based on this understanding, artificial agents that can successfully and safely interact with humans. Here we take a psychologically grounded and human-centered approach that addresses these two challenges. We test the hypothesis that optimal cooperation between a naive human and a robot requires that the robot can acquire and execute a joint plan, and that it communicates this joint plan through ecologically valid modalities including spoken language, gesture and gaze. We developed a cognitive system that comprises the human-like control of social actions, the ability to acquire and express shared plans and a spoken language stage. In order to test the psychological validity of our approach we tested 12 naive subjects in a cooperative task with the robot. We experimentally manipulated the presence of a joint plan (vs. a solo plan), the use of task-oriented gaze and gestures, and the use of language accompanying the unfolding plan. The quality of cooperation was analyzed in terms of proper turn taking, collisions and cognitive errors. Results showed that while successful turn taking could take place in the absence of the explicit use of a joint plan, its presence yielded significantly greater success. One advantage of the solo plan was that the robot would always be ready to generate actions, and could thus adapt if the human intervened at the wrong time, whereas in the joint plan the robot expected the human to take his/her turn. Interestingly, when the robot represented the action as involving a joint plan, gaze provided a highly potent nonverbal cue that facilitated successful collaboration and reduced errors in the absence of verbal communication. These results support the cooperative stance in human social cognition, and suggest that cooperative robots should employ joint plans, fully communicate them in order to sustain effective collaboration while being ready to adapt if the human makes a midstream mistake.

International audience; Sentence processing takes place in real-time. Previous words in the sentence can influence the processing of the current word in the timescale of hundreds of milliseconds. Recent neurophysiological studies in humans suggest that the fronto-striatal system (frontal cortex, and striatum-the major input locus of the basal ganglia) plays a crucial role in this process. The current research provides a possible explanation of how certain aspects of this real-time processing can occur, based on the dynamics of recurrent cortical networks, and plasticity in the cortico-striatal system. We simulate prefrontal area BA47 as a recurrent network that receives on-line input about word categories during sentence processing, with plastic connections between cortex and striatum. We exploit the homology between the cortico-striatal system and reservoir computing, where recurrent frontal cortical networks are the reservoir, and plastic cortico-striatal synapses are the readout. The system is trained on sentence-meaning pairs, where meaning is coded as activation in the striatum corresponding to the roles that different nouns and verbs play in the sentences. The model learns an extended set of grammatical constructions, and demonstrates the ability to generalize to novel constructions. It demonstrates how early in the sentence, a parallel set of predictions are made concerning the meaning, which are then confirmed or updated as the processing of the input sentence proceeds. It demonstrates how on-line responses to words are influenced by previous words in the sentence, and by previous sentences in the discourse, providing new insight into the neurophysiology of the P600 ERP scalp response to grammatical complexity. This demonstrates that a recurrent neural network can decode grammatical structure from sentences in real-time in order to generate a predictive representation of the meaning of the sentences. This can provide insight into the underlying mechanisms of human cortico-striatal function in sentence processing. Citation: Hinaut X, Dominey PF (2013) Real-Time Parallel Processing of Grammatical Structure in the Fronto-Striatal System: A Recurrent Network Simulation Study Using Reservoir Computing. PLoS ONE 8(2): e52946.

International audience; Converging evidence suggest that the medial prefrontal cortex (MPFC) is involved in feedback categorization, performance monitoring and task monitoring, and may contribute to the online regulation of reinforcement learning (RL) parameters that would affect decision-making processes in the lateral prefrontal cortex (LPFC). Previous neurophysiological experiments have shown MPFC activities encoding error likelihood, uncertainty, reward volatility as well as neural responses categorizing different types of feedback, for instance distinguishing between choice errors and execution errors. Rushworth and colleagues have proposed that the involvement of MPFC in tracking the volatility of the task could contribute to the regulation of one of RL parameters called the learning rate. We extend this hypothesis by proposing that MPFC could contribute to the regulation of other RL parameters such as the exploration rate and default action values in case of task shifts. Here we analyze the sensitivity to RL parameters of behavioral performance in two monkey decision-making tasks, one with a deterministic reward schedule and the other with a stochastic one. We show that there exist optimal parameters values specific to each of these tasks, that need to be found for optimal performance and that are usually hand-tuned in computational models. In contrast, automatic online regulation of these parameters using some heuristics can help producing a good, although non-optimal, behavioral performance in each task. We finally describe our computational model of MPFC-LPFC interaction used for online regulation of the exploration rate and its application to a human-robot interaction scenario. There, unexpected uncertainties are introduced by the human introducing cued task changes or by cheating. The model enables the robot to autonomously learn to reset exploration in response to such uncertain cues and events. The combined results provide concrete evidence specifying how prefrontal cortical subregions may cooperate to regulate RL parameters. It also shows how such neurophysiologically inspired mechanisms can control advanced robots in the real-world. Finally, the model's learning mechanisms that were challenged in the last robotic scenario provide testable predictions on the way monkeys may learn the structure of the task during the pre-training phase of the previous laboratory experiments.

As there is “dark matter” in the neuroscience of individuals engaged in dynamic interactions, similar dark matter is present in the domain of interaction between humans and cognitive robots. Progress in second-person neuroscience will contribute to the development of robotic cognitive systems, and such developed robotic systems will be used to test the validity of the underlying theories.

Understanding the world involves extracting the regularities that define the interaction of the behaving organism within this world, and computing the statistical structure characterizing these regularities. This can be based on contingencies of phenomena at various scales ranging from correlations between sensory signals (e.g., motor-proprioceptive loops) to high-level conceptual links (e.g., vocabulary grounding). Multiple cortical areas contain neurons whose receptive fields are tuned for signals co-occurring in multiple modalities. Moreover, the hierarchical organization of the cortex, described within the Convergence Divergence Zone framework, defines an ideal architecture to extract and make use of contingency at increasing levels of complexity. We present an artificial neural network model of the early cortical amodal computations, which we have demonstrated on the humanoid robot iCub. This model explains and predicts findings in neurophysiology and neuropsychology along with being an efficient tool to control the robot. In particular, through exploratory use of the body, the system learns a form of body schema in terms of specific modalities (e.g., arm proprioception, gaze proprioception, vision) and their multimodal contingencies. Once multimodal contingencies have been learned, the system is capable of generating and exploiting internal representations or mental images based on inputs in one of these multiple dimensions. The system thus provides insight on a possible neural substrate for mental imagery within the context of multimodal convergence.

International audience; We examined seven right-handed, asymmetrical (right side affected) Parkinson's disease patients and seven age-matched controls in a manual finger sequencing test using left and right hands in vision, no vision, and motor imagery conditions. All patients displayed motor asymmetry, favoring the left hand. They also displayed motor imagery asymmetry, mentally simulating movement more slowly with their right affected hand than with their left hand. Additionally, impairment in mental hand rotation correlated significantly with the imagery asymmetry. These data support two related hypotheses: (a) Motor sequence imagery and execution share common neural structures. (b) The frontostriatal system is among these shared structures.

International audience; Human-human interaction in natural environments relies on a variety of perceptual cues. Humanoid robots are becoming increasingly refined in their sensorimotor capabilities, and thus should now be able to manipulate and exploit these social cues in cooperation with their human partners. Previous studies have demonstrated that people follow human and robot gaze, and that it can help them to cope with spatially ambiguous language. Our goal is to extend these findings into the domain of action, to determine how human and robot gaze can influence the speed and accuracy of human action. We report on results from a human-human cooperation experiment demonstrating that an agent's vision of her/his partner's gaze can significantly improve that agent's performance in a cooperative task. We then implement a heuristic capability to generate such gaze cues by a humanoid robot that engages in the same cooperative interaction. The subsequent human-robot experiments demonstrate that a human agent can indeed exploit the predictive gaze of their robot partner in a cooperative task. This allows us to render the humanoid robot more human-like in its ability to communicate with humans. The long term objectives of the work are thus to identify social cooperation cues, and to validate their pertinence through implementation in a cooperative robot. The current research provides the robot with the capability to produce appropriate speech and gaze cues in the context of human-robot cooperation tasks. Gaze is manipulated in three conditions: Full gaze (coordinated eye and head), eyes hidden with sunglasses, and head-fixed. We demonstrate the pertinence of these cues in terms of statistical measures of action times for humans in the context of a cooperative task, as gaze significantly facilitates cooperation as measured by human response times.

International audience

International audience; A major challenge in modern robotics is to liberate robots from controlled industrial settings, and allow them to interact with humans and changing environments in the real world. The current research attempts to determine if a neurophysiologically motivated model of cortical function in the primate can help to address this challenge. Primates are endowed with cognitive systems that allow them to maximize the feedback from their environment by learning the values of actions in diverse situations and by adjusting their behavioral parameters (i.e. cognitive control) to accommodate unexpected events. In such contexts uncertainty can arise from at least two distinct sources - expected uncertainty resulting from noise during sensory-motor interaction in a known context, and unexpected uncertainty resulting from the changing probabilistic structure of the environment. However, it is not clear how neurophysiological mechanisms of reinforcement learning and cognitive control integrate in the brain to produce efficient behavior. Based on primate neuroanatomy and neurophysiology, we propose a novel computational model for the interaction between lateral prefrontal and anterior cingulate cortex (LPFC and ACC) reconciling previous models dedicated to these two functions. We deployed the model in two robots and demonstrate that, based on adaptive regulation of a meta-parameter β that controls the exploration rate, the model can robustly deal with the two kinds of uncertainties in the real world. In addition the model could reproduce monkey behavioral performance and neurophysiological data in two problem-solving tasks. A last experiment extends this to human-robot interaction with the iCub humanoid, and novel sources of uncertainty corresponding to "cheating" by the human. The combined results provide concrete evidence for the ability of neurophysiologically inspired cognitive systems to control advanced robots in the real world.

The current research extends our framework for embodied language and action comprehension to include a teleological representation that allows goal-based reasoning for novel actions. The objective of this work is to implement and demonstrate the advantages of a hybrid, embodied-teleological approach to action-language interaction, both from a theoretical perspective, and via results from human-robot interaction experiments with the iCub robot. We first demonstrate how a framework for embodied language comprehension allows the system to develop a baseline set of representations for processing goal-directed actions such as “take”, “cover”, and “give”. Spoken language and visual perception are input modes for these representations, and the generation of spoken language is the output mode. Moving towards a teleological (goal-based reasoning) approach, a crucial component of the new system is the representation of the subcomponents of these actions, which includes relations between initial enabling states, and final resulting states for these actions. We demonstrate how grammatical categories including causal connectives (e.g. because, if-then) can allow spoken language to enrich the learned set of state-action-state (SAS) representations. We then examine how this enriched SAS inventory enhances the robot’s ability to represent perceived actions in which the environment inhibits goal achievement. The paper addresses how language comes to reflect the structure of action, and how it can subsequently be used as an input and output vector for embodied and teleological aspects of action.

Taking inspiration from neural principles of decision-making is of particular interest to help improve adaptivity of artificial systems. Research at the crossroads of neuroscience and artificial intelligence in the last decade has helped understanding how the brain organizes reinforcement learning (RL) processes (the adaptation of decisions based on feedback from the environment). The current challenge is now to understand how the brain flexibly regulates parameters of RL such as the exploration rate based on the task structure, which is called meta-learning ([1]: Doya, 2002). Here, we propose a computational mechanism of exploration regulation based on real neurophysiological and behavioral data recorded in monkey prefrontal cortex during a visuo-motor task involving a clear distinction between exploratory and exploitative actions. We first fit trial-by-trial choices made by the monkeys with an analytical reinforcement learning model. We find that the model which has the highest likelihood of predicting monkeys' choices reveals different exploration rates at different task phases. In addition, the optimized model has a very high learning rate, and a reset of action values associated to a cue used in the task to signal condition changes. Beyond classical RL mechanisms, these results suggest that the monkey brain extracted task regularities to tune learning parameters in a task-appropriate way. We finally use these principles to develop a neural network model extending a previous cortico-striatal loop model. In our prefrontal cortex component, prediction error signals are extracted to produce feedback categorization signals. The latter are used to boost exploration after errors, and to attenuate it during exploitation, ensuring a lock on the currently rewarded choice. This model performs the task like monkeys, and provides a set of experimental predictions to be tested by future neurophysiological recordings.

International audience; Human interaction in natural environments relies on a variety of perceptual cues to guide and stabilize the interaction. Humanoid robots are becoming increasingly refined in their sensorimotor capabilities, and thus should be able to manipulate and exploit these communicative cues in cooperation with their human partners. In the current research we identify a set of principal communicative speech and gaze cues in human-human interaction, and then formalize and implement these cues in a humanoid robot. The objective of the work is to render the humanoid robot more human-like in its ability to communicate with humans. The first phase of this research, described here, is to provide the robot with a generative capability - that is to produce appropriate speech and gaze cues in the context of human-robot cooperation tasks. . We demonstrate the pertinence of these cues in terms of statistical measures of action times for humans in the context of a cooperative task, as gaze significantly facilitates cooperation as measured by human response times.

International audience; The determining of brain regions that exhibit specific activity during sentence comprehension compared to other non-linguistic cognitive tasks constitutes one of the important challenges in the domain of functional neuroimaging of the faculty of language. In the current paper we report an event-related functional magnetic resonance imaging (ER-fMRI) experiment, in which we directly compared the cerebral basis of sentence comprehension on the one hand, and of abstract sequence processing on the other hand. Previous experimental work done in our group, as well as different observations from recent behavioural, neurophysiological and functional neuroimaging experiments led us to propose the hypothesis that both of these tasks would share certain computational properties. Thus, this experiment was designed to show which brain regions would be implicated in both tasks and compare them to brain regions that would be specifically engaged in sentence comprehension. Results from this experiment suggest that distinct sub-regions in the left prefrontal cortex, potentially including Broca's area show distinct activation patterns during both of these tasks. Results are discussed in the context of a construction-based model of sentence processing (see Dominey and Hoen, 2006, this issue) that is based on a dual-path processing mechanism separating function and content information processing. We propose and discuss the hypothesis that subparts of Broca's area BA 44 and BA 45 would respectively be implicated in two different aspects of sentence comprehension: i) a general structure mapping capability and ii) the online integration of semantic representations onto structural constraints.

International audience; In the context of language processing, the N280 is an anterior negative event-related potential profile associated with the lexical categorization of grammatical function words versus content words. Subsequent studies suggested that this effect was related to word statistics including length and frequency in the lexicon. The current research tests the hypothesis that the N280 effect is related to an index of grammatical complexity. We recorded event-related potentials during a sentence reading task. Comparing content versus function words revealed the classic N280. Within function words, we compared the relative pronouns 'qui' and 'que' (which are identical for length and frequency) that in French indicate a subsequent simple (subject-subject) and complex (subject-object) relative clause, respectively. A left anterior N280 effect was observed only for 'que', supporting our hypothesis that the N280 reflects grammatical complexity that can be confounded with lexical category and statistical properties.

Connectionist networks have been criticized for their inability to represent complex structures with systematicity. That is, while they can be trained to represent and manipulate complex objects made of several constituents, they generally fail to generalize to novel combinations of the same constituents. This paper presents a modification of Pollack's Recursive Auto-Associative Memory (RAAM), that addresses this criticism. The network uses linear units and is trained with Oja's rule, in which it generalizes PCA to tree-structured data. Learned representations may be linearly combined, in order to represent new complex structures. This results in unprecedented generalization capabilities. Capacity is orders of magnitude higher than that of a RAAM trained with back-propagation. Moreover, regularities of the training set are preserved in the new formed objects. The formation of new structures displays developmental effects similar to those observed in children when learning to generalize about the argument structure of verbs.

A method is proposed where static patterns or snapshots of cortical activity that could be stored as hyperassociative indices in hippocampus can subsequently be retrieved and reinjected into the neocortex in order to enable neocortex to then proceed to unfold the corresponding sequence, thus implementing an index-based sequence memory storage and retrieval capability.

A sensorimotor sequence may contain information structure at several different levels. In this study, we investigated the hypothesis that two dissociable processes are required for the learning of surface structure and abstract structure, respectively, of sensorimotor sequences. Surface structure is the simple serial order of the sequence elements, whereas abstract structure is defined by relationships between repeating sequence elements. Thus, sequences ABCBAC and DEFEDF have different surface structures but share a common abstract structure, 123213, and are therefore isomorphic. Our simulations of sequence learning performance in serial reaction time (SRT) tasks demonstrated that (1) an existing model of the primate fronto-striatal system is capable of learning surface structure but fails to learn abstract structure, which requires an additional capability, (2) surface and abstract structure can be learned independently by these independent processes, and (3) only abstract structure transfers to isomorphic sequences. We tested these predictions in human subjects. For a sequence with predictable surface and abstract structure, subjects in either explicit or implicit conditions learn the surface structure, but only explicit subjects learn and transfer the abstract structure. For sequences with only abstract structure, learning and transfer of this structure occurs only in the explicit group. These results are parallel to those from the simulations and support our dissociable process hypothesis. Based on the synthesis of the current simulation and empirical results with our previous neuropsychological findings, we propose a neuro-physiological basis for these dissociable processes: Surface structure can be learned by processes that operate under implicit conditions and rely on the fronto-striatal system, whereas learning abstract structure requires a more explicit activation of dissociable processes that rely on a distributed network that includes the left anterior cortex.