Search

Brain research has in recent years indisputably entered a new epoch, driven bysubstantialmethodologicaladvances and digitallyenabled data integration and modeling at multiple scales –from molecules to the whole system. Major advances are emerging at theintersection of neurosciencewith technology and computing. This new science of the brain integrates high-quality basic research,systematic data integration across multiple scales, a new culture of large-scale collaboration andtranslation into applications. A systematic approach, as pioneered in Europe’s Human Brain Project(HBP), will be essential in meeting the pressing medical andtechnological challenges of the comingdecade. The aims of this paper are To develop a concept for the coming decade of digital brain research To discuss it with the research community at large, with the aim of identifying points of convergence and common goals To provide a scientific framework for current and future development of EBRAINS To inform and engage stakeholders, funding organizations and research institutions regarding future digital brain research To identify and address key ethical and societal issues While we do not claim that there is a ‘one size fits all’ approach to addressing these aspects, we areconvinced that discussions around the theme of digital brain research will help drive progress in thebroader field of neuroscience. You can submit your comments here and be considered for co-authorship Or become a supporter by clicking here More information below Comments on this manuscript are welcome This manuscriptis a living document that is being further developed in a participatory process. The work has been initiated by the Science and Infrastructure Board of the Human Brain Project (HBP). Now, the entire research community is invited to contribute to shaping the vision by submitting comments.Comments can be submitted via an online commentary form here. All submitted comments will be considered and discussed. The final decision on whether edits or additions will be made to the next version of the manuscript based on an individual comment will be made by the Science and Infrastructure Board (SIB) of the Human Brain Project (HBP) at regular intervals. The final version of the manuscript will be prepared and published in the next months. Comments may be submitted until the end of May, 2023. Every researcher is welcome to provide comment or indicate support for the contents of this paper and jointly shape the future of brain research. Supporters of the paper: Pietro Avanzini, Marc Beyer, Maria Del Vecchio, Jitka Annen, Maurizio Mattia, Steven Laureys, Rosanne Edelenbosch, Rafael Yuste, Jean-Pierre Changeux, Linda Richards, Hye Weon Jessica Kim, Chrysoula Samara, Luis Miguel González de la Garza, Nikoleta Petalidou, Vasudha Kulkarni, Cesar David Rincon, Isabella O'Shea, Munira Tamim Electricwala, Nicola Palomero-Gallagher, Bernd Carsten Stahl, Bahar Hazal Yalcinkaya, Meysam Hashemi, Carola Sales Carbonell, Marcel Carrère, Anthony Randal McIntosh, Hiba Sheheitli, Abolfazl Ziaeemehr, Martin Breyton, Giovanna Ramos Queda, Anirudh NIhalani Vattikonda, Gyorgy Buzsaki George Ogoh, William Knight Torbjørn V Ness, MIchiel van der Vlag, Mu-ming Poo.

Creating autonomous robots that can actively explore the environment, acquire knowledge and learn skills continuously is the ultimate achievement envisioned in cognitive and developmental robotics. Their learning processes should be based on interactions with their physical and social world in the manner of human learning and cognitive development. Based on this context, in this paper, we focus on the two concepts of world models and predictive coding. Recently, world models have attracted renewed attention as a topic of considerable interest in artificial intelligence. Cognitive systems learn world models to better predict future sensory observations and optimize their policies, i.e., controllers. Alternatively, in neuroscience, predictive coding proposes that the brain continuously predicts its inputs and adapts to model its own dynamics and control behavior in its environment. Both ideas may be considered as underpinning the cognitive development of robots and humans capable of continual or lifelong learning. Although many studies have been conducted on predictive coding in cognitive robotics and neurorobotics, the relationship between world model-based approaches in AI and predictive coding in robotics has rarely been discussed. Therefore, in this paper, we clarify the definitions, relationships, and status of current research on these topics, as well as missing pieces of world models and predictive coding in conjunction with crucially related concepts such as the free-energy principle and active inference in the context of cognitive and developmental robotics. Furthermore, we outline the frontiers and challenges involved in world models and predictive coding toward the further integration of AI and robotics, as well as the creation of robots with real cognitive and developmental capabilities in the future., Comment: 28 pages, 3 figures

We advance a novel computational model of multi-agent, cooperative joint actions that is grounded in the cognitive framework of active inference. The model assumes that to solve a joint task, such as pressing together a red or blue button, two (or more) agents engage in a process of interactive inference. Each agent maintains probabilistic beliefs about the goal of the joint task (e.g., should we press the red or blue button?) and updates them by observing the other agent's movements, while in turn selecting movements that make his own intentions legible and easy to infer by the other agent (i.e., sensorimotor communication). Over time, the interactive inference aligns both the beliefs and the behavioral strategies of the agents, hence ensuring the success of the joint action. We exemplify the functioning of the model in two simulations. The first simulation illustrates a ''leaderless'' joint action. It shows that when two agents lack a strong preference about their joint task goal, they jointly infer it by observing each other's movements. In turn, this helps the interactive alignment of their beliefs and behavioral strategies. The second simulation illustrates a "leader-follower" joint action. It shows that when one agent ("leader") knows the true joint goal, it uses sensorimotor communication to help the other agent ("follower") infer it, even if doing this requires selecting a more costly individual plan. These simulations illustrate that interactive inference supports successful multi-agent joint actions and reproduces key cognitive and behavioral dynamics of "leaderless" and "leader-follower" joint actions observed in human-human experiments. In sum, interactive inference provides a cognitively inspired, formal framework to realize cooperative joint actions and consensus in multi-agent systems., 32 pages, 16 figures

Processing faces accurately and efficiently is a key capability of humans and other animals that engage in sophisticated social tasks. Recent studies reported a decoupled coding for faces in the primate inferotemporal cortex, with two separate neural populations coding for the geometric position of (texture-free) facial landmarks and for the image texture at fixed landmark positions, respectively. Here, we formally assess the efficiency of this decoupled coding by appealing to the information-theoretic notion of description length, which quantifies the amount of information that is saved when encoding novel facial images, with a given precision. We show that despite decoupled coding describes the facial images in terms of two sets of principal components (of landmark shape and image texture), it is more efficient (i.e., yields more information compression) than the encoding in terms of the image principal components only, which corresponds to the widely used eigenface method. The advantage of decoupled coding over eigenface coding increases with image resolution and is especially prominent when coding variants of training set images that only differ in facial expressions. Moreover, we demonstrate that decoupled coding entails better performance in three different tasks: the representation of facial images, the (daydream) sampling of novel facial images, and the recognition of facial identities and gender. In summary, our study provides a first principle perspective on the efficiency and accuracy of the decoupled coding of facial stimuli reported in the primate inferotemporal cortex., 26 pages, 8 figures (+11 pages, 7 figures in the supporting information section). In v3: new figure 8 in section 3.2.3; further details added to the supporting information; title changed

Emotional concepts and their mental representations have been extensively studied. Yet, some ecologically relevant aspects, such as how they are processed in ambiguous contexts (e.g., in relation to other emotional stimuli that share similar characteristics), are incompletely known. We employed a similarity judgement of emotional concepts and manipulated the contextual congruency of the responses along the two main affective dimensions of hedonic valence and physiological activation, respectively. Behavioural and kinematics (mouse-tracking) measures were combined to gather a novel ‘similarity index’ between emotional concepts, to derive topographical maps of their mental representations. Self-report (interoceptive sensibility, positive–negative affectivity, depression) and physiological measures (heart rate variability, HRV) have been collected to explore their possible association with emotional conceptual representation. Results indicate that emotional concepts typically associated with low arousal profit by contextual congruency, with faster responses and reduced uncertainty when contextual ambiguity decreases. The emotional maps recreate two almost orthogonal axes of valence and arousal, and the similarity measure captures the smooth boundaries between emotions. The emotional map of a subgroup of individuals with low positive affectivity reveals a narrower conceptual distribution, with variations in positive emotions and in individuals with reduced arousal (such as those individuals with reduced HRV). Our work introduces a novel methodology to study emotional conceptual representations, bringing the behavioural dynamics of decision-making processes and choice uncertainty into the affective domain.This article is part of the theme issue ‘Concepts in interaction: social engagement and inner experiences’.

Decades of research has shown that people rate things more favorably if they have recently chosen them and less favorably if they have recently rejected them. This spreading apart of the ratings of the alternative options (“spreading of alternatives” or SoA) has been described as choice-induced preference change, where the act of choosing causes evaluations to systematically differ after choices relative to before. An important study pointed out that SoA can be detected in the data even without it being caused by a choice, as the result of a statistical artifact. This potential alternative source of SoA has encumbered researchers who investigate the phenomenon, impeding progress in the field field – and a deeper investigation of the cognitive mechanisms behind SoA. Here, we present a novel approach to control for the statistical artifact explanation through the use of regression analysis supported by computational simulations. Our approach provides a logical and computational manner of distinguishing the possible sources of SoA in experimental data that is quick and easy to implement. We demonstrate our approach by comparing theoretical predictions of the statistical artifact and other possible explanations of SoA with empirical data, and show that the statistical artifact alone cannot account for the data.

Decision-making is traditionally described as a cognitive process of deliberation followed by commitment to an action choice, preceding the planning and execution of the chosen action. However, this is challenged by recent data suggesting that during situated decisions, multiple options are specified simultaneously and compete in premotor cortical areas for selection and execution. Previous studies focused on the competition during planning and left unaddressed the dynamics of decisions during movement. Does deliberation extend into the execution phase? Are nonselected options still considered? Here we studied a decision-making task in which human participants were instructed to select a reaching path trajectory from an origin to a rectangular target, where reward was distributed nonuniformly at the target. Critically, we applied mechanical perturbations to the arm during movement to study under which conditions such perturbations produce changes of mind. Our results show that participants initially selected the direction of movement toward the highest reward region and changed their mind most frequently when the two choices offered the same reward, showing that deliberation continues and follows cost–benefit considerations during movement. Furthermore, changes of mind were dependent on the intensity of the perturbation and the current state of the motor system, including velocity and distance to targets. Although reward remains most relevant, our results indicate tthat the state of the motor system when the perturbation occurs is a crucial determinant of changes of mind.

Contains fulltext : 250990.pdf (Publisher’s version ) (Open Access) The field of motor control has long focused on the achievement of external goals through action (e.g., reaching and grasping objects). However, recent studies in conditions of multisensory conflict, such as when a subject experiences the rubber hand illusion or embodies an avatar in virtual reality, reveal the presence of unconscious movements that are not goal-directed, but rather aim at resolving multisensory conflicts; for example, by aligning the position of a person's arm with that of an embodied avatar. This second, conflict-resolution imperative of movement control did not emerge in classical studies of motor adaptation and online corrections, which did not allow movements to reduce the conflicts; and has been largely ignored so far in formal theories. Here, we propose a model of movement control grounded in the theory of active inference that integrates intentional and conflict-resolution imperatives. We present three simulations showing that the active inference model is able to characterize movements guided by the intention to achieve an external goal, by the necessity to resolve multisensory conflict, or both. Furthermore, our simulations reveal a fundamental difference between the (active) inference underlying intentional and conflict-resolution imperatives by showing that it is driven by two different (model and sensory) kinds of prediction errors. Finally, our simulations show that when movement is only guided by conflict resolution, the model incorrectly infers that is velocity is zero, as if it was not moving. This result suggests a novel speculative explanation for the fact that people are unaware of their subtle compensatory movements to avoid multisensory conflict. Furthermore, it can potentially help shed light on deficits of motor awareness that arise in psychopathological conditions. 23 december 2021 29 p.

The ability to plan ahead efficiently is key for both living organisms and artificial systems. Model-based planning and prospection are widely studied in cognitive neuroscience and artificial intelligence (AI), but from different perspectives--and with different desiderata in mind (biological realism versus scalability) that are difficult to reconcile. Here, we introduce a novel method to plan in POMDPs--Active Inference Tree Search (AcT)--that combines the normative character and biological realism of a leading planning theory in neuroscience (Active Inference) and the scalability of tree search methods in AI. This unification enhances both approaches. On the one hand, tree searches enable the biologically grounded, first principle method of active inference to be applied to large-scale problems. On the other hand, active inference provides a principled solution to the exploration-exploitation dilemma, which is often addressed heuristically in tree search methods. Our simulations show that AcT successfully navigates binary trees that are challenging for sampling-based methods, problems that require adaptive exploration, and the large POMDP problem 'RockSample'--in which AcT reproduces state-of-the-art POMDP solutions. Furthermore, we illustrate how AcT can be used to simulate neurophysiological responses (e.g., in the hippocampus and prefrontal cortex) of humans and other animals that solve large planning problems. These numerical analyses show that Active Tree Search is a principled realisation of neuroscientific and AI planning theories, which offer both biological realism and scalability., 50 pages, 9 figures, 1 Appendix of two sections with pseudocodes and one encoding example , submitted preprint

The mastery of skills such as playing tennis or balancing an inverted pendulum implies a very accurate control of movements to achieve the task goals. Traditional accounts of skilled action control that focus on either routinization or perceptual control make opposite predictions about the ways we achieve mastery. The notion of routinization emphasizes the decrease of the variance of our actions, whereas the notion of perceptual control emphasizes the decrease of the variance of the states we visit, but not of the actions we execute. Here, we studied how participants managed control tasks of varying levels of complexity, which consisted in controlling inverted pendulums of different lengths. We used information-theoretic measures to compare the predictions of alternative theoretic accounts that focus on routinization and perceptual control, respectively. Our results indicate that the successful performance of the control task strongly correlates with the decrease of state variability and the increase of action variability. As postulated by perceptual control theory, the mastery of skills consists in achieving stable control goals by flexible means., Comment: 15 pages, 7 figures

We discuss the relevance of modelling the sense of should or normative pressure as a social prediction error minimization strategy, its role in the creation of spontaneous order and its intrinsic fragility as an epistemic motivation.

The moral decisions we make during this period, such as deciding whether to comply with quarantine rules, have unprecedented societal effects. We simulate the "escape from Milan" that occurred on March 7th-8th 2020, when many travelers moved from a high-risk zone (Milan) to southern regions of Italy (Campania and Lazio) immediately after an imminent lockdown was announced. Our simulations show that fewer than 50 active cases might have caused the sudden spread of the virus observed afterwards in these regions. The surprising influence of the actions of few individuals on societal dynamics challenges our cognitive expectations -- as in normal conditions, collective dynamics are rather robust to the decisions of few "cheaters". This situation therefore requires novel educational strategies that increase our awareness and understanding of the unprecedented effects of our individual moral decisions., Comment: Published in "Social Sciences & Humanities Open" journal (Elsevier)

This paper presents a spatial navigation task on a mobile robot by employing a deep reinforcement learning algorithm (DQN) in the Neurorobotics Platform (NRP). The navigation task carried out with a Husky robot equipped with camera and laser sensors. The robot employs sensory readings and position information to select an action and detect a target. The preliminary results show that the NRP can be considered as a realistic virtual environment for robotic agents to perform a navigation task.

Highlights • An Active Inference account of homeostatic regulation and behavioural control. • Pavlovian, habitual and goal-directed behaviours explained with one scheme. • A possible phylogenetic trajectory from simpler to hierarchical controllers. • Precision-dependent processes regulate habitual and goal-directed behaviour., We review a theory of homeostatic regulation and adaptive behavioural control within the Active Inference framework. Our aim is to connect two research streams that are usually considered independently; namely, Active Inference and associative learning theories of animal behaviour. The former uses a probabilistic (Bayesian) formulation of perception and action, while the latter calls on multiple (Pavlovian, habitual, goal-directed) processes for homeostatic and behavioural control. We offer a synthesis these classical processes and cast them as successive hierarchical contextualisations of sensorimotor constructs, using the generative models that underpin Active Inference. This dissolves any apparent mechanistic distinction between the optimization processes that mediate classical control or learning. Furthermore, we generalize the scope of Active Inference by emphasizing interoceptive inference and homeostatic regulation. The ensuing homeostatic (or allostatic) perspective provides an intuitive explanation for how priors act as drives or goals to enslave action, and emphasises the embodied nature of inference.

International audience

Can PP (Predictive Processing) help us understand "the roots of cognition", and how we may have acquired (during evolution and/or development) our sophisticated cognitive abilities from the relatively simpler adaptive control mechanisms of our early evolutionary ancestors? Here I make the case that some cognitive operations may be constructed as detached actions - where the detachment process rests on the construction of generative PP models, which permit one to internalize action-environment dynamics. I provide three examples. The first example focuses on the role of internally-generated sequences of (hippocampal) neuronal activity across goal-directed navigation and detached tasks such as planning. This example illustrates how neuronal sequences (putatively forming an internal model for spatial navigation) may have a “dual use”: they may support both overt navigation and covert cognitive operations while running, respectively, in stimulus-based and internally-generated modes. Furthermore, the latter (internally-generated) mode may be considered a form of internalization of the former (stimulus-based) mode. The second example focuses on actions to resolve epistemic uncertainty, and the formal similarity in PP between epistemic actions that are executed overtly (e.g., exploration) and those executed covertly (e.g., episodic retrieval). This example illustrates the possibility of defining mental actions, such as reducing one’s uncertainty before making a choice, as internalized information foraging acts that have the same intentionality as externally-directed actions. The third example focuses on the detachment of cognitive goals such as “eating in a fancy restaurant” from homeostatic drives such as “being satiated”. This example illustrates that by internalizing regulatory loops within hierarchical PP models, one can build cognitive goals that can in turn enjoy some form of detachment – for example, one can go to a restaurant or buy food even when one is not hungry. I discuss these examples in relation to alternative theories of how higher cognition originates from (or is independent of) action-perception loops, including various versions of action-oriented, embodied and enactivist views., Philosophy and Predictive Processing

International audience; Numerous studies indicate that lexical decision is a dynamic process in which various sources of information (e.g., orthographic, phonological, lexical) need to be integrated over time and that mismatches between these sources produce uncertainty during decision-making. However, the specific contribution of these different sources of information to the decision process, and the specific ways they interact over time, are still incompletely understood. Here we report two experiments conducted to investigate the role of a key characteristic of language-orthographic consistency-in modulating reading strategies in a lexical decision task. We aimed to test different theories of how exactly orthographic consistency influences decisions, and the ways lexical and su-blexical representations are matched within it. For this, we exploited a key characteristic of the French language: the fact that retrieving spelling from phonology is hard, whereas the reverse is not the case (i.e., French is feedback-inconsistent but feedforward-consistent). Our results indicate a feedback inconsistency effect: uncertainty in our lexical decision task originates from the process of retrieving spelling from phonology, not when retrieving phonology from spelling. These results support the idea of a verification mechanism that compares orthographic sublexical information with lexical information during lexical decisions.

We present a novel computational model that describes action perception as an active inferential process that combines motor prediction (the reuse of our own motor system to predict perceived movements) and hypothesis testing (the use of eye movements to disambiguate amongst hypotheses). The system uses a generative model of how (arm and hand) actions are performed to generate hypothesis-specific visual predictions, and directs saccades to the most informative places of the visual scene to test these predictions - and underlying hypotheses. We test the model using eye movement data from a human action observation study. In both the human study and our model, saccades are proactive whenever context affords accurate action prediction; but uncertainty induces a more reactive gaze strategy, via tracking the observed movements. Our model offers a novel perspective on action observation that highlights its active nature based on prediction dynamics and hypothesis testing.

Controlling the body – given its huge number of degrees of freedom – poses severe computational challenges. Mounting evidence suggests that the brain alleviates this problem by exploiting “synergies”, or patterns of muscle activities (and/or movement dynamics and kinematics) that can be combined to control action, rather than controlling individual muscles of joints [1–10]. D’Ausilio et al. [11] explain how this view of motor organization based on synergies can profoundly change the way we interpret studies of action recognition in humans and monkeys, and in particular the controversy on the “granularity” of the mirror neuron system (MNs): whether it encodes either (lower) kinematic aspects of movements, or (higher) goal representations, or both but at different hierarchical levels [12]. Here we offer a complementary, computational perspective on the role of synergies for action recognition and the MNs. In computational modeling and robotics, it is widely assumed that a control scheme using synergies simplifies movement planning and execution. This scheme permits to use elemental behaviors or primitives as “building blocks” to be composed (e.g., combined linearly, sequenced) to produce more complex behaviors, thus controlling relatively few degrees of freedom [13–15]. Do synergies yield equivalent benefits for action recognition? To answer this question from a computational viewpoint, we frame the concept of synergies within generative architectures of action execution and recognition [16–20]. According to two leading theories of motor control, optimal feedback control [21] and active inference [22], the motor system can be conceptualized as a (hierarchical) generative model, which encodes a (probabilistic) mapping between “task goals” specified at a higher level (e.g., grasping a cup) and states of the “plant” to be controlled (i.e., (Less)

How do humans and other animals face novel problems for which predefined solutions are not available? Human problem solving links to flexible reasoning and inference rather than to slow trial-and-error learning. It has received considerable attention since the early days of cognitive science, giving rise to well known cognitive architectures such as SOAR and ACT-R, but its computational and brain mechanisms remain incompletely known. Furthermore, it is still unclear whether problem solving is a “specialized” domain or module of cognition, in the sense that it requires computations that are fundamentally different from those supporting perception and action systems. Here we advance a novel view of human problem solving as probabilistic inference with subgoaling. In this perspective, key insights from cognitive architectures are retained such as the importance of using subgoals to split problems into subproblems. However, here the underlying computations use probabilistic inference methods analogous to those that are increasingly popular in the study of perception and action systems. To test our model we focus on the widely used Tower of Hanoi (ToH) task, and show that our proposed method can reproduce characteristic idiosyncrasies of human problem solvers: their sensitivity to the “community structure” of the ToH and their difficulties in executing so-called “counterintuitive” movements. Our analysis reveals that subgoals have two key roles in probabilistic inference and problem solving. First, prior beliefs on (likely) useful subgoals carve the problem space and define an implicit metric for the problem at hand—a metric to which humans are sensitive. Second, subgoals are used as waypoints in the probabilistic problem solving inference and permit to find effective solutions that, when unavailable, lead to problem solving deficits. Our study thus suggests that a probabilistic inference scheme enhanced with subgoals provides a comprehensive framework to study problem solving and its deficits., Author Summary How humans solve challenging problems such as the Tower of Hanoi (ToH) or related puzzles is still largely unknown. Here we advance a computational model that uses the same probabilistic inference methods as those that are increasingly popular in the study of perception and action systems, thus making the point that problem solving does not need to be a specialized module or domain of cognition, but it can use the same computations underlying sensorimotor behavior. Crucially, we augment the probabilistic inference methods with subgoaling mechanisms that essentially permit to split the problem space into more manageable subparts, which are easier to solve. We show that our computational model can correctly reproduce important characteristics (and pitfalls) of human problem solving, including the sensitivity to the “community structure” of the ToH and the difficulty of executing so-called “counterintuitive” movements that require to (temporarily) move away from the final goal to successively achieve it.

Active inference is a general framework for perception and action that is gaining prominence in computational and systems neuroscience but is less known outside these fields. Here, we discuss a proof-of-principle implementation of the active inference scheme for the control or the 7-DoF arm of a (simulated) PR2 robot. By manipulating visual and proprioceptive noise levels, we show under which conditions robot control under the active inference scheme is accurate. Besides accurate control, our analysis of the internal system dynamics (e.g. the dynamics of the hidden states that are inferred during the inference) sheds light on key aspects of the framework such as the quintessentially multimodal nature of control and the differential roles of proprioception and vision. In the discussion, we consider the potential importance of being able to implement active inference in robots. In particular, we briefly review the opportunities for modelling psychophysiological phenomena such as sensory attenuation and related failures of gain control, of the sort seen in Parkinson's disease. We also consider the fundamental difference between active inference and optimal control formulations, showing that in the former the heavy lifting shifts from solving a dynamical inverse problem to creating deep forward or generative models with dynamics, whose attracting sets prescribe desired behaviours.

International audience; Despite decades of research, we lack a comprehensive framework to study and explain cognitive development. The emerging " paradigm " of action-based cognition implies that cognitive development is an active rather than a passive, automatic, and self-paced maturational process. Importantly, " active " refers to both sensorimotor activity (in the narrow sense) as well as to autonomous exploration (e.g., as found in active perception or active learning). How does this emphasis on action affect our understanding of cog-nitive development? Can an action-based approach provide a much-needed integrative theory of cognitive development? This chapter reviews key factors that infl uence development (including sensorimo-tor skills as well as genetic, social, and cultural factors) and their associated brain mechanisms. Discussion focuses on how these factors can be incorporated into a comprehensive action-based framework. Challenges are highlighted for future research (e.g., problems associated with explaining higher-level cognitive abilities and devising novel experimental methodologies). Although still in its infancy, an action-based approach to cognitive development holds promise to improve scientifi c understanding of cognitive development and to impact education and technology.

International audience

International audience

International audience

International audience

Understanding how organisms establish their form during embryogenesis and regeneration represents a major knowledge gap in biological pattern formation. It has been recently suggested that morphogenesis could be understood in terms of cellular information processing and the ability of cell groups to model shape. Here, we offer a proof of principle that self-assembly is an emergent property of cells that share a common (genetic and epigenetic) model of organismal form. This behaviour is formulated in terms of variational free-energy minimization-of the sort that has been used to explain action and perception in neuroscience. In brief, casting the minimization of thermodynamic free energy in terms of variational free energy allows one to interpret (the dynamics of) a system as inferring the causes of its inputs-and acting to resolve uncertainty about those causes. This novel perspective on the coordination of migration and differentiation of cells suggests an interpretation of genetic codes as parametrizing a generative model-predicting the signals sensed by cells in the target morphology-and epigenetic processes as the subsequent inversion of that model. This theoretical formulation may complement bottom-up strategies-that currently focus on molecular pathways-with (constructivist) top-down approaches that have proved themselves in neuroscience and cybernetics.

International audience

An important part of human intelligence, both historically and operationally, is our ability to communicate. We learn how to communicate, and maintain our communicative skills, in a society of communicators – a highly effective way to reach and maintain proficiency in this complex skill. Principles that might allow artificial agents to learn language this way are in completely known at present – the multi-dimensional nature of socio-communicative skills are beyond every machine learning framework so far proposed. Our work begins to address the challenge of proposing a way for observation-based machine learning of natural language and communication. Our framework can learn complex communicative skills with minimal up-front knowledge. The system learns by incrementally producing predictive models of causal relationships in observed data, guided by goal-inference and reasoning using forward-inverse models. We present results from two experiments where our S1 agent learns human communication by observing two humans interacting in a realtime TV-style interview, using multimodal communicative gesture and situated language to talk about recycling of various materials and objects. S1 can learn multimodal complex language and multimodal communicative acts, a vocabulary of 100 words forming natural sentences with relatively complex sentence structure, including manual deictic reference and anaphora. S1 is seeded only with high-level information about goals of the interviewer and interviewee, and a small ontology; no grammar or other information is provided to S1 a priori. The agent learns the pragmatics, semantics, and syntax of complex utterances spoken and gestures from scratch, by observing the humans compare and contrast the cost and pollution related to recycling aluminum cans, glass bottles, newspaper, plastic, and wood. After 20 hours of observation S1 can perform an unscripted TV interview with a human, in the same style, without making mistakes.

The article explores the possibilities of formalizing and explaining the mechanisms that support spatial and social perspective alignment sustained over the duration of a social interaction. The basic proposed principle is that in social contexts the mechanisms for sensorimotor transformations and multisensory integration (learn to) incorporate information relative to the other actor(s), similar to the "re-calibration" of visual receptive fields in response to repeated tool use. This process aligns or merges the co-actors' spatial representations and creates a "Shared Action Space" (SAS) supporting key computations of social interactions and joint actions; for example, the remapping between the coordinate systems and frames of reference of the co-actors, including perspective taking, the sensorimotor transformations required for lifting jointly an object, and the predictions of the sensory effects of such joint action. The social re-calibration is proposed to be based on common basis function maps (BFMs) and could constitute an optimal solution to sensorimotor transformation and multisensory integration in joint action or more in general social interaction contexts. However, certain situations such as discrepant postural and viewpoint alignment and associated differences in perspectives between the co-actors could constrain the process quite differently. We discuss how alignment is achieved in the first place, and how it is maintained over time, providing a taxonomy of various forms and mechanisms of space alignment and overlap based, for instance, on automaticity vs. control of the transformations between the two agents. Finally, we discuss the link between low-level mechanisms for the sharing of space and high-level mechanisms for the sharing of cognitive representations.

The document describes the architecture of the system being developed in the project. The principal contribution of this work is to the engineering of autonomous systems in a general fashion, i.e. independently of the target domain. This work is not “biologically inspired” and contributing to modeling natural intelligences is not an objective of the project. The main challenge set for the system is to adapt in dynamic open-ended environments with insufficient knowledge and limited resources. The system is to perform in real-time and extract knowledge from the domain it operates in. In particular, this means discovering meaningful states in the environment and learning skills by observing intentional agents in the domain. No system facing the complexity of the real world is able to learn effectively and efficiently from scratch. For each domain our system is plunged in, we hand craft a bootstrap code (called the Masterplan) that consists of the necessary initial and minimal knowledge to observe and act in the domain. The system is model-based and model-driven, meaning that its architecture is unified and consists essentially of dynamic hierarchies of models that capture knowledge in an executable form. The system is thus composed of executable models. From this perspective, learning translates into building models, integrating them into the existing hierarchies and revising them continuously. This perpetual rearranging of the internal agency of the system addresses the first objective of the HUMANOBS project:  to design an auto-reconfigurable architecture. Learning is based on model building, which in turn is driven by goals and predictions, i.e. the evaluation by the system of the observed phenomenology of the domain. In other words, the system infers what it shall do (specification) and observes ways to reach these goals (implementation). In that regard, this addresses the second objective of the project:  to build a system that can auto-generate specifications for skills and behaviors based on their observation. Learned specifications and implementations are highly context-dependent, which raises the challenge of identifying when to reuse (or refrain from reusing) learned models. Specifications and implementations are built hierarchically, which means that the system is able to reuse previously learned skills, however said skills are by design executable in parallel: this raises the challenge of coordinating (or sequentializing) the operation of the models that implement said skills. The architecture has been designed from the onset to solve these issues, and this addresses the last objective of the project:  to build behavior generation and coordination mechanisms for the reproduction and reuse of observed skills. The architecture has been designed in a principled way (each part of the architecture is based on the same architectural template), and organizes the cooperation of four continuous processes: Model Acquisition, Model Revision, Compaction (or model compression) and Reaction (reactive behavior in the domain). It is the interplay of said processes that not only ensure the viability of the whole system but also improves its performance. For example, acquiring models only requires a few examples, performs in real-time and is fast, but this process requires another process that revises models also rapidly and in real-time, and this in turn is supported by the reactive behavior of the system that pays attention only to meaningful entities and phenomena – which it has learned previously, the whole cycle having been bootstrapped by the Masterplan. This is in sharp contrast with traditional Machine Learning approaches which ignore the other cognitive processes of a system and thus, left on their own, require enormous quantities of training examples and cannot perform in real-time. A functional prototype has been developed as a proof of concept of the architecture, and the preliminary results reported in this deliverable strongly indicate that the architecture is sound and that our approach towards the engineering of autonomous systems is tractable and promising. 5/80 This prototype has been expanded, generalized and optimized furthermore into the final state of the architecture (with respect to the time frame of the project). The implemented final system satisfies the requirements of the project (although the evaluation results are presented in a separate deliverable) and, in particular, is completely domain-independent. Future developments and related research avenues have been identified and will bring the architecture beyond the requirements of this project. In the main, these developments are aimed at addressing the issues of (a) adding curiosity and imagination to the system's capabilities and, (b) controlling the autonomous bottom-up growth by means of top-down allonomic constraints.

A key goal in designing an artificial intelligence capable of performing complex tasks is a mechanism that allows it to efficiently choose appropriate and relevant actions in a variety of situations and contexts. Nowhere is this more obvious than in the case of building a general intelligence, where the contextual choice and application of actions must be done in the presence of large numbers of alternatives, both subtly and obviously distinct from each other. We present a framework for action selection based on the concurrent activity of multiple forward and inverse models. A key characteristic of the proposed system is the use of simulation to choose an action: the system continuously simulates the external states of the world (proximal and distal) by internally emulating the activity of its sensors, adopting the same decision process as if it were actually operating in the world, and basing subsequent choice of action on the outcome of such simulations. The work is part of our larger effort to create new observation-based machine learning techniques. We describe our approach, an early implementation, and an evaluation in a classical AT problem-solving domain: the Sokoban puzzle.

Research on action understanding in cognitive neuroscience has led to the identification of a wide "action understanding network" mainly encompassing parietal and premotor cortical areas. Within this cortical network mirror neurons are critically involved implementing a neural mechanism according to which, during action understanding, observed actions are reflected in the motor patterns for the same actions of the observer. We suggest that focusing only on cortical areas and processes could be too restrictive to explain important facets of action understanding regarding, for example, the influence of the observer's motor experience, the multiple levels at which an observed action can be understood, and the acquisition of action understanding ability. In this respect, we propose that aside from the cortical action understanding network, sub-cortical processes pivoting on cerebellar and basal ganglia cortical loops could crucially support both the expression and the acquisition of action understanding abilities. Within the paper we will discuss how this extended view can overcome some limitations of the "pure" cortical perspective, supporting new theoretical predictions on the brain mechanisms underlying action understanding that could be tested by future empirical investigations. © 2013 The Authors.

Visual lexical decision is a classical paradigm in Psycholinguistic, and numerous studies have assessed a so-called "lexicality effect" (i.e., better performance with lexical over non-lexical stimuli). Far less is know relative to the dynamics of choice, as many studies measure overal reaction times which are not informative of the underlying processes. To unfold visual lexical decision in time, we measured participants' hand movements toward one of two items alternatives by recording the streaming x,y coordinates of the computer mouse. Participants categorized as 'lexical' or 'non-lexical' four kinds of stimuli: high and low frequency words, pseudowords, and letter strings. Spatial attraction toward the opposite category was present for low frequency words and pseudowords. Increasing stimuli ambiguity lead to enhcanced movements' complexity and trajectories' attraction to competitors, as no such effect was present for high frequency words and letter strings. Results fit well with dynamic models of perceptual decision-making describing the process as a competition between alternatives guided by the continuous accumulation of evidence, as well as with a recent neural model of visual word recognition that highlights the role of top-down influences and predictions on perceptual processes. More broadly, our results point to a key role of statistical decision theory to study linguistic processing in terms of dynamic and non-modular mechanisms. Finally, we discuss two aspects that make our set-up challenging for current dynamical models of decision-making: 1) not all information (e.g. ortographic, phonological and semantic) is available at the same time, therefore the accumulation process is nonstationary; 2) the choice is not completed at the action onset, but can be revised at any time during the movement.

Visual lexical decision is a classical paradigm in psycholinguistics, and numerous studies have assessed the so-called ''lexicality effect'' (i.e., better performance with lexical than non-lexical stimuli). Far less is known about the dynamics of choice, because many studies measured overall reaction times, which are not informative about underlying processes. To unfold visual lexical decision in (over) time, we measured participants' hand movements toward one of two item alternatives by recording the streaming x,y coordinates of the computer mouse. Participants categorized four kinds of stimuli as 'lexical' or 'non-lexical': high and low frequency words, pseudowords, and letter strings. Spatial attraction toward the opposite category was present for low frequency words and pseudowords. Increasing the ambiguity of the stimuli led to greater movement complexity and trajectory attraction to competitors, whereas no such effect was present for high frequency words and letter strings. Results fit well with dynamic models of perceptual decision-making, which describe the process as a competition between alternatives guided by the continuous accumulation of evidence. More broadly, our results point to a key role of statistical decision theory in studying linguistic processing in terms of dynamic and non-modular mechanisms.

NA

Recent neuroscientific models of human behavior distinguish between different cognitive controllers: two instrumental systems (goal-directed and habitual) that maximize utility through learned actions, and a so-called Pavlovian system, which implements innate reactive responses. Although the interaction between instrumental and Pavlovian controllers has been suggested as a key process underlying emotional phenomena and surprising forms of misbehavior, few is known about it, especially in the sensorimotor aversive domain. With a combined experimental and computational approach, we study the interactions between instrumental (goal-directed) and Pavlovian processes in the aversive domain. First, we present a human experiment in which goal-directed and Pavlovian systems compete in order to control responses. The results indicate that Pavlovian processes can significantly interfere with goal-directed behavior. Second, we compare four alternative Bayesian models for their accuracy in modeling human performance. The results indicate a better fit for an architecture in which the Pavlovian controller can use both model-based and model-free features.

We describe a generative Bayesian model for action understanding in which inverse-forward internal model pairs are considered 'hypotheses' of plausible action goals that are explored in parallel via an approximate inference mechanism based on sequential Monte Carlo methods. The reenactment of internal model pairs can be considered a form of motor simulation, which supports both perceptual prediction and action understanding at the goal level. However, this procedure is generally considered to be computationally inefficient. We present a model that dynamically reallocates computational resources to more accurate internal models depending on both the available prior information and the prediction error of the inverse-forward models, and which leads to successful action recognition. We present experimental results that test the robustness and efficiency of our model in real-world scenarios.

A crucial part of the intelligence that smart environments should display is a specific form of social intelligence: the ability to read human behavior and its traces in terms of underlying intentions and assumptions. Such ability is crucial to enable human users to tacitly coordinate and negotiate with smart and proactive digital environments. In this article, the authors argue that the necessary tool for this ability is behavioral and stigmergic implicit (i.e. non-conventional) communication. The authors present a basic theory of such a fundamental interactive means-the theory of Behavioral Implicit Communication (BIC).

International audience

This paper presents a model of brain systems underlying reaching in monkeys based on the idea that complex behaviors are built on the basis of a repertoire of motor primitives organized around specific goals (in this case, arm's postures). The architecture of the system is based on an actor-critic reinforcement-learning model, enhanced with an accumulator model for action selection, capable of selecting sensorimotor primitives so as to accomplish a discrimination reaching task that has been used in physiological studies of monkeys' premotor cortex. The results show that the proposed architecture is a first important step towards the construction of a biologically plausible integrated motor-primitive based model of the hierarchical organization of mammals' sensorimotor systems.