Search

In this paper, we present some recent cognitive robotics studies on language and cognition integration to demonstrate how the language acquired by robotic agents can be directly grounded in action representations. These studies are characterized by the hypothesis that symbols are directly grounded into the agents' own categorical representations, while at the same time having logical (e.g. syntactic) relationships with other symbols. The two robotics studies are based on the combination of cognitive robotics with neural modeling methodologies, such as connectionist models and modeling field theory. Simulations demonstrate the efficacy of the mechanisms of action grounding of language and the symbol grounding transfer in agents that acquire lexicon via imitation and linguistic instructions. The paper also discusses the scientific and technological implications of such an approach.

This paper investigates the prospects of Rodney Brooks' proposal for AI without representation. It turns out that the supposedly characteristic features of "new AI" (embodiment, situatedness, absence of reasoning, and absence of representation) are all present in conventional systems: "New AI" is just like old AI. Brooks proposal boils down to the architectural rejection of central control in intelligent agents--Which, however, turns out to be crucial. Some of more recent cognitive science suggests that we might do well to dispose of the image of intelligent agents as central representation processors. If this paradigm shift is achieved, Brooks' proposal for cognition without representation appears promising for full-blown intelligent agents--Though not for conscious agents.

The traditional view of symbol grounding seeks to connect an a priori internal representation or ‘form’ to its external referent. But such a ‘form’ is usually itself systematically composed out of more primitive parts (i.e., it is ‘symbolic’), so this view ignores its grounding in the physics of the world. Some previous work simulating multiple talking/listening agents has effectively taken this stance, and shown how a shared discrete speech code (i.e., vowel system) can emerge. Taking the earlier work of Oudeyer, we have extended his model to include a dispersive force intended to account broadly for a speaker’s motivation to increase auditory distinctiveness. New simulations show that vowel systems result that are more representative of the range seen in human languages. These simulations make many profound abstractions and assumptions. Relaxing these by including more physically and physiologically realistic mechanisms for talking and listening is seen as the key to replicating more complex and dynamic aspects of speech, such as consonant-vowel patterning.

Taking a distributed view of language, this paper naturalizes symbol grounding. Learning to talk is traced to — not categorizing speech sounds — but events that shape the rise of human-style autonomy. On the extended symbol hypothesis, this happens as babies integrate micro-activity with slow and deliberate adult action. As they discover social norms, intrinsic motive formation enables them to reshape co-action. Because infants link affect to contingencies, dyads develop norm-referenced routines. Over time, infant doings become analysis amenable. The caregiver of a nine-month-old may, for example, prompt the baby to fetch objects. Once she concludes that the baby uses ‘words’ to understand what she says, the infant can use this belief in orienting to more abstract contingencies. New cognitive powers will develop as the baby learns to act in ways that are consistent with a caregiver’s false belief that her baby uses ‘words.’

I present the symbol grounding problem in the larger context of a materialist theory of content and then present two problems for causal, teleo-functional accounts of content. This leads to a distinction between two kinds of mental representations: presentations and symbols; only the latter are cognitive. Based on Milner and Goodale’s dual route model of vision, I posit the existence of precise interfaces between cognitive systems that are activated during object recognition. Interfaces are constructed as a child learns, and is taught, how to interact with its environment; hence, interface structure has a social determinant essential for symbol grounding. Symbols are encoded in the brain to exploit these interfaces, by having projections to the interfaces that are activated by what they symbolise. I conclude by situating my proposal in the context of Harnad’s (1990) solution to the symbol grounding problem and responding to three standard objections.

This paper compares the nascent theory of the ‘semiotic symbol’ in cognitive science with its computational relative. It finds that the semiotic symbol as it is understood in recent practical and theoretical work does not have the resources to explain the role of symbols in cognition. In light of this argument, an alternative model of symbol internalisation, based on Vygotsky, is put forward which goes further in showing how symbols can go from playing intersubjective communicative roles to intrasubjective cognitive ones. Such a formalisation restores the symbol’s cognitive and communicative dimensions to their proper roles.

Contains fulltext : 44290.pdf (Publisher’s version ) (Closed access) A substantial part of language understanding depends on our previous experiences, but part of it consists of the creation of new meanings. Such new meanings cannot be retrieved from memory but still have to be constructed. The goals of this article were: first, to explore the nature of new meaning creation, and second, to test abstract symbol theories against embodied theories of meaning. We presented context-setting sentences followed by a test sentence to which ERPs were recorded that described a novel sensible or novel senseless situation (e.g., "The boys searched for branches/bushes [sensible/senseless] with which they went drumming..."). Novel sensible contexts that were not associatively nor semantically related were matched to novel senseless contexts in terms of familiarity and semantic similarity by Latent Semantic Analysis (LSA). Abstract symbol theories like LSA cannot explain facilitation for novel sensible situations, whereas the embodied theory of Glenberg and Robertson [Glenberg, A.M., Robertson, D.A., 2000. Symbol grounding and meaning: A comparison of high-dimensional and embodied theories of meaning. Journal of Memory and Language, 43, 379-401.] in which meaning is grounded in perception and action can account for facilitation. Experiment 1 revealed an N400 effect in a sensibility judgment task. Experiment 2 demonstrated that this effect generalizes to a situation in which participants read for comprehension. Our findings support the following conclusions: First, participants can establish new meanings not stored in memory. Second, this is the first ERP study that shows that N400 is sensitive to new meanings and that these are created immediately - that is, in the same time frame as associative and semantic relations. Third, our N400 effects support embodied theories of meaning and challenge abstract symbol theories that can only discover meaningfulness by consulting stored symbolic knowledge.

Recently, studies on learning of word meanings by agents have begun. In these studies, a human shows objects to an agent and utters words such as ``red'' or ``box''. The agent finds out object's feature represented by each spoken word. In our method, firstly, the agent learns probability distribution p(x) and conditional probability distribution p(x|w), where x is an object feature and w is a word. If a word w does not represent a feature x, p(x) and p(x|w) will be almost same distribution because x is independent of w. This fact enables the agent to use distance between p(x) and p(x|w) when inferring which feature the word represents. Previous works also employ similar stochastic approaches to detect the feature. However, such approaches need a lot of examples to learn correct distributions.

In this paper, we extend a symbolic association framework for being able to handle missing elements in multimodal sequences. The general scope of the work is the symbolic associations of object-word mappings as it happens in language development in infants. In other words, two different representations of the same abstract concepts can associate in both directions. This scenario has been long interested in Artificial Intelligence, Psychology, and Neuroscience. In this work, we extend a recent approach for multimodal sequences (visual and audio) to also cope with missing elements in one or both modalities. Our method uses two parallel Long Short-Term Memories (LSTMs) with a learning rule based on EM-algorithm. It aligns both LSTM outputs via Dynamic Time Warping (DTW). We propose to include an extra step for the combination with the max operation for exploiting the common elements between both sequences. The motivation behind is that the combination acts as a condition selector for choosing the best representation from both LSTMs. We evaluated the proposed extension in the following scenarios: missing elements in one modality (visual or audio) and missing elements in both modalities (visual and sound). The performance of our extension reaches better results than the original model and similar results to individual LSTM trained in each modality., Under review on Journal of Artificial Intelligence Research (JAIR) -- Special Track on Deep Learning, Knowledge Representation, and Reasoning

Humans can learn the use of language through physical interaction with their environment and semiotic communication with other people. It is very important to obtain a computational understanding of how humans can form a symbol system and obtain semiotic skills through their autonomous mental development. Recently, many studies have been conducted on the construction of robotic systems and machine-learning methods that can learn the use of language through embodied multimodal interaction with their environment and other systems. Understanding human social interactions and developing a robot that can smoothly communicate with human users in the long term, requires an understanding of the dynamics of symbol systems and is crucially important. The embodied cognition and social interaction of participants gradually change a symbol system in a constructive manner. In this paper, we introduce a field of research called symbol emergence in robotics (SER). SER is a constructive approach towards an emergent symbol system. The emergent symbol system is socially self-organized through both semiotic communications and physical interactions with autonomous cognitive developmental agents, i.e., humans and developmental robots. Specifically, we describe some state-of-art research topics concerning SER, e.g., multimodal categorization, word discovery, and a double articulation analysis, that enable a robot to obtain words and their embodied meanings from raw sensory--motor information, including visual information, haptic information, auditory information, and acoustic speech signals, in a totally unsupervised manner. Finally, we suggest future directions of research in SER., submitted to Advanced Robotics

Natural language interfaces are powerful tools that enables humans and robots to convey information without the need for extensive training or complex graphical interfaces. Statistical techniques that employ probabilistic graphical models have proven effective at interpreting symbols that represent commands and observations for robot direction-following and object manipulation. A limitation of these approaches is their inefficiency in dealing with larger and more complex symbolic representations. Herein, we present a model for language understanding that uses parse trees and environment models both to learn the structure of probabilistic graphical models and to perform inference over this learned structure for symbol grounding. This model, called the Hierarchical Distributed Correspondence Graph (HDCG), exploits information about symbols that are expressed in the corpus to construct minimalist graphical models that are more efficient to search. In a series of comparative experiments, we demonstrate a significant improvement in efficiency without loss in accuracy over contemporary approaches for human-robot interaction.

Compositionality is a property of natural language which is of prime importance: It enables humans to form and conceptualize potentially novel and complex ideas, by combining words. On the other hand, the symbol grounding problem examines the way meaning is anchored to entities external to language, such as sensory percepts and sensory-motor routines. In this paper we aim towards the exploration of the intersection of compositionality and symbol grounding. We thus propose a methodology for constructing empirically derived models of grounded meaning, which afford composition of grounded semantics. We illustrate our methodology for the case of adjectival modifiers. Grounded models of adjectively modified and unmodified colors are acquired through a specially designed procedure with 134 participants, and then computational models of the modifiers “dark” and “light” are derived. The generalization ability of these learnt models is quantitatively evaluated, and their usage is demonstrated in a real-world physical humanoid robot. We regard this as an important step towards extending empirical approaches for symbol grounding so that they can accommodate compositionality: a necessary step towards the deep understanding of natural language for situated embodied agents, such as sensor-enabled ambient intelligence and interactive robots.

Learning from demonstration (LfD) aims at robots learning skills from human-demonstrated tasks. Robots should be able to learn at all levels of abstraction. Unlike at the level of motor primitives, high-level LfD requires symbolic representations. It thus faces the classical problem of symbol grounding. Furthermore, it requires the robot to interpret human-demonstrated actions at a higher, conceptual abstraction level. We present a method, that enables a robot to recognize human-demonstrated pick-and-place task goals on an object-relational abstraction layer. The robot can reproduce the task goals in new situations using a symbolic planner. We show that in a robotic context conceptual spaces can serve as a mean for symbol grounding at an object-relational level as well as for the recognition of conceptual similarities in effects of human-demonstrated actions. The method is evaluated in experiments on a real robot.

This chapter focuses on the relationship between embodiment and the phenomenology of selfhood. Specifically, it looks at the notion of minimal phenomenal selfhood (MPS; see pp. 000-00, below), which defines the research target of isolating the minimal set of conditions constituting the conscious experience of “being a self,” relative to a given class of systems. In preparation, I will introduce three new working concepts: “first-order embodiment” (1E), “second-order embodiment” (2E), and “third-order embodiment” (3E).1 Each of these concepts picks out a rather well-defined class of systems: 1E-systems are reactive, adaptive systems, achieving intelligent behavior without explicit computation; 2E-systems increase their level of causal selfcontrol by explicitly and holistically representing themselves as embodied; and 3E-systems integrate certain aspects of their body model into a unified ontology, thereby elevating them to the level of conscious experience. A normal, conscious human being walking down the street in an ordinary, non-pathological state of consciousness simultaneously is a member of all three classes at the same time. This may be interestingly different, or not true at all, in certain alternative configurations like dreamless sleep, epileptic absence seizures, dreaming, or experimentally induced full-body illusions (FBIs). Here, my goal lies in drawing attention to a new version of the mind-body problem: How, precisely, do we describe the grounding relations holding between 1E, 2E, and 3E? In analogy to the “symbol grounding problem” one might also call this the “self grounding problem” (Harnad, 1990; Barsalou, 2008). It is the problem of describing the abstract computational principles as well as the implementational mechanics by which a system’s phenomenal self-model (PSM; cf. Metzinger, 2003a, 2007) is anchored in low-level physical dynamics, in a maximally parsimonious way, and without assuming a single, central module for global self-representation.

word learning and comprehension is a very crucial and important issue because of its application and problematic nature. This problem does not just belong to the cognitive robotics field, as it also has significance in neuroscience and cognitive science. There are many issues like symbol grounding problem and sensory motor processing within grounded cognition framework and conceptual knowledge representation methods that have to be addressed and solved for the acquisition of abstract words in cognitive robots. This paper explains these concepts and matters, and also elucidates how these are linked to this problem. In this paper, first symbol grounding problem is discussed, and after that an overview of grounded cognition be given along with detail of methods/ideas that suggest how abstract word representation could use sensory motor system. Finally, the computation methods used for the representation of conceptual knowledge are discussed. Two cognitive robotics models based on Neural network and Semantic network that ground abstract words are presented and compared via simulation experiment to find out the pros and cons of computation methods for this problem. The aim of this paper is to explore the building blocks of cognitive robotics model at theoretical and experimental level, for grounding of abstract words.

Solving the visual symbol grounding problem has long been a goal of artificial intelligence. The field appears to be advancing closer to this goal with recent breakthroughs in deep learning for natural language grounding in static images. In this paper, we propose to translate videos directly to sentences using a unified deep neural network with both convolutional and recurrent structure. Described video datasets are scarce, and most existing methods have been applied to toy domains with a small vocabulary of possible words. By transferring knowledge from 1.2M+ images with category labels and 100,000+ images with captions, our method is able to create sentence descriptions of open-domain videos with large vocabularies. We compare our approach with recent work using language generation metrics, subject, verb, and object prediction accuracy, and a human evaluation., Comment: NAACL-HLT 2015 camera ready

Creating, maintaining, and deducing accurate world knowledge in a dynamic, complex, adversarial, and stochastic environment such as the RoboCup environment is a demanding task. Knowledge should be represented in real-time (i.e., within ms) and deduction from knowledge should be inferred within the same time constraints. We propose an extended assertional formalism for an expressive \(\mathcal {SROIQ}(\mathcal {D})\) Description Logic to represent asserted entities in a lattice structure. This structure can represent temporal-like information. Since the computational complexity of the classes of description logic increases with its expressivity, the problem demands either a restriction in the expressivity or an empirical upper bound on the maximum number of axioms in the knowledge base. We assume that the terminological/relational knowledge changes significantly slower than the assertional knowledge. Henceforth, using a fixed terminological and relational formalisms and the proposed lattice structure, we empirically bound the size of the knowledge bases to find the best trade-off in order to achieve deduction capabilities of an existing description logic reasoner in real-time. The queries deduce instances using the equivalent class expressions defined in the terminology. We have conducted all our experiments in the RoboCup 3D Soccer Simulation League environment and provide justifications of the usefulness of the proposed assertional extension. We show the feasibility of our new approach under real-time constraints and conclude that a modified FaCT++ reasoner empirically outperforms other reasoners within the given class of complexity.

Many recent papers claim, that the symbol grounding problem (SGP) remains unsolved. Most AI researchers ignore that and the autonomous agents (or robots) they design indeed do not seem to have any “problem”. Anyway, these claims should be taken rationally, since nearly all these papers make “robots” a subject of the discussion - leaving some kind of impression that what many roboticists do in the long run has to fail because of the SGP not yet being solved. Starting from Searle’s chinese room argument (CRA) and Harnad’s reformulation of the problem, we take a look on proposed solutions and the concretization of the problem by Taddeo’s and Floridi’s “Z condition”. We then refer to two works, which have recently shown that the Z-conditioned SGP is unsolvable. We conclude, that the original, hard SGP is not relevant in the context of designing goal-directed autonomous agents.

This article reviews several canonical signatures of number processing and the proposed theoretical interpretations of these signatures. It examines the biological origins of number sense in humans and nonhuman species alike, how to conceptualize the notion of numbers without symbolic form, and the role that number sense may play in human development. Symbolic numbers are discussed in terms of their history and the leading ideas about how the human brain assigns meaning to these crucial building blocks of modern human society.

We address the symbol grounding problem for robot perception through a data-driven approach to deriving categories from robot sensor data. Unlike model-based approaches, where human intuitive correspondences are sought between sensor readings and features of an environment (corners, doors, etc.), our method learns intrinsic categories (or natural kinds) from the raw data itself. We approximate a manifold underlying sensor data using Isomap nonlinear dimension reduction and apply Bayesian clustering (Gaussian mixture models) with model identification techniques to discover categories (or kinds). We demonstrate our method through the learning of sensory kinds from trials in various indoor and outdoor environments with different sensor modalities. Learned kinds are then used to classify new sensor data (out-of-sample readings). We present results indicating greater consistency in classifying sensor data employing mixture models in nonlinear low-dimensional embeddings. © 2007 Wiley Periodicals, Inc.

Previous simulation work on the evolution of communication has not shown how a large signal repertoire could emerge in situated agents. We present an artificial life simulation of agents, situated in a two-dimensional world, that must search for other agents with whom they can trade resources. With strong restrictions on which resources can be traded for others, initially non-communicating agents evolve/learn a signal system that describes the resource they seek and the resource they are willing to offer in return. A large signal repertoire emerges mainly through an evolutionary process. Agents whose production and comprehension abilities rely on a single mechanism fare best, although learning enables agents with separate mechanisms to achieve some measure of success. These results demonstrate that substantial signaling repertoires can evolve in situated multi-agent systems, and suggest that simulated social interactions such as trading may provide a useful context for further computational studies of the evolution of communication.

The failure of classical AI in delivering intelligent robots has forced many researchers to address the Symbol Grounding Problem (SGP) seriously. In this paper we present a methodology to solve this problem by facilitating a human instructor to interact with a robot using a Microsoft Kinect sensor so as to ground symbols. The instructor exhibits the physical objects or actions and also provides the corresponding symbols using speech to a robot which in turn uses and elaborates its own semantics for the symbol autonomously and grounds the same within. Grounding is achieved by toiling on sensory information while symbol theft enables the grounding of meta-level symbols or aliases. Apart from the description of the inherent grounding mechanism, the paper also discusses the results obtained from actual experiments that eventually make the robot identify and point to the concerned physical objects or perform the actions.

Solving the Symbol Grounding Problem (SGP) is crucial for the autonomous robot learning. The failure of classical AI in delivering smart robots has forced many researchers to address the problem earnestly. In this paper we present a methodology to solve this problem by facilitating a human instructor to interact with a robot using a Microsoft Kinect™ sensor so as to ground symbols. The instructor exhibits the physical objects and also provides the corresponding symbols using voice commands to a robot. The robot captures the physical attributes of different objects for the symbol to elaborate its own semantics autonomously and grounds the same within. Grounding is achieved by toiling on sensory information to acquire the commonalities from the attributes while symbol theft enables the grounding of meta-level symbols or aliases. Apart from the formal description of the inherent grounding mechanism, the paper also discusses the results obtained from actual experiments that eventually make the robot identify and point to the concerned physical objects.

The young math learner must make the transition from a concrete number situation, such as that of counting objects (fingers often being the most readily available), to that of using a written symbolic form that stands for the quantities the sets of objects come to represent. This challenging process is often coupled to that of learning a verbal number system that is not always transparent to children. A number of theoretical approaches have been advanced to explain aspects of how this transition takes place in cognitive development. The results obtained with the model presented here, show that a symbol grounding approach can be used to implement aspects of this transition in a cognitive robot. In the current extended version, the model develops finger and word representations, through the use of finger counting and verbal counting strategies, together with the visual representations of learned number symbols, which it uses to perform basic arithmetic operations. In the final training phases, the model is able to do this using only the number symbols as addends. We consider this an example of symbolic grounding, in that through the direct sensory experience with the body (finger counting), a category of linguistic symbol is learned (number words), and both types of representations subsequently serve to ground higher level (numerical) symbols, which are later used exclusively to perform the arithmetic operations.

This article reviews eight proposed strategies for solving the symbol grounding problem (SGP), which was given its classic formulation in Harnad (1990). After a concise introduction, the paper provides an analysis of the requirement that must be satisfied by any hypothesis seeking to solve the SGP, the zero semantical commitment condition. It is then used to assess the eight strategies, which are organized into three main approaches: representationalism, semi-representationalism and non-representationalism. The conclusion is that all the strategies are semantically committed and hence that none of them provides a valid solution to the SGP, which remains an open problem.

This paper discusses a relationship between symbol emergence and embodiment. Symbol emergence is an alternative approach to overcome symbol grounding problem. We believe human beings' symbols constructed and held internally are originally based on our own embodiment. Therefore, autonomous social robots will also have to construct their own symbols by themselves. In addition to this, we have a nonlinear hypothesis about symbol emergence. To realize symbol emergence inside of autonomous robots, we propose Dual-Schemata model. This model enables an autonomous robot to generate its own symbols, called perceptional schemata, depending on its embodiment. In the end, we show that this symbol emergence has a nonlinear relationship with embodiment.

How can we achieve artificial systems that are capable of understanding human language? This question has been addressed by the field of artificial intelligence for decades and has undergone several paradigm shifts from rule-based approaches, proposing sets of symbolic rules to process language input to produce an intelligent behavior, to the insight that symbolic rules are not sufficient to deal with new situations, let alone new symbols, that have not been encountered before. Rather, it has been proposed that the symbols of a language need to be grounded in perception in order to allow for generalisation to new situations. This new formulation of the problem has been termed “Symbol Grounding” and proposes that any system capable of understanding symbols needs to be embodied in the sense that it has to be able to perceive its environment and to produce actions within this environment that cause significant changes and that these percepts and actions need to be tied to symbols that allow to abstract and thus to transfer knowledge. Research thus focuses either on simulated embodied agents or robots that are situated in a physical environment. Given the complex nature of the challenge, a range of different strands of research has emerged from which we tried to capture the most relevant ones in this Special Issue. Focusing on the development of grounding capabilities in infants brings in new perspectives and has, for example, lead to the insight that while the learning system has to be extremely adaptive to the environment, the social environment itself also adapts to the learner. In the contribution by Paul Vogt & J. Douglas Mastin it is thus discussed which aspects of such interactions should be modelled in artificial systems with a special focus on data “from the wild”.

Communication among adaptive agents can be framed as language acquisition and broken down into three problems; symbol grounding, language learning, and language evolution. We propose that this view clarifies many of the difficulties framing issues of collaboration and self-organization. Additionally, we demonstrate simple classification systems that can provide the first step in grounding real-world data and provide general schema for constructing other such systems. The first system classifies auditory input from frog calls and is presented as a model of grounding objects. The second system uses the minimum description length framework to distinguish patterns of robot movement as a model of grounding actions.

A mechanism for planning ahead would appear to be essential to any creature with more than insect level intelligence. In this paper it is shown how planning, using full means‐ends analysis, can be had while avoiding the so called symbol grounding problem. The key role of knowledge representation in intelligence has been acknowledged since at least the enlightenment, but the advent of the computer has made it possible to explore the limits of alternate schemes, and to explore the nature of our everyday understanding of the world around us. In particular, artificial intelligence (AI) and robotics has forced a close examination, by people other than philosophers, of what it means to say for instance that “snow is white.” One interpretation of the “new AI” is that it is questioning the need for representation altogether. Brooks and others have shown how a range of intelligent behaviors can be had without representation, and this paper goes one step further showing how intending to do things can be achieved wi...

Empirical findings of cross-linguistic studies reveal three different frames of spatial reference: intrinsic, relative, and absolute. Of special interest are relative and absolute systems because they have antagonistic logical implications concerning the dependence on standpoint and orientation of the speaker/hearer. On the background of these findings it becomes crucial to show how an agent can form such language-specific spatial representations. In this paper, the system Locator is introduced as a model of concept formation in the spatial domain. It is assumed that an agent creates necessary discriminative features in processes of self-organization and selection and cannot just discover or “find” them in its environment. A number of simulations show that agents successfully create concepts of either a relative system (German) or an absolute system (Marquesan), relying solely on multimodal input (visual and linguistic).

In this paper, a new machine-learning method, called Dual-Schemata model, is presented. Dual-Schemata model is a kind of self-organizational machine learning methods for an autonomous robot interacting with an unknown dynamical environment. This is based on Piaget's Schema model, that is a classical psychological model to explain memory and cognitive development of human beings. Our Dual-Schemata model is developed as a computational model of Piaget's Schema model, especially focusing on sensori-motor developing period. This developmental process is characterized by a couple of two mutually-interacting dynamics; one is a dynamics formed by assimilation and accommodation, and the other dynamics is formed by equilibration and differentiation. By these dynamics schema system enables an agent to act well in a real world. This schema's differentiation process corresponds to a symbol formation process occurring within an autonomous agent when it interacts with an unknown, dynamically changing environment. Experiment results obtained from an autonomous facial robot in which our model is embedded are presented; an autonomous facial robot becomes able to chase a ball moving in various ways without any rewards nor teaching signals from outside. Moreover, emergence of concepts on the target movements within a robot is shown and discussed in terms of fuzzy logics on set-subset inclusive relationships.

Anchoring is the problem of connecting, inside an artificial system, symbols and sensor data that refer to the same physical objects in the external world. This problem needs to be solved in any robotic system that incorporates a symbolic component. However, it is only recently that the anchoring problem has started to be addressed as a problem per se, and a few general solutions have begun to appear in the literature. This paper introduces the special issue on perceptual anchoringof the Robotics and Autonomous Systemsjournal. Our goal is to provide a general overview of the anchoring problem, and highlight some of its subtle points. © 2003 Elsevier Science B.V. All rights reserved.

In this paper we design an experiment which can be depicted as a simple scenario, a very limited ‘world’. In this world, there are an actor that can pursue a project and an observer that is keeping its eyes on the actor. We try to show in the experiment that the observer can to some degree understand the actor based on its knowledge and some metaphors, i.e. understand what the actor is doing and why. As the conclusion of this experiment, we try to show some features of ‘understanding’. These are (1) that ‘understanding’ has to be based on some preliminary knowledge; (2) that ‘understanding’ is a process of incremental learning; (3) that, as for symbolic systems, some metaphors are necessary for mapping real entities into concepts in mind.

The GLAIR grounded layered architecture with integrated reasoning for cognitive robots and intelligent autonomous agents has been used in a series of projects in which Cassie, the SNePS cognitive agent, has been incorporated into hardware- or software-simulated cognitive robots. In this paper, we present an informal, but coherent, overview of the GLAIR approach to anchoring the abstract symbolic terms that denote an agent’s mental entities in the lower-level structures used by the embodied agent to operate in the real (or simulated) world. We discuss anchoring in the domains of: perceivable entities and properties, actions, time, and language. © 2003 Elsevier Science B.V. All rights reserved.

A fundamental problem for ecological and cognitive psychology alike is to explain how agents are situated, that is, functionally coupled to their environments so as to facilitate adaptive actions. Herbert Simon (1969/1996) argued that such coupling is artifactual (rule governed), being mediated by symbol functions and necessarily involving information processing. An alternative to this computational approach is offered by James Gibson's (1979/1986) view that the interface is natural (law governed), being a direct informational coupling rather than a symbolically mediated one. This latter view necessarily involves the agent's awareness, whereas the former, being mechanistic, does not. I review the coupling problem from historical, logical, and semantic perspectives. I give arguments that the computational approach provides an inadequate account of situated adaptive actions and founders on the symbol grounding problem, whereas the ecological approach does a better job on both. Personal comments are interspe...

Symbols are reductionist in nature. When used to influence thought and behavior, symbols cause the receiver to hold critical faculties in abeyance. In politics symbols cause the viewer or listener (the receiver) to acquiesce and to submit to the overpowering and diminishing nature of the symbol. In the field of public-sector marketing, the manipulation of uniquely political symbols (words, ceremonies, objects, rituals) is emerging as an important subject for study. The manipulation of public symbols in the highly dramatized Florida recount case provides an important context for analyzing how public symbols were marketed to produce desired results. © 2002 John Wiley & Sons, Inc.

This paper presents an approach to solve the symbol grounding problem within the framework of embodied cognitive science. It will be argued that symbolic structures can be used within the paradigm of embodied cognitive science by adopting an alternative definition of a symbol. In this alternative definition, the symbol may be viewed as a structural coupling between an agent's sensorimotor activations and its environment. A robotic experiment is presented in which mobile robots develop a symbolic structure from scratch by engaging in a series of language games. In this experiment it is shown that robots can develop a symbolic structure with which they can communicate the names of a few objects with a remarkable degree of success. It is further shown that, although the referents may be interpreted differently on different occasions, the objects are usually named with only one form.

According to an “external grounding” theory of meaning, a concept’s meaning depends on its connection to the external world. By a “conceptual web” account, a concept’s meaning depends on its relations to other concepts within the same system. We explore one aspect of meaning, the identification of matching concepts across systems (e.g. people, theories, or cultures). We present a computational algorithm called ABSURDIST (Aligning Between Systems Using Relations Derived Inside Systems for Translation) that uses only within-system similarity relations to find between-system translations. While illustrating the sufficiency of a conceptual web account for translating between systems, simulations of ABSURDIST also indicate powerful synergistic interactions between intrinsic, within-system information and extrinsic information. q 2002 Elsevier Science B.V. All rights reserved.

Aesthetics plays a central role in human life. Given its ubiquity across cultures, there is no shortage of theories about its origin, function, underlying mechanisms, purpose, and so on. While we applaud the diversity of these approaches and their commitment to shedding light on this mysterious and abstract conceptual domain, many of them are unabashedly top-down, centering on the role of higher-order, reason-based assumptions about how the mind works. In contrast to this view, over the past decade, findings across the cognitive sciences have provided considerable support for the thesis that cognition is fundamentally grounded in sensorimotor and perceptual states. The now popular view of embodied cognition – a species of grounded cognition – has energized many of the creative insights that have helped breathe life into traditionally intractable cognitive problems (e.g., symbol grounding). However, insightful critics like Mahon and Caramazza (J Physiol 102:59–70, 2008) and Dove (Cognition 110:412–431, 2009) have argued that grounded accounts of cognition fail to adequately explain the representation and processing of abstract concepts like AESTHETICS, which give no unified perceptual experiences. In this chapter, we argue that aesthetics (like other abstract conceptual representations) can be accommodated by an embodied theory that uses two classes of perceptual information (sensorimotor and affective) to explain art representation, production, and evaluation.

In this paper, we propose an approach inspired by Gardenfors conceptual spaces, to represent concepts (specifically colour) in a situated embodied agent. Through this we also extend the work of Braitenberg by proposing an alternate vehicle capable of recognising colour concepts. Furthermore, we demonstrate how it can learn to distinguish between different colours by evolving our vehicle’s conceptual space using Template Based Evolution. Background and motivation Categorisation and the ability to recognise and represent concepts has been described as a fundamental cognitive ability and a primary research goal of AI (Gardenfors and Williams, 2001). The traditional approach from classical AI is to represent concepts symbolically using a top-down design process. The bottom-up subsymbolic (connectionist) approach, in contrast, emphasizes the importance of symbol grounding and the crucial role that the body has to play, by having the agent situated in the world and embodied— able to perceive and interact with the world through sensorymotor co-ordination (Pfeifer and Scheier, 2001). Being able to recognise similarity is key to many biological phenomena, such as how simplistic agents are able to learn what food is safe to eat. Whilst prototypical items are easily represented through the symbolic approach, this does not take into account possible variations that naturally occur in real-life. Take the problem of colour recognition, for example. In natural systems, it is very unlikely to find exactly the same colour—no leaf on a tree is ever the same shade of green, or a berry is ever the same shade of red. Gardenfors (2004) presents an alternative methodology, called Conceptual Spaces which models knowledge in the form of regions in a geometric space made up of quality dimensions. By modeling in this way, Gardenfors argues that an agent is able to recognise variations on a defined concept prototype. A primary motivation for this paper is to see if a Conceptual Spaces representation can be implemented in a situated, embodied agent in a manner analogous to the approach adopted by Pfeifer and Scheier (2001) in order to demonstrate how concepts such as colour can be learnt in an unsupervised way.

The Robotics Collaborative Technology Alliance (RCTA) seeks to provide adaptive robot capabilities which move beyond traditional metric algorithms to include cognitive capabilities. Key to this effort is the Common World Model, which moves beyond the state-of-the-art by representing the world using semantic and symbolic as well as metric information. It joins these layers of information to define objects in the world. These objects may be reasoned upon jointly using traditional geometric, symbolic cognitive algorithms and new computational nodes formed by the combination of these disciplines to address Symbol Grounding and Uncertainty. The Common World Model must understand how these objects relate to each other. It includes the concept of Self-Information about the robot. By encoding current capability, component status, task execution state, and their histories we track information which enables the robot to reason and adapt its performance using Meta-Cognition and Machine Learning principles. The world model also includes models of how entities in the environment behave which enable prediction of future world states. To manage complexity, we have adopted a phased implementation approach. Phase 1, published in these proceedings in 2013 [1], presented the approach for linking metric with symbolic information and interfaces for traditional planners and cognitive reasoning. Here we discuss the design of “Phase 2” of this world model, which extends the Phase 1 design API, data structures, and reviews the use of the Common World Model as part of a semantic navigation use case.

The article addresses two questions. First, if knowledge is accounted information, how are we supposed (to apply this analysis in order) to understand perceptual knowledge and knowledge by testimony? In the first part of the article, I articulate an answer in terms of a re-interpretation of perception and testimony as data providers rather than full-blown cases of knowledge. Second, if perception and testimony are correctly understood as data providers, how are we supposed (to apply this analysis in order) to understand the semantic value of the data provided by such processes? In the second part of the article, I argue in favour of a constructionist hypothesis about how data may become meaningful for human cognitive agents through a process of repurposing of natural data/signals. The conclusion of the paper is that human agents are natural-born data hackers. © 2014 by Peeters Publishers. All rights reserved.

This thesis focuses on learning recurring patterns in multimodal perception. For that purpose it develops cognitive systems that model the mechanisms providing such capabilities to infants; a methodology that fits into thefield of developmental robotics.More precisely, this thesis revolves around two main topics that are, on the one hand the ability of infants or robots to imitate and understand human behaviors, and on the other the acquisition of language. At the crossing of these topics, we study the question of the how a developmental cognitive agent can discover a dictionary of primitive patterns from its multimodal perceptual flow. We specify this problem and formulate its links with Quine's indetermination of translation and blind source separation, as studied in acoustics.We sequentially study four sub-problems and provide an experimental formulation of each of them. We then describe and test computational models of agents solving these problems. They are particularly based on bag-of-words techniques, matrix factorization algorithms, and inverse reinforcement learning approaches. We first go in depth into the three separate problems of learning primitive sounds, such as phonemes or words, learning primitive dance motions, and learning primitive objective that compose complex tasks. Finally we study the problem of learning multimodal primitive patterns, which corresponds to solve simultaneously several of the aforementioned problems. We also details how the last problems models acoustic words grounding.; Cette thèse considère l'apprentissage de motifs récurrents dans la perception multimodale. Elle s'attache à développer des modèles robotiques de ces facultés telles qu'observées chez l'enfant, et elle s'inscrit en cela dans le domaine de la robotique développementale.Elle s'articule plus précisément autour de deux thèmes principaux qui sont d'une part la capacité d'enfants ou de robots à imiter et à comprendre le comportement d'humains, et d'autre part l'acquisition du langage. A leur intersection, nous examinons la question de la découverte par un agent en développement d'un répertoire de motifs primitifs dans son flux perceptuel. Nous spécifions ce problème et établissons son lien avec ceux de l'indétermination de la traduction décrit par Quine et de la séparation aveugle de source tels qu'étudiés en acoustique.Nous en étudions successivement quatre sous-problèmes et formulons une définition expérimentale de chacun. Des modèles d'agents résolvant ces problèmes sont également décrits et testés. Ils s'appuient particulièrement sur des techniques dites de sacs de mots, de factorisation de matrices et d'apprentissage par renforcement inverse. Nous approfondissons séparément les trois problèmes de l'apprentissage de sons élémentaires tels les phonèmes ou les mots, de mouvements basiques de danse et d'objectifs primaires composant des tâches motrices complexes. Pour finir nous étudions le problème de l'apprentissage d'éléments primitifs multimodaux, ce qui revient à résoudre simultanément plusieurs des problèmes précédents. Nous expliquons notamment en quoi cela fournit un modèle de l'ancrage de mots acoustiques

The symbol grounding problem is the problem of specifying a semantics for the representations employed by a physical symbol system in a way that is neither circular nor regressive. The quantum system identification problem is the problem of relating observational outcomes to specific collections of physical degrees of freedom, i.e., to specific Hilbert spaces. It is shown that with reasonable physical assumptions these problems are equivalent. As the quantum system identification problem is demonstrably unsolvable by finite means, the symbol grounding problem is similarly unsolvable.

Numerous impressive proposals addressing the title question have been made that appear to be rich, coherent, and stimulating but are either theoretically incomplete or empirically weak. It is proposed here that recent scientific results provide the elements for an empirically robust hypothesis that fills some theoretical gaps. Prompted by Henshilwood’s archeological discovery at Blombos SA and drawing from Tomasello’s results in comparative psychology on shared intentionality and from Rizzolotti and Arbib’s results in neuropsychology on mirror neurons, and adapting Sterelny’s notion of decoupling, an empirically robust hypothesis is proposed: deliberate symbolic artifacts of material culture preceded, triggered, and facilitated the origination of modern language and abstract thought. Of the three classic features of modern humans—art, language, and abstract thought—language has been an exclusive focus of many twentieth-century theorists, a focus attributed to advances in logic and computational modeling, with background assumptions from a simplistic epistemology. Hypotheses by Cassirer and Langer that art was prior to and facilitated the origination of language were dismissed by what we now know were false premises. It is proposed here that art enabled the critical features of displacement and communally shared semantic content required by human languages. The proposal also provides a causal answer to Harnad’s “symbol grounding problem.” A first-approximation sketch of the empirical model is given.

This thesis focuses on learning recurring patterns in multimodal perception. For that purpose it develops cognitive systems that model the mechanisms providing such capabilities to infants; a methodology that fits into thefield of developmental robotics.More precisely, this thesis revolves around two main topics that are, on the one hand the ability of infants or robots to imitate and understand human behaviors, and on the other the acquisition of language. At the crossing of these topics, we study the question of the how a developmental cognitive agent can discover a dictionary of primitive patterns from its multimodal perceptual flow. We specify this problem and formulate its links with Quine's indetermination of translation and blind source separation, as studied in acoustics.We sequentially study four sub-problems and provide an experimental formulation of each of them. We then describe and test computational models of agents solving these problems. They are particularly based on bag-of-words techniques, matrix factorization algorithms, and inverse reinforcement learning approaches. We first go in depth into the three separate problems of learning primitive sounds, such as phonemes or words, learning primitive dance motions, and learning primitive objective that compose complex tasks. Finally we study the problem of learning multimodal primitive patterns, which corresponds to solve simultaneously several of the aforementioned problems. We also details how the last problems models acoustic words grounding., Cette thèse considère l'apprentissage de motifs récurrents dans la perception multimodale. Elle s'attache à développer des modèles robotiques de ces facultés telles qu'observées chez l'enfant, et elle s'inscrit en cela dans le domaine de la robotique développementale.Elle s'articule plus précisément autour de deux thèmes principaux qui sont d'une part la capacité d'enfants ou de robots à imiter et à comprendre le comportement d'humains, et d'autre part l'acquisition du langage. A leur intersection, nous examinons la question de la découverte par un agent en développement d'un répertoire de motifs primitifs dans son flux perceptuel. Nous spécifions ce problème et établissons son lien avec ceux de l'indétermination de la traduction décrit par Quine et de la séparation aveugle de source tels qu'étudiés en acoustique.Nous en étudions successivement quatre sous-problèmes et formulons une définition expérimentale de chacun. Des modèles d'agents résolvant ces problèmes sont également décrits et testés. Ils s'appuient particulièrement sur des techniques dites de sacs de mots, de factorisation de matrices et d'apprentissage par renforcement inverse. Nous approfondissons séparément les trois problèmes de l'apprentissage de sons élémentaires tels les phonèmes ou les mots, de mouvements basiques de danse et d'objectifs primaires composant des tâches motrices complexes. Pour finir nous étudions le problème de l'apprentissage d'éléments primitifs multimodaux, ce qui revient à résoudre simultanément plusieurs des problèmes précédents. Nous expliquons notamment en quoi cela fournit un modèle de l'ancrage de mots acoustiques

International audience; Number skills are popularly bound to arithmetic knowledge in its symbolic form, such as " five + nine = fourteen, " but mounting evidence suggests that these symbolic relations are actually grounded, i.e., computed (see Harnad, 1990) on noisy internal magnitude representations that bear our general understanding of numbers and further improve with math experience (Figure 1). Multiple lines of evidence support the idea of semantics-based arithmetic, including behavioral research on humans (Gallistel and Gelman, 1992), animals (Gallistel and Gelman, 2000; Rugani et al., 2009), development (Halberda et al., 2008), mathematical disability, i.e., dyscalculia (Butterworth, 1999; review, Butterworth et al., 2011), and computational model-ing (Stoianov et al., 2004; review, Zorzi et al., 2005). Even more intimate relation between the number skills and the internal noisy magnitudes was recently demonstrated in several studies showing finer magnitude representations in subjects with greater arithmetic fluency (e.g., Nys et al., 2013; Piazza et al., 2013), also caused by extensive math studying during higher education (Lindskog et al., 2014). Here we discuss how these findings could be explained within a generative framework of cognition, according to which top-down predictive connections play a key role in the computing of low-to high-level representations (e.g., Friston, 2010; Clark, 2013). The noisy internal magnitude representations also known as Approximate Number System (ANS), or Number Sense are systematically found in the intraparietal silcus and prefrontal cor-tices (Dehaene, 1997; Viswanathan and Nieder, 2013) and one principle method to investigate them is to characterize the ability to quickly and approximately estimate the number of objects seen (Jevons, 1871). This phylogenetic ability is qualified as a visual sense (Burr and Ross, 2008), the mechanism of which emerged in generative neural networks that learn to efficiently encode visual numerosities (Stoianov and Zorzi, 2012). One crucial property of the internal magnitude representations is their systematically increasing imprecision (Figure 1; Gallistel and Gelman, 2000; Dehaene, 2003) characterized by a subject-specific constant known as ANS acuity (Halberda et al., 2008), which at the behavioral level is associated with log-linear performance decrement as the magnitude increases. In numerosity comparison, the probability to select the greater numerosity is a sigmoid function of the log-ratio of the compared magnitudes that is characterized by a dis-criminability (Weber) fraction w describing the slope of the sigmoid, whereby the better the discriminability, the closer is the sigmoid to a step-function, for which w = 0 (Piazza et al., 2013; Cappelletti et al., 2014). The behavioral discriminabil-ity coefficient w is closely related to the internal ANS acuity (Piazza et al., 2013). The ANS acuity progressively improves along with development, with corresponding w = 1 in the first few months of life to about w = 0.24 in healthy adults (Piazza et al., 2010), to worsen then with ageing to more than w = 0.30 (Cappelletti et al., 2014). The intriguing question we explore here is whether ANS improves along with refinement of the mathematical knowledge it supports, that is, whether math-studying improves general quantity understanding. A hint about this was provided by a study on a curious Amazonian Mundurucù population with two levels of math education (Piazza et al., 2013). The effect of math-studying on ANS acu-ity, controlling for age, was impressive: w = 0.31 for adults that had never studied math and w = 0.19 for math-educated adults. Sure, this is a study on a particular population, with an educational system that permitted to find subjects allowing the dissociation between age and education (see also Nys et al., 2013), and it remained unclear whether prolonged math schooling in cultures with broad educational system is associated with further improvement of the ANS. Lindskog et al. (2014) investigated this issue with first-and third-year university students majoring in disciplines with various levels of initial math-expertise and amount of math-studying, ranked in the order: humanity-disciplines, with expected basic math-background, and no math-studying, business-disciplines with average math-background and applied math-studying, and math-disciplines with high math proficiency and mostly theoretical math-studying. The ANS-acuity of the students was evaluated using visual numerosity comparison tasks and supplementary assessment measured their arithmetic fluency. Overall, the arithmetic fluency increased along with the rank of the math-expertise and similarly, the

We show how event extraction can be used for handling delayed response tasks with arbitrary delay periods between the stimulus and the cue for response. We use a simple recurrent network for solving the task. Our approach is based on a number of information processing levels, where the lowest level works on raw time-step based sensory data. This data is classified using an unsupervised clustering mechanism. The second level works on this classified data, but still on the individual time-step basis. An event extraction mechanism detects and signals transitions between classes; this forms the basis for the third level. As this level only is updated when events occur, it is independent of the time-scale of the lower level interaction. We also sketch how an event filtering mechanism could be constructed which discards irrelevant data from the event stream. Such a mechanism would output a fourth level representation which could be used for delayed response tasks where irrelevant, or distracting, events could occur during the delay.