Search

Neuronal populations code similar concepts by similar activity patterns across the human brain's semantic networks. However, it is unclear to what extent such meaning-to-symbol mapping reflects distributional statistics, or experiential information grounded in sensorimotor and emotional knowledge. We asked whether integrating distributional and experiential data better distinguished conceptual categories than each method taken separately. We examined the similarity structure of fMRI patterns elicited by visually presented action- and object-related words using representational similarity analysis (RSA). We found that the distributional and experiential/integrative models respectively mapped the high-dimensional semantic space in left inferior frontal, anterior temporal, and in left precentral, posterior inferior/middle temporal cortex. Furthermore, results from model comparisons uncovered category-specific similarity patterns, as both distributional and experiential models matched the similarity patterns for action concepts in left fronto-temporal cortex, whilst the experiential/integrative (but not distributional) models matched the similarity patterns for object concepts in left fusiform and angular gyrus. [ABSTRACT FROM AUTHOR]

Kemmerer captured the drastic change in theories of word meaning representations, contrasting the view that word meaning representations are amodal and universal, with the view that they are grounded and language‐specific. However, he does not address how language can be simultaneously grounded and language‐specific. Here, we approach this question from the perspective of language acquisition and evolution. We argue that adding a new element—iconicity—is critically beneficial and offer the iconicity ring hypothesis, which explains how language‐specific, secondary iconicity might emerge from biologically grounded and universally shared iconicity in the course of language acquisition and evolution. To complement Kemmerer's main argument that language can be grounded to body but simultaneously language specific, we offer the iconicity ring hypothesis, which explains how language‐specific, secondary iconicity might emerge from biologically grounded and universally shared iconicity in the course of language acquisition and evolution. [ABSTRACT FROM AUTHOR]

Many approaches were proposed to model complex socially emotional behavior in virtual actors, such as intelligent tutors, creative assistants, or team partners. They still lack the 'magic' of human-level cognition: systems built for one paradigm appear clueless outside of its boundary. Natural cognitive systems, on the other hand, can adapt to unexpected environments and paradigms. To capture the robustness of natural cognitive development, a new approach is proposed here that enables the formation of new higher cognitive abilities in a model system, embedded in an unexpected environment. This is achieved based on the naturally developing grounding of innate abstract constructs (schemas). The mechanism producing this binding is that of creative insight. In this study, principles of insight processes borrowed from psychology are formalized and adapted for computer modeling. To do this, several examples of insight phenomena at different evolutionary levels and in different species are analyzed before the model is formulated based on the dual process theory, the signal model of insight, and the eBICA cognitive architecture framework. Results of its computer simulations prove the concept. One specific finding is that the accumulation of activation during the incubation period increases creative abilities of the system. It is argued that the proposed approach can explain a range of facts and mysteries associated with the human cognitive ontogeny and can provide the basis for a self-sustained evolution of future Artificial Intelligence. [ABSTRACT FROM AUTHOR]

In order to fulfil their essential roles as the bearers of truth and the relata of logical relations, propositions must be public and shareable. That requirement has favoured Platonist and other non-mental views of them, despite the well-known problems of Platonism in general. Views that propositions are mental entities have correspondingly fallen out of favour, as they have difficulty in explaining how propositions could have shareable, objective properties. We revive a mentalist view of propositions, inspired by Artificial Intelligence work on perceptual algorithms, which shows how perception causes persistent mental entities with shareable properties that allow them to fulfil the traditional roles of (one core kind of) propositions. The clustering algorithms implemented in perception produce outputs which are (implicit) atomic propositions in different minds. Coordination of them across minds proceeds by game-theoretic processes of communication. The account does not rely on any unexplained notions such as mental content, representation, or correspondence (although those notions are applicable in philosophical analysis of the result).

This research investigated the acquisition process of thecolor lexicon, specifically how color words are initiallygrounded and develop into the lexical system possessedby the adults in the ambient language. We conducted alongitudinal study in which Japanese-learning 2-year-olds were tested every month on their understanding ofbasic words denoting 8 chromatic colors, continuinguntil they were able to map these words onto theirreferents consistently. The results strongly endorse theview that acquisition of the color lexicon should becharacterized as a process of system construction,through which children reorganize prelinguistic colorcategories onto the linguistic categories of the ambientlanguage, thereby representations of individual wordsare continuously refined along with the refinement ofthe representation of the system as a whole.

Humans create internal models of an environment (i.e. cognitive maps) through subjective sensorimotor experiences and can also understand spatial locations by looking at an external map as a symbol of an environment. We simulate the development of the cognitive map from sensorimotor experiences and grounding of the external map in a single deep neural network model. Our proposed network has a shared module that processes the features of multiple modalities (i.e. vision, hearing, and touch) and even external maps in the same manner. The multiple modalities are encoded into feature vectors by modality-specific encoders, and the encoded features are processed by the same shared module. The proposed network was trained to predict the sensory inputs of a simulated mobile robot. After the predictive learning, the spatial representation was developed in the internal states of the shared module, and the same spatial representation was used for predicting multiple modalities, including the external map. The network can also perform spatial navigation by associating the external map with the cognitive map. This implies that the external maps are grounded in subjective sensorimotor experiences, being bridged through the developed internal spatial representation in the shared module. [ABSTRACT FROM AUTHOR]

Deacon speculates on the origin of interpretation of signs using autocatalytic origin of life models and Peircean terminology. I explain why interpretation evolved only later as a triadic intervention between symbols and actions. In all organisms the passive one-dimensional genetic informational symbol sequences are converted to active functional proteins or nucleic acids by three-dimensional folding. This symbol grounding is a direct symbol-to-action conversion. It is universal throughout all evolution. Folding is entirely a lawful physical process, leaving neither freedom nor necessity for interpretation. Similarly, the initial converse action-to-symbol conversion of sensory inputs also leaves no freedom for interpretation until after the action-to-symbol conversion. [ABSTRACT FROM AUTHOR]

Recent advances in robotics allow for collaboration between humans and machines in performing tasks at home or in industrial settings without harming the life of the user. While humans can easily adapt to each other and work in team, it is not as trivial for robots. In their case, interaction skills typically come at the cost of extensive programming and teaching. Besides, understanding the semantics of a task is necessary to work efficiently and react to changes in the task execution process. As a result, in order to achieve seamless collaboration, appropriate reasoning, learning skills and interaction capabilities are needed. For us humans, a cornerstone of our communication is language that we use to teach, coordinate and communicate. In this paper we thus propose a system allowing (i) to teach new action semantics based on the already available knowledge and (ii) to use natural language communication to resolve ambiguities that could arise while giving commands to the robot. Reasoning then allows new skills to be performed either autonomously or in collaboration with a human. Teaching occurs through a web application and motions are learned with physical demonstration of the robotic arm. We demonstrate the utility of our system in two scenarios and reflect upon the challenges that it introduces.

Many approaches were proposed to model socially emotional behavior in virtual actors, such as intelligent tutors, creative assistants or team partners. They still lack the 'magic' of human-level cognition and emotionality: systems built for one paradigm appear uncanny outside of its boundary. Natural cognitive systems, on the other hand, can adapt to unexpected environments and paradigms. To capture the robustness of natural cognitive development, a new approach is proposed here that enables the formation of higher cognitive abilities in a model cognitive system, embedded in an unexpected environment. This is achieved on the basis of a naturally developing binding of innate constructs to channels of the interface and features of the environment. The eBICA cognitive architecture used as a basis for this study combines rational, cognitive and somatic factors. Its building blocks are moral schemas and semantic maps. Building our previous study of the eBICA cognitive architecture, the present work takes it to the new level. It is argued that the proposed approach is capable of explaining, based on a unified standpoint, a number of facts and mysteries associated with the human cognitive ontogeny and can provide a basis for the strong Artificial Intelligence. [ABSTRACT FROM AUTHOR]

As a full-blown research topic, numerical cognition is investigated by a variety of disciplines including cognitive science, developmental and educational psychology, linguistics, anthropology and, more recently, biology and neuroscience. However, despite the great progress achieved by such a broad and diversified scientific inquiry, we are still lacking a comprehensive theory that could explain how numerical concepts are learned by the human brain. In this perspective, I argue that computer simulation should have a primary role in filling this gap because it allows identifying the finer-grained computational mechanisms underlying complex behavior and cognition. Modeling efforts will be most effective if carried out at cross-disciplinary intersections, as attested by the recent success in simulating human cognition using techniques developed in the fields of artificial intelligence and machine learning. In this respect, deep learning models have provided valuable insights into our most basic quantification abilities, showing how numerosity perception could emerge in multi-layered neural networks that learn the statistical structure of their visual environment. Nevertheless, this modeling approach has not yet scaled to more sophisticated cognitive skills that are foundational to higher-level mathematical thinking, such as those involving the use of symbolic numbers and arithmetic principles. I will discuss promising directions to push deep learning into this uncharted territory. If successful, such endeavor would allow simulating the acquisition of numerical concepts in its full complexity, guiding empirical investigation on the richest soil and possibly offering far-reaching implications for educational practice.

As a full-blown research topic, numerical cognition is investigated by a variety of disciplines including cognitive science, developmental and educational psychology, linguistics, anthropology and, more recently, biology and neuroscience. However, despite the great progress achieved by such a broad and diversified scientific inquiry, we are still lacking a comprehensive theory that could explain how numerical concepts are learned by the human brain. In this perspective, I argue that computer simulation should have a primary role in filling this gap because it allows identifying the finer-grained computational mechanisms underlying complex behavior and cognition. Modeling efforts will be most effective if carried out at cross-disciplinary intersections, as attested by the recent success in simulating human cognition using techniques developed in the fields of artificial intelligence and machine learning. In this respect, deep learning models have provided valuable insights into our most basic quantification abilities, showing how numerosity perception could emerge in multi-layered neural networks that learn the statistical structure of their visual environment. Nevertheless, this modeling approach has not yet scaled to more sophisticated cognitive skills that are foundational to higher-level mathematical thinking, such as those involving the use of symbolic numbers and arithmetic principles. I will discuss promising directions to push deep learning into this uncharted territory. If successful, such endeavor would allow simulating the acquisition of numerical concepts in its full complexity, guiding empirical investigation on the richest soil and possibly offering far-reaching implications for educational practice. [ABSTRACT FROM AUTHOR]

A significant body of research links individual differences in symbolic numerical abilities, such as arithmetic, to number sense, the neurobiological system used to approximate and manipulate quantities without language or symbols. However, recent findings from cognitive neuroscience challenge this influential theory. Our current review presents an overview of evidence for the number sense account of symbolic numerical abilities and then reviews recent studies that challenge this account, organized around the following four assertions. (1) There is no number sense as traditionally conceived. (2) Neural substrates of number sense are more widely distributed than common consensus asserts, complicating the neurobiological evidence linking number sense to numerical abilities. (3) The most common measures of number sense are confounded by other cognitive demands, which drive key correlations. (4) Number sense and symbolic number systems (Arabic digits, number words, and so on) rely on distinct neural mechanisms and follow independent developmental trajectories. The review follows each assertion with comments on future directions that may bring resolution to these issues. [ABSTRACT FROM AUTHOR]

Modern computing is generally taken to consist primarily of symbol manipulation. But symbols are abstract, and computers are physical. How can a physical device manipulate abstract symbols? Neither Church nor Turing considered this question. My answer is that the bit, as a hardware-implemented abstract data type, serves as a bridge between materiality and abstraction. Computing also relies on three other primitive—but more straightforward—abstractions: Sequentiality, State, and Transition. These physically-implemented abstractions define the borderline between hardware and software and between physicality and abstraction. At a deeper level, asking how a physical device can interact with abstract symbols is the wrong question. The relationship between symbols and physical devices begins with the realization that human beings already know what it means to manipulate symbols. We build and program computers to do what we understand to be symbol manipulation. To understand what that means, consider a light switch. A light switch doesn't turn a light on or off. Those are abstractions. Light switches don't operate with abstractions. We build light switches (and their associated circuitry), so that when flipped, the world is changed in such a way that we understand the light to be on or off. Similarly, we build computers to perform operations that we understand as manipulating symbols. [ABSTRACT FROM AUTHOR]

From neuroscience, brain imaging and the psychology of memory, we are beginning to assemble an integrated theory of the brain subsystems and pathways that allow the compression, storage and reconstruction of memories for past events and their use in contextualizing the present and reasoning about the future-mental time travel (MTT). Using computational models, embedded in humanoid robots, we are seeking to test the sufficiency of this theoretical account and to evaluate the usefulness of braininspired memory systems for social robots. In this contribution, we describe the use of machine learning techniques-Gaussian process latent variable models-to build a multimodal memory system for the iCub humanoid robot and summarize results of the deployment of this system for human-robot interaction. We also outline the further steps required to create a more complete robotic implementation of human-like autobiographical memory and MTT. We propose that generative memory models, such as those that form the core of our robot memory system, can provide a solution to the symbol grounding problem in embodied artificial intelligence. [ABSTRACT FROM AUTHOR]

In a multirobot system, using shared symbols for objects in the environment is a prerequisite for collaboration. Sharing symbols requires that each agent has anchored a symbol with an internal, sensor level representation, as well as that these symbols match between the agents. The problem can be solved easily when the internal representations can be communicated between the agents. However, with heterogeneous embodiments the available sensors are likely to differ, making it impossible to share the internal representations directly. We propose the use of pointing gestures to align symbols between a heterogeneous group of robots. We describe a planning framework that minimizes the required effort for anchoring representations across robots. The framework allows planning for both the gesturing and observing agents in a decentralized fashion. It considers both implicit sources of failure, such as ambiguous pointing, as well as costs required by actions. Simulation experiments demonstrate that the resulting planning problem has a complex solution structure with multiple local minima. Demonstration with a heterogeneous two-robot system shows the practical viability of this approach. [ABSTRACT FROM AUTHOR]

Automatic knowledge grounding is still an open problem in cognitive robotics. Recent research in developmental robotics suggests that a robot's interaction with its environment is a valuable source for collecting such knowledge about the effects of robot's actions. A useful concept for this process is that of an affordance, defined as a relationship between an actor, an action performed by this actor, an object on which the action is performed, and the resulting effect. This paper proposes a formalism for defining and identifying affordance equivalence. By comparing the elements of two affordances, we can identify equivalences between affordances, and thus acquire grounded knowledge for the robot. This is useful when changes occur in the set of actions or objects available to the robot, allowing to find alternative paths to reach goals. In the experimental validation phase we verify if the recorded interaction data is coherent with the identified affordance equivalences. This is done by querying a Bayesian Network that serves as container for the collected interaction data, and verifying that both affordances considered equivalent yield the same effect with a high probability.

An important characteristic of human language is compositionality. We can efficiently express a wide variety of real-world situations, events, and behaviors by compositionally constructing the meaning of a complex expression from a finite number of elements. Previous studies have analyzed how machine-learning models, particularly neural networks, can learn from experience to represent compositional relationships between language and robot actions with the aim of understanding the symbol grounding structure and achieving intelligent communicative agents. Such studies have mainly dealt with the words (nouns, adjectives, and verbs) that directly refer to real-world matters. In addition to these words, the current study deals with logic words, such as “not,” “and,” and “or” simultaneously. These words are not directly referring to the real world, but are logical operators that contribute to the construction of meaning in sentences. In human–robot communication, these words may be used often. The current study builds a recurrent neural network model with long short-term memory units and trains it to learn to translate sentences including logic words into robot actions. We investigate what kind of compositional representations, which mediate sentences and robot actions, emerge as the network's internal states via the learning process. Analysis after learning shows that referential words are merged with visual information and the robot's own current state, and the logical words are represented by the model in accordance with their functions as logical operators. Words such as “true,” “false,” and “not” work as non-linear transformations to encode orthogonal phrases into the same area in a memory cell state space. The word “and,” which required a robot to lift up both its hands, worked as if it was a universal quantifier. The word “or,” which required action generation that looked apparently random, was represented as an unstable space of the network's dynamical system.

In this paper, we propose a Bayesian generative model that can form multiple categories based on each sensory-channel and can associate words with any of the four sensory-channels (action, position, object, and color). This paper focuses on cross-situational learning using the co-occurrence between words and information of sensory-channels in complex situations rather than conventional situations of cross-situational learning. We conducted a learning scenario using a simulator and a real humanoid iCub robot. In the scenario, a human tutor provided a sentence that describes an object of visual attention and an accompanying action to the robot. The scenario was set as follows: the number of words per sensory-channel was three or four, and the number of trials for learning was 20 and 40 for the simulator and 25 and 40 for the real robot. The experimental results showed that the proposed method was able to estimate the multiple categorizations and to learn the relationships between multiple sensory-channels and words accurately. In addition, we conducted an action generation task and an action description task based on word meanings learned in the cross-situational learning scenario. The experimental results showed that the robot could successfully use the word meanings learned by using the proposed method.

Abstract This paper describes a method for processing natural text and assigning meaning to it without the need of any a priori linguistic knowledge. The semiotic cognitive automaton is driven only by the observations it makes and operates based solely on grounded symbols. An experiment of learning from one book in the English language is detailed, where certain paradigms of words are first formed thanks to side effects occurring in samples of text and then connected to functions and structures part of the internal reality of the automaton by way of meta-reasoning. Making use of the reflectivity of a functional language is envisaged to fully specify the reflection engine component of the automaton. In wait of a complete software specification, it is possible to carry out a mental experiment to validate the capabilities of the automaton. The method is general enough to learn semantics in multiple domains and lies the foundation of Semiotic Artificial Intelligence. [ABSTRACT FROM AUTHOR]

Abstract Language acquisition theories classically distinguish passive language understanding from active language production. However, recent findings show that brain areas such as Broca’s region are shared in language understanding and production. Furthermore, these areas are also implicated in understanding and producing goal-oriented actions. These observations question the passive view of language development. In this work, we propose a cognitive developmental model of symbol acquisition, coherent with an active view of language learning. For that purpose, we introduce the concept of social babbling. In this view, symbols are learned in the same way as goal-oriented actions in the context of specific caregiver–infant interactions. We show that this model allows a virtual agent to learn both symbolic words and gestures to refer to objects while interacting with a caregiver. We validate our model by reproducing results from studies on the influence of parental responsiveness on infants language acquisition. [ABSTRACT FROM AUTHOR]

This paper reviews computational modelling approaches to the learning of abstract concepts and words in embodied agents such as humanoid robots. This will include a discussion of the learning of abstract words such as 'use' and 'make' in humanoid robot experiments, and the acquisition of numerical concepts via gesture and finger counting strategies. The current approaches share a strong emphasis on embodied cognition aspects for the grounding of abstract concepts, and a continuum, rather than dichotomy, view of concrete/abstract concepts differences. [ABSTRACT FROM AUTHOR]

Automatic knowledge grounding is still an open problem in cognitive robotics. Recent research in developmental robotics suggests that a robot's interaction with its environment is a valuable source for collecting such knowledge about the effects of robot's actions. A useful concept for this process is that of an affordance, defined as a relationship between an actor, an action performed by this actor, an object on which the action is performed, and the resulting effect. This paper proposes a formalism for defining and identifying affordance equivalence. By comparing the elements of two affordances, we can identify equivalences between affordances, and thus acquire grounded knowledge for the robot. This is useful when changes occur in the set of actions or objects available to the robot, allowing to find alternative paths to reach goals. In the experimental validation phase we verify if the recorded interaction data is coherent with the identified affordance equivalences. This is done by querying a Bayesian Network that serves as container for the collected interaction data, and verifying that both affordances considered equivalent yield the same effect with a high probability. [ABSTRACT FROM AUTHOR]

Various perceptual domains have underlying compositional semantics that are rarely captured in current models. We suspect this is because directly learning the compositional structure has evaded these models. Yet, the compositional structure of a given domain can be grounded in a separate domain thereby simplifying its learning. To that end, we propose a new approach to modeling bimodal perceptual domains that explicitly relates distinct projections across each modality and then jointly learns a bimodal sparse representation. The resulting model enables compositionality across these distinct projections and hence can generalize to unobserved percepts spanned by this compositional basis. For example, our model can be trained on red triangles and blue squares; yet, implicitly will also have learned red squares and blue triangles. The structure of the projections and hence the compositional basis is learned automatically; no assumption is made on the ordering of the compositional elements in either modality. Although our modeling paradigm is general, we explicitly focus on a tabletop building-blocks setting. To test our model, we have acquired a new bimodal dataset comprising images and spoken utterances of colored shapes (blocks) in the tabletop setting. Our experiments demonstrate the benefits of explicitly leveraging compositionality in both quantitative and human evaluation studies. [ABSTRACT FROM PUBLISHER]

This article focuses on the interweaving of constructive praxis with communication in ontogenesis, in phylogenesis and in biocultural niche evolution (ecogenesis), within an EvoDevoSocio framework. I begin by discussing the nature of symbolization, its evolution from communicative signaling and its elaboration into semantic systems. I distinguish between the symbol-ready and the language-ready brain, leading to a discussion of linguistic conceptualization and its dual grounding in organism and language system. There follows an outline account of the interpenetration in the human biocultural niche-complex of semiosphere and technosphere, mediated by the evolution of the niche of infancy. Symbolization (the foundation of the semiosphere) is by definition normative; the normative character of the technosphere is demonstrated by the interrelations in human development between affordance, action schema and canonical functional object schema. A model of the neuro-computational implementation of dual grounding is proposed. [ABSTRACT FROM AUTHOR]