Search

As a full-blown research topic, numerical cognition is investigated by a variety of disciplines including cognitive science, developmental and educational psychology, linguistics, anthropology and, more recently, biology and neuroscience. However, despite the great progress achieved by such a broad and diversified scientific inquiry, we are still lacking a comprehensive theory that could explain how numerical concepts are learned by the human brain. In this perspective, I argue that computer simulation should have a primary role in filling this gap because it allows identifying the finer-grained computational mechanisms underlying complex behavior and cognition. Modeling efforts will be most effective if carried out at cross-disciplinary intersections, as attested by the recent success in simulating human cognition using techniques developed in the fields of artificial intelligence and machine learning. In this respect, deep learning models have provided valuable insights into our most basic quantification abilities, showing how numerosity perception could emerge in multi-layered neural networks that learn the statistical structure of their visual environment. Nevertheless, this modeling approach has not yet scaled to more sophisticated cognitive skills that are foundational to higher-level mathematical thinking, such as those involving the use of symbolic numbers and arithmetic principles. I will discuss promising directions to push deep learning into this uncharted territory. If successful, such endeavor would allow simulating the acquisition of numerical concepts in its full complexity, guiding empirical investigation on the richest soil and possibly offering far-reaching implications for educational practice.

Infants use exploratory behaviors to learn about the objects around them. Psychologists have theorized that behaviors such as grasping touching, pressing, and lifting, coupled with the visual, tactile, haptic and auditory sensory modalities, enable infants to form grounded object representations. For example, scratching an object can provide information about its roughness, while lifting it can provide information about its weight. In a sense, the exploratory behavior acts as a "question'' to the object, which is subsequently "answered" by the sensory stimuli produced during the execution of the behavior. In contrast, most object representations used by robots today rely solely on computer vision or laser scan data, gathered through passive observation. Such disembodied approaches to robotic perception may be useful for recognizing an object using a 3D model database, but nevertheless, will fail to infer object properties that cannot be detected using vision alone. To bridge this gap, our research has pursued a developmental framework for object perception and exploration in which the robot's representation of objects is grounded in its own sensorimotor experience with them \citesinapov2014grounding. In this framework, an object is represented by sensorimotor contingencies that span a diverse set of exploratory behaviors and sensory modalities. In this talk, I will highlight results from several large-scale experimental studies which show that the behavior-grounded object representation enables a robot to solve a wide variety of perceptual and cognitive tasks relevant to object learning \citesinapov2014learning,sinapov2011interactive. I will discuss recent work on how robots can ground language in multisensory experience with objects \citethomason2016learning and will conclude with a discussion on open problems in multisensory symbol grounding, which, if solved, could result in the large-scale deployment of robotic systems in real-world domains.

We address one-shot imitation learning, where the goal is to execute a previously unseen task based on a single demonstration. While there has been exciting progress in this direction, most of the approaches still require a few hundred tasks for meta-training, which limits the scalability of the approaches. Our main contribution is to formulate one-shot imitation learning as a symbolic planning problem along with the symbol grounding problem. This formulation disentangles the policy execution from the inter-task generalization and leads to better data efficiency. The key technical challenge is that the symbol grounding is prone to error with limited training data and leads to subsequent symbolic planning failures. We address this challenge by proposing a continuous relaxation of the discrete symbolic planner that directly plans on the probabilistic outputs of the symbol grounding model. Our continuous relaxation of the planner can still leverage the information contained in the probabilistic symbol grounding and significantly improve over the baseline planner for the one-shot imitation learning tasks without using large training data., IROS 2019

Human language is capable of communicating mental models between speakers of a language. The question why and how this works is closely tied to a specific variant of the symbol grounding problem that still leaves many open questions. This paper presents the core mechanism of the TextMap system, a logic-based system for generating visuospatial representations from textual input. The system leverages a recent discovery linking logical truth tables of formulae to images: a simple model counting mechanism that automatically extracts coordinate information from propositional Horn-logic knowledge bases encoding spatial predications. The system is based on a biologically inspired low-level bit vector mechanism, the activation bit vector machine (ABVM). It does not require an ontology apart from a list of which tokens indicate relations. Its minimalism and simplicity make TextMap a general purpose visualization or imagery tool. This paper demonstrates the core model counting mechanism and the results of a larger case study of a geographic layout of 13 cities.

The symbol grounding problem raises its head in the fields of the philosophy of AI, philosophy of psychology and philosophy of cognitive sciences. The solution to the symbol grounding problem must account for the genesis of mental representations in the world. It has to offer a strategy for grounding mental representations in the objective domain. Orthodox representationalist theories do not provide a satisfactory reply to the symbol grounding problem. On the other hand, there are embodied-enactivist approaches that dissolve the problem but only at the cost of representations and internal phenomenal states. The code model of biosemiotics provides a biologically viable (i.e., mechanistic) venue for developing a new solution to the problem. For the same reason, it could reconcile representationalism to the embodied approach.

The speed and accuracy with which robots are able to interpret natural language is fundamental to realizing effective human-robot interaction. A great deal of attention has been paid to developing models and approximate inference algorithms that improve the efficiency of language understanding. However, existing methods still attempt to reason over a representation of the environment that is flat and unnecessarily detailed, which limits scalability. An open problem is then to develop methods capable of producing the most compact environment model sufficient for accurate and efficient natural language understanding. We propose a model that leverages environment-related information encoded within instructions to identify the subset of observations and perceptual classifiers necessary to perceive a succinct, instruction-specific environment representation. The framework uses three probabilistic graphical models trained from a corpus of annotated instructions to infer salient scene semantics, perceptual classifiers, and grounded symbols. Experimental results on two robots operating in different environments demonstrate that by exploiting the content and the structure of the instructions, our method learns compact environment representations that significantly improve the efficiency of natural language symbol grounding., Accepted to ICRA 2019

The cognitive science research aims to make the machines to understand the human (natural) languages such as English and Tamil as the way humans do. As a result, machines can be programmed and instructed by human languages. As a starting point, the machine should understand the word meanings before it starts to understand the sentences. In the last fifty years, research community has put lots of effort to understand how to represent the meanings of words so that machine can manipulate the meanings in order to understand the sentences. Current research works represent the meaning of the word (i.e. symbol) in terms of other words like in dictionaries and this is useful only for humans to understand the meanings of the symbol and this approach is not suitable for machines. To overcome this issue, the concept of symbol grounding is proposed. Symbol grounding is the task of assigning meaning(s) to the word and physical symbol grounding is the task of assigning meaning to the word that refers to physical world object such as apple, cup, bottle, etc. The physical symbol grounding research focuses more on how to model or represent physical world object. Many research works have been done in the physical symbol grounding and they have the following limitations: a) the feature values can be seen as another way of representing the entity. Just capturing the color and two-dimensional shape parameters does not constitute the complete representation or modeling of the entity. So, the feature values extracted from the signal are incomplete. b) The feature values are helpful for the robot only to recognize the entity in the signals. The above systems assume that recognition is understanding, but actually it is not so. c) They use preprogrammed semantic rules to understand the meaning of the set of words (i.e. sentence) and also preprogrammed action rules to execute the behaviors in robot. Because of these limitations, machines using the above computational models for physical symbol grounding cannot become an autonomous or intelligent machine to understand the human language. To overcome the above limitations, a new architecture based on 3D object’s geometrical properties is proposed for the physical symbol grounding.

Ninety years ago, psychology had been characterized in terms of a history of critiques: scientific psychology, established by Wilhelm Wundt and his followers, and its empirical methods focusing on biological processes (e.g. reaction time experiments) was opposed to mythological or interpretive psychology aimed at the (meta-physical) meanings (Politzer 1929; Vygotsky 1997). One of the essential schisms associated with the two forms of psychology continues to the present day: the dichotomy of body and mind. It exists because scientific psychology is concerned with biological processes whereas interpretive psychology is concerned with the ideal meanings. The schism is a direct consequence of the “tacit presupposition of the mind with its private ideas which are in fact qualities without intelligible connection with the entities represented” (Whitehead 1929/1978, 76). Even though the problem was framed such a long time ago, psychologists and cognitive scientists have been wondering much more recently about how the (mental, immaterial) world of (abstract) ideas is connected to the physical world of the body, a problem that has come to be known as the symbol grounding problem (Harnad 1990). That psychophysiological schism manifests itself in the traditionally separate psychological treatments of intellect and affect (Vygotsky 1987). This separate treatment has continued to the present day, where mental-cognitive processes are thought to be influenced by affective states only from the outside, generally in terms of decreasing mental performance. The “affective domain (system)” is theorized separately from the “psychomotor domain (system),” which is in turn separate from the “cognitive domain (system)” (e.g. Sternberg and Williams 2010). We thereby get something like a system composed of subsystems, each of which exists in its own right (Fig. 1.1). The ways in which the independent subsystems come to interact does not tend to be further specified. This leaves open the question how an affective system, which quite obviously has bodily-physical qualities (e.g. blushing, increased heart rates) can ever influence a non-bodily system (ideal meanings). Thus, for example, how would affects, which are manifestations of physiological events, come to bear on something that is not material physiological, such as thinking and ideas? In other words and using a computer analogy, we have to ask, “How does the hardware affect the software?” In computing, of course, the former does not affect the latter – or the results of a computation would (might) be different depending on the chip used because of material differences.

A major weakness of all theories in educational psychology – including those of sociocultural and cultural-historical inclinations – is their intellectualist bent. Even Lev S. Vygotsky never explained how the scientific concepts that the children in his studies were learning came about in the first place, and how any abstract universal concept could have emerged in the course of human history – though he did intimate that all higher psychological functions were relations with others first before they were functions of the individual (Vygotsky 1989). Piaget’s main problem was the reduction of development to biological maturation underlying the different stages; and he used biological concepts for describing the development of qualities that are recognized to be purely societal or cultural in the approaches developed by social psychologists such as L. S. Vygotsky, A. N. Leont’ev, A. R. Luria, and their students and followers. Mainstream psychological theories also have an intellectualist bent in that they conceive of what we know in terms of ideal things, such as concepts, which are abstracted from the physical world. In the cognitive sciences and psychology, these ideal things exist somehow exist independent of the world, which, as suggested in Chap. 1, leads to the symbol grounding problem, that is, of the connection between the material world and ideal concepts. The problem of the missing connection has been made thematic in different names: the psychophysical or body–mind problem. The problem, however, is one of method and presupposition. As soon as we shift to a transactional approach, where everything is conceived in terms of events that together make the world-as-event, and where phenomena are the result of recurrences in percipient events and experiences, the existence of ideal things as an ensemble effect of material processes no longer is a mystery. This is so because in this approach, “all awareness, even awareness of concepts, requires at least the synthesis of physical feelings with conceptual feeling” (Whitehead 1929/1978, 243). Just as stereovision arises as a new quality in the cooperation of two eyes, mind is a new quality arising from the cooperation of people. This chapter is concerned with how ideals and universals come to exist in human history and in the life of young people (students in early elementary school). Even the birth of ideals is an event, and any ideal exists in and as a twofold relation – as intimated in the introductory quotation and, as seen below, in statements that Marx and Engels made in the mid-19th century: relation among people and relation between people and things.

From neuroscience, brain imaging and the psychology of memory, we are beginning to assemble an integrated theory of the brain subsystems and pathways that allow the compression, storage and reconstruction of memories for past events and their use in contextualizing the present and reasoning about the future—mental time travel (MTT). Using computational models, embedded in humanoid robots, we are seeking to test the sufficiency of this theoretical account and to evaluate the usefulness of brain-inspired memory systems for social robots. In this contribution, we describe the use of machine learning techniques—Gaussian process latent variable models—to build a multimodal memory system for the iCub humanoid robot and summarize results of the deployment of this system for human–robot interaction. We also outline the further steps required to create a more complete robotic implementation of human-like autobiographical memory and MTT. We propose that generative memory models, such as those that form the core of our robot memory system, can provide a solution to the symbol grounding problem in embodied artificial intelligence.This article is part of the theme issue ‘From social brains to social robots: applying neurocognitive insights to human–robot interaction’.

In Chap. 1, I show how classical psychological theorizing is ontologically based on the primacy of material and ideal things. The material things are located in Cartesian space, which is treated as the substrate for the relationship between material and geometrical bodies. Ideal things, on the other hand, are non-spatial and ephemeral. In cognitive psychology, this eventually has led to the question about how thought and mind are grounded and how concepts (theory, ideals) come to be related to the material world – the symbol grounding problem in the cognitive sciences. Interpreters of cultural psychology do not operate differently, so that the thing (object) becomes a Trojan horse by means of which Cartesian thinking enters and comes to dominate even those theories said to be about “(practical) activity.” This includes (rightly or wrongly) the theory that Vygotsky developed and also the one by his student A. N. Leont’ev, which today goes under the name of cultural-historical activity theory. As shown in Fig. 2.6 below, a mediational triangle –whether in Vygotsky’s original form or that build on the Helsinki interpretation of Leont’ev’s work – epitomizes the thinking in terms of objects; and the relations in those triangles also are things. Sign-things and tool-things come to stand between a person-thing and another person-thing or a material object-thing; or a tool stands between subject and object, a subject stands between division of labor and the object of activity. All these are things with external relation-things that are such so that other things can come to stand between and thereby mediate them.

This chapter constitutes the second of the four main chapters in the concept of Intelligent Physical Compliance. It addresses the question on how symbolic task descriptions can be translated into meaningful robot operations in the context of wiping tasks. To achieve this, three main elements are investigated, namely, symbol grounding and planning with symbols, planning algorithms for mobile manipulation, and effect-space planning for wiping tasks.

Recent research has suggested that numeral order processing - the speed and accuracy with which individuals can determine whether a set of digits is in numerical order or not - is related to arithmetic and mathematics outcomes. It has therefore been proposed that ordinal relations are a fundamental property of symbolic numeral representations. However, order information is also inherent in the verbal count sequence, and thus verbal count sequence knowledge may instead explain the relationship between performance on numeral order tasks and arithmetic. We explored this question with 62 children aged 6- to 8-years-old. We found that performance on a verbal count sequence knowledge task explained the relationship between numeral order processing and arithmetic. Moreover many children appeared to explicitly base their judgments of numerical order on count sequence information. This suggests that insufficient attention may have been paid to verbal number knowledge in understanding the sources of information that give meaning to numbers.

How many words (and which ones) are sufficient to define all other words? When dictionaries are analyzed as directed graphs with links from defining words to defined words, they reveal a latent structure. Recursively removing all words that are reachable by definition but that do not define any further words reduces the dictionary to a Kernel of about 10%. This is still not the smallest number of words that can define all the rest. About 75% of the Kernel turns out to be its Core, a Strongly Connected Subset of words with a definitional path to and from any pair of its words and no word's definition depending on a word outside the set. But the Core cannot define all the rest of the dictionary. The 25% of the Kernel surrounding the Core consists of small strongly connected subsets of words: the Satellites. The size of the smallest set of words that can define all the rest (the graph's Minimum Feedback Vertex Set or MinSet) is about 1% of the dictionary, 15% of the Kernel, and half-Core, half-Satellite. But every dictionary has a huge number of MinSets. The Core words are learned earlier, more frequent, and less concrete than the Satellites, which in turn are learned earlier and more frequent but more concrete than the rest of the Dictionary. In principle, only one MinSet's words would need to be grounded through the sensorimotor capacity to recognize and categorize their referents. In a dual-code sensorimotor-symbolic model of the mental lexicon, the symbolic code could do all the rest via re-combinatory definition., 38 pages, 10 figures, 2 tables, 73 references

A natural language interface exploits the conceptual simplicity and naturalness of the language to create a high-level user-friendly communication channel between humans and machines. One of the promising applications of such interfaces is generating visual interpretations of semantic content of a given natural language that can be then visualized either as a static scene or a dynamic animation. This survey discusses requirements and challenges of developing such systems and reports 26 graphical systems that exploit natural language interfaces and addresses both artificial intelligence and visualization aspects. This work serves as a frame of reference to researchers and to enable further advances in the field., Due to copyright most of the figures only appear in the journal version

Actual robot systems are typically used within an industrial environment, where they perform tasks in the areas of manufacturing, assembly, or inspection. Many of these tasks require the definition and execution of sensor-based motions. Such motions are typically defined using a subsymbolic representation like manipulation primitive nets [1] or iTaSC [2]. These representations are only suitable for experts in the domain of robotics because they require the definition of low-level parameters like setpoints, control strategies, or task frames. Since the application domains of robot systems shall be extended to SMEs or private households, existing representations lack in an intuitive interface for users without expert knowledge in the area of robotics. In this work, we describe an intuitive interface for the definition of sensor-based actions. Our approach is based on manipulation primitive nets and consists of a transformation between the subsymbolic manipulation primitive nets and a symbolic user interface, described by a domain specific language.

Intelligent systems need to understand and respond to human words to enable them to interact with humans in a natural way. Several studies attempted to realize these abilities by investigating the symbol grounding problem. For example, we proposed multilayered multimodal latent Dirichlet allocation (mMLDA) to enable the formation of various concepts and inference using grounded concepts. We previously reported on the issue of connecting words to various hierarchical concepts and also proposed a simple preliminary algorithm for generating sentences. This paper proposes a novel method that enables a sensing system to verbalize an everyday scene it observes. The method uses mMLDA and Bayesian hidden Markov models (BHMM) and the proposed algorithm improves the word inference of our previous work. The advantage of our approach is that grammar learning based on BHMM not only boosts concept selection results but enables our method to process functional words. The proposed verbalization algorithm produces results...

Through semiotic modelling, a system can retrieve and manipulate its own representational formats to interpret a series of observations; this is in contrast to information processing approaches that require representational formats to be specified beforehand and thus limit the semantic properties that the system can experience. Our semiotic cognitive automaton is driven only by the observations it makes and therefore operates based on grounded symbols. A best-case scenario for our automaton involves observations that are univocally interpreted—i.e. distinct observation symbols—and that make reference to a reality characterised by “hard constraints”. Arithmetic offers such a scenario. The gap between syntax and semantics is also subtle in the case of calculations. Our automaton starts without any a priori knowledge of mathematical formalisms and not only learns the syntactical rules by which arithmetic operations are solved but also reveals the true meaning of numbers by means of second-order reasoning.

How do numerical symbols, such as number words, acquire semantic meaning? This question, also referred to as the "symbol-grounding problem," is a central problem in the field of numerical cognition. Present theories suggest that symbols acquire their meaning by being mapped onto an approximate system for the nonsymbolic representation of number (Approximate Number System or ANS). In the present literature review, we first asked to which extent current behavioural and neuroimaging data support this theory, and second, to which extent the ANS, upon which symbolic numbers are assumed to be grounded, is numerical in nature. We conclude that (a) current evidence that has examined the association between the ANS and number symbols does not support the notion that number symbols are grounded in the ANS and (b) given the strong correlation between numerosity and continuous variables in nonsymbolic number processing tasks, it is next to impossible to measure the pure association between symbolic and nonsymbolic numerosity. Instead, it is clear that significant cognitive control resources are required to disambiguate numerical from continuous variables during nonsymbolic number processing. Thus, if there exists any mapping between the ANS and symbolic number, then this process of association must be mediated by cognitive control. Taken together, we suggest that studying the role of both cognitive control and continuous variables in numerosity comparison tasks will provide a more complete picture of the symbol-grounding problem.

One of the challenges of linguistic creativity is to use words in a way that is novel and striking and even whimsical, to convey meanings that remain stubbornly grounded in the very same world of familiar experiences as serves to anchor the most literal and unimaginative language. The challenge remains unmet by systems that merely shuttle or arrange words to achieve novel arrangements without concern as to how those arrangements are to spur the processes of meaning construction in a listener. In this paper we explore a problem of lexical invention that cannot be solved without a model – explicit or implicit – of the perceptual grounding of language: the invention of apt new names for colours. To solve this problem here we shall call upon the notion of a linguistic readymade, a phrase that is wrenched from its original context of use to be given new meaning and new resonance in new settings. To ensure that our linguistic readymades – which owe a great deal to Marcel Duchamp's notion of found art – are anchored in a consensus model of perception, we introduce the notion of a lexicalised colour stereotype.

Humans can acquire language through physical interaction with their environment and semiotic interaction with other people. It is very important to understand how humans can form a symbol system and obtain semiotic skills through their autonomous mental development from a computational point of view. A machine learning system that enables a robot to obtain and modulate its symbol system is crucially important to develop robotic systems that achieve long-term human-robot communication and collaboration. In this paper, I introduce the basis of our research field and related topics. Specifically, I describe the concept of symbol emergence systems and the recent research topics , e.g., multimodal categorization, spatial concept formation, language acquisition, and double articulation analysis, that will contribute to future human-robot communication and collaboration.

Nowadays, the research that robots learn the meaning of a word by relating speech instructions with signals of the sensorimotor system has started. Lots of pre-research on learning ‘concept’ that means the meaning of a word by robot have used the co-occurrence information of words and sensor signals directly. However, relative concepts, such as color, size or location, require comparing with other objects or concepts. The objects or concepts used for comparison are called “reference object.” In human-computer interaction, reference point may not be explicitly communicated, because human utterance is ambiguous. Therefore, estimating a true reference object from the multi-candidate is necessary to learn relative concepts. In this paper, we propose a learning method based on EM algorithm to estimate the reference point and learn relative concepts. In our experiments, a robot learned concepts expressing relative location, such as ‘in front of or ‘behind.’ The robot was given text labels expressing its locations from a human teacher in several situations. Nevertheless, the instructions didn’t include the true reference points and the coordinate system of each concept. Under these conditions, the robot could learn the concepts based on coordinate systems have the orientations of the reference objects or the teacher.

In this paper, I juxtapose the Symbol Grounding Problem and causal theories of reference. In the first part of the paper, I show some basic assumptions they share in order to show, in the second part, some difficulties implied by these assumptions. These difficulties are: the meaning determination problem, the easy and hard disjunction problem, and the trivialization problem. My diagnosis is that both the easy and hard disjunction problem result from a more general difficulty with causal theories and the SGP solution, which is the possibility of misrepresenting, and in particular of accounting for system-detectable error. I emphasize some implications they have for the notion of representation. Finally, I enumerate some theoretical desiderata for a satisfactory account of naturalized semantics (and solutions to SGP) that would be free of the problems mentioned above.

This paper reviews computational modelling approaches to the learning of abstract concepts and words in embodied agents such as humanoid robots. This will include a discussion of the learning of abstract words such as ‘use’ and ‘make’ in humanoid robot experiments, and the acquisition of numerical concepts via gesture and finger counting strategies. The current approaches share a strong emphasis on embodied cognition aspects for the grounding of abstract concepts, and a continuum, rather than dichotomy, view of concrete/abstract concepts differences. This article is part of the theme issue ‘Varieties of abstract concepts: development, use and representation in the brain’.

This chapter presents some difficulties with the characterisation of autonomous agents as information processing systems arising from the work on real autonomous mobile robots. It argues that the modern fashion for situatedness. The chapter also argues that the so-called symbol grounding problem is an inappropriate response to those who have pointed out that the world of an autonomous agent does not come ready categorized and neatly labeled. It presents a series of sketches based on the experience of trying to build real autonomous systems: simple mobile robots. In each sketch, it identifies some problems with characterising these robots as information processing systems. The chapter concludes with some speculations on what might be a more satisfactory characterisation, which has much in common with some ideas of Humberto Maturana and Francisco Varela.

This study aims at explaining the symbols used in the novel and describing the values of optimistic reflected in the symbols used in Hemingway’s The Old man and The Sea. This research is classified as a descriptive qualitative method. The primary data source of this study is the novel The Old Man and the sea by Ernest Hemingway. The collecting data process is note-taking technique. This study is also categorized as library research. Based on the findings and discussion, the researcher draws the conclusion such follow. First, it is found that there are eight symbols used in this novel. They are classified into Individual symbol and Universal symbol. In the novel the Old Man and the Sea the symbols which the researcher considers as individual symbols. Individual symbols are the old man as a symbol of someone who has optimistic, Manolin as a symbol of hope, Marlin as a symbol Struggle of life, Shark as a symbol of Destruction and problem, joe Di Maggio as a symbol Strong desire, bird as a symbol of help. The second category is Universal symbol. universal symbol are Sea as symbol of universe, Lion as symbol of strength. there are two elements or values which characterize someone as the one who has optimistic life which is reflected in the symbols used in this novel. They are the values of struggle and positive thinking, Second, Those symbols reflect the values of optimistic life which is to be the theme of the novel. Keywords: optimistic, symbol

Cognitive science is a form of "reverse engineering" (as Dennett has dubbed it). We are trying to explain the mind by building (or explaining the functional principles of) systems that have minds. A "Turing" hierarchy of empirical constraints can be applied to this task, from t1, toy models that capture only an arbitrary fragment of our performance capacity, to T2, the standard "pen-pal" Turing Test (total symbolic capacity), to T3, the Total Turing Test (total symbolic plus robotic capacity), to T4 (T3 plus internal [neuromolecular] indistinguishability). All scientific theories are underdetermined by data. What is the right level of empirical constraint for cognitive theory? I will argue that T2 is underconstrained (because of the Symbol Grounding Problem and Searle's Chinese Room Argument) and that T4 is overconstrained (because we don't know what neural data, if any, are relevant). T3 is the level at which we solve the "other minds" problem in everyday life, the one at which evolution operates (the Blind Watchmaker is no mind-reader either) and the one at which symbol systems can be grounded in the robotic capacity to name and manipulate the objects their symbols are about. I will illustrate this with a toy model for an important component of T3 -- categorization -- using neural nets that learn category invariance by "warping" similarity space the way it is warped in human categorical perception: within-category similarities are amplified and between-category similarities are attenuated. This analog "shape" constraint is the grounding inherited by the arbitrarily shaped symbol that names the category and by all the symbol combinations it enters into. No matter how tightly one constrains any such model, however, it will always be more underdetermined than normal scientific and engineering theory. This will remain the ineliminable legacy of the mind/body problem.

Humans use signs, e.g., sentences in a spoken language, for communication and thought. Hence, symbol systems like language are crucial for our communication with other agents and adaptation to our real-world environment. The symbol systems we use in our human society adaptively and dynamically change over time. In the context of artificial intelligence (AI) and cognitive systems, the symbol grounding problem has been regarded as one of the central problems related to {\it symbols}. However, the symbol grounding problem was originally posed to connect symbolic AI and sensorimotor information and did not consider many interdisciplinary phenomena in human communication and dynamic symbol systems in our society, which semiotics considered. In this paper, we focus on the symbol emergence problem, addressing not only cognitive dynamics but also the dynamics of symbol systems in society, rather than the symbol grounding problem. We first introduce the notion of a symbol in semiotics from the humanities, to leave the very narrow idea of symbols in symbolic AI. Furthermore, over the years, it became more and more clear that symbol emergence has to be regarded as a multifaceted problem. Therefore, secondly, we review the history of the symbol emergence problem in different fields, including both biological and artificial systems, showing their mutual relations. We summarize the discussion and provide an integrative viewpoint and comprehensive overview of symbol emergence in cognitive systems. Additionally, we describe the challenges facing the creation of cognitive systems that can be part of symbol emergence systems., 23 pages, 6 figures. Submitted to IEEE Transactions on Cognitive and Developmental Systems

I propose a formal approach towards solving Harnad’s “Symbol Grounding Problem” (SGP) through epistemological analogy. (Sect. 1) The SGP and Taddeo and Floridi’s “Zero Semantical Commitment Condition” (z-condition) for its solution are both revisited using Frege’s philosophy of language, in such a way that the SGP is converted into two circumscribed tasks. (Sect. 2) The ground for studying these tasks within human cognition is that both the human mind and AI are conceivable, as in Newell’s “physical symbol systems” (PSSs), and that they share the core of the SGP: the problem of constructing an objective reference. (Sect. 3) After two forms of reference have been identified in the human mind, I then show why the latter may constitute a model for facing the SGP.

`No' belongs to the first ten words used by children and embodies the first active form of linguistic negation. Despite its early occurrence the details of its acquisition process remain largely unknown. The circumstance that `no' cannot be construed as a label for perceptible objects or events puts it outside of the scope of most modern accounts of language acquisition. Moreover, most symbol grounding architectures will struggle to ground the word due to its non-referential character. In an experimental study involving the child-like humanoid robot iCub that was designed to illuminate the acquisition process of negation words, the robot is deployed in several rounds of speech-wise unconstrained interaction with na\"ive participants acting as its language teachers. The results corroborate the hypothesis that affect or volition plays a pivotal role in the socially distributed acquisition process. Negation words are prosodically salient within prohibitive utterances and negative intent interpretations such that they can be easily isolated from the teacher's speech signal. These words subsequently may be grounded in negative affective states. However, observations of the nature of prohibitive acts and the temporal relationships between its linguistic and extra-linguistic components raise serious questions over the suitability of Hebbian-type algorithms for language grounding., Comment: Submitted journal article. 21 pages main paper plus 28 pages supplementary information / appendix. 8 figures in main paper

The utility of collaborative manipulators for shared tasks is highly dependent on the speed and accuracy of communication between the human and the robot. The run-time of recently developed probabilistic inference models for situated symbol grounding of natural language instructions depends on the complexity of the representation of the environment in which they reason. As we move towards more complex bi-directional interactions, tasks, and environments, we need intelligent perception models that can selectively infer precise pose, semantics, and affordances of the objects when inferring exhaustively detailed world models is inefficient and prohibits real-time interaction with these robots. In this paper we propose a model of language and perception for the problem of adapting the configuration of the robot perception pipeline for tasks where constructing exhaustively detailed models of the environment is inefficient and inconsequential for symbol grounding. We present experimental results from a synthetic corpus of natural language instructions for robot manipulation in example environments. The results demonstrate that by adapting perception we get significant gains in terms of run-time for perception and situated symbol grounding of the language instructions without a loss in the accuracy of the latter.

An important characteristic of human language is compositionality. We can efficiently express a wide variety of real-world situations, events, and behaviors by compositionally constructing the meaning of a complex expression from a finite number of elements. Previous studies have analyzed how machine-learning models, particularly neural networks, can learn from experience to represent compositional relationships between language and robot actions with the aim of understanding the symbol grounding structure and achieving intelligent communicative agents. Such studies have mainly dealt with the words (nouns, adjectives, and verbs) that directly refer to real-world matters. In addition to these words, the current study deals with logic words, such as “not,” “and,” and “or” simultaneously. These words are not directly referring to the real world, but are logical operators that contribute to the construction of meaning in sentences. In human–robot communication, these words may be used often. The current study builds a recurrent neural network model with long short-term memory units and trains it to learn to translate sentences including logic words into robot actions. We investigate what kind of compositional representations, which mediate sentences and robot actions, emerge as the network's internal states via the learning process. Analysis after learning shows that referential words are merged with visual information and the robot's own current state, and the logical words are represented by the model in accordance with their functions as logical operators. Words such as “true,” “false,” and “not” work as non-linear transformations to encode orthogonal phrases into the same area in a memory cell state space. The word “and,” which required a robot to lift up both its hands, worked as if it was a universal quantifier. The word “or,” which required action generation that looked apparently random, was represented as an unstable space of the network's dynamical system.

From the start, Kurt Godel observed that computer and brain paradigms were considered on a par by researchers and that researchers had misunderstood his theorems. He hailed with displeasure that the brain transcends computers. In this brief article, we point out that Artificial Intelligence (AI) comprises multitudes of human-made methodologies, systems, and languages, and implemented with computer technology. These advances enhance development in the electron and quantum realms. In the biological realm, animal neurons function, also utilizing electron flow, and are products of evolution. Mirror neurons are an important paradigm in neuroscience research. Moreover, the paradigm shift proposed here - 'hall of mirror neurons' - is a potentially further productive research tactic. These concepts further expand AI and brain research.

A standard approach in the simulation of language evolution is the use of Language Games to model communicative interactions between intelligent agents. Usually, in such language games, an agent uses the results from its perceptual layer to categorize and to conceptualize world objects, a process named categorization. In this paper, we develop an approach to the categorization process, where the decomposition of reality in meaningful experiences is co-evolved with the lexicon formation in the language game. This approach brings some insights on how meaning might be assigned to symbols, in a dynamic and continuously changing environment being experienced by an agent. In order to do that, we use Barsalou’s notion of mental simulation and Gardenfors’ notion of conceptual spaces such that, together with ESOM neural networks, a cognitive architecture can be developed, where mental concepts formation and lexicon formation are able to co-evolve during a language game. The performance of our cognitive architecture is evaluated and the results show that the architecture is able to fulfill its semantics function, by allowing a population of agents to exchange the meaning of linguistic symbols during a naming game, without relying on “a priori” categorization scheme provided by an external expert or a set of examples for training a neural network in a previous discrimination game. These results, beyond bringing evidence on potential ways for symbols to get meaning on a biologically realistic way, open a set of possibilities for further uses of conceptual spaces on a much more complex problem: the grounding of a grammatical language.

As the accompanying articles in the Special Issue on Semiotic Scaffolding will attest, my colleagues in biosemiotics have done an exemplary job in showing us how to think about the critically generative role that semiotic scaffolding plays “vertically” – i.e., in evolutionary and developmental terms – by “allowing access to the upper floors” (Hoffmeyer Semiotica 198: 11–31, 2014a) of biological complexity, cognition and evolution.

How do abstract words get meaning? And how could sensorimotor experience based representation be used for abstract words? This is a very important problem for cognitive science, neuroscience and cognitive robotics because these words are far from perception and are perceived through the mind, not through the senses.. In this paper a cognitive robotic model is proposed to obtain abstract words semantic representation through indirect grounding in sensorimotor experience. The model employs the symbolic knowledge representation method (graph-based semantic network) for the robot's conceptual knowledge. The robot perceives abstract words in this model by referring to already grounded concrete words with sensorimotor experience, so it is not subject to the symbol grounding problem. An algorithm is written to obtain semantic referents of abstract words through transitive inference method. A simulation experiment has been designed for DARwIn-OP robot model for verification.