Search

Brain imaging studies of English past tense inflection have found dissociations between regular and irregular verbs, but no coherent picture has emerged to explain how these dissociations arise. Here we use synthetic brain imaging on a neural network model to provide a mechanistic account of the origins of such dissociations. The model suggests that dissociations between regional activation patterns in verb inflection emerge in an adult processing system that has been shaped through experience-dependent structural brain development. Although these dissociations appear to be between regular and irregular verbs, they arise in the model from a combination of statistical properties including frequency, relationships to other verbs, and phonological complexity, without a causal role for regularity or semantics. These results are consistent with the notion that all inflections are produced in a single associative mechanism. The model generates predictions about the patterning of active brain regions for different verbs that can be tested in future imaging studies. [ABSTRACT FROM AUTHOR]

Knowledge of common events is central to many aspects of cognition. Intuitively, it seems as though events are linear chains of the activities of which they are comprised. In line with this intuition, a number of theories of the temporal structure of event knowledge have posited mental representations (data structures) consisting of linear chains of activities. Competing theories focus on the hierarchical nature of event knowledge, with representations comprising ordered scenes, and chains of activities within those scenes. We present evidence that the temporal structure of events typically is not well‐defined, but it is much richer and more variable both within and across events than has usually been assumed. We also present evidence that prediction‐based neural network models can learn these rich and variable event structures and produce behaviors that reflect human performance. We conclude that knowledge of the temporal structure of events in the human mind emerges as a consequence of prediction‐based learning. McRae, Brown and Elman argue against the view that events are structured as frequently‐occurring sequences of world stimuli. They underline the importance of temporal structure defining event types and advance a more complex temporal structure, which allows for some variance in the component elements. [ABSTRACT FROM AUTHOR]

While the ability to acquire non-linearly separable (NLS) classifications is well documented in the study of human category learning, the relative ease of learning compared to a linear separable structure is difficult to evaluate without potential confounds. Medin and Schwanenflugel (Journal of Experimental Psychology: Human Learning and Memory, 7, 355-368, 1981) were the first to demonstrate that NLS classifications are not more difficult to acquire than linearly separable ones when structures are equated in terms of within- and between-category similarities. However, their evidence is less sturdy than might be expected due to non-standard methodology and low sample size. We conducted a conceptual replication to clarify the behavioral picture and perform qualitative testing of formal models. The behavioral results not only showed a lack of advantage for the linearly separable (LS) structure, but revealed a stronger finding: the NLS structure was reliably easier to acquire. Differences in the relative ease of NLS learners to master certain items yielded evidence for the existence of distinct learner subgroups, one marked by significantly easier (not harder) learning of exception items. Comparing the qualitative fits of leading computational models to the human learning performance confirmed that a pure prototype account, even with contemporary updates, remains incompatible with the data. However, exemplar models and similarity-based models grounded in sophisticated forms of abstraction-based learning successfully account for the NLS advantage. In sum, evidence against a linear separability constraint is redoubled, and the observed NLS advantage along with behavioral patterns seen at the subgroup and item level provide a valuable basis for comprehensive evaluation of competing theoretical accounts and models. [ABSTRACT FROM AUTHOR]

The connectionist model is one approach to studying language processing from a computational perspective. And building a representation in the connectionist model study is just as important as making the structure of the model in that it determines the level of learning and performance of the model. The connectionist model has been constructed in two different ways: localist representation and distributed representation. However, the localist representation used in the previous studies had limitations in that the unit of the output layer having a rare target activation value is inactivated, and the past distributed representation has the limitation of difficulty in confirming the result by the opacity of the displayed information. This has been a limitation of the overall connection model study. In this paper, we present a new method to induce distributed representation with local representation using abstraction of information, which is a feature of restricted Boltzmann machine, with respect to the limitation of such representation of the past. As a result, our proposed method effectively solves the problem of conventional representation by using the method of information compression and inverse transformation of distributed representation into local representation. [ABSTRACT FROM AUTHOR]

The recent "deep learning revolution" in artificial neural networks had strong impact and widespread deployment for engineering applications, but the use of deep learning for neurocomputational modeling has been so far limited. In this article we argue that unsupervised deep learning represents an important step forward for improving neurocomputational models of perception and cognition, because it emphasizes the role of generative learning as opposed to discriminative (supervised) learning. As a case study, we present a series of simulations investigating the emergence of neural coding of visual space for sensorimotor transformations. We compare different network architectures commonly used as building blocks for unsupervised deep learning by systematically testing the type of receptive fields and gain modulation developed by the hidden neurons. In particular, we compare Restricted Boltzmann Machines (RBMs), which are stochastic, generative networks with bidirectional connections trained using contrastive divergence, with autoencoders, which are deterministic networks trained using error backpropagation. For both learning architectures we also explore the role of sparse coding, which has been identified as a fundamental principle of neural computation. The unsupervised models are then compared with supervised, feed-forward networks that learn an explicit mapping between different spatial reference frames. Our simulations show that both architectural and learning constraints strongly influenced the emergent coding of visual space in terms of distribution of tuning functions at the level of single neurons. Unsupervised models, and particularly RBMs, were found to more closely adhere to neurophysiological data from single-cell recordings in the primate parietal cortex. These results provide new insights into how basic properties of artificial neural networks might be relevant for modeling neural information processing in biological systems. [ABSTRACT FROM AUTHOR]

The extent to which meaning is involved in reading aloud has proven an area of longstanding debate, and current computational models differ on this dimension. The connectionist triangle model proposes that normal individuals rely on semantic information for correct reading of words with atypical spelling-sound relationships, but to varying degrees. This proposed individual difference would account for the varying stage of decline at which patients with semantic dementia first show the reading impairment known as surface dyslexia. Recent neuroimaging data has provided validation of this view, showing that individual differences in degree of semantic reliance during exception word reading predict the amount of activation in left anterior temporal regions associated with semantic processing. This study aimed to establish the cognitive correlates of individual differences in semantic reliance during exception word reading. Experiment 1 used a subgrouping approach with 32 participants and found larger imageability and semantic priming effects specifically for exception word reading amongst high relative to low semantic reliance readers. High semantic reliance readers also tended to read nonwords more slowly than low semantic reliance readers. A second experiment used a regression approach with 129 readers and confirmed the relationship of degree of semantic reliance both to imageability effects in exception word reading and speed of nonword reading. Further, while the performance of the higher semantic readers revealed no significant association with semantic processing tasks, there was a negative relationship with rhyme processing tasks. We therefore speculate that differences in phonological abilities may be responsible for varying degrees of semantic reliance in reading aloud. This proposal accords with the results of functional imaging showing that higher semantic reliance during exception word reading corresponds to lower activation in left pre-central gyrus, an area associated with direct spelling sound mapping and phonological processing. Our results therefore establish the nature of systematic individual differences in degree of semantic involvement amongst normal readers, and suggest directions for future neuroimaging and computational modeling research to uncover their origins. [ABSTRACT FROM AUTHOR]

Connectionist modeling was used to investigate the brain mechanisms responsible for pain's ability to shift attention away from another stimulus modality and toward itself. Different connectionist model architectures were used to simulate the different possible brain mechanisms underlying this attentional bias, where nodes in the model simulated the brain areas thought to mediate the attentional bias, and the connections between the nodes simulated the interactions between the brain areas. Mathematical optimization techniques were used to find the model parameters, such as connection strengths, that produced the best quantitative fits of reaction time and event-related potential data obtained in our previous work. Of the several architectures tested, two produced excellent quantitative fits of the experimental data. One involved an unexpected pain stimulus activating somatic threat detectors in the dorsal posterior insula. This threat detector activity was monitored by the medial prefrontal cortex, which in turn evoked a phasic response in the locus coeruleus. The locus coeruleus phasic response resulted in a facilitation of the cortical areas involved in decision and response processes time-locked to the painful stimulus. The second architecture involved the presence of pain causing an increase in general arousal. The increase in arousal was mediated by locus coeruleus tonic activity, which facilitated responses in the cortical areas mediating the sensory, decision, and response processes involved in the task. These two neural network architectures generated competing predictions that can be tested in future studies. [ABSTRACT FROM AUTHOR]

Connectionist models can be characterized within the more general framework of probabilistic graphical models, which allow to efficiently describe complex statistical distributions involving a large number of interacting variables. This integration allows building more realistic computational models of cognitive functions, which more faithfully reflect the underlying neural mechanisms at the same time providing a useful bridge to higher-level descriptions in terms of Bayesian computations. Here we discuss a powerful class of graphical models that can be implemented as stochastic, generative neural networks. These models overcome many limitations associated with classic connectionist models, for example by exploiting unsupervised learning in hierarchical architectures (deep networks) and by taking into account top-down, predictive processing supported by feedback loops. We review some recent cognitive models based on generative networks, and we point out promising research directions to investigate neuropsychological disorders within this approach. Though further efforts are required in order to fill the gap between structured Bayesian models and more realistic, biophysical models of neuronal dynamics, we argue that generative neural networks have the potential to bridge these levels of analysis, thereby improving our understanding of the neural bases of cognition and of pathologies caused by brain damage. [ABSTRACT FROM AUTHOR]

An influential view of the nature of the language system is that of an evolved biological system in which a set of rules is combined with a lexicon that contains the words of the language together with a representation of their context. Alternative views, usually based on connectionist modeling, attempt to explain the structure of language on the basis of complex associative processes. Here, I put forward a third view that stresses experience-dependent structural development of the brain circuits supporting language as a core principle of the organization of the language system. In this view, embodied in a recent neuroconstructivist neural network of past tense development and processing, initial domain-general predispositions enable the development of functionally specialized brain structures through interactions between experience-dependent brain development and statistical learning in a structured environment. Together, these processes shape a biological adult language system that appears to separate into distinct mechanism for processing rules and exceptions, whereas in reality those subsystems co-develop and interact closely. This view puts experience-dependent brain development in response to a specific language environment at the heart of understanding not only language development but adult language processing as well. [ABSTRACT FROM AUTHOR]

Learning the structure of event sequences is a ubiquitous problem in cognition and particularly in language. One possible solution is to learn a probabilistic generative model of sequences that allows making predictions about upcoming events. Though appealing from a neurobiological standpoint, this approach is typically not pursued in connectionist modeling. Here, we investigated a sequential version of the restricted Boltzmann machine (RBM), a stochastic recurrent neural network that extracts high-order structure from sensory data through unsupervised generative learning and can encode contextual information in the form of internal, distributed representations. We assessed whether this type of network can extract the orthographic structure of English monosyllables by learning a generative model of the letter sequences forming a word training corpus. We show that the network learned an accurate probabilistic model of English graphotactics, which can be used to make predictions about the letter following a given context as well as to autonomously generate high-quality pseudowords. The model was compared to an extended version of simple recurrent networks, augmented with a stochastic process that allows autonomous generation of sequences, and to non-connectionist probabilistic models (n-grams and hidden Markov models). We conclude that sequential RBMs and stochastic simple recurrent networks are promising candidates for modeling cognition in the temporal domain. [ABSTRACT FROM AUTHOR]

Understanding spoken words involves a rapid mapping from speech to conceptual representations. One distributed feature-based conceptual account assumes that the statistical characteristics of concepts' features-the number of concepts they occur in ( distinctiveness/ sharedness) and likelihood of co-occurrence ( correlational strength)-determine conceptual activation. To test these claims, we investigated the role of distinctiveness/sharedness and correlational strength in speech-to-meaning mapping, using a lexical decision task and computational simulations. Responses were faster for concepts with higher sharedness, suggesting that shared features are facilitatory in tasks like lexical decision that require access to them. Correlational strength facilitated responses for slower participants, suggesting a time-sensitive co-occurrence-driven settling mechanism. The computational simulation showed similar effects, with early effects of shared features and later effects of correlational strength. These results support a general-to-specific account of conceptual processing, whereby early activation of shared features is followed by the gradual emergence of a specific target representation. [ABSTRACT FROM AUTHOR]

The question of whether, and if so how, learning can be transfered from previously experienced games to novel games has recently attracted the attention of the experimental game theory literature. Existing research presumes that learning operates over actions, beliefs or decision rules. This study instead uses a connectionist approach that learns a direct mapping from game payoffs to a probability distribution over own actions. Learning is operationalized as a backpropagation rule that adjusts the weights of feedforward neural networks in the direction of increasing the probability of an agent playing a myopic best response to the last game played. One advantage of this approach is that it expands the scope of the model to any possible n × n normal-form game allowing for a comprehensive model of transfer of learning. Agents are exposed to games drawn from one of seven classes of games with significantly different strategic characteristics and then forced to play games from previously unseen classes. I find significant transfer of learning, i.e., behavior that is path-dependent, or conditional on the previously seen games. Cooperation is more pronounced in new games when agents are previously exposed to games where the incentive to cooperate is stronger than the incentive to compete, i.e., when individual incentives are aligned. Prior exposure to Prisoner's dilemma, zero-sum and discoordination games led to a significant decrease in realized payoffs for all the game classes under investigation. A distinction is made between superficial and deep transfer of learning both-the former is driven by superficial payoff similarities between games, the latter by differences in the incentive structures or strategic implications of the games. I examine whether agents learn to play the Nash equilibria of games, how they select amongst multiple equilibria, and whether they transfer Nash equilibrium behavior to unseen games. Sufficient exposure to a strategically heterogeneous set of games is found to be a necessary condition for deep learning (and transfer) across game classes. Paradoxically, superficial transfer of learning is shown to lead to better outcomes than deep transfer for a wide range of game classes. The simulation results corroborate important experimental findings with human subjects, and make several novel predictions that can be tested experimentally. [ABSTRACT FROM AUTHOR]

Consciousness remains a mystery-'a phenomenon that people do not know how to think about-yet' (Dennett, , p. 21). Here, I consider how the connectionist perspective on information processing may help us progress toward the goal of understanding the computational principles through which conscious and unconscious processing differ. I begin by delineating the conceptual challenges associated with classical approaches to cognition insofar as understanding unconscious information processing is concerned, and to highlight several contrasting computational principles that are constitutive of the connectionist approach. This leads me to suggest that conscious and unconscious processing are fundamentally connected, that is, rooted in the very same computational principles. I further develop a perspective according to which the brain continuously and unconsciously learns to redescribe its own activity itself based on constant interaction with itself, with the world, and with other minds. The outcome of such interactions is the emergence of internal models that are metacognitive in nature and that function so as to make it possible for an agent to develop a (limited, implicit, practical) understanding of itself. In this light, plasticity and learning are constitutive of what makes us conscious, for it is in virtue of our own experiences with ourselves and with other people that our mental life acquires its subjective character. The connectionist framework continues to be uniquely positioned in the Cognitive Sciences to address the challenge of identifying what one could call the 'computational correlates of consciousness' (Mathis & Mozer, ) because it makes it possible to focus on the mechanisms through which information processing takes place. [ABSTRACT FROM AUTHOR]

We analyzed stimulus class formation in a human study and in a connectionist model (CM) with a yes/no procedure, using compound stimuli. In the human study, the participants were six female undergraduate students; the CM was a feed-forward back-propagation network. Two 3-member stimulus classes were trained with a similar procedure in both the human participants and the model. During training, the compound stimuli A1B1, A2B2, B1C1, B2C2 were associated with the yes response, and A1B2, A2B1, B1C2, B2C1 were associated with the no response. During testing, we probe the emergence of the yes and no responses in the presence of new compound stimuli (A1C1, A2C2, A1C2, A2C1). Four participants and five out of six runs of the CM exhibited stimulus class formation. These results show the usefulness of the yes/no procedure for simulating stimulus class formation in a CM in contrast with the models inspired in matching-to-sample procedures. [ABSTRACT FROM AUTHOR]

It has been argued that the mental representation resulting from sentence comprehension is not (just) an abstract symbolic structure but a "mental simulation" of the state-of-affairs described by the sentence. We present a particular formalization of this theory and show how it gives rise to quantifications of the amount of syntactic and semantic information conveyed by each word in a sentence. These information measures predict simulated word-processing times in a dynamic connectionist model of sentence comprehension as mental simulation. A quantitatively similar relation between information content and reading time is known to be present in human reading-time data. [ABSTRACT FROM AUTHOR]

A substantial body of experimental evidence has demonstrated that labels have an impact on infant categorization processes. Yet little is known regarding the nature of the mechanisms by which this effect is achieved. We distinguish between two competing accounts: supervised name-based categorization and unsupervised feature-based categorization. We describe a neurocomputational model of infant visual categorization, based on self-organizing maps, that implements the unsupervised feature-based approach. The model successfully reproduces experiments demonstrating the impact of labeling on infant visual categorization reported in Plunkett, Hu, and Cohen (2008) . It mimics infant behavior in both the familiarization and testing phases of the procedure, using a training regime that involves only single presentations of each stimulus and using just 24 participant networks per experiment. The model predicts that the observed behavior in infants is due to a transient form of learning that might lead to the emergence of hierarchically organized categorical structure and that the impact of labels on categorization is influenced by the perceived similarity and the sequence in which the objects are presented. The results suggest that early in development, say before 12 months old, labels need not act as invitations to form categories nor highlight the commonalities between objects, but they may play a more mundane but nevertheless powerful role as additional features that are processed in the same fashion as other features that characterize objects and object categories. [ABSTRACT FROM AUTHOR]

Evidence from numerous studies using the visual world paradigm has revealed both that spoken language can rapidly guide attention in a related visual scene and that scene information can immediately influence comprehension processes. These findings motivated the coordinated interplay account ( Knoeferle & Crocker, 2006 ) of situated comprehension, which claims that utterance-mediated attention crucially underlies this closely coordinated interaction of language and scene processing. We present a recurrent sigma-pi neural network that models the rapid use of scene information, exploiting an utterance-mediated attentional mechanism that directly instantiates the CIA. The model is shown to achieve high levels of performance (both with and without scene contexts), while also exhibiting hallmark behaviors of situated comprehension, such as incremental processing, anticipation of appropriate role fillers, as well as the immediate use, and priority, of depicted event information through the coordinated use of utterance-mediated attention to the scene. [ABSTRACT FROM AUTHOR]

This article presents a sobering view of the discipline of cognitive neuropsychology as practiced over the last three or four decades. Our judgment is that, although the study of abnormal cognition resulting from brain injury or disease in previously normal adults has produced a catalogue of fascinating and highly selective deficits, it has yielded relatively little advance in understanding how the brain accomplishes its cognitive business. We question the wisdom of the following three “choices” in mainstream cognitive neuropsychology: (a) single-case methodology, (b) dissociation between functions as the most important source of evidence, and (c) a central goal of diagramming the functional architecture of cognition rather than specifying how its components work. These choices may all stem from an excessive commitment to strict and fine-grained modularity. Although different brain regions are undoubtedly specialized for different functions, we argue that parallel and interactive processing is a better assumption about cognitive processing. The essential goal of specifying representations and processes can, we claim, be significantly assisted by computational modeling which, by its very nature, requires such specification. [ABSTRACT FROM AUTHOR]

Two studies provided evidence that holsters the Marsh, Adams, and Kleck hypothesis that the morphology of certain emotion expressions reflects an evolved adaptation to mimic babies or mature adults. Study 1 found differences in emotion expressions' resemblance to babies using objective indices of babyfaceness provided by connectionist models that are impervious to overlapping cultural stereotypes about babies and the emotions. Study 2 not only replicated parallels between impressions of certain emotions and babies versus adults hut also showed that objective indices of babyfaceness partially mediated impressions of the emotion expressions. babyface effects were independent of strong effects of attractiveness, and babyfaceness did not mediate impressions of happy expressions, to which the evolutionary hypothesis would not apply. [ABSTRACT FROM AUTHOR]

This paper provides an overview of developments in a dual processing theory of automatic and controlled processing that began with the empirical and theoretical work described by Schneider and Shiffrin (1977) and Shiffrin and Schneider (1977) over a quarter century ago. A review of relevant empirical findings suggests that there is a set of core behavioral phenomena reflecting differences between controlled and automatic processing that must be addressed by a successful theory. These phenomena relate to: consistency in training, serial versus parallel processing, level of effort, robustness to stressors, degree of control, effects on long-term memory, and priority encoding. We detail a computational model of controlled processing, CAP2, that accounts for these phenomena as emergent properties of an underlying hybrid computational architecture. The model employs a large network of distributed data modules that can categorize, buffer, associate, and prioritize information. Each module is a connectionist network with input and output layers, and each module communicates with a central Control System by outputting priority and activity report signals, and by receiving control signals. The Control System is composed of five processors including a Goal Processor, an Attention Controller, an Activity Monitor, an Episodic Store, and a Gating & Report Relay. The transition from controlled to automatic processing occurs in this model as the data modules become capable of transmitting their output without mediation by the Control System. We describe recent progress in mapping the components of this model onto specific neuroanatomical substrates, briefly discuss the potential for applying functional neuroimaging techniques to test the model’s predictions, and its relation to other models. [Copyright &y& Elsevier]

The facilitation and analytical support of argumentation-based collaborative decision making is the focus of this research. We model collaborative decision making as an argumentation process. We develop a connectionist modeling framework, a network representation formalism for argument structures, connectionist network mechanisms, and their models of computations to extract the behavior of argument structures. We use two examples from the case study literature to illustrate the concepts. Several interesting properties of the connectionist network models are observed from our computational results. We find that although the length of the computation is affected by parametric values, the final activation levels of the units are largely unaffected. We observe that the initial activation levels of the defeasible units seem to have no effect on their final activation levels. The proposed modeling approach generates valuable insights into the characteristics of specific argumentative discussions. While the intention of this work is not to introduce the connectionist paradigm as a means to bring arguments to a closure (resolution), we show that certain resolution mechanisms can be easily implemented under the connectionist framework. [ABSTRACT FROM AUTHOR]

Two experiments used classical conditioning to examine transfer of response features specific to the warning interval between a conditioned stimulus (CS) and an unconditioned stimulus (US). Rabbits were given initial training with a stimulus (CSA) in one modality (tone) at a designated interval (e.g., 200 ms). In a second stage, the conditioned response (CR) to CSA was extinguished. Finally, training was shifted to a new stimulus (CSB) in another modality (light) at a new interval (e.g., 400 ms). Compared to rest controls, there was an enhanced rate of CR acquisition to CSB and a tendency for early CRs to CSB to reflect the CSA-US interval. However, the extinction procedure and the change in CS-US interval together reduced these two aspects of transfer across stimulus modalities. The findings are discussed in terms of their implications for theories of cross-modal transfer. A layered network with real-time features is offered. [ABSTRACT FROM PUBLISHER]

High phonological neighborhood density has been associated with both advantages and disadvantages in early word learning. High density may support the formation and fine‐tuning of new word sound memories—a process termed lexical configuration (e.g., Storkel, 2004). However, new high‐density words are also more likely to be misunderstood as instances of known words, and may therefore fail to trigger the learning process (e.g., Swingley & Aslin, 2007). To examine these apparently contradictory effects, we trained an autoencoder neural network on 587,954 word tokens (5,497 types, including mono‐ and multisyllabic words of all grammatical classes) spoken by 279 caregivers to English‐speaking children aged 18–24 months. We then simulated a communicative development inventory administration and compared network performance to that of 2,292 children aged 18–24 months. We argue that autoencoder performance illustrates concurrent density advantages and disadvantages, in contrast to prior behavioral and computational literature treating such effects independently. Low network error rates signal a configuration advantage for high‐density words, while high network error rates signal a triggering advantage for low‐density words. This interpretation is consistent with the application of autoencoders in academic research and industry, for simultaneous feature extraction (i.e., configuration) and anomaly detection (i.e., triggering). Autoencoder simulation therefore illustrates how apparently contradictory density and distinctiveness effects can emerge from a common learning mechanism. Open Research Badges: This article has earned an Open Data badge for making publicly available the digitally‐shareable data necessary to reproduce the reported results. The data is available at https://osf.io/2qk5j/. Learn more about the Open Practices badges from the Center for Open Science: https://osf.io/tvyxz/wiki. [ABSTRACT FROM AUTHOR]