Search

Computational approaches such as genome and metabolome mining are becoming essential to natural products (NPs) research. Consequently, a need exists for an automated structure-type classification system to handle the massive amounts of data appearing for NP structures. An ideal semantic ontology for the classification of NPs should go beyond the simple presence/absence of chemical substructures, but also include the taxonomy of the producing organism, the nature of the biosynthetic pathway, and/or their biological properties. Thus, a holistic and automatic NP classification framework could have considerable value to comprehensively navigate the relatedness of NPs, and especially so when analyzing large numbers of NPs. Here, we introduce NPClassifier, a deep-learning tool for the automated structural classification of NPs from their counted Morgan fingerprints. NPClassifier is expected to accelerate and enhance NP discovery by linking NP structures to their underlying properties.

A thiazole-containing cyclic depsipeptide with 11 amino acid residues, named pagoamide A (1), was isolated from laboratory cultures of a marine Chlorophyte, Derbesia sp. This green algal sample was collected from America Samoa, and pagoamide A was isolated using guidance by MS/MS-based molecular networking. Cultures were grown in a light- and temperature-controlled environment and harvested after several months of growth. The planar structure of pagoamide A (1) was characterized by detailed 1D and 2D NMR experiments along with MS and UV analysis. The absolute configurations of its amino acid residues were determined by advanced Marfey's analysis following chemical hydrolysis and hydrazinolysis reactions. Two of the residues in pagoamide A (1), phenylalanine and serine, each occurred twice in the molecule, once in the d- and once in the l-configuration. The biosynthetic origin of pagoamide A (1) was considered in light of other natural products investigations with coenocytic green algae.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

This report describes the first application of the novel NMR-based machine learning tool "Small Molecule Accurate Recognition Technology" (SMART 2.0) for mixture analysis and subsequent accelerated discovery and characterization of new natural products. The concept was applied to the extract of a filamentous marine cyanobacterium known to be a prolific producer of cytotoxic natural products. This environmental Symploca extract was roughly fractionated, and then prioritized and guided by cancer cell cytotoxicity, NMR-based SMART 2.0, and MS2-based molecular networking. This led to the isolation and rapid identification of a new chimeric swinholide-like macrolide, symplocolide A, as well as the annotation of swinholide A, samholides A-I, and several new derivatives. The planar structure of symplocolide A was confirmed to be a structural hybrid between swinholide A and luminaolide B by 1D/2D NMR and LC-MS2 analysis. A second example applies SMART 2.0 to the characterization of structurally novel cyclic peptides, and compares this approach to the recently appearing "atomic sort" method. This study exemplifies the revolutionary potential of combined traditional and deep learning-assisted analytical approaches to overcome longstanding challenges in natural products drug discovery.

Echo state networks (ESNs) are randomly connected recurrent neural networks (RNNs) that can be used as a temporal kernel for modeling time series data, and have been successfully applied on time series prediction tasks. Recently, ESNs have been applied to time series classification (TSC) tasks. However, previous ESN-based classifiers involve either training the model by predicting the next item of a sequence, or predicting the class label at each time step. The former is essentially a predictive model adapted from time series prediction work, rather than a model designed specifically for the classification task. The latter approach only considers local patterns at each time step and then averages over the classifications. Hence, rather than selecting the most discriminating sections of the time series, this approach will incorporate non-discriminative information into the classification, reducing accuracy. In this paper, we propose a novel end-to-end framework called the Echo Memory Network (EMN) in which the time series dynamics and multi-scale discriminative features are efficiently learned from an unrolled echo memory using multi-scale convolution and max-over-time pooling. First, the time series data are projected into the high dimensional nonlinear space of the reservoir and the echo states are collected into the echo memory matrix, followed by a single multi-scale convolutional layer to extract multi-scale features from the echo memory matrix. Max-over-time pooling is used to maintain temporal invariance and select the most important local patterns. Finally, a fully-connected hidden layer feeds into a softmax layer for classification. This architecture is applied to both time series classification and human action recognition datasets. For the human action recognition datasets, we divide the action data into five different components of the human body, and propose two spatial information fusion strategies to integrate the spatial information over them. With one training-free recurrent layer and only one layer of convolution, the EMN is a very efficient end-to-end model, and ranks first in overall classification ability on 55 TSC benchmark datasets and four 3D skeleton-based human action recognition tasks.

Various algorithms comparing 2D NMR spectra have been explored for their ability to dereplicate natural products as well as determine molecular structures. However, spectroscopic artefacts, solvent effects, and the interactive effect of functional group(s) on chemical shifts combine to hinder their effectiveness. Here, we leveraged Non-Uniform Sampling (NUS) 2D NMR techniques and deep Convolutional Neural Networks (CNNs) to create a tool, SMART, that can assist in natural products discovery efforts. First, an NUS heteronuclear single quantum coherence (HSQC) NMR pulse sequence was adapted to a state-of-the-art nuclear magnetic resonance (NMR) instrument, and data reconstruction methods were optimized, and second, a deep CNN with contrastive loss was trained on a database containing over 2,054 HSQC spectra as the training set. To demonstrate the utility of SMART, several newly isolated compounds were automatically located with their known analogues in the embedded clustering space, thereby streamlining the discovery pipeline for new natural products.

The Nonlinear autoregressive exogenous (NARX) model, which predicts the current value of a time series based upon its previous values as well as the current and past values of multiple driving (exogenous) series, has been studied for decades. Despite the fact that various NARX models have been developed, few of them can capture the long-term temporal dependencies appropriately and select the relevant driving series to make predictions. In this paper, we propose a dual-stage attention-based recurrent neural network (DA-RNN) to address these two issues. In the first stage, we introduce an input attention mechanism to adaptively extract relevant driving series (a.k.a., input features) at each time step by referring to the previous encoder hidden state. In the second stage, we use a temporal attention mechanism to select relevant encoder hidden states across all time steps. With this dual-stage attention scheme, our model can not only make predictions effectively, but can also be easily interpreted. Thorough empirical studies based upon the SML 2010 dataset and the NASDAQ 100 Stock dataset demonstrate that the DA-RNN can outperform state-of-the-art methods for time series prediction.

What are the roles of central and peripheral vision in human scene recognition? Larson and Loschky (2009) showed that peripheral vision contributes more than central vision in obtaining maximum scene recognition accuracy. However, central vision is more efficient for scene recognition than peripheral, based on the amount of visual area needed for accurate recognition. In this study, we model and explain the results of Larson and Loschky (2009) using a neurocomputational modeling approach. We show that the advantage of peripheral vision in scene recognition, as well as the efficiency advantage for central vision, can be replicated using state-of-the-art deep neural network models. In addition, we propose and provide support for the hypothesis that the peripheral advantage comes from the inherent usefulness of peripheral features. This result is consistent with data presented by Thibaut, Tran, Szaffarczyk, and Boucart (2014), who showed that patients with central vision loss can still categorize natural scenes efficiently. Furthermore, by using a deep mixture-of-experts model ("The Deep Model," or TDM) that receives central and peripheral visual information on separate channels simultaneously, we show that the peripheral advantage emerges naturally in the learning process: When trained to categorize scenes, the model weights the peripheral pathway more than the central pathway. As we have seen in our previous modeling work, learning creates a transform that spreads different scene categories into different regions in representational space. Finally, we visualize the features for the two pathways, and find that different preferences for scene categories emerge for the two pathways during the training process.

An active object recognition system has the advantage of acting in the environment to capture images that are more suited for training and lead to better performance at test time. In this paper, we utilize deep convolutional neural networks for active object recognition by simultaneously predicting the object label and the next action to be performed on the object with the aim of improving recognition performance. We treat active object recognition as a reinforcement learning problem and derive the cost function to train the network for joint prediction of the object label and the action. A generative model of object similarities based on the Dirichlet distribution is proposed and embedded in the network for encoding the state of the system. The training is carried out by simultaneously minimizing the label and action prediction errors using gradient descent. We empirically show that the proposed network is able to predict both the object label and the actions on GERMS, a dataset for active object recognition. We compare the test label prediction accuracy of the proposed model with Dirichlet and Naive Bayes state encoding. The results of experiments suggest that the proposed model equipped with Dirichlet state encoding is superior in performance, and selects images that lead to better training and higher accuracy of label prediction at test time.

Categorical and coordinate stimulus processing were hypoth-esized by Kosslyn (1987) to be lateralized visual tasks, dif-ferentiated by task-relevant spatial frequencies. Slotnick et al.(2001) directly tested Kosslyn’s hypothesis and concluded thatthe lateralization presents only when tasks are sufficiently dif-ficult. Our differential encoding model is a three layer neuralnetwork that accounts for lateralization in visual processingvia the biologically plausible mechanism of differences in con-nection spread of long-range lateral neural connections (Hsiao,Cipollini, & Cottrell, 2013). We show that our model accountsfor Slotnick’s data and that Slotnick’s analysis does not con-vincingly explain their results. Instead, we propose that Koss-lyn’s initial hypothesis was based on an incorrect assumption:categorical and coordinate stimuli are not solely differentiatedby spatial frequencies. The results that our model capturescannot be reproduced by Ivry and Robertson’s (1998) Dou-ble Filtering by Frequency theory, which is driven solely bylateralized spatial frequency processing.

In previous work, we showed that a simple neurocomputa-tional model The Model, or TM) trained on the Ekman &Friesen Pictures of Facial Affect (POFA) dataset to catego-rize the images into the six basic expressions can account forwide array of data (albeit from a single study) on facial ex-pression processing. The model demonstrated categorical per-ception of facial expressions, as well as the so-called facialexpression circumplex, a circular configuration based on MDSresults that places the categories in the order happy, surprise,fear, sadness, anger and disgust. Somewhat ironically, the cir-cumplex in TM was generated from the similarity between thecategorical outputs of the network, i.e., the six numbers rep-resenting the probability of the category given the face. Here,we extend this work by 1) using a new dataset, NimsStims,that is much larger than POFA, and is not as tightly controlledfor the correct Facial Action Units; 2) using a completely dif-ferent neural network architecture, a Siamese Neural Network(SNN) that maps two faces through twin networks into a 2Dsimilarity space; and 3) training the network only implicitly,based on a teaching signal that pairs of faces are in either inthe same or different categories. Our results show that in thissetting, the network learns a representation that is very similarto the original circumplex. Fear and surprise overlap, whichis consistent with the inherent confusability between these twofacial expressions. Our results suggest that humans evolvedin such a way that nearby emotions are represented by similarappearances.

It is commonly believed that the central visual field (fovea andparafovea) is important for recognizing objects and faces, andthe peripheral region is useful for scene recognition. However,the relative importance of central versus peripheral informa-tion for object, scene, and face recognition is unclear. Larsonand Loschky (2009) investigated this question in the context ofscene processing using experimental conditions where a cir-cular region only reveals the central visual field and blocksperipheral information (”Window”), and in a ”Scotoma” con-dition, where only the peripheral region is available. Theymeasured the scene recognition accuracy as a function of vi-sual angle, and demonstrated that peripheral vision was indeedmore useful in recognizing scenes than central vision in termsof achieving maximum recognition accuracy. In this work,we modeled and replicated the result of Larson and Loschky(2009), using deep convolutional neural networks (CNNs).Having fit the data for scenes, we used the model to predictfuture data for large-scale scene recognition as well as for ob-jects and faces. Our results suggest that the relative order ofimportance of using central visual field information is facerecognition>object recognition>scene recognition, and vice-versa for peripheral information. Furthermore, our results pre-dict that central information is more efficient than peripheralinformation on a per-pixel basis across all categories, which isconsistent with Larson and Loschky’s data.

The visual word form area (VWFA) is a region of the cortex lo-cated in the left fusiform gyrus, that appears to be a waystationin the reading pathway. The discovery of the VWFA occurredin the late twentieth century with the advancement of func-tional magnetic resonance imaging (fMRI). Since then, therehas been an increasing number of neuroimaging studies to un-derstand the VWFA, and there are disagreements as to its prop-erties. One such disagreement is regarding whether or not theVWFA is more selective for real words over pseudowords1. Arecent study using fMRI adaptation (Glezer, et al., 2009) pro-vided evidence that neurons in the VWFA are selectively tunedto real words. This contradicts the hypothesis that the VWFAis tuned to the sublexical structure of visual words, and there-fore has no preference for real words over pseudowords. Inthis paper, we develop a realistic model of the VWFA by train-ing a deep convolutional neural network to map printed wordsto their labels. The network is able to achieve an accuracy of98.5% on the test set. We then analyze this network to see ifit can account for the data Glezer et al. found for words andpseudowords, and find that it does.

Previous functional magnetic resonance imaging (fMRI) research on action observation has emphasized the role of putative mirror neuron areas such as Broca's area, ventral premotor cortex, and the inferior parietal lobule. However, recent evidence suggests action observation involves many distributed cortical regions, including dorsal premotor and superior parietal cortex. How these different regions relate to traditional mirror neuron areas, and whether traditional mirror neuron areas play a special role in action representation, is unclear. Here we use multi-voxel pattern analysis (MVPA) to show that action representations, including observation, imagery, and execution of reaching movements: (1) are distributed across both dorsal (superior) and ventral (inferior) premotor and parietal areas; (2) can be decoded from areas that are jointly activated by observation, execution, and imagery of reaching movements, even in cases of equal-amplitude blood oxygen level-dependent (BOLD) responses; and (3) can be equally accurately classified from either posterior parietal or frontal (premotor and inferior frontal) regions. These results challenge the presumed dominance of traditional mirror neuron areas such as Broca's area in action observation and action representation more generally. Unlike traditional univariate fMRI analyses, MVPA was able to discriminate between imagined and observed movements from previously indistinguishable BOLD activations in commonly activated regions, suggesting finer-grained distributed patterns of activation.

Since Yarbus's seminal work, vision scientists have argued that our eye movement patterns differ depending upon our task. This has recently motivated the creation of multi-fixation pattern analysis algorithms that try to infer a person's task (or mental state) from their eye movements alone. Here, we introduce new algorithms for multi-fixation pattern analysis, and we use them to argue that people have scanpath routines for judging faces. We tested our methods on the eye movements of subjects as they made six distinct judgments about faces. We found that our algorithms could detect whether a participant is trying to distinguish angriness, happiness, trustworthiness, tiredness, attractiveness, or age. However, our algorithms were more accurate at inferring a subject's task when only trained on data from that subject than when trained on data gathered from other subjects, and we were able to infer the identity of our subjects using the same algorithms. These results suggest that (1) individuals have scanpath routines for judging faces, and that (2) these are diagnostic of that subject, but that (3) at least for the tasks we used, subjects do not converge on the same "ideal" scanpath pattern. Whether universal scanpath patterns exist for a task, we suggest, depends on the task's constraints and the level of expertise of the subject.

The processing of Navon figures (Navon, 1977), i.e.,hierarchical letter stimuli, has been studied in experimentalsettings for many years. In particular, they have been studiedin the context of visual hemifield studies and yielded aninteraction between hemifield and whether a target is at thelocal or global level, with a right hemisphere advantage forthe global level, and a left hemisphere advantage for thetargets at the local level (Sergent, 1982). This is a ventralstream process, however, and we were interested in whetherthere might be a similar interaction for hierarchical motionstimuli, presumably a dorsal stream process. Hence wedeveloped a series of dynamic geometric Navon figures inorder to study global/local rotation processing. These figuresconsist of a global figure (a triangle or a square) made up oflocal figures (also triangles or squares). Both global and localfigures can rotate in either clockwise or counterclockwisedirections independently. We found that there is no right orleft visual field perceptual advantage for either the global orlocal levels of these figures. However, curiously enough, wefound that there is a significant processing advantage forclockwise motion compared to counterclockwise motion. Wealso found a highly significant interaction between thedetection of a particular rotational motion and the presence orabsence of that motion in the figure being examined. Finally,our data strongly support the Global Precedence Hypothesiswhich says that people generally tend to focus on the globalproperties of an object before local properties and thatprocessing proceeds in a global-to-local direction.

Recognition of social styles of people is an interesting but relatively unexplored task. Recognizing "style" appears to be a quite different problem than categorization, it is like recognizing a letter's font as opposed to recognizing the letter itself. Similar-looking things must be mapped to different categories. Hence a priori it would appear that features that are good for categorization should not be good for style recognition. Here we show this is not the case by starting with a convolutional deep network pre-trained on Image Net (Caffe), a categorization problem, and using the features as input to a classifier for urban tribes. Combining the results from individuals in group pictures and the group itself, with some fine-tuning of the network, we reduce the previous state of the art error by almost half, going from 46% recognition rate to 71%. To explore how the networks perform this task, we compute the mutual information between the Image Net output category activations and the urban tribe categories, and find, for example, that bikers are well categorized as whiptail lizards by Caffe, and that better recognized social groups have more highly-correlated Image Net categories. This gives us insight into the features useful for categorizing urban tribes.

We show that Gabor filter representations give quantitativelyindistinguishable results for classification of facial expressions as localPCA representations, in contrast to other recent work. We also show that asimple discriminant analysis automatically locates regions roughlycorresponding to relevant Facial Actions. Finally, we in troduce a method thattypically boosts generalization performance 9% by "peeking" at all of theunlabeled training patt erns before classifying them.Pre-2018 CSE ID: CS1999-0629

We propose an account for the acquisition of grammatical relations using the concepts of connectionist learning and a construction-based theory of grammar. The proposal is based on the observation that early production of childhood speech is formulaic and the assumption that the purpose of language is communication. If one assumes that children's comprehension of multiword speech is not globally systematic, but based initially on semi-rote knowledge (socalled "pivot grammars"), a pathway through small-scale systematicity to grammatical relations appropriate to the child's target language can be seen. W e propose such a system and demonstrate a portion of the emergence of grammatical relations using a connectionist network.

In this paper, we make a methodological point concerning the contribution of the representation of the output of a neural network model when using the model to compare to human error performance. We replicate part of Dell, Juliano & Govindjee's work on modeling speech errors using recurrent networks (Dell et al., 1993). We find that 1) the error patterns reported by Dell et al. do not appear to remain when more networks are used; and 2) some components of the error patterns that are found can be accounted for by simply adding Gaussian noise to the output representation they used. We suggest that when modeling error behavior, the technique of adding noise to the output representation of a network should be used as a control to assess to what degree errors may be attributed to the underlying network.

A previous experiment tested subjects' new/old judgments of previously-studied faces, distractors, and morphs between pairs of studied parents. We examine the extent to which models based on principal component analysis (eigenfaces) can predict human recognition of studied faces and false alarms to the distractors and morphs. We also compare eigenface models to the predictions of previous models based on the positions of faces in a multidimensional "face space" derived from a multidimensional scaling (MDS) of human similarity ratings. We find that the error in reconstructing a test face from its position in an "eigenface space" provides a good overall prediction of human familiarity ratings. However, the model has difficulty accounting for the fact that humans false alarm to morphs with similar parents more frequently than they false alarm to morphs with dissimilar parents. We ascribe this to the limitations of the simple reconstruction error-based model. We then outline preliminary work to improve the fine-grained fit within the eigenface-based modeling framework, and discuss the results' implicadons for exemplar- and face space-based models of face processing.

In this paper we investigate representational and methodological issues in a attractor network model of the mapping from orthography to semantics based on [Plaut, 1995]. We find that, contrary to psycholinguistic studies, the response time to concrete words (represented by more 1 bits in the output pattern) is slower than for abstract words. This model also predicts that response times to words in a dense semantic neighborhood will be faster than words which have few semantically similar neighbors in the language. This is conceptually consistent with the neighborhood effect seen in the mapping from orthography to phonology [Seidenberg & McClelland, 1989, Plaut etal., 1996] in that patterns with many neighbors are faster in both pathways, but since there is no regularity in the random mapping used here, it is clear that the cause of this effect is different than that of previous experiments. We also report a rather distressing finding. Reaction time in this model is measured by the time it takes the network to settle after being presented with a new input. When the criterion used to determine when the network is "settled" is changed to include testing of the hidden units, each of the results reported above change the direction of effect - abstract words are now slower, as are words in dense semantic neighborhoods. Since there are independent reasons to exclude hidden units from the stopping criterion, and this is what is done in common practice, we believe this phenomenon to be of interest mostly to neural network practitioners. However, it does provide some insight into the interaction between the hidden and output units during settling.

How do my mental states compare to yours? We suggest that, while we may not be able to compare experiences, we can compare neural representations, and that the correct way to compare neural representations is through analysis of the distances between them. In this paper, we present a technique for measuring the similarities between representations at various layers of neural networks. We then use the measure to demonstrate empirically that different artificial neural networks trained by backpropagation on the same categorization task, even with different representational encodings of the input patterns and different numbers of hidden units, reach states in which representations at the hidden units are similar.

The performance of a neural network that categorizes facial expressions is compared with human subjects over a set of experiments using interpolated imagery. The experiments for both the human subjects and neural networks make use of interpolations of facial expressions from the Pictures of Facial Affect Database [Ekman and Friesen, 1976]. The only difference in materials between those used in the human subjects experiments [Young et al., 1997] and our materials are the manner in which the interpolated images are constructed - image-quality morphs versus pixel averages. Nevertheless, the neural network accurately captures the categorical nature of the human responses, showing sharp transitions in labeling of images along the interpolated sequence. Crucially for a demonstration of categorical perception [Hamad, 1987], the model shows the highest discrimination between transition images at the crossover point. The model also captures the shape of the reaction time curves of the human subjects along the sequences. Finally, the network matches human subjects' judgements of which expressions are being mixed in the images. The main failing of the model is that there are intrusions of "neutral" responses in some transitions, which are not seen in the human subjects. We attribute this difference to the difference between the pixel average stimuli and the image quality morph stimuli. These results show that a simple neural network classifier, with no access to the biological constraints that are presumably imposed on the human emotion processor, and whose only access to the surrounding culture is the category labels placed by American subjects on the facial expressions, can nevertheless simulate fairiy well the human responses to emotional expressions.

A considerable body of evidence from prosopagnosia, a deficit in face recognition dissociable from nonface object recognition, indicates that the visual system devotes a specialized functional area to mechanisms appropriate for face processing. We present a modular neural network composed of two "expert" networks and one mediating "gate" network with the task of learning to recognize the faces of 12 individuals and classifying 36 nonface objects as members of one of three classes. While learning the task, the network tends to divide labor between the two expert modules, with one expert specializing in face processing and the other specializing in nonface object processing. After training, we observe the network's performance on a test set as one of the experts is progressively damaged. The results roughly agree with data reported for prosopagnosic patients: as damage to the "face" expert increases, the network's face recognition performance decreases dramatically while its object classification performance drops slowly. We conclude that data-driven competitive learning between two unbiased functional units can give rise to localized face processing, and that selective damage in such a system could underlie prosopagnosia.

Plunkett & Marchman (1993) showed that a neural network trained on an incrementally expanded training set was able to master the past tense and show the U-shaped learning pattern characteristic of children. In Jackson, Constandse & Cottrell (1996) we argued that Plunkett & Marchman's restriction of the training set was unrealistic and proposed a model of selective attention that enabled our network to master the past tense without external restrictions on its training set. Analysis in the present paper shows that the network in Jackson, Constandse & Cottrell (1996) does not exhibit appropriate U-shaped learning, however. We propose a modified model of selective attention that results in the mastery of the past tense as well as the kind of U-shaped learning observed in children.

It is well known that children generally exhibit a "U-shaped" pattern of development in the process of acquiring the past tense. Piunkett & Marchman (1991) showed that a connectionist network, trained on the past tense, would exhibit U-shaped learning effects. This network did not completely master the past tense mapping, however. Piunkett & Marchman (1993) showed that a network trained with an incrementally expanded training set was able to achieve acceptable levels of mastery, as well as show the desired U-shaped pattern. In this paper, we point out some problems with using an incrementally expanded training set. We propose a model of selective attention that enables our network to completely master the past tense mapping and exhibit U-shaped learning effects without requiring external manipulation of its training set.

We present an automated emotion recognition system that is capable of identifying six basic emotions (happy, surprise, sad, angry, fear, disgust) in novel face images. An ensemble of simple feed-forward neural networks are used to rate each of the images. The outputs of these networks are then combined to generate a score for each emotion. The networks were trained on a database of face images that human subjects consistently rated as portraying a single emotion. Such a system achieves 86% generalization on novel face images (individuals the networks were not trained on) drawn from the same database. The neural network model exhibits categorical perception between some emotion pairs. A linear sequence of morph images is created between two expressions of an individual's face and this sequence is analyzed by the model. Sharp transitions in the output response vector occur in a single step in the sequence for some emotion pairs and not for others. We plan to us the model's response to limit and direct testing in detennining if human subjects exhibit categorical perception in morph image sequences.

In recent years, a number of models of lexical access based on attractor networks have appeared. These models reproduce a number of effects seen in psycholinguistic experiments, but all suffer from unrealistic representations of lexical semantics. In an effort to improve this situation we are looking at techniques developed in the information retrieval literature that use the statistics found in large corpora to automatically produce vector representations for large numbers of words. This paper concentrates on the problem of transforming the real-valued cooccurrence vectors produced by these statistical techniques into the binary- or bipolar-valued vectors required by attractor network models, while maintaining the important inter-vector distance relationships. We describe an algorithm we call discrete multidimensional scaling which accomplishes this, and present the results of a set of experiments using this algorithm.