Search

Technological innovations have become a key driver of societal advancements. Nowhere is this more evident than in the field of machine learning (ML), which has developed algorithmic models that shape our decisions, behaviors, and outcomes. These tools have widespread use, in part, because they can synthesize massive amounts of data to make seemingly objective recommendations. Yet, in the past few years, the ML community has been drawing attention to the need for caution when interpreting and using these models. This is because these models are created by humans, from data generated by humans, whose psychology allows for various biases that impact how the models are developed, trained, tested, and interpreted. As psychologists, we thus face a fork in the road: Down the first path, we can continue to use these models without examining and addressing these critical flaws and rely on computer scientists to try to mitigate them. Down the second path, we can turn our expertise in bias toward this growing field, collaborating with computer scientists to reduce the models' deleterious outcomes. This article serves to light the way down the second path by identifying how extant psychological research can help examine and curtail bias in ML models.

Introduction: Despite the assumed importance of school-focused possible identities for academic motivation and outcomes, interventions rarely assess the effect of the intervention on possible identities. This may be due to difficulty coding open-ended text at scale but leaves open a number of questions: 1) how do school-focused possible identities change over the course of the school year, 2) whether these changes are associated with changes in school outcomes, and 3) whether a machine coding approach is viable. Methods: In Study 1 (n = 247 Chicago 8th-graders) we assess fall-to-spring change in school-focused possible identities. We test whether a change in school-focused possible identities predicts 8th-grade academic outcomes. We include robustness checks. Then we examine school context effects. In Study 2 (n = 1006 Chicago 8th-graders) we address the problem of coding at scale, using a separate data set to train a machine-learning algorithm. Results: On average, school-focused possible identities decline over the school year. But nearly a third of students have increasing school-focused possible identity scores. This increase is associated with improved grades. School context influences whether linked strategies matter. Our machine-learning algorithm accurately classifies school-focused possible identities in our original sample and this school-focused classification reliably predicts academic trajectories. Conclusions: Change in school-focused possible identities is normative over the course of the school year, interventions should take this into account. On average, students have fewer school-focused possible identities by spring. This decline is associated with declining academic trajectories. However, when school-focused possible identities increase, so do grades. Whether strategies matter is context-dependent.

Stroke is the leading cause of adult disability. Recovery of function after stroke involves signaling events that are mediated by cAMP and cGMP pathways, such as axonal sprouting, neurogenesis, and synaptic plasticity. cAMP and cGMP are degraded by phosphodiesterases (PDEs), which are differentially expressed in brain regions. PDE10A is highly expressed in the basal ganglia/striatum. We tested a novel PDE10A inhibitor (TAK-063) for its effects on functional recovery. Stroke was produced in mice in the cortex or the striatum. Behavioral recovery was measured to 9 weeks. Tissue outcome measures included analysis of growth factor levels, angiogenesis, neurogenesis, gliogenesis, and inflammation. TAK-063 improved motor recovery after striatal stroke in a dose-related manner, but not in cortical stroke. Recovery of motor function correlated with increases in striatal brain-derived neurotrophic factor. TAK-063 treatment also increased motor system axonal connections. Stroke affects distinct brain regions, with each comprising different cellular and molecular elements. Inhibition of PDE10A improved recovery of function after striatal but not cortical stroke, consistent with its brain localization. This experiment is the first demonstration of brain region-specific enhanced functional recovery after stroke, and indicates that differential molecular signaling between brain regions can be exploited to improve recovery based on stroke subtype.

Social stereotypes impact individuals’ judgement about different social groups. One area where such stereotyping has acritical impact is in hate speech detection, in which human annotations of text are used to train machine learning models.Such models are likely to be biased in the same ways that humans are biased in their judgments of social groups. Inthis research, we investigate the effect of stereotypes of social groups on the performance of expert annotators in a largecorpus of annotated hate speech. We also examine the effect of these stereotypes on unintended bias of hate speechclassifiers. To this end, we show how language-encoded stereotypes, associated with social groups, lead to disagreementsin identifying hate speech. Lastly, we analyze how inconsistencies in annotations propagate to a supervised classifier whenhuman-generated labels are used to train a hate speech detection model.

Cognitive tests used to measure individual differences are gen-erally designed with equality in mind: the same “broadly ac-ceptable” items are used for all participants. This has unknownconsequences for equity, particularly when a single set of lin-guistic stimuli are used for a diverse population of languageusers. We hypothesized that differences in language varietywould result in disparities in psycholinguistically meaningfulproperties of test items in two widely-used cognitive tasks, re-sulting in large differences in performance. As a proxy for in-dividuals’ language use, we administered a self-report surveyof media consumption. We identified two substantial clustersfrom the survey data, roughly orthogonal to a priori groups re-cruited into the study (university students and members of thesurrounding community). We found effects of both populationand cluster membership. Comparing item-wise differences be-tween the clusters’ language models did not identify specificitems driving performance differences.

Meaning depends on context. This applies both in obviouscases like deictics or sarcasm as well as more subtle situationslike framing or persuasion. One key characteristic of context isthe identity of the participants in an interaction. Our interpre-tation of an utterance depends on a variety of factors such asour personal history, background knowledge, and our relation-ship to the source. While demographics allow us to capturesome of this variance, the relevance of specific demographicfactors varies across contexts. To address these challenges, weintroduce a method for combining demographics and contextinto situated demographic embeddings—mapping representa-tions onto a continuous space appropriate for the given domain.We further demonstrate how to make use of related externalresources so as to apply this approach in low-resource situa-tions. We show the resulting representations to be interpretableand consider domain-specific similarity. Finally, we show howthese representations can be incorporated to improve modelingof a real-world natural language understanding task.

Communication accommodation is a phenomenon in social in-teractions in which people adjust their language to that of theirinterlocutor. A component of communication accommodationis research on power and dominance relations which suggestslanguage use is dependent on power position. There are differ-ent linguistic markers which imply power standing of people.For example, when high power individuals interact with peoplein low power positions, the language of the interaction tends tofollow the language of the high power individuals. While pre-vious studies have mostly focused on the word-level features,we show that not only people in low power mirror word usageof people in high power, but they also adjust their syntacticstructures to those in high power. Notably, we apply a compu-tational tool on two corpora and show that individuals in lowpower align their syntactic structures to those in high powerwhile people in high power do not.

Analysis of language provides important insights into the un-derlying psychological properties of individuals and groups.While the majority of language analysis work in psychologyhas focused on semantics, psychological information is en-coded not just in what people say, but how they say it. Inthe current work, we propose Conversation Level Syntax Simi-larity Metric-Group Representations (CASSIM-GR). This toolbuilds generalized representations of syntactic structures ofdocuments, thus allowing researchers to distinguish betweenpeople and groups based on syntactic differences. CASSIM-GR builds off of Conversation Level Syntax Similarity Metricby applying spectral clustering to syntactic similarity matricesand calculating the center of each cluster of documents. Thisresulting cluster centroid then represents the syntactical struc-ture of the group of documents. To examine the effectivenessof CASSIM-GR, we conduct three experiments across threeunique corpora. In each experiment, we calculate the cluster-ing accuracy and compare our proposed technique to a bag-of-words approach. Our results provide evidence for the ef-fectiveness of CASSIM-GR and demonstrate that combiningsyntactic similarity and tf-idf semantic information improvesthe total accuracy of group classification.

People’s beliefs about what is morally right and wrong vary widely between individuals, contexts, and cultures;however it is thought that they are governed by core latent constructs. While there is evidence that these constructs are reflectedin natural language, this requires further testing. We demonstrate that the structure of moral values in natural discourse can bemodeled by applying factor analyses to distributed representations of morally relevant terms learned by a neural network. Wefirst demonstrate that robust latent constructs can be estimated from the covariance of distributed representations of constructexemplars. We then test whether the factor structure proposed by Moral Foundations Theory (MFT) is reflected in naturallanguage. Finally, we conduct a bottom-up investigation of the structure of moral values in natural language using free-responses reported by participants. Ultimately, we find evidence that the representation of moral values in natural languagepartially corresponds to MFT.

The Linguistic Category Model (LCM) was developed as a manual coding scheme for quantifying abstract mindsetsin human language. Previous attempts to computationally automate the LCM have relied primarily on pre-coded semanticfeatures, which fail to incorporate important contextual information integral to the LCM coding scheme. In this paper, weintroduce Syntax-LCM, a novel method for automating LCM coding using syntax and dependency tree features as predictors ofconstrual level. We compare the accuracy of Syntax-LCM to that of two previously used automated methods: LIWC LCM andBrysbaert concreteness ratings. We find support that the Syntax-LCM approximates the hand-coded LCM with higher accuracycompared to both the Brysbaert and the LIWC LCM. We also provide evidence that the syntactic features accounted for bySyntax-LCM mirror the inclusion criteria in the original coding manual and support theoretical relationships between distanceand abstract thinking.

Recent advances in technology have paved the way for human-agent interactions to become ubiquitous in our daily lives, anddecades worth of research on virtual agents have enhancedthese interactions. However, for the most part, the effect of dif-ferent types of agents on the human brain is unknown, and theneuroscience of human-agent interactions is rarely studied. Inthis study, we examine the underlying neural systems involvedin processing and responding to different types of negotiatingagents. More specifically, we show that different brain patternsare observed for various types of virtual agents; consequently,we can decode the strategy and emotional display of the agentbased on the counterpart’s brain activity. Using fMRI data, weanalyzed participants’ brain activity during negotiations withagents who show three different emotional expressions and usetwo different types of negotiation strategies. We demonstratethat, using Multi-Voxel Pattern Analysis, we can reliably de-code agents’ emotional expressions based on the activity in theleft dorsal anterior insula, and also agents’ strategies based onthe activity in the frontal pole.

People interact differently with humans than they do with com-puters, but there is minimal research on what brings aboutthese differences. Using agents labeled as either “another par-ticipant” or a “computer program”, we investigated the differ-ences in people’s behavior and brain activity during the courseof a negotiation paradigm. Our results indicate that people per-ceive human-labeled agents more human-like than computer-labeled agents, and the level of concession in the negotiationsis dependent on agent type. We have also found that these dif-ferences can be captured in brain activation by showing thatparts of the Theory of Mind neural correlates are activated inhuman-labeled agent conditions, but not in computer-labeledagent conditions. We further demonstrate that brain activitycan predict whether the negotiation agent was introduced asa competing human player or a computer program. Overall,our study suggests that labeling an interaction partner as ei-ther another human or a computer program leads to significantimpacts on one’s decision making.

The importance of human-agent negotiation, and the role ofemotion in such negotiations, have been emphasized in human-agent interaction research. Thus far, studies have focused onbehavioral effects, rather than examining the neural underpin-nings of different behaviors shown in human-agent interac-tions. Here, we used a multi-round negotiation platform, in-stead of the more common single-shot negotiation, and wereable to find distinct brain patterns in emotion-related regionsof the brain during different types of offers. Using multi-voxelpattern analysis to analyze brain imaging data acquired duringfunctional MRI scanning, we show that it is possible to pre-dict whether the negotiator concedes, does not change, or asksfor more during the negotiation. Most importantly, we demon-strate that left dorsal anterior insula, which is known to be anemotion-related brain region, shows a different pattern of ac-tivity for each of the three offer types.

The psycholinguistic theory of Communication Accommoda-tion proposes that people modify communication dynamics(e.g. vocal patterns, gesture, word choice, syntax, etc.) tominimize (or maximize) their social differences. Research oncommunication accommodation has shown that people whowant social approval will modify their linguistic style to matchthat of their interactant; however, most studies have been con-ducted on small-scale datasets and in laboratory situations. Inthis work, we investigate the relationship between linguisticsyntactic usage and conversation participation in a more nat-uralistic conversational setting: social media conversations onReddit.com. We introduce a novel approach for calculatingdocument-level syntactic similarity by relying on natural lan-guage processing methods (parse tree generators) and graphtheory techniques (minimum weight perfect matching on com-plete bipartite graphs). Using the proposed method, we presentthe results of two experiments which demonstrate that userswho comment on a post tend to use syntax similar to that ofthe original post. Specifically, we provide evidence that com-ments on a post are more likely to follow the syntactic structureof the original post, compared to both random comments andalso posts by the author of the comment.

It has been shown that prior knowledge and information areorganized according to categories, and that also backgroundknowledge plays an important role in classification. The purposeof this study is first, to investigate the relationship betweenbackground knowledge and text classification, and second,to incorporate this relationship in a computational model.Our behavioral results demonstrate that participants with accessto background knowledge (experts), overall performedsignificantly better than those without access to this knowledge(novices). More importantly, we show that experts rely moreon relational features than surface features, an aspect that bagof-words methods fail to capture. We then propose a computationalmodel for text classification which incorporates backgroundknowledge. This model is built upon vector-based representationmethods and achieves significantly more accurateresults over other models that were tested.