1,162 results on '"Frank, Michael"'
Search Results
2. Deep Transfer Hashing for Adaptive Learning on Federated Streaming Data
- Author
-
Röder, Manuel and Schleif, Frank-Michael
- Subjects
Computer Science - Machine Learning ,Computer Science - Cryptography and Security ,Computer Science - Distributed, Parallel, and Cluster Computing - Abstract
This extended abstract explores the integration of federated learning with deep transfer hashing for distributed prediction tasks, emphasizing resource-efficient client training from evolving data streams. Federated learning allows multiple clients to collaboratively train a shared model while maintaining data privacy - by incorporating deep transfer hashing, high-dimensional data can be converted into compact hash codes, reducing data transmission size and network loads. The proposed framework utilizes transfer learning, pre-training deep neural networks on a central server, and fine-tuning on clients to enhance model accuracy and adaptability. A selective hash code sharing mechanism using a privacy-preserving global memory bank further supports client fine-tuning. This approach addresses challenges in previous research by improving computational efficiency and scalability. Practical applications include Car2X event predictions, where a shared model is collectively trained to recognize traffic patterns, aiding in tasks such as traffic density assessment and accident detection. The research aims to develop a robust framework that combines federated learning, deep transfer hashing and transfer learning for efficient and secure downstream task execution., Comment: Presented at ECML2024: 8th Intl. Worksh. and Tutorial on Interactive Adaptive Learning, Sep. 9th, 2024, Vilnius, Lithuania
- Published
- 2024
3. Sparse Uncertainty-Informed Sampling from Federated Streaming Data
- Author
-
Röder, Manuel and Schleif, Frank-Michael
- Subjects
Computer Science - Machine Learning ,Computer Science - Computer Vision and Pattern Recognition - Abstract
We present a numerically robust, computationally efficient approach for non-I.I.D. data stream sampling in federated client systems, where resources are limited and labeled data for local model adaptation is sparse and expensive. The proposed method identifies relevant stream observations to optimize the underlying client model, given a local labeling budget, and performs instantaneous labeling decisions without relying on any memory buffering strategies. Our experiments show enhanced training batch diversity and an improved numerical robustness of the proposal compared to existing strategies over large-scale data streams, making our approach an effective and convenient solution in FL environments., Comment: Preprint, 6 pages, 3 figures, Accepted for ESANN 2024
- Published
- 2024
4. Is Child-Directed Speech Effective Training Data for Language Models?
- Author
-
Feng, Steven Y., Goodman, Noah D., and Frank, Michael C.
- Subjects
Computer Science - Computation and Language ,Computer Science - Artificial Intelligence ,Computer Science - Machine Learning - Abstract
While high-performing language models are typically trained on hundreds of billions of words, human children become fluent language users with a much smaller amount of data. What are the features of the data they receive, and how do these features support language modeling objectives? To investigate this question, we train GPT-2 and RoBERTa models on 29M words of English child-directed speech and a new matched, synthetic dataset (TinyDialogues), comparing to OpenSubtitles, Wikipedia, and a heterogeneous blend of datasets from the BabyLM challenge. We evaluate the syntactic and semantic knowledge of these models using developmentally-inspired evaluations. Through pretraining experiments, we test whether the global developmental ordering or the local discourse ordering of children's training data supports high performance relative to other datasets. The local properties of the data affect model results, but surprisingly, global properties do not. Further, child language input is not uniquely valuable for training language models. These findings support the hypothesis that, rather than proceeding from better data, the child's learning algorithm is substantially more data-efficient than current language modeling techniques., Comment: EMNLP 2024. Code and data at https://github.com/styfeng/TinyDialogues
- Published
- 2024
5. A systematic review on expert systems for improving energy efficiency in the manufacturing industry
- Author
-
Ioshchikhes, Borys, Frank, Michael, and Weigold, Matthias
- Subjects
Computer Science - Artificial Intelligence ,Computer Science - Computers and Society - Abstract
Against the backdrop of the European Union's commitment to achieve climate neutrality by 2050, efforts to improve energy efficiency are being intensified. The manufacturing industry is a key focal point of these endeavors due to its high final electrical energy demand, while simultaneously facing a growing shortage of skilled workers crucial for meeting established goals. Expert systems (ESs) offer the chance to overcome this challenge by automatically identifying potential energy efficiency improvements and thereby playing a significant role in reducing electricity consumption. This paper systematically reviews state-of-the-art approaches of ESs aimed at improving energy efficiency in industry, with a focus on manufacturing. The literature search yields 1692 results, of which 54 articles published between 1987 and 2023 are analyzed in depth. These publications are classified according to the system boundary, manufacturing type, application perspective, application purpose, ES type, and industry. Furthermore, we examine the structure, implementation, utilization, and development of ESs in this context. Through this analysis, the review reveals research gaps, pointing toward promising topics for future research., Comment: 23 pages, 7 figures, journal
- Published
- 2024
- Full Text
- View/download PDF
6. The BabyView dataset: High-resolution egocentric videos of infants' and young children's everyday experiences
- Author
-
Long, Bria, Xiang, Violet, Stojanov, Stefan, Sparks, Robert Z., Yin, Zi, Keene, Grace E., Tan, Alvin W. M., Feng, Steven Y., Zhuang, Chengxu, Marchman, Virginia A., Yamins, Daniel L. K., and Frank, Michael C.
- Subjects
Computer Science - Computer Vision and Pattern Recognition - Abstract
Human children far exceed modern machine learning algorithms in their sample efficiency, achieving high performance in key domains with much less data than current models. This ''data gap'' is a key challenge both for building intelligent artificial systems and for understanding human development. Egocentric video capturing children's experience -- their ''training data'' -- is a key ingredient for comparison of humans and models and for the development of algorithmic innovations to bridge this gap. Yet there are few such datasets available, and extant data are low-resolution, have limited metadata, and importantly, represent only a small set of children's experiences. Here, we provide the first release of the largest developmental egocentric video dataset to date -- the BabyView dataset -- recorded using a high-resolution camera with a large vertical field-of-view and gyroscope/accelerometer data. This 493 hour dataset includes egocentric videos from children spanning 6 months - 5 years of age in both longitudinal, at-home contexts and in a preschool environment. We provide gold-standard annotations for the evaluation of speech transcription, speaker diarization, and human pose estimation, and evaluate models in each of these domains. We train self-supervised language and vision models and evaluate their transfer to out-of-distribution tasks including syntactic structure learning, object recognition, depth estimation, and image segmentation. Although performance in each scales with dataset size, overall performance is relatively lower than when models are trained on curated datasets, especially in the visual domain. Our dataset stands as an open challenge for robust, humanlike AI systems: how can such systems achieve human-levels of success on the same scale and distribution of training data as humans?, Comment: 9 pages, 2 figures, 4 tables and SI. Submitted to NeurIPS Datasets and Benchmarks
- Published
- 2024
7. DevBench: A multimodal developmental benchmark for language learning
- Author
-
Tan, Alvin Wei Ming, Yu, Sunny, Long, Bria, Ma, Wanjing Anya, Murray, Tonya, Silverman, Rebecca D., Yeatman, Jason D., and Frank, Michael C.
- Subjects
Computer Science - Computation and Language ,Computer Science - Machine Learning - Abstract
How (dis)similar are the learning trajectories of vision-language models and children? Recent modeling work has attempted to understand the gap between models' and humans' data efficiency by constructing models trained on less data, especially multimodal naturalistic data. However, such models are often evaluated on adult-level benchmarks, with limited breadth in language abilities tested, and without direct comparison to behavioral data. We introduce DevBench, a multimodal benchmark comprising seven language evaluation tasks spanning the domains of lexical, syntactic, and semantic ability, with behavioral data from both children and adults. We evaluate a set of vision-language models on these tasks, comparing models and humans not only on accuracy but on their response patterns. Across tasks, models exhibit variation in their closeness to human response patterns, and models that perform better on a task also more closely resemble human behavioral responses. We also examine the developmental trajectory of OpenCLIP over training, finding that greater training results in closer approximations to adult response patterns. DevBench thus provides a benchmark for comparing models to human language development. These comparisons highlight ways in which model and human language learning processes diverge, providing insight into entry points for improving language models.
- Published
- 2024
8. Auxiliary task demands mask the capabilities of smaller language models
- Author
-
Hu, Jennifer and Frank, Michael C.
- Subjects
Computer Science - Computation and Language ,Computer Science - Artificial Intelligence - Abstract
Developmental psychologists have argued about when cognitive capacities such as language understanding or theory of mind emerge. These debates often hinge on the concept of "task demands" -- the auxiliary challenges associated with performing a particular evaluation -- that may mask the child's underlying ability. The same issues arise when measuring the capacities of language models (LMs): performance on a task is a function of the model's underlying knowledge, combined with the model's ability to interpret and perform the task given its available resources. Here, we show that for analogical reasoning, reflective reasoning, word prediction, and grammaticality judgments, evaluation methods with greater task demands yield lower performance than evaluations with reduced demands. This "demand gap" is most pronounced for models with fewer parameters and less training data. Our results illustrate that LM performance should not be interpreted as a direct indication of intelligence (or lack thereof), but as a reflection of capacities seen through the lens of researchers' design choices., Comment: Published at the 1st Conference on Language Modeling (COLM 2024)
- Published
- 2024
9. Curriculum effects and compositionality emerge with in-context learning in neural networks
- Author
-
Russin, Jacob, Pavlick, Ellie, and Frank, Michael J.
- Subjects
Computer Science - Neural and Evolutionary Computing ,Computer Science - Machine Learning ,Quantitative Biology - Neurons and Cognition - Abstract
Human learning embodies a striking duality: sometimes, we appear capable of following logical, compositional rules and benefit from structured curricula (e.g., in formal education), while other times, we rely on an incremental approach or trial-and-error, learning better from curricula that are unstructured or randomly interleaved. Influential psychological theories explain this seemingly disparate behavioral evidence by positing two qualitatively different learning systems -- one for rapid, rule-based inferences and another for slow, incremental adaptation. It remains unclear how to reconcile such theories with neural networks, which learn via incremental weight updates and are thus a natural model for the latter type of learning, but are not obviously compatible with the former. However, recent evidence suggests that both metalearning neural networks and large language models are capable of "in-context learning" (ICL) -- the ability to flexibly grasp the structure of a new task from a few examples given at inference time. Here, we show that networks capable of ICL can reproduce human-like learning and compositional behavior on rule-governed tasks, while at the same time replicating human behavioral phenomena in tasks lacking rule-like structure via their usual in-weight learning (IWL). Our work shows how emergent ICL can equip neural networks with fundamentally different learning properties than those traditionally attributed to them, and that these can coexist with the properties of their native IWL, thus offering a novel perspective on dual-process theories and human cognitive flexibility., Comment: 27 pages (including appendix), 10 figures, 7 tables. Previous version accepted as a talk + full paper at CogSci 2024
- Published
- 2024
10. Transformer Mechanisms Mimic Frontostriatal Gating Operations When Trained on Human Working Memory Tasks
- Author
-
Traylor, Aaron, Merullo, Jack, Frank, Michael J., and Pavlick, Ellie
- Subjects
Computer Science - Artificial Intelligence ,I.2.6 - Abstract
Models based on the Transformer neural network architecture have seen success on a wide variety of tasks that appear to require complex "cognitive branching" -- or the ability to maintain pursuit of one goal while accomplishing others. In cognitive neuroscience, success on such tasks is thought to rely on sophisticated frontostriatal mechanisms for selective \textit{gating}, which enable role-addressable updating -- and later readout -- of information to and from distinct "addresses" of memory, in the form of clusters of neurons. However, Transformer models have no such mechanisms intentionally built-in. It is thus an open question how Transformers solve such tasks, and whether the mechanisms that emerge to help them to do so bear any resemblance to the gating mechanisms in the human brain. In this work, we analyze the mechanisms that emerge within a vanilla attention-only Transformer trained on a simple sequence modeling task inspired by a task explicitly designed to study working memory gating in computational cognitive neuroscience. We find that, as a result of training, the self-attention mechanism within the Transformer specializes in a way that mirrors the input and output gating mechanisms which were explicitly incorporated into earlier, more biologically-inspired architectures. These results suggest opportunities for future research on computational similarities between modern AI architectures and models of the human brain., Comment: 8 pages, 4 figures
- Published
- 2024
11. A Meta-analysis of Syntactic Satiation in Extraction from Islands
- Author
-
Lu, Jiayi, Frank, Michael, and Degen, Judith
- Abstract
Sentence acceptability judgments are often affected by a pervasive phenomenon called satiation: native speakers give increasingly higher ratings to initially degraded sentences after repeated exposure. Various studies have investigated the satiation effect experimentally, the vast majority of which focused on different types of island-violating sentences in English (sentences with illicit long-distance syntactic movements). However, mixed findings are reported regarding which types of island violations are affected by satiation and which ones are not. This article presents a meta-analysis of past experimental studies on the satiation of island effects in English, with the aim of providing accurate estimates of the rate of satiation for each type of island, testing whether different island effects show different rates of satiation, exploring potential factors that contributed to the heterogeneity in past results, and spotting possible publication bias. The meta-analysis shows that adjunct islands, the Complex NP Constraint (CNPC), subject islands, the that-trace effect, the want-for construction, and whether-islands reliably exhibit satiation, albeit at different rates. No evidence for satiation is found for the Left Branch Condition (LBC). Whether context sentences were presented in the original acceptability judgment experiments predicts the differences in the rates of satiation reported across studies. Potential publication bias is found among studies testing the CNPC and whether-islands. These meta-analytic results can be used to inform debates regarding the nature of island effects and serve as a proof of concept that meta-analysis can be a valuable tool for linguistic research.
- Published
- 2024
12. Efficient Cross-Domain Federated Learning by MixStyle Approximation
- Author
-
Röder, Manuel, Heller, Leon, Münch, Maximilian, and Schleif, Frank-Michael
- Subjects
Computer Science - Machine Learning ,Computer Science - Computer Vision and Pattern Recognition - Abstract
With the advent of interconnected and sensor-equipped edge devices, Federated Learning (FL) has gained significant attention, enabling decentralized learning while maintaining data privacy. However, FL faces two challenges in real-world tasks: expensive data labeling and domain shift between source and target samples. In this paper, we introduce a privacy-preserving, resource-efficient FL concept for client adaptation in hardware-constrained environments. Our approach includes server model pre-training on source data and subsequent fine-tuning on target data via low-end clients. The local client adaptation process is streamlined by probabilistic mixing of instance-level feature statistics approximated from source and target domain data. The adapted parameters are transferred back to the central server and globally aggregated. Preliminary results indicate that our method reduces computational and transmission costs while maintaining competitive performance on downstream tasks., Comment: Accepted at the Adapting to Change: Reliable Multimodal Learning Across Domains Workshop @ ECML PKKD 2023
- Published
- 2023
13. The BabyView camera: Designing a new head-mounted camera to capture children’s early social and visual environments
- Author
-
Long, Bria, Goodin, Sarah, Kachergis, George, Marchman, Virginia A., Radwan, Samaher F., Sparks, Robert Z., Xiang, Violet, Zhuang, Chengxu, Hsu, Oliver, Newman, Brett, Yamins, Daniel L. K., and Frank, Michael C.
- Published
- 2024
- Full Text
- View/download PDF
14. Neural scaling laws for phenotypic drug discovery
- Author
-
Linsley, Drew, Griffin, John, Brown, Jason Parker, Roose, Adam N, Frank, Michael, Linsley, Peter, Finkbeiner, Steven, and Linsley, Jeremy
- Subjects
Computer Science - Machine Learning ,Computer Science - Artificial Intelligence ,Computer Science - Computer Vision and Pattern Recognition ,Quantitative Biology - Quantitative Methods - Abstract
Recent breakthroughs by deep neural networks (DNNs) in natural language processing (NLP) and computer vision have been driven by a scale-up of models and data rather than the discovery of novel computing paradigms. Here, we investigate if scale can have a similar impact for models designed to aid small molecule drug discovery. We address this question through a large-scale and systematic analysis of how DNN size, data diet, and learning routines interact to impact accuracy on our Phenotypic Chemistry Arena (Pheno-CA) benchmark: a diverse set of drug development tasks posed on image-based high content screening data. Surprisingly, we find that DNNs explicitly supervised to solve tasks in the Pheno-CA do not continuously improve as their data and model size is scaled-up. To address this issue, we introduce a novel precursor task, the Inverse Biological Process (IBP), which is designed to resemble the causal objective functions that have proven successful for NLP. We indeed find that DNNs first trained with IBP then probed for performance on the Pheno-CA significantly outperform task-supervised DNNs. More importantly, the performance of these IBP-trained DNNs monotonically improves with data and model scale. Our findings reveal that the DNN ingredients needed to accurately solve small molecule drug development tasks are already in our hands, and project how much more experimental data is needed to achieve any desired level of improvement. We release our Pheno-CA benchmark and code to encourage further study of neural scaling laws for small molecule drug discovery.
- Published
- 2023
15. Diagnosing and exploiting the computational demands of videos games for deep reinforcement learning
- Author
-
Govindarajan, Lakshmi Narasimhan, Liu, Rex G, Linsley, Drew, Ashok, Alekh Karkada, Reuter, Max, Frank, Michael J, and Serre, Thomas
- Subjects
Computer Science - Machine Learning ,Computer Science - Artificial Intelligence ,Computer Science - Computer Vision and Pattern Recognition ,Computer Science - Robotics - Abstract
Humans learn by interacting with their environments and perceiving the outcomes of their actions. A landmark in artificial intelligence has been the development of deep reinforcement learning (dRL) algorithms capable of doing the same in video games, on par with or better than humans. However, it remains unclear whether the successes of dRL models reflect advances in visual representation learning, the effectiveness of reinforcement learning algorithms at discovering better policies, or both. To address this question, we introduce the Learning Challenge Diagnosticator (LCD), a tool that separately measures the perceptual and reinforcement learning demands of a task. We use LCD to discover a novel taxonomy of challenges in the Procgen benchmark, and demonstrate that these predictions are both highly reliable and can instruct algorithmic development. More broadly, the LCD reveals multiple failure cases that can occur when optimizing dRL algorithms over entire video game benchmarks like Procgen, and provides a pathway towards more efficient progress.
- Published
- 2023
16. Parallel developmental changes in childrens production and recognition of line drawings of visual concepts.
- Author
-
Chai, Zixian, Frank, Michael, Long, Bria, Fan, Judith, and Huey, Holly
- Abstract
Childhood is marked by the rapid accumulation of knowledge and the prolific production of drawings. We conducted a systematic study of how children create and recognize line drawings of visual concepts. We recruited 2-10-year-olds to draw 48 categories via a kiosk at a childrens museum, resulting in >37K drawings. We analyze changes in the category-diagnostic information in these drawings using vision algorithms and annotations of object parts. We find developmental gains in childrens inclusion of category-diagnostic information that are not reducible to variation in visuomotor control or effort. Moreover, even unrecognizable drawings contain information about the animacy and size of the category children tried to draw. Using guessing games at the same kiosk, we find that children improve across childhood at recognizing each others line drawings. This work leverages vision algorithms to characterize developmental changes in childrens drawings and suggests that these changes reflect refinements in childrens internal representations.
- Published
- 2024
17. Using Psychometrics to Improve Cognitive Models--and Theory
- Author
-
Tan, Alvin Wei Ming, Kachergis, George, and Frank, Michael C.
- Subjects
Psychology ,Cognitive development ,Skill acquisition and learning ,Computational Modeling ,Statistics - Abstract
The field of psychometrics has undergone substantial evolution over the past several decades, both in terms of advances in methodology and improved software and hardware for deploying new methods. Despite these strides, many of these developments have not been integrated into the broader field of psychology, as highlighted by Embretson (2005) and Borsboom (2006). Understanding and incorporating these psychometric advances is crucial to enable cognitive scientists to address growing concerns about validity and reliability, as well as to develop robust theoretical frameworks for understanding cognition.
- Published
- 2024
18. Cognitive diversity in context: US-China differences in children's reasoning, visual attention, and social cognition
- Author
-
Carstensen, Alexandra, Cao, Anjie, Tan, Alvin Wei Ming, Liu, Di, Liu, Yichun, Bui, Minh Khong, Wang-Zhao, Jiayi, Diep, Ai Nghi, Han, Qi, Frank, Michael C., and Walker, Caren M.
- Subjects
Attention ,Causal reasoning ,Cognitive development ,Social cognition ,Cross-cultural analysis - Abstract
Outward differences between cultures are very salient, with Western and East Asian cultures as a prominent comparison pair. A large literature describes cross-cultural variation in cognition, but relatively less research has explored the developmental origins of this variation. This study helps to fill the empirical gap by replicating four prominent findings documenting cross-cultural differences in children's reasoning, visual attention, and social cognition in a cross-sectional sample of 240 3-12-year-olds from the US and China. We observe cross-cultural differences in three of the four tasks and describe the distinct developmental trajectory that each task follows throughout early and middle childhood.
- Published
- 2024
19. Show or Tell? Preschool-aged children adapt their communication to their partner's auditory access
- Author
-
Chuey, Aaron, Qing, Catherine, Williams, Rondeline M., Frank, Michael C., and Gweon, Hyowon
- Subjects
Psychology ,Cognitive development ,Social cognition ,Theory of Mind - Abstract
Adults routinely tailor their communication to others' auditory access, such as substituting gestures for speech in noisy environments. Yet, assessing the effectiveness of different communicative acts given others' perceptual access—especially when it differs from one's own—requires mental-state reasoning, which undergoes significant developmental change. Can young children tailor their communication to others' auditory access? In Study 1, parental report (n=98) indicated that most children, by age 4, adjust their communicative behaviors in noisy settings. Study 2 elicited these behaviors experimentally with 4- to 5-year-olds (n=68). Children taught how a novel toy works to a learner who wore headphones playing either loud music or nothing. Children were more likely to use physical demonstrations, and less likely to use verbal explanations, when the learner's auditory access was obstructed. These findings illustrate how mental-state reasoning might support children's ability to communicate successfully across perceptually-compromised contexts and individuals.
- Published
- 2024
20. Predicting ages of acquisition for children's early vocabulary across 27 languages and dialects
- Author
-
Tan, Alvin Wei Ming, Loukatou, Georgia, Braginsky, Mika, Mankewitz, Jess, and Frank, Michael C.
- Subjects
Psychology ,Development ,Language development ,Language learning ,Bayesian modeling ,Big data ,Cross-linguistic analysis ,Developmental analysis ,Statistics - Abstract
What factors contribute to the growth of children's early vocabulary? One method for exploring this question is investigating predictors (e.g., frequency) that differentiate words learnt earlier from those learnt later. A more comprehensive account requires the examination of multiple language families and multiple levels of linguistic representation (e.g., phonological, morphosyntactic, semantic). Here, we studied 10 predictors of word ages of acquisition across 27 languages and dialects. We found that words that were more frequent, concrete, and associated with babies were learnt earlier, whereas words that had greater length in phonemes and mean length of utterance were learnt later. There was no reliable effect of other morphosyntactic predictors, or of phonological neighbourhood. We found evidence of a tradeoff between a language's morphological complexity and the effect of syntactic complexity for predicates, supporting the competition model. Predictor coefficients revealed broad consistency across all languages, along with variability that reflected language family classifications.
- Published
- 2024
21. The Perils of Omitting Omissions when Modeling Evidence Accumulation
- Author
-
Leng, Xiamin, Fengler, Alexander, Shenhav, Amitai, and Frank, Michael J.
- Subjects
Cognitive Neuroscience ,Decision making ,Bayesian modeling ,Computational Modeling ,Mathematical modeling - Abstract
Choice deadlines are commonly imposed in decision-making research to incentivize speedy responses and sustained attention to the task settings. However, computational models of choice and response times routinely overlook this deadline, instead simply omitting trials past the deadline from further analysis. This choice is made under the implicit assumption that parameter estimation is not significantly affected by ignoring these omissions. Using new tools from likelihood-free inference, here we elucidate the degree to which omitting omissions, even in seemingly benign settings, can lead researchers astray. We explore the phenomenon using a Sequential Sampling Model (SSM) with collapsing boundaries as a test-bed.
- Published
- 2024
22. Predicting graded dishabituation in a rational learning model using perceptual stimulus embeddings
- Author
-
Cao, Anjie, Raz, Gal, Saxe, Rebecca, and Frank, Michael C.
- Subjects
Artificial Intelligence ,Computer Science ,Psychology ,Attention ,Behavioral Science ,Decision making ,Learning ,Representation ,Bayesian modeling ,Computer-based experiment - Abstract
How do humans decide what to look at and when to stop looking? The Rational Action, Noisy Choice for Habituation (RANCH) model formulates looking behaviors as a rational information acquisition process. RANCH instantiates a hypothesis about the perceptual encoding process using a neural network-derived embedding space, which allows it to operate on raw images. In this paper, we show that the model not only captures key looking time patterns such as habituation and dishabituation, but also makes fine-grained, out-of-sample predictions about magnitudes of dishabituation to previously unseen stimuli. We validated those predictions experimentally with a self-paced looking time task in adults (N = 468). We also show that model fits are robust across parameters, but that assumptions about the perceptual encoding process, the learning process and the decision process are all critical for predicting human performance.
- Published
- 2024
23. Human Curriculum Effects Emerge with In-Context Learning in Neural Networks
- Author
-
Russin, Jacob, Pavlick, Ellie, and Frank, Michael J.
- Subjects
Learning ,Problem Solving ,Computational neuroscience ,Large Language Models ,Neural Networks - Abstract
Human learning is sensitive to rule-like structure and the curriculum of examples used for training. In tasks governed by succinct rules, learning is more robust when related examples are blocked across trials, but in the absence of such rules, interleaving is more effective. To date, no neural model has simultaneously captured these seemingly contradictory effects. Here we show that this same tradeoff spontaneously emerges with “in-context learning” (ICL) both in neural networks trained with metalearning and in large language models (LLMs). ICL is the ability to learn new tasks “in context” — without weight changes — via an inner-loop algorithm implemented in activation dynamics. Experiments with pretrained LLMs and metalearning transformers show that ICL exhibits the blocking advantage demonstrated in humans on a task involving rule-like structure, and conversely, that concurrent in-weight learning reproduces the interleaving advantage observed in humans on tasks lacking such structure.
- Published
- 2024
24. Modeling Social Learning Through Demonstration in Multi-Armed Bandits
- Author
-
Martinez, Julio, Frank, Michael C., and Haber, Nick
- Subjects
Psychology ,Decision making ,Learning ,Social cognition ,Computational Modeling - Abstract
Humans are efficient social learners who leverage social information to rapidly adapt to new environments, but the computations by which we combine social information with prior knowledge are poorly understood. We study social learning within the context of multi-armed bandits using a novel “asteroid mining” video game where participants learn through active play and passive observation of expert and novice players. We simulate human exploration and social learning using naive versions of Thompson and Upper Confidence Bound (UCB) solvers and hybrid models that use Thompson and UCB solvers for direct learning together with a multi-layer perceptron to estimate what should be learned from other players. Two variants of the hybrid models provide good, parameter-free fits to human performance across a range of learning conditions. Our work shows a route for integrating social learning into reinforcement learning models and suggests that human social learning conforms to the predictions of such models.
- Published
- 2024
25. Young children strategically adapt to unreliable social partners
- Author
-
Shannon, Katherine Adams, Conine-Nakano, Aneesa, Frankenhuis, Willem E., Frank, Michael C., and Gweon, Hyowon
- Subjects
Cognitive development ,Decision making ,Social cognition - Abstract
Children learn a lot from others, but the effectiveness of their social learning depends on the reliability of others' help. How do children adapt their future learning decisions based on the past reliability of receiving help? In two experiments, 4- to 6-year-olds (N = 60 each) interacted with a researcher who either followed through on promised help (Reliable condition) or failed to do so (Unreliable condition). Experiment 1 was inconclusive. However, with an improved design, Experiment 2 found that children in the Unreliable condition were more likely to forego a harder but more rewarding puzzle as their next task and choose an easier, less rewarding puzzle instead compared to those in the Reliable condition. Such decisions, while seemingly maladaptive at face value, likely reflect an adaptive response to the low likelihood of receiving help. These results extend our understanding of social learning across diverse ecological contexts.
- Published
- 2024
26. Characterizing Contextual Variation in Children's Preschool Language Environment Using Naturalistic Egocentric Videos
- Author
-
Sparks, Robert Z., Long, Bria, Keene, Grace E, Perez, Malia J., Tan, Alvin Wei Ming, Marchman, Virginia A, and Frank, Michael C.
- Subjects
Cognitive development ,Language development ,Language learning ,Classroom studies ,Corpus studies - Abstract
What structures children's early language environment? Large corpora of child-centered naturalistic recordings provide an important window into this question, but most available data centers on young children within the home or in lab contexts interacting primarily with a single caregiver. Here, we characterize children's language experience in a very different kind of environment: the preschool classroom. Children ages 3 – 5 years (N = 26) wore a head-mounted camera in their preschool class, yielding a naturalistic, egocentric view of children's everyday experience across many classroom activity contexts (e.g., sand play, snack time), with >30 hours of video data. Using semi-automatic transcriptions (227,624 words), we find that activity contexts in the preschool classroom vary in both the quality and quantity of the language that children both hear and produce. Together, these findings reinforce prior theories emphasizing the contribution of activity contexts in structuring the variability in children's early learning environments.
- Published
- 2024
27. Examining the robustness and generalizability of the shape bias: a meta-analysis
- Author
-
Abdelrahim, Samah O and Frank, Michael C.
- Subjects
Psychology ,Concepts and categories ,Language development ,Cross-cultural analysis ,Cross-linguistic analysis - Abstract
The "shape bias" -- the bias to generalize new nouns by their shape rather than other features such as color or texture -- has been argued to facilitate early noun learning for children. However, there is conflicting evidence about the magnitude and nature of this bias, as well as how it changes developmentally and how it varies across cultures. In this paper, we synthesize evidence about the shape bias using meta-analysis and meta-regression. We find strong overall evidence for the shape bias, but the literature is dominated by studies of English-speaking children, making it difficult to assess cross-cultural differences. Large between-study heterogeneity also highlights procedural variation in the literature. Overall, publication bias, heterogeneity, and data sparsity may limit the ability to distinguish theoretical accounts of the shape bias.
- Published
- 2024
28. A large-scale comparison of cross-situational word learning models
- Author
-
Kachergis, George and Frank, Michael C.
- Subjects
Psychology ,Language learning ,Computational Modeling - Abstract
One problem language learners face is extracting word meanings from scenes with many possible referents. Despite the ambiguity of individual situations, a large body of empirical work shows that people are able to learn cross-situationally when a word occurs in different situations. Many computational models of cross-situational word learning have been proposed, yet there is little consensus on the main mechanisms supporting learning, in part due to the profusion of disparate studies and models, and lack of systematic model comparisons across a wide range of studies. This study compares the performance of several extant models on a dataset of 44 experimental conditions and a total of 1,696 participants. Using cross-validation, we fit multiple models representing theories of both associative learning and hypothesis-testing theories of word learning, find two best-fitting models, and discuss issues of model and mechanism identifiability. Finally, we test the models' ability to generalize to additional experiments, including develop- mental data.
- Published
- 2024
29. Simulating Infants' Attachment: Behavioral Patterns of Caregiver Proximity Seeking and Environment Exploration Using Reinforcement Learning Models.
- Author
-
Zhou, Xi Jia, Doyle, Chris, Frank, Michael C., and Haber, Nick
- Subjects
Psychology ,Cognitive development ,Decision making ,Learning ,Agent-based Modeling - Abstract
Attachment is crucial for infants' cognitive development and social relationships. Traditional attachment research has been qualitative, lacking a model to explain how infants' attachment styles develop from experience and how these are influenced by personal traits and environmental factors. We propose such a model, predicting how infants balance interaction with caregivers against exploring their surroundings. Our study is based in a grid-world environment containing an infant and caregiver agent. We vary the infant's temperamental factors (e.g., ability to regulate emotions and preferences for social vs. environmental reward), and caregiver behavior (whether positive or negative interactions are more likely). We find that different equilibria result that qualitatively correspond to different attachment styles. Our findings suggest that the characteristic exploratory behavior of each attachment style in real infants may arise from interactions of infant temperament and caregiver behaviors.
- Published
- 2024
30. Learning the meanings of function words from grounded language using a visual question answering model
- Author
-
Portelance, Eva, Frank, Michael C., and Jurafsky, Dan
- Subjects
Computer Science - Computation and Language ,I.2.7 ,I.2.6 ,I.2.10 - Abstract
Interpreting a seemingly-simple function word like "or", "behind", or "more" can require logical, numerical, and relational reasoning. How are such words learned by children? Prior acquisition theories have often relied on positing a foundation of innate knowledge. Yet recent neural-network based visual question answering models apparently can learn to use function words as part of answering questions about complex visual scenes. In this paper, we study what these models learn about function words, in the hope of better understanding how the meanings of these words can be learnt by both models and children. We show that recurrent models trained on visually grounded language learn gradient semantics for function words requiring spatial and numerical reasoning. Furthermore, we find that these models can learn the meanings of logical connectives and and or without any prior knowledge of logical reasoning, as well as early evidence that they are sensitive to alternative expressions when interpreting language. Finally, we show that word learning difficulty is dependent on frequency in models' input. Our findings offer proof-of-concept evidence that it is possible to learn the nuanced interpretations of function words in visually grounded context by using non-symbolic general statistical learning algorithms, without any prior knowledge of linguistic meaning., Comment: Published in Cognitive Science 2024
- Published
- 2023
31. Managing EEG studies: How to prepare and what to do once data collection has begun
- Author
-
Boudewyn, Megan A, Erickson, Molly A, Winsler, Kurt, Ragland, John Daniel, Yonelinas, Andrew, Frank, Michael, Silverstein, Steven M, Gold, Jim, MacDonald, Angus W, Carter, Cameron S, Barch, Deanna M, and Luck, Steven J
- Subjects
Biological Sciences ,Biomedical and Clinical Sciences ,Psychology ,Neurosciences ,Clinical Research ,Electroencephalography ,Humans ,Data Collection ,Software ,Research Design ,EEG methods ,guidelines ,large-scale ,multisite ,protocol ,recommendations ,Medical and Health Sciences ,Psychology and Cognitive Sciences ,Experimental Psychology ,Biological sciences ,Biomedical and clinical sciences - Abstract
In this paper, we provide guidance for the organization and implementation of EEG studies. This work was inspired by our experience conducting a large-scale, multi-site study, but many elements could be applied to any EEG project. Section 1 focuses on study activities that take place before data collection begins. Topics covered include: establishing and training study teams, considerations for task design and piloting, setting up equipment and software, development of formal protocol documents, and planning communication strategy with all study team members. Section 2 focuses on what to do once data collection has already begun. Topics covered include: (1) how to effectively monitor and maintain EEG data quality, (2) how to ensure consistent implementation of experimental protocols, and (3) how to develop rigorous preprocessing procedures that are feasible for use in a large-scale study. Links to resources are also provided, including sample protocols, sample equipment and software tracking forms, sample code, and tutorial videos (to access resources, please visit: https://osf.io/wdrj3/).
- Published
- 2023
32. Optimizing YOLOv5 for Green AI: A Study on Model Pruning and Lightweight Networks
- Author
-
Xu, Bangguo, Yan, Simei, Liu, Liang, Schleif, Frank-Michael, Kacprzyk, Janusz, Series Editor, Gomide, Fernando, Advisory Editor, Kaynak, Okyay, Advisory Editor, Liu, Derong, Advisory Editor, Pedrycz, Witold, Advisory Editor, Polycarpou, Marios M., Advisory Editor, Rudas, Imre J., Advisory Editor, Wang, Jun, Advisory Editor, Villmann, Thomas, editor, Kaden, Marika, editor, Geweniger, Tina, editor, and Schleif, Frank-Michael, editor
- Published
- 2024
- Full Text
- View/download PDF
33. Practical Approaches to Approximate Dominant Eigenvalues in Large Matrices
- Author
-
Schleif, Frank-Michael, Kacprzyk, Janusz, Series Editor, Gomide, Fernando, Advisory Editor, Kaynak, Okyay, Advisory Editor, Liu, Derong, Advisory Editor, Pedrycz, Witold, Advisory Editor, Polycarpou, Marios M., Advisory Editor, Rudas, Imre J., Advisory Editor, Wang, Jun, Advisory Editor, Villmann, Thomas, editor, Kaden, Marika, editor, Geweniger, Tina, editor, and Schleif, Frank-Michael, editor
- Published
- 2024
- Full Text
- View/download PDF
34. Can perioperative pCO2 gaps predict complications in patients undergoing major elective abdominal surgery randomized to goal-directed therapy or standard care? A secondary analysis
- Author
-
de Keijzer, Ilonka N., Kaufmann, Thomas, de Waal, Eric E.C., Frank, Michael, de Korte-de Boer, Dianne, Montenij, Leonard M., Buhre, Wolfgang, and Scheeren, Thomas W.L.
- Published
- 2024
- Full Text
- View/download PDF
35. Virtual Product Optimization of Cyclone Pre-separators for Heavy Machinery
- Author
-
Ehrle, Maximilian, Colletto, Calogero, Frank, Michael, and Wierse, Andreas
- Published
- 2024
- Full Text
- View/download PDF
36. Tensor Slicing and Optimization for Multicore NPUs
- Author
-
Sousa, Rafael, Pereira, Marcio, Kwon, Yongin, Kim, Taeho, Jung, Namsoon, Kim, Chang Soo, Frank, Michael, and Araujo, Guido
- Subjects
Computer Science - Performance ,Computer Science - Hardware Architecture ,Computer Science - Computer Vision and Pattern Recognition - Abstract
Although code generation for Convolution Neural Network (CNN) models has been extensively studied, performing efficient data slicing and parallelization for highly-constrai\-ned Multicore Neural Processor Units (NPUs) is still a challenging problem. Given the size of convolutions' input/output tensors and the small footprint of NPU on-chip memories, minimizing memory transactions while maximizing parallelism and MAC utilization are central to any effective solution. This paper proposes a TensorFlow XLA/LLVM compiler optimization pass for Multicore NPUs, called Tensor Slicing Optimization (TSO), which: (a) maximizes convolution parallelism and memory usage across NPU cores; and (b) reduces data transfers between host and NPU on-chip memories by using DRAM memory burst time estimates to guide tensor slicing. To evaluate the proposed approach, a set of experiments was performed using the NeuroMorphic Processor (NMP), a multicore NPU containing 32 RISC-V cores extended with novel CNN instructions. Experimental results show that TSO is capable of identifying the best tensor slicing that minimizes execution time for a set of CNN models. Speed-ups of up to 21.7\% result when comparing the TSO burst-based technique to a no-burst data slicing approach. To validate the generality of the TSO approach, the algorithm was also ported to the Glow Machine Learning framework. The performance of the models were measured on both Glow and TensorFlow XLA/LLVM compilers, revealing similar results.
- Published
- 2023
- Full Text
- View/download PDF
37. Developmental Changes in Drawing Production under Different Memory Demands in a U.S. and Chinese Sample
- Author
-
Long, Bria, Wang, Ying, Christie, Stella, Frank, Michael C., and Fan, Judith E.
- Abstract
Children's drawings of common object categories become dramatically more recognizable across childhood. What are the major factors that drive developmental changes in children's drawings? To what degree are children's drawings a product of their changing internal category representations versus limited by their visuomotor abilities or their ability to recall the relevant visual information? To explore these questions, we examined the degree to which developmental changes in drawing recognizability vary across different drawing tasks that vary in memory demands (i.e., drawing from verbal vs. picture cues) and with children's shape-tracing abilities across two geographical locations (San Jose, United States, and Beijing, China). We collected digital shape tracings and drawings of common object categories (e.g., cat, airplane) from 4- to 9-year-olds (N = 253). The developmental trajectory of drawing recognizability was remarkably similar when children were asked to draw from pictures versus verbal cues and across these two geographical locations. In addition, our Beijing sample produced more recognizable drawings but showed similar tracing abilities to children from San Jose. Overall, this work suggests that the developmental trajectory of children's drawings is remarkably consistent and not easily explainable by changes in visuomotor control or working memory; instead, changes in children's drawings over development may at least partly reflect changes in the internal representations of object categories.
- Published
- 2023
- Full Text
- View/download PDF
38. Predicting Age of Acquisition for Children's Early Vocabulary in Five Languages Using Language Model Surprisal
- Author
-
Portelance, Eva, Duan, Yuguang, Frank, Michael C., and Lupyan, Gary
- Abstract
What makes a word easy to learn? Early-learned words are frequent and tend to name concrete referents. But words typically do not occur in isolation. Some words are predictable from their contexts; others are less so. Here, we investigate whether predictability relates to when children start producing different words (age of acquisition; AoA). We operationalized predictability in terms of a word's surprisal in child-directed speech, computed using n-gram and long-short-term-memory (LSTM) language models. Predictability derived from LSTMs was generally a better predictor than predictability derived from n-gram models. Across five languages, average surprisal was positively correlated with the AoA of predicates and function words but not nouns. Controlling for concreteness and word frequency, more predictable predicates and function words were learned earlier. Differences in predictability between languages were associated with cross-linguistic differences in AoA: the same word (when it was a predicate) was produced earlier in languages where the word was more predictable.
- Published
- 2023
- Full Text
- View/download PDF
39. Near-Landauer Reversible Skyrmion Logic with Voltage-Based Propagation
- Author
-
Walker, Benjamin W., Edwards, Alexander J., Hu, Xuan, Frank, Michael P., Garcia-Sanchez, Felipe, and Friedman, Joseph S.
- Subjects
Condensed Matter - Mesoscale and Nanoscale Physics ,Computer Science - Emerging Technologies ,Physics - Applied Physics - Abstract
Magnetic skyrmions are topological quasiparticles whose non-volatility, detectability, and mobility make them exciting candidates for low-energy computing. Previous works have demonstrated the feasibility and efficiency of current-driven skyrmions in cascaded logic structures inspired by reversible computing. As skyrmions can be propelled through the voltage-controlled magnetic anisotropy (VCMA) effect with much greater efficiency, this work proposes a VCMA-based skyrmion propagation mechanism that drastically reduces energy dissipation. Additionally, we demonstrate the functionality of skyrmion logic gates enabled by our novel voltage-based propagation and estimate its energy efficiency relative to other logic schemes. The minimum dissipation of this VCMA-driven magnetic skyrmion logic at 0 K is found to be $\sim$6$\times$ the room-temperature Landauer limit, indicating the potential for sub-Landauer dissipation through further engineering., Comment: 4 pages, 6 figures
- Published
- 2023
40. Federated Learning -- Methods, Applications and beyond
- Author
-
Heusinger, Moritz, Raab, Christoph, Rossi, Fabrice, and Schleif, Frank-Michael
- Subjects
Computer Science - Machine Learning ,Statistics - Machine Learning - Abstract
In recent years the applications of machine learning models have increased rapidly, due to the large amount of available data and technological progress.While some domains like web analysis can benefit from this with only minor restrictions, other fields like in medicine with patient data are strongerregulated. In particular \emph{data privacy} plays an important role as recently highlighted by the trustworthy AI initiative of the EU or general privacy regulations in legislation. Another major challenge is, that the required training \emph{data is} often \emph{distributed} in terms of features or samples and unavailable for classicalbatch learning approaches. In 2016 Google came up with a framework, called \emph{Federated Learning} to solve both of these problems. We provide a brief overview on existing Methods and Applications in the field of vertical and horizontal \emph{Federated Learning}, as well as \emph{Fderated Transfer Learning}.
- Published
- 2022
- Full Text
- View/download PDF
41. Development of an in situ Mediator Dosing Concept for Scanning Electrochemical Microscopy in Lithium‐Ion Battery Research
- Author
-
Johannes Eidenschink and Prof. Frank‐Michael Matysik
- Subjects
Electrochemistry ,Interfacial Studies ,Lithium-Ion Batteries ,Scanning Probe Microscopy ,Surface Analysis ,Industrial electrochemistry ,TP250-261 ,Chemistry ,QD1-999 - Abstract
Abstract In scanning electrochemical microscopy (SECM), the addition of a redox active species plays an essential role. Those deliberately added mediators may alter results in SECM studies. In investigations of lithium‐ion battery (LIB) materials, especially of the positive electrode, the oxidation potentials of commonly used mediator substances such as ferrocene are located within the operation potential of the electrode. Thus, they possibly interfere with the regular charge/discharge processes. In situ studies are therefore in need of approaches reducing or eliminating the use of mediators. Within this publication, a novel mediator dosing (MD) concept is introduced. A capillary was closely positioned at the tip of the scanning probe. By gravity flow, stable flow rates of mediator solution of up to 32.4±0.6 μL h−1 were achieved. These low amounts were found to be sufficient to form a ferrocene zone at the probe tip enabling feedback mode SECM measurements with comparable quality to measurements directly in ferrocene solution. Proof of concept experiments were conducted by investigation of a thin‐film electrode with a micro‐structured surface. Furthermore, the MD concept was applied in imaging experiments of a commercially available LIB graphite electrode.
- Published
- 2024
- Full Text
- View/download PDF
42. THE AARP GUIDE TO MODERN PROBLEMS & THEIR SOLUTIONS 2024 EDITION: WE TAPPED TOP PROS TO TAKE ON TODAY'S MOST COMMON HEALTH, MONEY, HOME AND TECH ISSUES
- Author
-
Braverman, Beth, Fernandez, Maisy, Frank, Michael, Morris, Chris, Ogletree, Kelsey, Pandell, Lexi, Schmid, Pamela, Schiff, David, Spence, Evelyn, Westen, Robin, and Wolpin, Stewart
- Subjects
Personal finance -- Management ,Financial management -- Methods ,Health attitudes -- Evaluation ,Company business management ,General interest - Abstract
It's been a year and a half since we last tackled some of life's most vexing struggles in the pages of the AARP Bulletin. And in that time--what do you [...]
- Published
- 2024
43. Indikationen und Missbrauch von Testosteron
- Author
-
Köhn, Frank-Michael and Schuppe, Hans-Christian
- Published
- 2024
- Full Text
- View/download PDF
44. Deep Transfer Hashing for Adaptive Learning on Federated Streaming Data.
- Author
-
Manuel Röder and Frank-Michael Schleif
- Published
- 2024
45. Spatial Downscaling Frameworks for Satellite Based Land Surface Temperature to Support Permafrost Modelling.
- Author
-
Sonia Dupuis, Stefan Wunderle, and Frank-Michael Göttsche
- Published
- 2024
- Full Text
- View/download PDF
46. Crossing Domain Borders with Federated Few-Shot Adaptation.
- Author
-
Manuel Röder, Maximilian Münch, Christoph Raab, and Frank-Michael Schleif
- Published
- 2024
- Full Text
- View/download PDF
47. Experimental Toolchain for Evaluation of Mixture Formation and Combustion in Hydrogen Engines for Light Duty Applications
- Author
-
Lejsek, David, Seboldt, Dimitri, Leick, Philippe, Grzeszik, Roman, Frank, Michael, Stapf, Karl Georg, Kulzer, André Casal, editor, Reuss, Hans-Christian, editor, Wagner, Andreas, editor, and Liedecke, Franziska, With Contrib. by
- Published
- 2024
- Full Text
- View/download PDF
48. Exploring the shortcomings in formal criteria selection for multicriteria decision making based inventory classification models: a systematic review and future directions.
- Author
-
Theunissen, Frank Michael, Bezuidenhout, Carel Nicolaas, and Alam, Shafiq
- Subjects
MULTIPLE criteria decision making ,INVENTORY control ,ARTIFICIAL intelligence ,INVENTORIES ,CLASSIFICATION - Abstract
Criteria selection significantly impacts the reliability and utility of multicriteria decision making (MCDM) models. While criteria may vary across industries, a formalised criteria selection process is influential in determining MCDM model outcomes. This article analyses and compares the criteria selection approaches used in 62 articles that apply MCDM-based inventory classification models, contrasting them with methodologies outside the field. Our findings reveal a conspicuous absence of formal criteria selection methods within MCDM-based inventory classification research. The limited application of quantitative and qualitative approaches indicates that this field has not kept pace with methodological advances in criteria selection. To bridge this gap, we advocate for further research aimed at developing a conceptual framework for criteria selection tailored to inventory classification. We also suggest evaluating the impact of formal criteria selection processes on inventory management decisions and exploring the benefits of integrating artificial intelligence into criteria selection for inventory classification studies. Additionally, this article identifies several limitations related to criteria selection for practitioners employing MCDM-based inventory classification models. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
49. Neural Index of Reinforcement Learning Predicts Improved Stimulus-Response Retention under High Working Memory Load.
- Author
-
Rac-Lubashevsky, Rachel, Cremer, Anna, Collins, Anne GE, Frank, Michael J, and Schwabe, Lars
- Subjects
Biomedical and Clinical Sciences ,Neurosciences ,Behavioral and Social Science ,Basic Behavioral and Social Science ,Mental Health ,Clinical Research ,Male ,Humans ,Female ,Memory ,Short-Term ,Learning ,Reinforcement ,Psychology ,Choice Behavior ,Cognition ,EEG ,reinforcement learning ,retention ,stress ,working memory ,Medical and Health Sciences ,Psychology and Cognitive Sciences ,Neurology & Neurosurgery - Abstract
Human learning and decision-making are supported by multiple systems operating in parallel. Recent studies isolating the contributions of reinforcement learning (RL) and working memory (WM) have revealed a trade-off between the two. An interactive WM/RL computational model predicts that although high WM load slows behavioral acquisition, it also induces larger prediction errors in the RL system that enhance robustness and retention of learned behaviors. Here, we tested this account by parametrically manipulating WM load during RL in conjunction with EEG in both male and female participants and administered two surprise memory tests. We further leveraged single-trial decoding of EEG signatures of RL and WM to determine whether their interaction predicted robust retention. Consistent with the model, behavioral learning was slower for associations acquired under higher load but showed parametrically improved future retention. This paradoxical result was mirrored by EEG indices of RL, which were strengthened under higher WM loads and predictive of more robust future behavioral retention of learned stimulus-response contingencies. We further tested whether stress alters the ability to shift between the two systems strategically to maximize immediate learning versus retention of information and found that induced stress had only a limited effect on this trade-off. The present results offer a deeper understanding of the cooperative interaction between WM and RL and show that relying on WM can benefit the rapid acquisition of choice behavior during learning but impairs retention.SIGNIFICANCE STATEMENT Successful learning is achieved by the joint contribution of the dopaminergic RL system and WM. The cooperative WM/RL model was productive in improving our understanding of the interplay between the two systems during learning, demonstrating that reliance on RL computations is modulated by WM load. However, the role of WM/RL systems in the retention of learned stimulus-response associations remained unestablished. Our results show that increased neural signatures of learning, indicative of greater RL computation, under high WM load also predicted better stimulus-response retention. This result supports a trade-off between the two systems, where degraded WM increases RL processing, which improves retention. Notably, we show that this cooperative interplay remains largely unaffected by acute stress.
- Published
- 2023
50. Reward-Predictive Clustering
- Author
-
Lehnert, Lucas, Frank, Michael J., and Littman, Michael L.
- Subjects
Computer Science - Machine Learning ,Computer Science - Artificial Intelligence - Abstract
Recent advances in reinforcement-learning research have demonstrated impressive results in building algorithms that can out-perform humans in complex tasks. Nevertheless, creating reinforcement-learning systems that can build abstractions of their experience to accelerate learning in new contexts still remains an active area of research. Previous work showed that reward-predictive state abstractions fulfill this goal, but have only be applied to tabular settings. Here, we provide a clustering algorithm that enables the application of such state abstractions to deep learning settings, providing compressed representations of an agent's inputs that preserve the ability to predict sequences of reward. A convergence theorem and simulations show that the resulting reward-predictive deep network maximally compresses the agent's inputs, significantly speeding up learning in high dimensional visual control tasks. Furthermore, we present different generalization experiments and analyze under which conditions a pre-trained reward-predictive representation network can be re-used without re-training to accelerate learning -- a form of systematic out-of-distribution transfer.
- Published
- 2022
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.