Search

Toddlers evolve from free exploration with sparse feedback to exploiting prior experiences for goal-directed learning with denser rewards. Drawing inspiration from this Toddler-Inspired Reward Transition, we set out to explore the implications of varying reward transitions when incorporated into Reinforcement Learning (RL) tasks. Central to our inquiry is the transition from sparse to potential-based dense rewards, which share optimal strategies regardless of reward changes. Through various experiments, including those in egocentric navigation and robotic arm manipulation tasks, we found that proper reward transitions significantly influence sample efficiency and success rates. Of particular note is the efficacy of the toddler-inspired Sparse-to-Dense (S2D) transition. Beyond these performance metrics, using Cross-Density Visualizer technique, we observed that transitions, especially the S2D, smooth the policy loss landscape, promoting wide minima that enhance generalization in RL models., Comment: Accepted as a full paper at AAAI 2024 (Oral presentation): 7 pages (main paper), 2 pages (references), 17 pages (appendix) each

Recent machine learning algorithms have been developed using well-curated datasets, which often require substantial cost and resources. On the other hand, the direct use of raw data often leads to overfitting towards frequently occurring class information. To address class imbalances cost-efficiently, we propose an active data filtering process during self-supervised pre-training in our novel framework, Duplicate Elimination (DUEL). This framework integrates an active memory inspired by human working memory and introduces distinctiveness information, which measures the diversity of the data in the memory, to optimize both the feature extractor and the memory. The DUEL policy, which replaces the most duplicated data with new samples, aims to enhance the distinctiveness information in the memory and thereby mitigate class imbalances. We validate the effectiveness of the DUEL framework in class-imbalanced environments, demonstrating its robustness and providing reliable results in downstream tasks. We also analyze the role of the DUEL policy in the training process through various metrics and visualizations., Comment: Accepted as a full paper at AAAI 2024: The 38th Annual AAAI Conference on Artificial Intelligence (Main Tech Track). 7 pages (main paper), 2 pages (references), 11 pages (appendix) each

Understanding geometric concepts, such as distance and shape, is essential for understanding the real world and also for many vision tasks. To incorporate such information into a visual representation of a scene, we propose learning to represent the scene by sketching, inspired by human behavior. Our method, coined Learning by Sketching (LBS), learns to convert an image into a set of colored strokes that explicitly incorporate the geometric information of the scene in a single inference step without requiring a sketch dataset. A sketch is then generated from the strokes where CLIP-based perceptual loss maintains a semantic similarity between the sketch and the image. We show theoretically that sketching is equivariant with respect to arbitrary affine transformations and thus provably preserves geometric information. Experimental results show that LBS substantially improves the performance of object attribute classification on the unlabeled CLEVR dataset, domain transfer between CLEVR and STL-10 datasets, and for diverse downstream tasks, confirming that LBS provides rich geometric information., Comment: CVPR 2023

In Self-Supervised Learning (SSL), it is known that frequent occurrences of the collision in which target data and its negative samples share the same class can decrease performance. Especially in real-world data such as crawled data or robot-gathered observations, collisions may occur more often due to the duplicates in the data. To deal with this problem, we claim that sampling negative samples from the adaptively debiased distribution in the memory makes the model more stable than sampling from a biased dataset directly. In this paper, we introduce a novel SSL framework with adaptive Duplicate Elimination (DUEL) inspired by the human working memory. The proposed framework successfully prevents the downstream task performance from degradation due to a dramatic inter-class imbalance., Comment: 9 pages, 5 figures, submitted to NeurIPS 2022 Workshop on Self-supervised Learning: Theory and Practice.(Accepted, https://sslneurips22.github.io/)

Molecular dynamics simulations are performed to investigate the role of a coherent {\Sigma}3 (111) twin boundary on the plastic deformation behavior of Cu nanopillars. Our work reveals that the mechanical response of pillars with and without the twin boundary is decisively driven by the characteristics of initial dislocation sources. In the condition of comparably large pillar size and abundant initial mobile dislocations, overall yield and flow stresses are controlled by the longest, available mobile dislocation. An inverse correlation of the yield and flow stresses with the length of the longest dislocation is established, and its extrapolation agrees well with experimental yield stress data. The experimentally reported subtle differences in yield and flow stresses between pillars with and without the twin boundary are thus likely related to the maximum lengths of the mobile dislocations. In the condition of comparably small pillar size, for which a reduction of mobile dislocations during heat treatment and mechanical loading occurs, the mechanical response of pillars with and without the twin boundary can be clearly distinguished. Dislocation starvation during deformation is more clearly present in pillars without the twin boundary than in pillars with the twin boundary because the twin boundary acts as a pinning surface for the dislocation network., Comment: - Main document: 29 pages, 12 figures, 3 tables - Supplementary Material: 13 pages, 11 figures, 2tables

Molecular dynamics simulations are performed to provide a detailed understanding of the functional degradation of shape memory alloys at small scale. The origin of the experimentally reported accumulation of plastic deformation and the anomalous sudden increase of the residual strain under cyclic mechanical loading are explained by detailed insights into the relevant atomic scale processes. Our work reveals that the mechanical response of shape-memory-alloy pillars under cyclic compression is significantly influenced by the presence of an amorphous-like surface region as experimentally induced by focused ion beam milling. The main factor responsible for the observed degradation of superelasticity under cyclic loading is the accumulated plastic deformation and the resultant retained martensite originating from a synergetic contribution of the amorphous and crystalline shape-memory-alloy regions. We show that the reported sudden diminishment of the stress plateaus and hysteresis under cyclic loading is caused by the increased stability of the martensite phase due to the presence of the amorphous phase. Based on the identified mechanism responsible for the degradation, we validate reported methods of recovering the superelasticity and propose a new method to prohibit the synergetic contribution of the amorphous and crystalline regions, such as to achieve a sustainable operation of shape memory alloys at small scale., Comment: 32 pages, 15 figures, 1 table

Active learning is widely used to reduce labeling effort and training time by repeatedly querying only the most beneficial samples from unlabeled data. In real-world problems where data cannot be stored indefinitely due to limited storage or privacy issues, the query selection and the model update should be performed as soon as a new data sample is observed. Various online active learning methods have been studied to deal with these challenges; however, there are difficulties in selecting representative query samples and updating the model efficiently without forgetting. In this study, we propose Message Passing Adaptive Resonance Theory (MPART) that learns the distribution and topology of input data online. Through message passing on the topological graph, MPART actively queries informative and representative samples, and continuously improves the classification performance using both labeled and unlabeled data. We evaluate our model in stream-based selective sampling scenarios with comparable query selection strategies, showing that MPART significantly outperforms competitive models., Comment: Accepted to ICML 2021

Semi-supervised learning and continuous learning are fundamental paradigms for human-level intelligence. To deal with real-world problems where labels are rarely given and the opportunity to access the same data is limited, it is necessary to apply these two paradigms in a joined fashion. In this paper, we propose Label Propagation Adaptive Resonance Theory (LPART) for semi-supervised continuous learning. LPART uses an online label propagation mechanism to perform classification and gradually improves its accuracy as the observed data accumulates. We evaluated the proposed model on visual (MNIST, SVHN, CIFAR-10) and audio (NSynth) datasets by adjusting the ratio of the labeled and unlabeled data. The accuracies are much higher when both labeled and unlabeled data are used, demonstrating the significant advantage of LPART in environments where the data labels are scarce., Comment: 5 pages, 2 figures, 1 table, accepted in ICASSP 2020