Search

Continual self-supervised learning (CSSL) learns a series of tasks sequentially on the unlabeled data. Two main challenges of continual learning are catastrophic forgetting and task confusion. While CSSL problem has been studied to address the catastrophic forgetting challenge, little work has been done to address the task confusion aspect. In this work, we show through extensive experiments that self-supervised learning (SSL) can make CSSL more susceptible to the task confusion problem, particularly in less diverse settings of class incremental learning because different classes belonging to different tasks are not trained concurrently. Motivated by this challenge, we present a novel cross-model feature Mixup (CroMo-Mixup) framework that addresses this issue through two key components: 1) Cross-Task data Mixup, which mixes samples across tasks to enhance negative sample diversity; and 2) Cross-Model feature Mixup, which learns similarities between embeddings obtained from current and old models of the mixed sample and the original images, facilitating cross-task class contrast learning and old knowledge retrieval. We evaluate the effectiveness of CroMo-Mixup to improve both Task-ID prediction and average linear accuracy across all tasks on three datasets, CIFAR10, CIFAR100, and tinyImageNet under different class-incremental learning settings. We validate the compatibility of CroMo-Mixup on four state-of-the-art SSL objectives. Code is available at \url{https://github.com/ErumMushtaq/CroMo-Mixup}., Comment: ECCV 2024

Uncertainty estimation (UE) of generative large language models (LLMs) is crucial for evaluating the reliability of generated sequences. A significant subset of UE methods utilize token probabilities to assess uncertainty, aggregating multiple token probabilities into a single UE score using a scoring function. Existing scoring functions for probability-based UE, such as length-normalized scoring and semantic contribution-based weighting, are designed to solve certain aspects of the problem but exhibit limitations, including the inability to handle biased probabilities and complex semantic dependencies between tokens. To address these issues, in this work, we propose Learnable Response Scoring (LARS) function, a novel scoring function that leverages supervised data to capture complex dependencies between tokens and probabilities, thereby producing more reliable and calibrated response scores in computing the uncertainty of LLM generations. Our comprehensive experiments across question-answering and arithmetical reasoning tasks with various datasets demonstrate that LARS significantly outperforms existing scoring functions, achieving improvements of up to 16\% AUROC score.

Large language models (LLMs) have shown impressive capabilities in tasks such as machine translation, text summarization, question answering, and solving complex mathematical problems. However, their primary training on data-rich languages like English limits their performance in low-resource languages. This study addresses this gap by focusing on the Indexical Shift problem in Turkish. The Indexical Shift problem involves resolving pronouns in indexical shift contexts, a grammatical challenge not present in high-resource languages like English. We present the first study examining indexical shift in any language, releasing a Turkish dataset specifically designed for this purpose. Our Indexical Shift Dataset consists of 156 multiple-choice questions, each annotated with necessary linguistic details, to evaluate LLMs in a few-shot setting. We evaluate recent multilingual LLMs, including GPT-4, GPT-3.5, Cohere-AYA, Trendyol-LLM, and Turkcell-LLM, using this dataset. Our analysis reveals that even advanced models like GPT-4 struggle with the grammatical nuances of indexical shift in Turkish, achieving only moderate performance. These findings underscore the need for focused research on the grammatical challenges posed by low-resource languages. We released the dataset and code \href{https://anonymous.4open.science/r/indexical_shift_llm-E1B4} {here}.

Generative Large Language Models (LLMs) are widely utilized for their excellence in various tasks. However, their tendency to produce inaccurate or misleading outputs poses a potential risk, particularly in high-stakes environments. Therefore, estimating the correctness of generative LLM outputs is an important task for enhanced reliability. Uncertainty Estimation (UE) in generative LLMs is an evolving domain, where SOTA probability-based methods commonly employ length-normalized scoring. In this work, we propose Meaning-Aware Response Scoring (MARS) as an alternative to length-normalized scoring for UE methods. MARS is a novel scoring function that considers the semantic contribution of each token in the generated sequence in the context of the question. We demonstrate that integrating MARS into UE methods results in a universal and significant improvement in UE performance. We conduct experiments using three distinct closed-book question-answering datasets across five popular pre-trained LLMs. Lastly, we validate the efficacy of MARS on a Medical QA dataset. Code can be found https://github.com/Ybakman/LLM_Uncertainity.

Federated Learning (FL) has gained significant attraction due to its ability to enable privacy-preserving training over decentralized data. Current literature in FL mostly focuses on single-task learning. However, over time, new tasks may appear in the clients and the global model should learn these tasks without forgetting previous tasks. This real-world scenario is known as Continual Federated Learning (CFL). The main challenge of CFL is Global Catastrophic Forgetting, which corresponds to the fact that when the global model is trained on new tasks, its performance on old tasks decreases. There have been a few recent works on CFL to propose methods that aim to address the global catastrophic forgetting problem. However, these works either have unrealistic assumptions on the availability of past data samples or violate the privacy principles of FL. We propose a novel method, Federated Orthogonal Training (FOT), to overcome these drawbacks and address the global catastrophic forgetting in CFL. Our algorithm extracts the global input subspace of each layer for old tasks and modifies the aggregated updates of new tasks such that they are orthogonal to the global principal subspace of old tasks for each layer. This decreases the interference between tasks, which is the main cause for forgetting. We empirically show that FOT outperforms state-of-the-art continual learning methods in the CFL setting, achieving an average accuracy gain of up to 15% with 27% lower forgetting while only incurring a minimal computation and communication cost.

Federated Learning (FL) aims to train a machine learning (ML) model in a distributed fashion to strengthen data privacy with limited data migration costs. It is a distributed learning framework naturally suitable for privacy-sensitive medical imaging datasets. However, most current FL-based medical imaging works assume silos have ground truth labels for training. In practice, label acquisition in the medical field is challenging as it often requires extensive labor and time costs. To address this challenge and leverage the unannotated data silos to improve modeling, we propose an alternate training-based framework, Federated Alternate Training (FAT), that alters training between annotated data silos and unannotated data silos. Annotated data silos exploit annotations to learn a reasonable global segmentation model. Meanwhile, unannotated data silos use the global segmentation model as a target model to generate pseudo labels for self-supervised learning. We evaluate the performance of the proposed framework on two naturally partitioned Federated datasets, KiTS19 and FeTS2021, and show its promising performance., Comment: Camera Ready Version of ISBI2023 Accepted work

Edge localization is a fascinating quantum phenomenon. In this paper, the underlying mechanism generating it is presented analytically and verified numerically for a weakly kicked three-dimensional rotor. Analogy to tight binding model in solid state physics is used. The edge states result of the edge at zero angular momentum of the three-dimensional kicked rotor., Comment: 16 pages, 3 figures

When light is incident upon a diffraction grating, images of the grating appear at periodic intervals behind the grating. This phenomenon and the associated self-imaging distance were named after Talbot who first observed them in the nineteenth century. A century later, this effect held new surprises with the discovery of sub-images at regular fractional distances of the Talbot length. In this paper, we show that water waves enable one to observe the Talbot effect in a classroom experiment. Quantitative measurements, of for example the Talbot distances, can be performed with an easy to use digital Schlieren method.

The effect of quantum collapse and revival is a fascinating interference phenomenon. In this paper the phenomenon is demonstrated analytically and numerically for a simple system, a slightly anharmonic Hamiltonian. The initial wave-function is a displaced ground state of a harmonic oscillator. Possible experimental realizations for cold atoms are discussed in detail., Comment: 18 pages, 2 figures

The quantum fidelity was introduced by Peres to study some fingerprints of classically chaotic behavior in the quantum dynamics of the corresponding systems. In the present paper the signatures of classical dynamics near elliptic points and of interactions between particles are characterized for kicked systems. In particular, the period of the fidelity resulting of the interactions is found using analytical and numerical calculations. A mechanism leading to the oscillations with the intermediate period is proposed. It is of a semiclassical origin and results of the interplay between the oscillations of the width of the wave packets and the rotation of their center around the elliptic fixed point.

Measurement professionals cannot come to an agreement on the definition of the term 'item fairness'. In this paper a continuous measure of item unfairness is proposed. The more the unfairness measure deviates from zero, the less fair the item is. If the measure exceeds the cutoff value, the item is identified as definitely unfair. The new approach can identify unfair items that would not be identified with conventional procedures. The results are in accord with experts' judgments on the item qualities. Since no assumptions about scores distributions and/or correlations are assumed, the method is applicable to any educational test. Its performance is illustrated through application to scores of a real test., Comment: 14 pages, 5 figures

The basic statistical methods of data representation did not change since their emergence. Their simplicity was dictated by the intricacies of computations in the before computers epoch. It turns out that such approach is not uniquely possible in the presence of quick computers. The suggested here method improves significantly the reliability of data processing and their graphical representation. In this paper we show problems of the standard data processing which can bring to incorrect results. A method solving these problems is proposed. It is based on modification of data representation. The method was implemented in a computer program Consensus5. The program performances are illustrated through varied examples.

Understanding of free-format multi-step arithmetic word problems with extraneous information is discussed. A model including a full set of general skills necessary for understanding such problems was developed and computer implemented. The validity of the simulation was confirmed by testing on a variety of word problems with extraneous information. The results of this study show the way for moving from an intuitive understanding of word problems to a conscious and controlled process., Comment: Comments: 15 pages

Even though the concepts of time and space are basic concepts of physics, they have not been vouchsafed a constructive definition. As to space, this is no wonder because a correct notion of space cannot be formed in the frame of the existing physical paradigm. However, a definition of time can be given on the basis of the simple principle that each physical magnitude constitutes a feature of some physical carrier (Costa, 1987). This article deals with identifying that carrier for the feature "time". Finally this article affirms that because of the lack of a definition of time some physical experiments have been erroneously interpreted as signs of time dilation., Comment: 5 pages, v2: added one reference