Search

Recently, ultrasensitive calorimeters have been proposed as a resource-efficient solution for multiplexed qubit readout in superconducting large-scale quantum processors. However, experiments demonstrating frequency multiplexing of these superconductor-normal conductor-superconductor (SNS) sensors are coarse. To this end, we present the design, fabrication, and operation of three SNS sensors with frequency-multiplexed input and probe circuits, all on a single chip. These devices have their probe frequencies in the range \SI{150}{\mega\hertz} -- \SI{200}{\mega\hertz}, which is well detuned from the heater frequencies of \SI{4.4}{\giga\hertz} -- \SI{7.6}{\giga\hertz} compatible with typical readout frequencies of superconducting qubits. Importantly, we show on-demand triggering of both individual and multiple low-noise SNS bolometers with very low cross talk. These experiments pave the way for multiplexed bolometric characterization and calorimetric readout of multiple qubits, a promising step in minimizing related resources such as the number of readout lines and microwave isolators in large-scale superconducting quantum computers., Comment: 7 pages, 4 figures

This study evaluates metrics for tasks such as classification, regression, clustering, correlation analysis, statistical tests, segmentation, and image-to-image (I2I) translation. Metrics were compared across Python libraries, R packages, and Matlab functions to assess their consistency and highlight discrepancies. The findings underscore the need for a unified roadmap to standardize metrics, ensuring reliable and reproducible ML evaluations across platforms. This study examined a wide range of evaluation metrics across various tasks and found only some to be consistent across platforms, such as (i) Accuracy, Balanced Accuracy, Cohens Kappa, F-beta Score, MCC, Geometric Mean, AUC, and Log Loss in binary classification; (ii) Accuracy, Cohens Kappa, and F-beta Score in multi-class classification; (iii) MAE, MSE, RMSE, MAPE, Explained Variance, Median AE, MSLE, and Huber in regression; (iv) Davies-Bouldin Index and Calinski-Harabasz Index in clustering; (v) Pearson, Spearman, Kendall's Tau, Mutual Information, Distance Correlation, Percbend, Shepherd, and Partial Correlation in correlation analysis; (vi) Paired t-test, Chi-Square Test, ANOVA, Kruskal-Wallis Test, Shapiro-Wilk Test, Welchs t-test, and Bartlett's test in statistical tests; (vii) Accuracy, Precision, and Recall in 2D segmentation; (viii) Accuracy in 3D segmentation; (ix) MAE, MSE, RMSE, and R-Squared in 2D-I2I translation; and (x) MAE, MSE, and RMSE in 3D-I2I translation. Given observation of discrepancies in a number of metrics (e.g. precision, recall and F1 score in binary classification, WCSS in clustering, multiple statistical tests, and IoU in segmentation, amongst multiple metrics), this study concludes that ML evaluation metrics require standardization and recommends that future research use consistent metrics for different tasks to effectively compare ML techniques and solutions., Comment: This paper is 12 pages with 1 table and 10 figures

Academics increasingly acknowledge the predictive power of social media for a wide variety of events and, more specifically, for financial markets. Anecdotal and empirical findings show that cryptocurrencies are among the financial assets that have been affected by news and influencers' activities on Twitter. However, the extent to which Twitter crypto influencer's posts about trading signals and their effect on market prices is mostly unexplored. In this paper, we use LLMs to uncover buy and not-buy signals from influencers and news outlets' Twitter posts and use a VAR analysis with Granger Causality tests and cross-correlation analysis to understand how these trading signals are temporally correlated with the top nine major cryptocurrencies' prices. Overall, the results show a mixed pattern across cryptocurrencies and temporal periods. However, we found that for the top three cryptocurrencies with the highest presence within news and influencer posts, their aggregated LLM-detected trading signal over the preceding 24 hours granger-causes fluctuations in their market prices, exhibiting a lag of at least 6 hours. In addition, the results reveal fundamental differences in how influencers and news outlets cover cryptocurrencies.

In Large Language Model (LLM) development, Reinforcement Learning from Human Feedback (RLHF) is crucial for aligning models with human values and preferences. RLHF traditionally relies on the Kullback-Leibler (KL) divergence between the current policy and a frozen initial policy as a reference, which is added as a penalty in policy optimization algorithms like Proximal Policy Optimization (PPO). While this constraint prevents models from deviating too far from the initial checkpoint, it limits exploration of the reward landscape, reducing the model's ability to discover higher-quality solutions. As a result, policy optimization is often trapped in a narrow region of the parameter space, leading to suboptimal alignment and performance. This paper presents SALSA (Soup-based Alignment Learning for Stronger Adaptation), a novel approach designed to overcome these limitations by creating a more flexible and better located reference model through weight-space averaging of two independent supervised fine-tuned (SFT) models. This model soup allows for larger deviation in KL divergence and exploring a promising region of the solution space without sacrificing stability. By leveraging this more robust reference model, SALSA fosters better exploration, achieving higher rewards and improving model robustness, out-of-distribution generalization, and performance. We validate the effectiveness of SALSA through extensive experiments on popular open models (Llama2-7B, Mistral-7B, and Gemma-2B) across various benchmarks (MT-Bench, Arena-Hard, UltraFeedback), where it consistently surpasses PPO by fostering deeper exploration and achieving superior alignment in LLMs.

Deep reinforcement learning (DRL) is emerging as a promising method for adaptive robotic motion and complex task automation, effectively addressing the limitations of traditional control methods. However, ensuring safety throughout both the learning process and policy deployment remains a key challenge due to the risky exploration inherent in DRL, as well as the discrete nature of actions taken at intervals. These discontinuities, despite being part of a continuous action space, can lead to abrupt changes between successive actions, causing instability and unsafe intermediate states. To address these challenges, this paper proposes an integrated framework that combines DRL with a jerk-bounded trajectory generator (JBTG) and a robust low-level control strategy, significantly enhancing the safety, stability, and reliability of robotic manipulators. The low-level controller ensures the precise execution of DRL-generated commands, while the JBTG refines these motions to produce smooth, continuous trajectories that prevent abrupt or unsafe actions. The framework also includes pre-calculated safe velocity zones for smooth braking, preventing joint limit violations and ensuring compliance with kinematic constraints. This approach not only guarantees the robustness and safety of the robotic system but also optimizes motion control, making it suitable for practical applications. The effectiveness of the proposed framework is demonstrated through its application to a highly complex heavy-duty manipulator., Comment: This paper has been submitted to the IEEE for potential publication

While large language models (LLMs) dominate the AI landscape, Small-scale large Language Models (SLMs) are gaining attention due to cost and efficiency demands from consumers. However, there is limited research on the training behavior and computational requirements of SLMs. In this study, we explore the computational bottlenecks of training SLMs (up to 2B parameters) by examining the effects of various hyperparameters and configurations, including GPU type, batch size, model size, communication protocol, attention type, and the number of GPUs. We assess these factors on popular cloud services using metrics such as loss per dollar and tokens per second. Our findings aim to support the broader adoption and optimization of language model training for low-resource AI research institutes., Comment: 8 pages, 4 figures

This paper proposes an AI-based scheme for islanding detection in active distribution networks. By reviewing existing studies, it is clear that there are several gaps in the field to ensure reliable islanding detection, including (i) model complexity and stability concerns, (ii) limited accuracy under noisy conditions, and (iii) limited applicability to systems with different types of resources. Accordingly, this paper proposes a WaveNet classifier reinforced by a denoising U-Net model to address these shortcomings. The proposed scheme has a simple structure due to the use of 1D convolutional layers and incorporates residual connections that significantly enhance the model's generalization. Additionally, the proposed scheme is robust against noisy conditions by incorporating a denoising U-Net model. Furthermore, the model is sufficiently fast using a sliding window time series of 10 milliseconds for detection. Utilizing positive/negative/zero sequence components of voltages, superimposed waveforms, and the rate of change of frequency provides the necessary features to precisely detect the islanding condition. In order to assess the effectiveness of the suggested scheme, over 3k islanding/non-islanding cases were tested, considering different load active/reactive powers values, load switching transients, capacitor bank switching, fault conditions in the main grid, different load quality factors, signal-to-noise ratio levels, and both types of conventional and inverter-based sources.

Parameter-efficient fine-tuning (PEFT) methods typically assume that Large Language Models (LLMs) are trained on data from a single device or client. However, real-world scenarios often require fine-tuning these models on private data distributed across multiple devices. Federated Learning (FL) offers an appealing solution by preserving user privacy, as sensitive data remains on local devices during training. Nonetheless, integrating PEFT methods into FL introduces two main challenges: communication overhead and data heterogeneity. In this paper, we introduce FedTT and FedTT+, methods for adapting LLMs by integrating tensorized adapters into client-side models' encoder/decoder blocks. FedTT is versatile and can be applied to both cross-silo FL and large-scale cross-device FL. FedTT+, an extension of FedTT tailored for cross-silo FL, enhances robustness against data heterogeneity by adaptively freezing portions of tensor factors, further reducing the number of trainable parameters. Experiments on BERT and LLaMA models demonstrate that our proposed methods successfully address data heterogeneity challenges and perform on par or even better than existing federated PEFT approaches while achieving up to 10$\times$ reduction in communication cost.

Recent advancements in Large Language Models (LLMs) have sparked interest in their formal reasoning capabilities, particularly in mathematics. The GSM8K benchmark is widely used to assess the mathematical reasoning of models on grade-school-level questions. While the performance of LLMs on GSM8K has significantly improved in recent years, it remains unclear whether their mathematical reasoning capabilities have genuinely advanced, raising questions about the reliability of the reported metrics. To address these concerns, we conduct a large-scale study on several SOTA open and closed models. To overcome the limitations of existing evaluations, we introduce GSM-Symbolic, an improved benchmark created from symbolic templates that allow for the generation of a diverse set of questions. GSM-Symbolic enables more controllable evaluations, providing key insights and more reliable metrics for measuring the reasoning capabilities of models.Our findings reveal that LLMs exhibit noticeable variance when responding to different instantiations of the same question. Specifically, the performance of all models declines when only the numerical values in the question are altered in the GSM-Symbolic benchmark. Furthermore, we investigate the fragility of mathematical reasoning in these models and show that their performance significantly deteriorates as the number of clauses in a question increases. We hypothesize that this decline is because current LLMs cannot perform genuine logical reasoning; they replicate reasoning steps from their training data. Adding a single clause that seems relevant to the question causes significant performance drops (up to 65%) across all state-of-the-art models, even though the clause doesn't contribute to the reasoning chain needed for the final answer. Overall, our work offers a more nuanced understanding of LLMs' capabilities and limitations in mathematical reasoning., Comment: preprint

Existing 3D facial emotion modeling have been constrained by limited emotion classes and insufficient datasets. This paper introduces "Emo3D", an extensive "Text-Image-Expression dataset" spanning a wide spectrum of human emotions, each paired with images and 3D blendshapes. Leveraging Large Language Models (LLMs), we generate a diverse array of textual descriptions, facilitating the capture of a broad spectrum of emotional expressions. Using this unique dataset, we conduct a comprehensive evaluation of language-based models' fine-tuning and vision-language models like Contranstive Language Image Pretraining (CLIP) for 3D facial expression synthesis. We also introduce a new evaluation metric for this task to more directly measure the conveyed emotion. Our new evaluation metric, Emo3D, demonstrates its superiority over Mean Squared Error (MSE) metrics in assessing visual-text alignment and semantic richness in 3D facial expressions associated with human emotions. "Emo3D" has great applications in animation design, virtual reality, and emotional human-computer interaction., Comment: 11 pages, 10 figures

Large Language Models (LLMs) typically generate outputs token by token using a fixed compute budget, leading to inefficient resource utilization. To address this shortcoming, recent advancements in mixture of expert (MoE) models, speculative decoding, and early exit strategies leverage the insight that computational demands can vary significantly based on the complexity and nature of the input. However, identifying optimal routing patterns for dynamic execution remains an open challenge, limiting the full potential of these adaptive methods. To address this need, we study adaptive computation in LLMs more systematically. We propose a novel framework that integrates smaller auxiliary modules within each Feed-Forward Network layer of the LLM. This design enables dynamic routing of tokens based on task complexity: tokens can be processed by either the small or big modules at each layer, or even bypass certain layers entirely. This allows us to introduce a novel notion of a token's difficulty, defined by its potential to benefit from additional computational resources. Importantly, by employing oracles to identify optimal patterns of adaptive computations, we gain valuable insights into the internal workings of LLMs and the routing processes in a simplified heterogeneous MoE setup. We show that trained routers operate differently from oracles and often yield suboptimal solutions. Notably, activating a large module in just one layer outperforms models that use large modules across all layers, underscoring the gap between practical implementations of routing in MoE models and theoretical optima for adaptive computation.

We study the band structure and scattering of in-plane coupled longitudinal and shear stress waves in layered media and observe that exceptional points (EP) appear for elastic (lossless) media, when parameterized with real-valued frequency and tangential wave vector component. The occurrence of these EP pairs is not limited to the original stop bands. They could also appear in all mode pass bands, leading to the formation of new stop bands. The scattered energy near these locations is studied along with the associated polarization patterns. The broken phase symmetry is observed inside the frequency bands book-ended by these EP pairs. This is especially manifested by the chirality of the trajectory of the particle velocity, which gets selected by a ``direction'' of the wave, e.g. the imaginary part of normal component of the wave vector, or the energy flux direction just outside the band. Additionally, EP pairs also appear in the spectrum of the (modified) scattering matrix when mechanical gain is theoretically included to balance the loss in a parity-time symmetric finite structure. These EP pairs lead to amplification of transmission to above 1 and single-sided reflectivity, both phenomena associated with broken phase symmetry, with intriguing potential applications.

Trajectory prediction models in autonomous driving are vulnerable to perturbations from non-causal agents whose actions should not affect the ego-agent's behavior. Such perturbations can lead to incorrect predictions of other agents' trajectories, potentially compromising the safety and efficiency of the ego-vehicle's decision-making process. Motivated by this challenge, we propose $\textit{Causal tRajecTory predICtion}$ $\textbf{(CRiTIC)}$, a novel model that utilizes a $\textit{Causal Discovery Network}$ to identify inter-agent causal relations over a window of past time steps. To incorporate discovered causal relationships, we propose a novel $\textit{Causal Attention Gating}$ mechanism to selectively filter information in the proposed Transformer-based architecture. We conduct extensive experiments on two autonomous driving benchmark datasets to evaluate the robustness of our model against non-causal perturbations and its generalization capacity. Our results indicate that the robustness of predictions can be improved by up to $\textbf{54%}$ without a significant detriment to prediction accuracy. Lastly, we demonstrate the superior domain generalizability of the proposed model, which achieves up to $\textbf{29%}$ improvement in cross-domain performance. These results underscore the potential of our model to enhance both robustness and generalization capacity for trajectory prediction in diverse autonomous driving domains. Further details can be found on our project page: https://critic-model.github.io/., Comment: 6 pages with 3 figures

The pre-training phase of language models often begins with randomly initialized parameters. With the current trends in scaling models, training their large number of parameters can be extremely slow and costly. In contrast, small language models are less expensive to train, but they often cannot achieve the accuracy of large models. In this paper, we explore an intriguing idea to connect these two different regimes: Can we develop a method to initialize large language models using smaller pre-trained models? Will such initialization bring any benefits in terms of training time and final accuracy? In this paper, we introduce HyperCloning, a method that can expand the parameters of a pre-trained language model to those of a larger model with increased hidden dimensions. Our method ensures that the larger model retains the functionality of the smaller model. As a result, the larger model already inherits the predictive power and accuracy of the smaller model before the training starts. We demonstrate that training such an initialized model results in significant savings in terms of GPU hours required for pre-training large language models.

The flipped classroom model has been widely acknowledged as a practical pedagogical approach to enhancing student engagement and learning. However, it faces challenges such as improving student interaction with learning content and peers, particularly in Japanese universities where digital technologies are not always fully utilized. To address these challenges and identify potential solutions, a case study was conducted in which an online flipped course on academic skills was developed and implemented in an immersive virtual environment. The primary objective during this initial phase was not to establish a causal relationship between the use of immersive flipped learning and students' engagement and perceived learning outcomes. Instead, this initiative aimed to explore the benefits and challenges of the immersive flipped learning approach in relation to students' online engagement and their perceived learning outcomes. Following a mixed-methods research approach, quantitative and qualitative data were collected through a survey (N=50) and students' reflective reports (N=80). The study revealed high levels of student engagement and perceived learning outcomes, although it also identified areas needing improvement, particularly in supporting student interactions in the target language. Despite the exploratory nature of this study, the findings suggest that a well-designed flipped learning approach, set in an engaging immersive environment, can significantly enhance student engagement, thereby supporting the learning process. When creating an immersive flipped learning course, educators should incorporate best practices from the literature on both flipped learning and immersive learning design to ensure optimal learning outcomes. The findings of this study can serve as a valuable resource for educators seeking to design engaging and effective remote learning experiences., Comment: 14 pages

A $\nabla$-algebra is a natural generalization of a Heyting algebra, unifying several algebraic structures, including bounded lattices, Heyting algebras, temporal Heyting algebras, and the algebraic representation of dynamic topological systems. In the prequel to this paper [3], we explored the algebraic properties of various varieties of $\nabla$-algebras, their subdirectly-irreducible and simple elements, their closure under Dedekind-MacNeille completion, and their Kripke-style representation. In this sequel, we first introduce $\nabla$-spaces as a common generalization of Priestley and Esakia spaces, through which we develop a duality theory for certain categories of $\nabla$-algebras. Then, we reframe these dualities in terms of spectral spaces and provide an algebraic characterization of natural families of dynamic topological systems over Priestley, Esakia, and spectral spaces. Additionally, we present a ring-theoretic representation for some families of $\nabla$-algebras. Finally, we introduce several logical systems to capture different varieties of $\nabla$-algebras, offering their algebraic, Kripke, topological, and ring-theoretic semantics, and establish a deductive interpolation theorem for some of these systems., Comment: 55 pages

The spectral diameter of a symplectic ball is shown to be equal to its capacity; this result upgrades the known bound by a factor of two and yields a simple formula for the spectral diameter of a symplectic ellipsoid. We also study the relationship between the spectral diameter and packings by two balls., Comment: 21 pages

This paper develops a unified framework to maximize the network sum-rate in a multi-user, multi-BS downlink terahertz (THz) network by optimizing user associations, number and bandwidth of sub-bands in a THz transmission window (TW), bandwidth of leading and trailing edge-bands in a TW, sub-band assignment, and power allocations. The proposed framework incorporates multi-connectivity and captures the impact of molecular absorption coefficient variations in a TW, beam-squint, molecular absorption noise, and link blockages. To make the problem tractable, we first propose a convex approximation of the molecular absorption coefficient using curve fitting in a TW, determine the feasible bandwidths of the leading and trailing edge-bands, and then derive closed-form optimal solution for the number of sub-bands considering beam-squint constraints. We then decompose joint user associations, sub-band assignment, and power allocation problem into two sub-problems, i.e., \textbf{(i)} joint user association and sub-band assignment, and \textbf{(ii)} power allocation. To solve the former problem, we analytically prove the unimodularity of the constraint matrix which enables us to relax the integer constraint without loss of optimality. To solve power allocation sub-problem, a fractional programming (FP)-based centralized solution as well as an alternating direction method of multipliers (ADMM)-based light-weight distributed solution is proposed. The overall problem is then solved using alternating optimization until convergence. Complexity analysis of the algorithms and numerical convergence are presented. Numerical findings validate the effectiveness of the proposed algorithms and extract useful insights about the interplay of the density of base stations (BSs), Average order of multi-connectivity (AOM), molecular absorption, {hardware impairment}, {imperfect CSI}, and link blockages., Comment: This paper has been accepted for publication in IEEE journals

We present foundation language models developed to power Apple Intelligence features, including a ~3 billion parameter model designed to run efficiently on devices and a large server-based language model designed for Private Cloud Compute. These models are designed to perform a wide range of tasks efficiently, accurately, and responsibly. This report describes the model architecture, the data used to train the model, the training process, how the models are optimized for inference, and the evaluation results. We highlight our focus on Responsible AI and how the principles are applied throughout the model development.

The evolution of smart home technologies, particularly agentic ones such as conversational agents, robots, and virtual avatars, is reshaping our understanding of home and domestic life. This shift highlights the complexities of modern domestic life, with the household landscape now featuring diverse cohabiting units like co-housing and communal living arrangements. These agentic technologies present specific design challenges and opportunities as they become integrated into everyday routines and activities. Our workshop envisions smart homes as dynamic, user-shaped spaces, focusing on the integration of these technologies into daily life. We aim to explore how these technologies transform household dynamics, especially through boundary fluidity, by uniting researchers and practitioners from fields such as design, sociology, and ethnography. Together, we will develop a taxonomy of challenges and opportunities, providing a structured perspective on the integration of agentic technologies and their impact on contemporary living arrangements., Comment: 13 pages, no figures, ECSCW 2024 Workshop Proposal

Semantic Textual Relatedness holds significant relevance in Natural Language Processing, finding applications across various domains. Traditionally, approaches to STR have relied on knowledge-based and statistical methods. However, with the emergence of Large Language Models, there has been a paradigm shift, ushering in new methodologies. In this paper, we delve into the investigation of sentence-level STR within Track A (Supervised) by leveraging fine-tuning techniques on the RoBERTa transformer. Our study focuses on assessing the efficacy of this approach across different languages. Notably, our findings indicate promising advancements in STR performance, particularly in Latin languages. Specifically, our results demonstrate notable improvements in English, achieving a correlation of 0.82 and securing a commendable 19th rank. Similarly, in Spanish, we achieved a correlation of 0.67, securing the 15th position. However, our approach encounters challenges in languages like Arabic, where we observed a correlation of only 0.38, resulting in a 20th rank., Comment: 10 pages, 9 figures, 4 tables

Motivated by the stringent safety requirements that are often present in real-world applications, we study a safe online convex optimization setting where the player needs to simultaneously achieve sublinear regret and zero constraint violation while only using zero-order information. In particular, we consider a multi-point feedback setting, where the player chooses $d + 1$ points in each round (where $d$ is the problem dimension) and then receives the value of the constraint function and cost function at each of these points. To address this problem, we propose an algorithm that leverages forward-difference gradient estimation as well as optimistic and pessimistic action sets to achieve $\mathcal{O}(d \sqrt{T})$ regret and zero constraint violation under the assumption that the constraint function is smooth and strongly convex. We then perform a numerical study to investigate the impacts of the unknown constraint and zero-order feedback on empirical performance., Comment: 20 pages, 1 figure. Published in the proceedings of the Learning for Dynamics and Control Conference (L4DC) 2024

In this paper, we consider contamination by code generation test sets, in particular in their use in modern large language models. We discuss three possible sources of such contamination and show findings supporting each of them: (i) direct data leakage, (ii) indirect data leakage through the use of synthetic data and (iii) overfitting to evaluation sets during model selection. To address this, we release Less Basic Python Problems (LBPP): an uncontaminated new benchmark of 161 prompts with their associated Python solutions. LBPP is released at https://huggingface.co/datasets/CohereForAI/lbpp ., Comment: EMNLP 2024 Findings. 5 main pages, 9 in total

Deep neural networks (DNNs) continue to grow rapidly in size, making them infeasible to train on a single device. Pipeline parallelism is commonly used in existing DNN systems to support large-scale DNN training by partitioning a DNN into multiple stages, which concurrently perform DNN training for different micro-batches in a pipeline fashion. However, existing pipeline-parallel approaches only consider sequential pipeline stages and thus ignore the topology of a DNN, resulting in missed model-parallel opportunities. This paper presents graph pipeline parallelism (GPP), a new pipeline-parallel scheme that partitions a DNN into pipeline stages whose dependencies are identified by a directed acyclic graph. GPP generalizes existing sequential pipeline parallelism and preserves the inherent topology of a DNN to enable concurrent execution of computationally-independent operators, resulting in reduced memory requirement and improved GPU performance. In addition, we develop GraphPipe, a distributed system that exploits GPP strategies to enable performant and scalable DNN training. GraphPipe partitions a DNN into a graph of stages, optimizes micro-batch schedules for these stages, and parallelizes DNN training using the discovered GPP strategies. Evaluation on a variety of DNNs shows that GraphPipe outperforms existing pipeline-parallel systems such as PipeDream and Piper by up to 1.6X. GraphPipe also reduces the search time by 9-21X compared to PipeDream and Piper.

Twitter data has been widely used by researchers across various social and computer science disciplines. A common aim when working with Twitter data is the construction of a random sample of users from a given country. However, while several methods have been proposed in the literature, their comparative performance is mostly unexplored. In this paper, we implement four common methods to collect a random sample of Twitter users in the US: 1% Stream, Bounding Box, Location Query, and Language Query. Then, we compare the methods according to their tweet- and user-level metrics as well as their accuracy in estimating US population with and without using inclusion probabilities of various demographics. Our results show that the 1% Stream method performs differently than others in tweet- and user-level metrics, and best for the construction of a population representative sample. We discuss the conditions under which the 1% Stream method may not be suitable and suggest the Bounding Box method as the second-best method to use.

In this paper, we address variational inequalities (VI) with a finite-sum structure. We introduce a novel single-loop stochastic variance-reduced algorithm, incorporating the Bregman distance function, and establish an optimal convergence guarantee under a monotone setting. Additionally, we explore a structured class of non-monotone problems that exhibit weak Minty solutions, and analyze the complexity of our proposed method, highlighting a significant improvement over existing approaches. Numerical experiments are presented to demonstrate the performance of our algorithm compared to state-of-the-art methods

As distributed learning applications such as Federated Learning, the Internet of Things (IoT), and Edge Computing grow, it is critical to address the shortcomings of such technologies from a theoretical perspective. As an abstraction, we consider decentralized learning over a network of communicating clients or nodes and tackle two major challenges: data heterogeneity and adversarial robustness. We propose a decentralized minimax optimization method that employs two important modules: local updates and gradient tracking. Minimax optimization is the key tool to enable adversarial training for ensuring robustness. Having local updates is essential in Federated Learning (FL) applications to mitigate the communication bottleneck, and utilizing gradient tracking is essential to proving convergence in the case of data heterogeneity. We analyze the performance of the proposed algorithm, Dec-FedTrack, in the case of nonconvex-strongly concave minimax optimization, and prove that it converges a stationary point. We also conduct numerical experiments to support our theoretical findings.

Existing work in scientific machine learning (SciML) has shown that data-driven learning of solution operators can provide a fast approximate alternative to classical numerical partial differential equation (PDE) solvers. Of these, Neural Operators (NOs) have emerged as particularly promising. We observe that several uncertainty quantification (UQ) methods for NOs fail for test inputs that are even moderately out-of-domain (OOD), even when the model approximates the solution well for in-domain tasks. To address this limitation, we show that ensembling several NOs can identify high-error regions and provide good uncertainty estimates that are well-correlated with prediction errors. Based on this, we propose a cost-effective alternative, DiverseNO, that mimics the properties of the ensemble by encouraging diverse predictions from its multiple heads in the last feed-forward layer. We then introduce Operator-ProbConserv, a method that uses these well-calibrated UQ estimates within the ProbConserv framework to update the model. Our empirical results show that Operator-ProbConserv enhances OOD model performance for a variety of challenging PDE problems and satisfies physical constraints such as conservation laws., Comment: ICML 2024

We study the problem of online convex optimization (OCO) under unknown linear constraints that are either static, or stochastically time-varying. For this problem, we introduce an algorithm that we term Optimistically Safe OCO (OSOCO) and show that it enjoys $\tilde{O}(\sqrt{T})$ regret and no constraint violation. In the case of static linear constraints, this improves on the previous best known $\tilde{O}(T^{2/3})$ regret under the same assumptions. In the case of stochastic time-varying constraints, our work supplements existing results that show $O(\sqrt{T})$ regret and $O(\sqrt{T})$ cumulative violation under more general convex constraints and a different set of assumptions. In addition to our theoretical guarantees, we also give numerical results that further validate the effectiveness of our approach., Comment: 38 pages, 2 figures