Search

Practical superconducting nanowire single photon detectors (SNSPDs) demonstrate a strong trade-off between detection sensitivity and the reset time. Often, there are wide variations in sensitivity and response times from SNSPDs of the same superconducting material. Here, using detailed physical models, we show that the dirtiness in a superconductor enforces a sensitivity and bandwidth trade-off in all practical devices. More importantly, a certain degree of dirtiness is a necessary requirement for achieving single photon detection. Under typical bias conditions close to the transition setpoints, the minimum number of photons required to register a voltage pulse decreases by the dirtiness parameter (Ioffe-Regel parameter) and the reset time of SNSPD increases by the same dirtiness parameter, thereby giving a constant value for the sensitivity-bandwidth product. The constant is weakly modified by biasing current and the temperature. Since dirtiness in the superconducting nanowire is a physically controllable parameter with an important bearing on the final response of an SNSPD, this work opens new opportunities to develop SNSPD devices with engineered sensitivity-bandwidth setpoint as dictated by an application., Comment: 12 Pages, 05 Figures

Thermodynamic force and irreversible current are the foundational concepts of classical nonequilibrium thermodynamics. Entropy production rate is provided by their product in classical systems, ranging from mesoscopic to macroscopic systems. However, there is no complete quantum extension of such a structure that respects quantum mechanics. In this paper, we propose anti-Hermitian operators that represent currents and forces accompanied by a gradient structure in open quantum systems described by the quantum master equation. We prove that the entropy production rate is given by the product of the force and current operators, which extends the canonical expression of the entropy production rate in the classical systems. The framework constitutes a comprehensive analogy with the nonequilibrium thermodynamics of discrete classical systems. We also show that the structure leads to the extensions of some results in stochastic thermodynamics: the geometric housekeeping-excess decomposition of entropy production and thermodynamic trade-off relations such as the thermodynamic uncertainty relation and the dissipation-time uncertainty relation. In discussing the trade-off relations, we will introduce a measure of fluctuation, which we term the quantum diffusivity., Comment: 24 pages, 5 figures

Adversarial Training (AT) is one of the most effective methods to enhance the robustness of DNNs. However, existing AT methods suffer from an inherent trade-off between adversarial robustness and clean accuracy, which seriously hinders their real-world deployment. While this problem has been widely studied within the current AT paradigm, existing AT methods still typically experience a reduction in clean accuracy by over 10% to date, without significant improvements in robustness compared with simple baselines like PGD-AT. This inherent trade-off raises a question: whether the current AT paradigm, which assumes to learn the corresponding benign and adversarial samples as the same class, inappropriately combines clean and robust objectives that may be essentially inconsistent. In this work, we surprisingly reveal that up to 40% of CIFAR-10 adversarial samples always fail to satisfy such an assumption across various AT methods and robust models, explicitly indicating the improvement room for the current AT paradigm. Accordingly, to relax the tension between clean and robust learning derived from this overstrict assumption, we propose a new AT paradigm by introducing an additional dummy class for each original class, aiming to accommodate the hard adversarial samples with shifted distribution after perturbation. The robustness w.r.t. these adversarial samples can be achieved by runtime recovery from the predicted dummy classes to their corresponding original ones, eliminating the compromise with clean learning. Building on this new paradigm, we propose a novel plug-and-play AT technology named DUmmy Classes-based Adversarial Training (DUCAT). Extensive experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet demonstrate that the DUCAT concurrently improves clean accuracy and adversarial robustness compared with state-of-the-art benchmarks, effectively breaking the existing inherent trade-off., Comment: Preprint. Work in progress. The code is available at https://github.com/FlaAI/DUCAT

In many epidemiological and ecological contexts, there is a trade-off between infections and interaction. This arises because the links between individuals capable of spreading infections are also often associated with beneficial activities. Here, we consider how the presence of explicit network structure changes the optimal solution of a class of infection-interaction trade-offs. In order to do this, we develop and analyse a low-dimensional dynamical system approximating the network SIS epidemic. We find that network structure in the form of heterogeneous numbers of contacts can have a significant impact on the optimal number of contacts that comes out of a trade-off model., Comment: 16 pages; 5 figures

Feedback control is a renowned mechanism for attenuating intrinsic fluctuation in regulatory networks. However, its impact on the response sensitivity to external signals and the response timescale, which are also critical for signal transmission, has yet to be understood. In this letter, we study a general feedback-controlled network in which the feedback is achieved by a complex interactive module. By comparing the solution of Langevin equations with and without feedback, we analytically derive a fundamental trade-off between fluctuation, sensitivity, and timescale altered by the feedback. We show that feedback control cannot infinitely suppress fluctuation without the cost of reducing sensitivity or response speed. Furthermore, the lower bound for this trade-off can be reduced up to half in non-gradient dynamical systems compared to gradient systems. We validate this trade-off as a tight bound for high-dimensional systems in nonlinear regime through numerical simulations. These results elucidate the fundamental limitation of feedback control in enhancing the information transmission capacity of regulatory networks.

Quality-Diversity (QD) algorithms have exhibited promising results across many domains and applications. However, uncertainty in fitness and behaviour estimations of solutions remains a major challenge when QD is used in complex real-world applications. While several approaches have been proposed to improve the performance in uncertain applications, many fail to address a key challenge: determining how to prioritise solutions that perform consistently under uncertainty, in other words, solutions that are reproducible. Most prior methods improve fitness and reproducibility jointly, ignoring the possibility that they could be contradictory objectives. For example, in robotics, solutions may reliably walk at 90% of the maximum velocity in uncertain environments, while solutions that walk faster are also more prone to falling over. As this is a trade-off, neither one of these two solutions is "better" than the other. Thus, algorithms cannot intrinsically select one solution over the other, but can only enforce given preferences over these two contradictory objectives. In this paper, we formalise this problem as the performance-reproducibility trade-off for uncertain QD. We propose four new a-priori QD algorithms that find optimal solutions for given preferences over the trade-offs. We also propose an a-posteriori QD algorithm for when these preferences cannot be defined in advance. Our results show that our approaches successfully find solutions that satisfy given preferences. Importantly, by simply accounting for this trade-off, our approaches perform better than existing uncertain QD methods. This suggests that considering the performance-reproducibility trade-off unlocks important stepping stones that are usually missed when only performance is optimised.

In clinical studies upon which decisions are based there are two types of errors that can be made: a type I error arises when the decision is taken to declare a positive outcome when the truth is in fact negative, and a type II error arises when the decision is taken to declare a negative outcome when the truth is in fact positive. Commonly the primary analysis of such a study entails a two-sided hypothesis test with a type I error rate of 5% and the study is designed to have a sufficiently low type II error rate, for example 10% or 20%. These values are arbitrary and often do not reflect the clinical, or regulatory, context of the study and ignore both the relative costs of making either type of error and the sponsor's prior belief that the drug is superior to either placebo, or a standard of care if relevant. This simplistic approach has recently been challenged by numerous authors both from a frequentist and Bayesian perspective since when resources are constrained there will be a need to consider a trade-off between type I and type II errors. In this paper we review proposals to utilise the trade-off by formally acknowledging the costs to optimise the choice of error rates for simple, point null and alternative hypotheses and extend the results to composite, or interval hypotheses, showing links to the Probability of Success of a clinical study.

Quantum coherence, a fundamental resource in quantum computing and quantum information, often competes with localization effects that affects quantum states in disordered systems. In this work, we prove exact trade-off relations between quantum coherence and a measure of localization and many-body localization, namely, the inverse participation ratio (IPR).We prove that for a pure quantum state, l1-norm of quantum coherence and the relative entropy of coherence satisfy complementarity relations with IPR. For a mixed state, IPR and the l2-norm of quantum coherence as well as relative entropy of coherence satisfy trade-off inequalities. These relations suggest that quantum coherence, in disordered quantum systems is also an ideal characterization of the delocalisation to many-body localisation transition, much like IPR, which is a well-known diagnostic of MBL. These relations also provide insight into the unusual properties of bipartite entanglement entropy across the MBL transition. We believe that these trade-off relations can help in better understanding of how coherence can be preserved or lost in realistic many-body quantum systems, which is vital for developing robust quantum technologies and uncovering new phases of quantum matter., Comment: 6 pages, 1 figure+Supplementary Material

Due to the diverse functionalities of different thermal machines, their optimization relies on a case-by-case basis, lacking unified results. In this work, we propose a general approach to determine power-efficiency trade-off relation (PETOR) for any thermal machine. For cases where cycle (of duration $\tau$) irreversibility satisfies the typical $1/\tau$-scaling, we provide a unified PETOR which is applicable to heat engines, refrigerators, heat exchangers and heat pumps. It is shown that, some typical PETORs, such as those for low-dissipation Carnot cycles (including heat engine and refrigerator cycles) and the steady-state heat engines operating between finite-sized reservoirs are naturally recovered., Comment: 4+2 pages, 2 figures, comments are welcome

Large language models are probabilistic models, and the process of generating content is essentially sampling from the output distribution of the language model. Existing watermarking techniques inject watermarks into the generated content without altering the output quality. On the other hand, existing acceleration techniques, specifically speculative sampling, leverage a draft model to speed up the sampling process while preserving the output distribution. However, there is no known method to simultaneously accelerate the sampling process and inject watermarks into the generated content. In this paper, we investigate this direction and find that the integration of watermarking and acceleration is non-trivial. We prove a no-go theorem, which states that it is impossible to simultaneously maintain the highest watermark strength and the highest sampling efficiency. Furthermore, we propose two methods that maintain either the sampling efficiency or the watermark strength, but not both. Our work provides a rigorous theoretical foundation for understanding the inherent trade-off between watermark strength and sampling efficiency in accelerating the generation of watermarked tokens for large language models. We also conduct numerical experiments to validate our theoretical findings and demonstrate the effectiveness of the proposed methods.

We propose and analyze a new data-driven trade-off (TRO) approach for modeling uncertainty that serves as a middle ground between the optimistic approach, which adopts a distributional belief, and the pessimistic distributionally robust optimization approach, which hedges against distributional ambiguity. We equip the TRO model with a TRO ambiguity set characterized by a size parameter controlling the level of optimism and a shape parameter representing distributional ambiguity. We first show that constructing the TRO ambiguity set using a general star-shaped shape parameter with the empirical distribution as its star center is necessary and sufficient to guarantee the hierarchical structure of the sequence of TRO ambiguity sets. Then, we analyze the properties of the TRO model, including quantifying conservatism, quantifying bias and generalization error, and establishing asymptotic properties. Specifically, we show that the TRO model could generate a spectrum of decisions, ranging from optimistic to conservative decisions. Additionally, we show that it could produce an unbiased estimator of the true optimal value. Furthermore, we establish the almost-sure convergence of the optimal value and the set of optimal solutions of the TRO model to their true counterparts. We exemplify our theoretical results using an inventory control problem and a portfolio optimization problem.

Adaptation refers to the ability to recover and maintain ``normal'' function upon perturbations of internal of external conditions and is essential for sustaining life. Biological adaptation mechanisms are dissipative, i.e. they require a supply of energy such as the coupling to the hydrolysis of ATP. Via evolution the underlying biochemical machinery of living organisms evolved into highly optimized states. However, in the case of adaptation processes two quantities are optimized simultaneously, the adaptation speed or accuracy and the thermodynamic cost. In such cases one typically faces a trade-off, where improving one quantity implies worsening the other. The solution is no longer unique but rather a Pareto set -- the set of all physically attainable protocols along which no quantity can be improved without worsening another. Here we investigate Pareto fronts in adaptation-dissipation trade-offs for a cellular thermostat and a minimal ATP-driven receptor-ligand reaction network. We find convex sections of Pareto fronts to be interrupted by concave regions, implying the coexistence of distinct optimization mechanisms. We discuss the implications of such ``compromise-optimal'' solutions and argue that they may endow biological systems with a superior flexibility to evolve, resist, and adapt to different environments., Comment: 14 pages, 7 figures

In recent years, Vision-Language Models (VLMs) have demonstrated significant advancements in artificial intelligence, transforming tasks across various domains. Despite their capabilities, these models are susceptible to jailbreak attacks, which can compromise their safety and reliability. This paper explores the trade-off between jailbreakability and stealthiness in VLMs, presenting a novel algorithm to detect non-stealthy jailbreak attacks and enhance model robustness. We introduce a stealthiness-aware jailbreak attack using diffusion models, highlighting the challenge of detecting AI-generated content. Our approach leverages Fano's inequality to elucidate the relationship between attack success rates and stealthiness scores, providing an explainable framework for evaluating these threats. Our contributions aim to fortify AI systems against sophisticated attacks, ensuring their outputs remain aligned with ethical standards and user expectations.

When performing predictions that use Machine Learning (ML), we are mainly interested in performance and interpretability. This generates a natural trade-off, where complex models generally have higher skills but are harder to explain and thus trust. Interpretability is particularly important in the climate community, where we aim at gaining a physical understanding of the underlying phenomena. Even more so when the prediction concerns extreme weather events with high impact on society. In this paper, we perform probabilistic forecasts of extreme heatwaves over France, using a hierarchy of increasingly complex ML models, which allows us to find the best compromise between accuracy and interpretability. More precisely, we use models that range from a global Gaussian Approximation (GA) to deep Convolutional Neural Networks (CNNs), with the intermediate steps of a simple Intrinsically Interpretable Neural Network (IINN) and a model using the Scattering Transform (ScatNet). Our findings reveal that CNNs provide higher accuracy, but their black-box nature severely limits interpretability, even when using state-of-the-art Explainable Artificial Intelligence (XAI) tools. In contrast, ScatNet achieves similar performance to CNNs while providing greater transparency, identifying key scales and patterns in the data that drive predictions. This study underscores the potential of interpretability in ML models for climate science, demonstrating that simpler models can rival the performance of their more complex counterparts, all the while being much easier to understand. This gained interpretability is crucial for building trust in model predictions and uncovering new scientific insights, ultimately advancing our understanding and management of extreme weather events.

Machine learning is permeating every conceivable domain to promote data-driven decision support. The focus is often on advanced black-box models due to their assumed performance advantages, whereas interpretable models are often associated with inferior predictive qualities. More recently, however, a new generation of generalized additive models (GAMs) has been proposed that offer promising properties for capturing complex, non-linear patterns while remaining fully interpretable. To uncover the merits and limitations of these models, this study examines the predictive performance of seven different GAMs in comparison to seven commonly used machine learning models based on a collection of twenty tabular benchmark datasets. To ensure a fair and robust model comparison, an extensive hyperparameter search combined with cross-validation was performed, resulting in 68,500 model runs. In addition, this study qualitatively examines the visual output of the models to assess their level of interpretability. Based on these results, the paper dispels the misconception that only black-box models can achieve high accuracy by demonstrating that there is no strict trade-off between predictive performance and model interpretability for tabular data. Furthermore, the paper discusses the importance of GAMs as powerful interpretable models for the field of information systems and derives implications for future work from a socio-technical perspective., Comment: Accepted for publication in Business & Information Systems Engineering (2024)

General Data Protection Regulations (GDPR) aim to safeguard individuals' personal information from harm. While full compliance is mandatory in the European Union and the California Privacy Rights Act (CPRA), it is not in other places. GDPR requires simultaneous compliance with all the principles such as fairness, accuracy, and data minimization. However, it overlooks the potential contradictions within its principles. This matter gets even more complex when compliance is required from decision-making systems. Therefore, it is essential to investigate the feasibility of simultaneously achieving the goals of GDPR and machine learning, and the potential tradeoffs that might be forced upon us. This paper studies the relationship between the principles of data minimization and fairness in recommender systems. We operationalize data minimization via active learning (AL) because, unlike many other methods, it can preserve a high accuracy while allowing for strategic data collection, hence minimizing the amount of data collection. We have implemented several active learning strategies (personalized and non-personalized) and conducted a comparative analysis focusing on accuracy and fairness on two publicly available datasets. The results demonstrate that different AL strategies may have different impacts on the accuracy of recommender systems with nearly all strategies negatively impacting fairness. There has been no to very limited work on the trade-off between data minimization and fairness, the pros and cons of active learning methods as tools for implementing data minimization, and the potential impacts of AL on fairness. By exploring these critical aspects, we offer valuable insights for developing recommender systems that are GDPR compliant.

The threat posed to humanity by global warming has led scientists to question the nature of their activities and the need to reduce the greenhouse gas emissions from research. Until now, most studies have aimed at quantifying the carbon footprints and relatively less works have addressed the ways GHG emissions can be significantly reduced. A factor two reduction by 2030 implies to think beyond increases in the efficacy of current processes, which will have a limited effect, and beyond wishful thinking about large new sources of energy. Hence, choices among research questions or allocated means within a given field will be needed. They can be made in light of the perceived societal utility of research activities. Here, we addressed the question of how scientists perceive the impact of GHG reduction on their discipline and a possible trade-off between the societal utility of their discipline and an acceptable level of GHG emissions. We conducted 28 semi-directive interviews of French astrophysicists from different laboratories. Our most important findings are that, for most researchers, astronomy is considered to have a positive societal impact mainly regarding education but also because of the fascination it exerts on at least a fraction of the general public. Technological applications are also mentioned but with relatively less emphasis. The reduction of GHG emissions is believed to be necessary and most often reductions within the private-sphere have been achieved. However, the question of community-wide reductions in astrophysics research, and in particular the possible reductions of large facilities reveals much more contrasted opinions., Comment: Proceedings for the SF2A 2024 meeting - M. Bethermin, K. Bailli , N. Lagarde, J. Malzac, R.-M. Ouazzani, J. Richard, O. Venot, A. Siebert (eds); abstract abridged

The effective distribution of user transmit powers is essential for the significant advancements that the emergence of 6G wireless networks brings. In recent studies, Deep Neural Networks (DNNs) have been employed to address this challenge. However, these methods frequently encounter issues regarding fairness and computational inefficiency when making decisions, rendering them unsuitable for future dynamic services that depend heavily on the participation of each individual user. To address this gap, this paper focuses on the challenge of transmit power allocation in wireless networks, aiming to optimize $\alpha$-fairness to balance network utilization and user equity. We introduce a novel approach utilizing Kolmogorov-Arnold Networks (KANs), a class of machine learning models that offer low inference costs compared to traditional DNNs through superior explainability. The study provides a comprehensive problem formulation, establishing the NP-hardness of the power allocation problem. Then, two algorithms are proposed for dataset generation and decentralized KAN training, offering a flexible framework for achieving various fairness objectives in dynamic 6G environments. Extensive numerical simulations demonstrate the effectiveness of our approach in terms of fairness and inference cost. The results underscore the potential of KANs to overcome the limitations of existing DNN-based methods, particularly in scenarios that demand rapid adaptation and fairness., Comment: a conference paper, accepted for publication at IEEE VCC 2024

The detailed mechanism of bonding in the cold spray process has remained elusive for both experimental and theoretical parties. Adiabatic shear instability and hydrodynamic plasticity models have been so far the most popular explanations. Here, using molecular dynamics simulation, we investigate their validity at the nanoscale. The present study has potential application for the fabrication of ultra-thin layers for the electronics industry. For this aim, we considered Ti nanoparticles of different diameters and Si substrates of different orientations. It is shown that very high spray velocities are required for a jet to be observed at the nanoscale. We propose a method for thermostating the substrate that enables utilizing high spray velocities. For the first time, we demonstrate an oscillatory behavior in both the normal and radial stress components within the substrate that can propagate into the particle. We have shown that neither the adiabatic shear instability model nor the hydrodynamic plasticity model can be ignored at the nanoscale. Besides, the formation of a low-resistance titanium silicide proper for electronic application is illustrated., Comment: 14 pages, 15 figures

Algorithmic fairness in machine learning has recently garnered significant attention. However, two pressing challenges remain: (1) The fairness guarantees of existing fair classification methods often rely on specific data distribution assumptions and large sample sizes, which can lead to fairness violations when the sample size is moderate-a common situation in practice. (2) Due to legal and societal considerations, using sensitive group attributes during decision-making (referred to as the group-blind setting) may not always be feasible. In this work, we quantify the impact of enforcing algorithmic fairness and group-blindness in binary classification under group fairness constraints. Specifically, we propose a unified framework for fair classification that provides distribution-free and finite-sample fairness guarantees with controlled excess risk. This framework is applicable to various group fairness notions in both group-aware and group-blind scenarios. Furthermore, we establish a minimax lower bound on the excess risk, showing the minimax optimality of our proposed algorithm up to logarithmic factors. Through extensive simulation studies and real data analysis, we further demonstrate the superior performance of our algorithm compared to existing methods, and provide empirical support for our theoretical findings.

Recent advancements in Transformer-based large language models (LLMs) have set new standards in natural language processing. However, the classical softmax attention incurs significant computational costs, leading to a $O(T)$ complexity for per-token generation, where $T$ represents the context length. This work explores reducing LLMs' complexity while maintaining performance by introducing Rodimus and its enhanced version, Rodimus$+$. Rodimus employs an innovative data-dependent tempered selection (DDTS) mechanism within a linear attention-based, purely recurrent framework, achieving significant accuracy while drastically reducing the memory usage typically associated with recurrent models. This method exemplifies semantic compression by maintaining essential input information with fixed-size hidden states. Building on this, Rodimus$+$ combines Rodimus with the innovative Sliding Window Shared-Key Attention (SW-SKA) in a hybrid approach, effectively leveraging the complementary semantic, token, and head compression techniques. Our experiments demonstrate that Rodimus$+$-1.6B, trained on 1 trillion tokens, achieves superior downstream performance against models trained on more tokens, including Qwen2-1.5B and RWKV6-1.6B, underscoring its potential to redefine the accuracy-efficiency balance in LLMs. Model code and pre-trained checkpoints will be available soon.

With the explosive growth of deep learning applications, the right to be forgotten has become increasingly in demand in various AI industries. For example, given a facial recognition system, some individuals may wish to remove images that might have been used in the training phase from the trained model. Unfortunately, modern deep neural networks sometimes unexpectedly leak personal identities. Recent studies have presented various machine unlearning algorithms to make a trained model unlearn the data to be forgotten. While these methods generally perform well in terms of forgetting scores, we have found that an unexpected modelutility drop can occur. This phenomenon, which we term correlation collapse, happens when the machine unlearning algorithms reduce the useful correlation between image features and the true label. To address this challenge, we propose Distribution-Level Feature Distancing (DLFD), a novel method that efficiently forgets instances while preventing correlation collapse. Our method synthesizes data samples so that the generated data distribution is far from the distribution of samples being forgotten in the feature space, achieving effective results within a single training epoch. Through extensive experiments on facial recognition datasets, we demonstrate that our approach significantly outperforms state-of-the-art machine unlearning methods., Comment: 10 pages, 6 figures

To be safely and successfully deployed, LLMs must simultaneously satisfy truthfulness and utility goals. Yet, often these two goals compete (e.g., an AI agent assisting a used car salesman selling a car with flaws), partly due to ambiguous or misleading user instructions. We propose AI-LieDar, a framework to study how LLM-based agents navigate scenarios with utility-truthfulness conflicts in a multi-turn interactive setting. We design a set of realistic scenarios where language agents are instructed to achieve goals that are in conflict with being truthful during a multi-turn conversation with simulated human agents. To evaluate the truthfulness at large scale, we develop a truthfulness detector inspired by psychological literature to assess the agents' responses. Our experiment demonstrates that all models are truthful less than 50% of the time, although truthfulness and goal achievement (utility) rates vary across models. We further test the steerability of LLMs towards truthfulness, finding that models follow malicious instructions to deceive, and even truth-steered models can still lie. These findings reveal the complex nature of truthfulness in LLMs and underscore the importance of further research to ensure the safe and reliable deployment of LLMs and AI agents.

Differential privacy (DP) provides a provable framework for protecting individuals by customizing a random mechanism over a privacy-sensitive dataset. Deep learning models have demonstrated privacy risks in model exposure as an established learning model unintentionally records membership-level privacy leakage. Differentially private stochastic gradient descent (DP- SGD) has been proposed to safeguard training individuals by adding random Gaussian noise to gradient updates in the backpropagation. Researchers identify that DP-SGD typically causes utility loss since the injected homogeneous noise alters the gradient updates calculated at each iteration. Namely, all elements in the gradient are contaminated regardless of their importance in updating model parameters. In this work, we argue that the utility loss mainly results from the homogeneity of injected noise. Consequently, we propose a generic differential privacy framework with heterogeneous noise (DP-Hero) by defining a heterogeneous random mechanism to abstract its property. The insight of DP-Hero is to leverage the knowledge encoded in the previously trained model to guide the subsequent allocation of noise heterogeneity, thereby leveraging the statistical perturbation and achieving enhanced utility. Atop DP-Hero, we instantiate a heterogeneous version of DP-SGD, where the noise injected into gradients is heterogeneous and guided by prior-established model parameters. We conduct comprehensive experiments to verify and explain the effectiveness of the proposed DP-Hero, showing improved training accuracy compared with state-of-the-art works. Broadly, we shed light on improving the privacy-utility space by learning the noise guidance from the pre-existing leaked knowledge encoded in the previously trained model, showing a different perspective of understanding the utility-improved DP training., Comment: 13 pages

We consider the problem of federated Q-learning, where $M$ agents aim to collaboratively learn the optimal Q-function of an unknown infinite-horizon Markov decision process with finite state and action spaces. We investigate the trade-off between sample and communication complexities for the widely used class of intermittent communication algorithms. We first establish the converse result, where it is shown that a federated Q-learning algorithm that offers any speedup with respect to the number of agents in the per-agent sample complexity needs to incur a communication cost of at least an order of $\frac{1}{1-\gamma}$ up to logarithmic factors, where $\gamma$ is the discount factor. We also propose a new algorithm, called Fed-DVR-Q, which is the first federated Q-learning algorithm to simultaneously achieve order-optimal sample and communication complexities. Thus, together these results provide a complete characterization of the sample-communication complexity trade-off in federated Q-learning., Comment: Accepted to NeurIPS 2024

Fairness in clustering has been considered extensively in the past; however, the trade-off between the two objectives -- e.g., can we sacrifice just a little in the quality of the clustering to significantly increase fairness, or vice-versa? -- has rarely been addressed. We introduce novel algorithms for tracing the complete trade-off curve, or Pareto front, between quality and fairness in clustering problems; that is, computing all clusterings that are not dominated in both objectives by other clusterings. Unlike previous work that deals with specific objectives for quality and fairness, we deal with all objectives for fairness and quality in two general classes encompassing most of the special cases addressed in previous work. Our algorithm must take exponential time in the worst case as the Pareto front itself can be exponential. Even when the Pareto front is polynomial, our algorithm may take exponential time, and we prove that this is inevitable unless P = NP. However, we also present a new polynomial-time algorithm for computing the entire Pareto front when the cluster centers are fixed, and for perhaps the most natural fairness objective: minimizing the sum, over all clusters, of the imbalance between the two groups in each cluster.

A central problem in data science is to use potentially noisy samples of an unknown function to predict function values for unseen inputs. In classical statistics, the predictive error is understood as a trade-off between the bias and the variance that balances model simplicity with its ability to fit complex functions. However, over-parameterized models exhibit counter-intuitive behaviors, such as "double descent" in which models of increasing complexity exhibit decreasing generalization error. In contrast to the bias-variance trade-off, we introduce an alternative paradigm called the generalized aliasing decomposition (GAD). We explain the asymptotically small error of complex models as a systematic "de-aliasing" that occurs in the over-parameterized regime. In the limit of large models, the error contribution due to aliasing vanishes, leaving an expression for the asymptotic total error we call the data insufficiency failure of very large models on few training points. Because the generalized aliasing decomposition can be explicitly calculated from the relationship between model class and samples without seeing any data labels, it can answer questions related to experimental design and model selection before collecting data or performing experiments. We demonstrate this approach using several examples, including classical regression problems and a cluster expansion model used in materials science.

Elucidating fundamental limitations inherent in physical systems is a central subject in physics. For important thermodynamic operations such as information erasure, cooling, and copying, resources like time and energetic cost must be expended to achieve the desired outcome within a predetermined error margin. In this study, we introduce the concept of separated states, which consist of fully unoccupied and occupied states. This concept generalizes many critical states involved in relevant thermodynamic operations. We then uncover a three-way trade-off relation between time, cost, and error for a general class of thermodynamic operations aimed at creating separated states, simply expressed as $\tau\mathcal{C}\varepsilon_{\tau}\ge 1-\eta$. This fundamental relation is applicable to diverse thermodynamic operations, including information erasure, cooling, and copying. It provides a profound quantification of the unattainability principle in the third law of thermodynamics in a general form. Building upon this relation, we explore the quantitative limitations governing cooling operations, the preparation of separated states, and a no-go theorem for exact classical copying. Furthermore, we extend these findings to the quantum regime, encompassing both Markovian and non-Markovian dynamics. Specifically, within Lindblad dynamics, we derive a similar three-way trade-off relation that quantifies the cost of achieving a pure state with a given error. The generalization to general quantum dynamics involving a system coupled to a finite bath implies that heat dissipation becomes infinite as the quantum system is exactly cooled down to the ground state or perfectly reset to a pure state, thereby resolving an open question regarding the thermodynamic cost of information erasure., Comment: 21 pages, 6 figures

Over the last few years, federated learning (FL) has emerged as a prominent method in machine learning, emphasizing privacy preservation by allowing multiple clients to collaboratively build a model while keeping their training data private. Despite this focus on privacy, FL models are susceptible to various attacks, including membership inference attacks (MIAs), posing a serious threat to data confidentiality. In a recent study, Rezaei \textit{et al.} revealed the existence of an accuracy-privacy trade-off in deep ensembles and proposed a few fusion strategies to overcome it. In this paper, we aim to explore the relationship between deep ensembles and FL. Specifically, we investigate whether confidence-based metrics derived from deep ensembles apply to FL and whether there is a trade-off between accuracy and privacy in FL with respect to MIA. Empirical investigations illustrate a lack of a non-monotonic correlation between the number of clients and the accuracy-privacy trade-off. By experimenting with different numbers of federated clients, datasets, and confidence-metric-based fusion strategies, we identify and analytically justify the clear existence of the accuracy-privacy trade-off.

Recent years have seen a surge in methods for two-sample testing, among which the Maximum Mean Discrepancy (MMD) test has emerged as an effective tool for handling complex and high-dimensional data. Despite its success and widespread adoption, the primary limitation of the MMD test has been its quadratic-time complexity, which poses challenges for large-scale analysis. While various approaches have been proposed to expedite the procedure, it has been unclear whether it is possible to attain the same power guarantee as the MMD test at sub-quadratic time cost. To fill this gap, we revisit the approximated MMD test using random Fourier features, and investigate its computational-statistical trade-off. We start by revealing that the approximated MMD test is pointwise consistent in power only when the number of random features approaches infinity. We then consider the uniform power of the test and study the time-power trade-off under the minimax testing framework. Our result shows that, by carefully choosing the number of random features, it is possible to attain the same minimax separation rates as the MMD test within sub-quadratic time. We demonstrate this point under different distributional assumptions such as densities in a Sobolev ball. Our theoretical findings are corroborated by simulation studies.

To bypass using local observables as intermediate quantities in proving the eigenstate thermalization hypothesis (ETH), we have introduced an observable-independent measure of distinguishability. In this paper, we establish the connection between this measure and several other ETH measures in a more natural way. We first demonstrate a universal trade-off relation between the diagonal and off-diagonal elements of the measure. We then extend this discussion to eigenstate typicality and the average observable. This trade-off relationship reveals that the exponential growth of off-diagonal elements directly suppresses their own values, as well as indirectly suppressing the diagonal elements. This provides a new perspective on the physical mechanisms underlying ETH. Finally, through numerical calculations on a one-dimensional Ising spin chain, we explore various trade-off relationships and examine strong and weak ETH., Comment: 11 pages, 6 figures