Search

We establish optimal results on limits at infinity for functions in fractional Sobolev spaces., Comment: 25 pages

A collaboration between dataset owners and model owners is needed to facilitate effective machine learning (ML) training. During this collaboration, however, dataset owners and model owners want to protect the confidentiality of their respective assets (i.e., datasets, models and training code), with the dataset owners also caring about the privacy of individual users whose data is in their datasets. Existing solutions either provide limited confidentiality for models and training code, or suffer from privacy issues due to collusion. We present Citadel++, a scalable collaborative ML training system designed to simultaneously protect the confidentiality of datasets, models and training code, as well as the privacy of individual users. Citadel++ enhances differential privacy techniques to safeguard the privacy of individual user data while maintaining model utility. By employing Virtual Machine-level Trusted Execution Environments (TEEs) and improved integrity protection techniques through various OS-level mechanisms, Citadel++ effectively preserves the confidentiality of datasets, models and training code, and enforces our privacy mechanisms even when the models and training code have been maliciously designed. Our experiments show that Citadel++ provides privacy, model utility and performance while adhering to confidentiality and privacy requirements of dataset owners and model owners, outperforming the state-of-the-art privacy-preserving training systems by up to 543x on CPU and 113x on GPU TEEs.

The coalescent is a foundational model of latent genealogical trees under neutral evolution, but suffers from intractable sampling probabilities. Methods for approximating these sampling probabilities either introduce bias or fail to scale to large sample sizes. We show that a class of cost functionals of the coalescent with recurrent mutation and a finite number of alleles converge to tractable processes in the infinite-sample limit. A particular choice of costs yields insight about importance sampling methods, which are a classical tool for coalescent sampling probability approximation. These insights reveal that the behaviour of coalescent importance sampling algorithms differs markedly from standard sequential importance samplers, with or without resampling. We conduct a simulation study to verify that our asymptotics are accurate for algorithms with finite (and moderate) sample sizes. Our results also facilitate the a priori optimisation of computational resource allocation for coalescent sequential importance sampling. We do not observe the same behaviour for importance sampling methods under the infinite sites model of mutation, which is regarded as a good and more tractable approximation of finite alleles mutation in most respects., Comment: 32 pages, 7 figures

In this paper, we study the relationship between Sobolev extension domains and homogeneous Sobolev extension domains. Precisely, we obtain the following results. 1- Let $1\leq q\leq p\leq \infty$. Then a bounded $(L^{1, p}, L^{1, q})$-extension domain is also a $(W^{1, p}, W^{1, q})$-extension domain. 2- Let $1\leq q\leq p

Proton beam radiotherapy stands at the forefront of precision cancer treatment, leveraging the unique physical interactions of proton beams with human tissue to deliver minimal dose upon entry and deposit the therapeutic dose precisely at the so-called Bragg peak, with no residual dose beyond this point. The Bragg peak is the characteristic maximum that occurs when plotting the curve describing the rate of energy deposition along the length of the proton beam. Moreover, as a natural phenomenon, it is caused by an increase in the rate of nuclear interactions of protons as their energy decreases. From an analytical perspective, Bortfeld proposed a parametric family of curves that can be accurately calibrated to data replicating the Bragg peak in one dimension. We build, from first principles, the very first mathematical model describing the energy deposition of protons. Our approach uses stochastic differential equations and affords us the luxury of defining the natural analogue of the Bragg curve in two or three dimensions. This work is purely theoretical and provides a new mathematical framework which is capable of encompassing models built using Geant4 Monte Carlo, at one extreme, to pencil beam calculations with Bortfeld curves at the other.

We show that the volume of the boundary of a bounded Sobolev $(p,q)$-extension domain is zero when $1\leq q

Each homeomorphic parametrization of a Jordan curve via the unit circle extends to a homeomorphism of the entire plane. It is a natural question to ask if such a homeomorphism can be chosen so as to have some Sobolev regularity. This prompts the simplified question: for a homeomorphic embedding of the unit circle into the plane, when can we find a homeomorphism from the unit disk that has the same boundary values and integrable first-order distributional derivatives? We give the optimal geometric criterion for the interior Jordan domain so that there exists a Sobolev homeomorphic extension for any homeomorphic parametrization of the Jordan curve. The problem is partially motivated by trying to understand which boundary values can correspond to deformations of finite energy., Comment: 16 pages, 8 figures

The hidden state threat model of differential privacy (DP) assumes that the adversary has access only to the final trained machine learning (ML) model, without seeing intermediate states during training. Current privacy analyses under this model, however, are limited to convex optimization problems, reducing their applicability to multi-layer neural networks, which are essential in modern deep learning applications. Additionally, the most successful applications of the hidden state privacy analyses in classification tasks have been for logistic regression models. We demonstrate that it is possible to privately train convex problems with privacy-utility trade-offs comparable to those of one hidden-layer ReLU networks trained with DP stochastic gradient descent (DP-SGD). We achieve this through a stochastic approximation of a dual formulation of the ReLU minimization problem which results in a strongly convex problem. This enables the use of existing hidden state privacy analyses, providing accurate privacy bounds also for the noisy cyclic mini-batch gradient descent (NoisyCGD) method with fixed disjoint mini-batches. Our experiments on benchmark classification tasks show that NoisyCGD can achieve privacy-utility trade-offs comparable to DP-SGD applied to one-hidden-layer ReLU networks. Additionally, we provide theoretical utility bounds that highlight the speed-ups gained through the convex approximation.

We show that genealogical trees arising from a broad class of non-neutral models of population evolution converge to the Kingman coalescent under a suitable rescaling of time. As well as non-neutral biological evolution, our results apply to genetic algorithms encompassing the prominent class of sequential Monte Carlo (SMC) methods. The time rescaling we need differs slightly from that used in classical results for convergence to the Kingman coalescent, which has implications for the performance of different resampling schemes in SMC algorithms. In addition, our work substantially simplifies earlier proofs of convergence to the Kingman coalescent, and corrects an error common to several earlier results.

We present a novel method for accurately auditing the differential privacy (DP) guarantees of DP mechanisms. In particular, our solution is applicable to auditing DP guarantees of machine learning (ML) models. Previous auditing methods tightly capture the privacy guarantees of DP-SGD trained models in the white-box setting where the auditor has access to all intermediate models; however, the success of these methods depends on a priori information about the parametric form of the noise and the subsampling ratio used for sampling the gradients. We present a method that does not require such information and is agnostic to the randomization used for the underlying mechanism. Similarly to several previous DP auditing methods, we assume that the auditor has access to a set of independent observations from two one-dimensional distributions corresponding to outputs from two neighbouring datasets. Furthermore, our solution is based on a simple histogram-based density estimation technique to find lower bounds for the statistical distance between these distributions when measured using the hockey-stick divergence. We show that our approach also naturally generalizes the previously considered class of threshold membership inference auditing methods. We improve upon accurate auditing methods such as the $f$-DP auditing. Moreover, we address an open problem on how to accurately audit the subsampled Gaussian mechanism without any knowledge of the parameters of the underlying mechanism.

Private selection mechanisms (e.g., Report Noisy Max, Sparse Vector) are fundamental primitives of differentially private (DP) data analysis with wide applications to private query release, voting, and hyperparameter tuning. Recent work (Liu and Talwar, 2019; Papernot and Steinke, 2022) has made significant progress in both generalizing private selection mechanisms and tightening their privacy analysis using modern numerical privacy accounting tools, e.g., R\'enyi DP. But R\'enyi DP is known to be lossy when $(\epsilon,\delta)$-DP is ultimately needed, and there is a trend to close the gap by directly handling privacy profiles, i.e., $\delta$ as a function of $\epsilon$ or its equivalent dual form known as $f$-DPs. In this paper, we work out an easy-to-use recipe that bounds the privacy profiles of ReportNoisyMax and PrivateTuning using the privacy profiles of the base algorithms they corral. Numerically, our approach improves over the RDP-based accounting in all regimes of interest and leads to substantial benefits in end-to-end private learning experiments. Our analysis also suggests new distributions, e.g., binomial distribution for randomizing the number of rounds that leads to more substantial improvements in certain regimes.

In the arena of privacy-preserving machine learning, differentially private stochastic gradient descent (DP-SGD) has outstripped the objective perturbation mechanism in popularity and interest. Though unrivaled in versatility, DP-SGD requires a non-trivial privacy overhead (for privately tuning the model's hyperparameters) and a computational complexity which might be extravagant for simple models such as linear and logistic regression. This paper revamps the objective perturbation mechanism with tighter privacy analyses and new computational tools that boost it to perform competitively with DP-SGD on unconstrained convex generalized linear problems.

For a continuous function $f:\mathbb{R}\to\mathbb{R}$, define the corresponding graph by setting \[\Gamma_f := {(x1, f(x1)) : x_1\in\mathbb{R}} .\] In this paper, we study the Sobolev extension property for the upper and lower domains over the graph $\Gamma_{\psi^\alpha_c}$ for $\psi^\alpha_c(x_1):=d(x_1, \mathcal C)^\alpha$, where $\mathcal C$ is the classical ternary Cantor set in the unit interval and $\alpha\in(0, 1)$.

Inference of the reproduction number through time is of vital importance during an epidemic outbreak. Typically, epidemiologists tackle this using observed prevalence or incidence data. However, prevalence and incidence data alone is often noisy or partial. Models can also have identifiability issues with determining whether a large amount of a small epidemic or a small amount of a large epidemic has been observed. Sequencing data however is becoming more abundant, so approaches which can incorporate genetic data are an active area of research. We propose using particle MCMC methods to infer the time-varying reproduction number from a combination of prevalence data reported at a set of discrete times and a dated phylogeny reconstructed from sequences. We validate our approach on simulated epidemics with a variety of scenarios. We then apply the method to a real data set of HIV-1 in North Carolina, USA, between 1957 and 2019., Comment: 28 pages, 18 figures (44 pages, 35 figures including appendices)

In this paper, we study the behaviour at infinity of $p$-Sobolev functions in the setting of Ahlfors $Q$-regular metric measure spaces supporting a $p$-Poincar\'e inequality. By introducing the notions of sets which are $p$-thin at infinity, we show that functions in the homogeneous space $\dot N^{1,p}(X)$ necessarily have limits at infinity outside of $p$-thin sets, when $1\le pQ$, we show by example that uniqueness of limits at infinity may fail for functions in $\dot N^{1,p}(X)$. While functions in $\dot N^{1,p}(X)$ may not have any reasonable limit at infinity when $p=Q$, we introduce the notion of a $Q$-thick set at infinity, and characterize the limits of functions in $\dot N^{1,Q}(X)$ along infinite curves in terms of limits outside $Q$-thin sets and along $Q$-thick sets. By weakening the notion of a thick set, we show that a function in $\dot N^{1,Q}(X)$ with a limit along such an almost thick set may fail to have a limit along any infinite curve. While homogeneous $p$-Sobolev functions may have infinite limits at infinity when $p\ge Q$, we provide bounds on how quickly such functions may grow: when $p=Q$, functions in $\dot N^{1,p}(X)$ have sub-logarithmic growth at infinity, whereas when $p>Q$, such functions have growth at infinity controlled by $d(\cdot, O)^{1-Q/p}$, where $O$ is a fixed base point in $X$. For the inhomogeneous spaces $N^{1,p}(X)$, the phenomenon is different. We show that for $1\le p\le Q$, the limit of a function $u\in N^{1,p}(X)$ is zero outside of a $p$-thin set, whereas $\lim_{x\to+\infty}u(x)=0$ for all $u\in N^{1,p}(X)$ when $p>Q$.

In this work, we develop excursion theory for the Wright--Fisher diffusion with mutation. Our construction is intermediate between the classical excursion theory where all excursions begin and end at a single point and the more general approach considering excursions of processes from general sets. Since the Wright--Fisher diffusion has two boundary points, it is natural to construct excursions which start from a specified boundary point, and end at one of two boundary points which determine the next starting point. In order to do this we study the killed Wright--Fisher diffusion, which is sent to a cemetery state whenever it hits either endpoint. We then construct a marked Poisson process of such killed paths which, when concatenated, produce a pathwise construction of the Wright--Fisher diffusion., Comment: 29 pages, 3 figures

Mathematical models of genetic evolution often come in pairs, connected by a so-called duality relation. The most seminal example are the Wright-Fisher diffusion and the Kingman coalescent, where the former describes the stochastic evolution of neutral allele frequencies in a large population forwards in time, and the latter describes the genetic ancestry of randomly sampled individuals from the population backwards in time. As well as providing a richer description than either model in isolation, duality often yields equations satisfied by quantities of interest. We employ the so-called Bernoulli factory - a celebrated tool in simulation-based computing - to derive duality relations for broad classes of genetics models. As concrete examples, we present Wright-Fisher diffusions with general drift functions, and Allen-Cahn equations with general, nonlinear forcing terms. The drift and forcing functions can be interpreted as the action of frequency-dependent selection. To our knowledge, this work is the first time a connection has been drawn between Bernoulli factories and duality in models of population genetics., Comment: 11 pages, 1 figure

Tuning the hyperparameters of differentially private (DP) machine learning (ML) algorithms often requires use of sensitive data and this may leak private information via hyperparameter values. Recently, Papernot and Steinke (2022) proposed a certain class of DP hyperparameter tuning algorithms, where the number of random search samples is randomized itself. Commonly, these algorithms still considerably increase the DP privacy parameter $\varepsilon$ over non-tuned DP ML model training and can be computationally heavy as evaluating each hyperparameter candidate requires a new training run. We focus on lowering both the DP bounds and the computational cost of these methods by using only a random subset of the sensitive data for the hyperparameter tuning and by extrapolating the optimal values to a larger dataset. We provide a R\'enyi differential privacy analysis for the proposed method and experimentally show that it consistently leads to better privacy-utility trade-off than the baseline method by Papernot and Steinke.

The Wright--Fisher diffusion is important in population genetics in modelling the evolution of allele frequencies over time subject to the influence of biological phenomena such as selection, mutation, and genetic drift. Simulating paths of the process is challenging due to the form of the transition density. We present EWF, a robust and efficient sampler which returns exact draws for the diffusion and diffusion bridge processes, accounting for general models of selection including those with frequency-dependence. Given a configuration of selection, mutation, and endpoints, EWF returns draws at the requested sampling times from the law of the corresponding Wright--Fisher process. Output was validated by comparison to approximations of the transition density via the Kolmogorov--Smirnov test and QQ plots. All software is available at https://github.com/JaroSant/EWF, Comment: 34 pages, 12 figures

Individual privacy accounting enables bounding differential privacy (DP) loss individually for each participant involved in the analysis. This can be informative as often the individual privacy losses are considerably smaller than those indicated by the DP bounds that are based on considering worst-case bounds at each data access. In order to account for the individual privacy losses in a principled manner, we need a privacy accountant for adaptive compositions of randomised mechanisms, where the loss incurred at a given data access is allowed to be smaller than the worst-case loss. This kind of analysis has been carried out for the R\'enyi differential privacy (RDP) by Feldman and Zrnic (2021), however not yet for the so-called optimal privacy accountants. We make first steps in this direction by providing a careful analysis using the Gaussian differential privacy which gives optimal bounds for the Gaussian mechanism, one of the most versatile DP mechanisms. This approach is based on determining a certain supermartingale for the hockey-stick divergence and on extending the R\'enyi divergence-based fully adaptive composition results by Feldman and Zrnic. We also consider measuring the individual $(\varepsilon,\delta)$-privacy losses using the so-called privacy loss distributions. With the help of the Blackwell theorem, we can then make use of the RDP analysis to construct an approximative individual $(\varepsilon,\delta)$-accountant., Comment: 31 pages, 10 figures

We establish the existence and uniqueness of limits at infinity along infinite curves outside a zero modulus family for functions in a homogeneous Sobolev space under the assumption that the underlying space is equipped with a doubling measure which supports a Poincar\'e inequality. We also characterize the settings where this conclusion is nontrivial. Secondly, we introduce notions of weak polar coordinate systems and radial curves on metric measure spaces. Then sufficient and necessary conditions for existence of radial limits are given. As a consequence, we characterize the existence of radial limits in certain concrete settings.