Search

In November 2023, the UK and US announced the creation of their AI Safety Institutes (AISIs). Five other jurisdictions have followed in establishing AISIs or similar institutions, with more likely to follow. While there is considerable variation between these institutions, there are also key similarities worth identifying. This primer describes one cluster of similar AISIs, the "first wave," consisting of the Japan, UK, and US AISIs. First-wave AISIs have several fundamental characteristics in common: they are technical government institutions, have a clear mandate related to the safety of advanced AI systems, and lack regulatory powers. Safety evaluations are at the center of first-wave AISIs. These techniques test AI systems across tasks to understand their behavior and capabilities on relevant risks, such as cyber, chemical, and biological misuse. They also share three core functions: research, standards, and cooperation. These functions are critical to AISIs' work on safety evaluations but also support other activities such as scientific consensus-building and foundational AI safety research. Despite its growing popularity as an institutional model, the AISI model is not free from challenges and limitations. Some analysts have criticized the first wave of AISIs for specializing too much in a sub-area and for being potentially redundant with existing institutions, for example. Future developments may rapidly change this landscape, and particularities of individual AISIs may not be captured by our broad-strokes description. This policy brief aims to outline the core elements of first-wave AISIs as a way of encouraging and improving conversations on this novel institutional model, acknowledging this is just a simplified snapshot rather than a timeless prescription.

We report on the formation of multiple quantum droplets in a heteronuclear $^{41}$K-$^{87}$Rb mixture released in an optical waveguide. By a sudden change of the interspecies interaction from the non-interacting to the strongly attractive regime, we initially form a single droplet in an excited compression-elongation mode. The latter axially expands up to a critical length and then splits into two or more smaller fragments, recognizable as quantum droplets. We find that the number of formed droplets increases with decreasing interspecies attraction and increasing atom number. We show, by combining theory and experiment, that this behavior is consistent with capillary instability, which causes the breakup of the stretching droplet due to the surface tension. Our results open new possibilities to explore the properties of quantum liquids and systems of multiple quantum droplets in two-component bosonic mixtures.

International standards are crucial for ensuring that frontier AI systems are developed and deployed safely around the world. Since the AI Safety Institutes (AISIs) possess in-house technical expertise, mandate for international engagement, and convening power in the national AI ecosystem while being a government institution, we argue that they are particularly well-positioned to contribute to the international standard-setting processes for AI safety. In this paper, we propose and evaluate three models for AISI involvement: 1. Seoul Declaration Signatories, 2. US (and other Seoul Declaration Signatories) and China, and 3. Globally Inclusive. Leveraging their diverse strengths, these models are not mutually exclusive. Rather, they offer a multi-track system solution in which the central role of AISIs guarantees coherence among the different tracks and consistency in their AI safety focus., Comment: 13 pages

Adversarial examples pose a significant challenge to the robustness, reliability and alignment of deep neural networks. We propose a novel, easy-to-use approach to achieving high-quality representations that lead to adversarial robustness through the use of multi-resolution input representations and dynamic self-ensembling of intermediate layer predictions. We demonstrate that intermediate layer predictions exhibit inherent robustness to adversarial attacks crafted to fool the full classifier, and propose a robust aggregation mechanism based on Vickrey auction that we call \textit{CrossMax} to dynamically ensemble them. By combining multi-resolution inputs and robust ensembling, we achieve significant adversarial robustness on CIFAR-10 and CIFAR-100 datasets without any adversarial training or extra data, reaching an adversarial accuracy of $\approx$72% (CIFAR-10) and $\approx$48% (CIFAR-100) on the RobustBench AutoAttack suite ($L_\infty=8/255)$ with a finetuned ImageNet-pretrained ResNet152. This represents a result comparable with the top three models on CIFAR-10 and a +5 % gain compared to the best current dedicated approach on CIFAR-100. Adding simple adversarial training on top, we get $\approx$78% on CIFAR-10 and $\approx$51% on CIFAR-100, improving SOTA by 5 % and 9 % respectively and seeing greater gains on the harder dataset. We validate our approach through extensive experiments and provide insights into the interplay between adversarial robustness, and the hierarchical nature of deep representations. We show that simple gradient-based attacks against our model lead to human-interpretable images of the target classes as well as interpretable image changes. As a byproduct, using our multi-resolution prior, we turn pre-trained classifiers and CLIP models into controllable image generators and develop successful transferable attacks on large vision language models., Comment: 34 pages, 25 figures, appendix

We explore a class of adversarial attacks targeting the activations of language models. By manipulating a relatively small subset of model activations, $a$, we demonstrate the ability to control the exact prediction of a significant number (in some cases up to 1000) of subsequent tokens $t$. We empirically verify a scaling law where the maximum number of target tokens $t_\mathrm{max}$ predicted depends linearly on the number of tokens $a$ whose activations the attacker controls as $t_\mathrm{max} = \kappa a$. We find that the number of bits of control in the input space needed to control a single bit in the output space (what we call attack resistance $\chi$) is remarkably constant between $\approx 16$ and $\approx 25$ over 2 orders of magnitude of model sizes for different language models. Compared to attacks on tokens, attacks on activations are predictably much stronger, however, we identify a surprising regularity where one bit of input steered either via activations or via tokens is able to exert control over a similar amount of output bits. This gives support for the hypothesis that adversarial attacks are a consequence of dimensionality mismatch between the input and output spaces. A practical implication of the ease of attacking language model activations instead of tokens is for multi-modal and selected retrieval models, where additional data sources are added as activations directly, sidestepping the tokenized input. This opens up a new, broad attack surface. By using language models as a controllable test-bed to study adversarial attacks, we were able to experiment with input-output dimensions that are inaccessible in computer vision, especially where the output dimension dominates., Comment: 15 pages, 9 figures

Very distinct strategies can be deployed to recognize and characterize an unknown environment or a shape. A recent and promising approach, especially in robotics, is to reduce the complexity of the exploratory units to a minimum. Here, we show that this frugal strategy can be taken to the extreme by exploiting the power of statistical geometry and introducing new invariant features. We show that an elementary robot devoid of any orientation or observation system, exploring randomly, can access global information about an environment such as the values of the explored area and perimeter. The explored shapes are of arbitrary geometry and may even non-connected. From a dictionary, this most simple robot can thus identify various shapes such as famous monuments and even read a text.

We show that we can easily design a single adversarial perturbation $P$ that changes the class of $n$ images $X_1,X_2,\dots,X_n$ from their original, unperturbed classes $c_1, c_2,\dots,c_n$ to desired (not necessarily all the same) classes $c^*_1,c^*_2,\dots,c^*_n$ for up to hundreds of images and target classes at once. We call these \textit{multi-attacks}. Characterizing the maximum $n$ we can achieve under different conditions such as image resolution, we estimate the number of regions of high class confidence around a particular image in the space of pixels to be around $10^{\mathcal{O}(100)}$, posing a significant problem for exhaustive defense strategies. We show several immediate consequences of this: adversarial attacks that change the resulting class based on their intensity, and scale-independent adversarial examples. To demonstrate the redundancy and richness of class decision boundaries in the pixel space, we look for its two-dimensional sections that trace images and spell words using particular classes. We also show that ensembling reduces susceptibility to multi-attacks, and that classifiers trained on random labels are more susceptible. Our code is available on GitHub., Comment: Code at https://github.com/stanislavfort/multi-attacks

Recent advances in deep learning methods for natural language processing (NLP) have created new business opportunities and made NLP research critical for industry development. As one of the big players in the field of NLP, together with governments and universities, it is important to track the influence of industry on research. In this study, we seek to quantify and characterize industry presence in the NLP community over time. Using a corpus with comprehensive metadata of 78,187 NLP publications and 701 resumes of NLP publication authors, we explore the industry presence in the field since the early 90s. We find that industry presence among NLP authors has been steady before a steep increase over the past five years (180% growth from 2017 to 2022). A few companies account for most of the publications and provide funding to academic researchers through grants and internships. Our study shows that the presence and impact of the industry on natural language processing research are significant and fast-growing. This work calls for increased transparency of industry influence in the field., Comment: Published at ACL 2023

At the interface between two fluid layers in relative motion, infinitesimal fluctuations can be exponentially amplified, inducing vorticity and the breakdown of the laminar flow. This process, known as the Kelvin-Helmholtz instability, is responsible for many familiar phenomena observed in the atmosphere and in the oceans, as well as in astrophysical objects, being known as one of the paradigmatic routes to turbulence in fluid mechanics. While shear-flow instabilities in classical fluids have been extensively observed in various contexts, controlled experiments in the presence of quantized circulation are comparatively very few. Here, we engineer two counter-rotating atomic superflows, a configuration that in classical inviscid fluids is unstable via the Kelvin-Helmholtz instability. We observe how the contact interface, i.e. the shear layer, develops into an ordered circular array of quantized vortices, which loses stability and rolls up into vortex clusters. We extract the instability growth rates and find that they obey the same scaling relations across different superfluid regimes, ranging from weakly-interacting bosonic to strongly-correlated fermionic pair condensates. The measured scalings, reproduced by numerical simulations and well described by a microscopic point-vortex model, are consistent with the classical hydrodynamic Kelvin-Helmholtz instability of a finite-width shear layer. Our results establish interesting connections between vortex arrays and shear-flow instabilities, suggesting a possible interpretation of the observed quantized vortex dynamics as a manifestation of the underlying unstable flow. Moreover, they open the way for exploring a wealth of out-of-equilibrium phenomena, from vortex-matter phase transitions to the spontaneous emergence and decay of two-dimensional quantum turbulence.

Stochastic Approximation (SA) is a classical algorithm that has had since the early days a huge impact on signal processing, and nowadays on machine learning, due to the necessity to deal with a large amount of data observed with uncertainties. An exemplar special case of SA pertains to the popular stochastic (sub)gradient algorithm which is the working horse behind many important applications. A lesser-known fact is that the SA scheme also extends to non-stochastic-gradient algorithms such as compressed stochastic gradient, stochastic expectation-maximization, and a number of reinforcement learning algorithms. The aim of this article is to overview and introduce the non-stochastic-gradient perspectives of SA to the signal processing and machine learning audiences through presenting a design guideline of SA algorithms backed by theories. Our central theme is to propose a general framework that unifies existing theories of SA, including its non-asymptotic and asymptotic convergence results, and demonstrate their applications on popular non-stochastic-gradient algorithms. We build our analysis framework based on classes of Lyapunov functions that satisfy a variety of mild conditions. We draw connections between non-stochastic-gradient algorithms and scenarios when the Lyapunov function is smooth, convex, or strongly convex. Using the said framework, we illustrate the convergence properties of the non-stochastic-gradient algorithms using concrete examples. Extensions to the emerging variance reduction techniques for improved sample complexity will also be discussed., Comment: Accepted for publication at IEEE Transactions on Signal Processing; 31 pages, 7 pages of supplementary materials

This paper introduces a novel algorithm, the Perturbed Proximal Preconditioned SPIDER algorithm (3P-SPIDER), designed to solve finite sum non-convex composite optimization. It is a stochastic Variable Metric Forward-Backward algorithm, which allows approximate preconditioned forward operator and uses a variable metric proximity operator as the backward operator; it also proposes a mini-batch strategy with variance reduction to address the finite sum setting. We show that 3P-SPIDER extends some Stochastic preconditioned Gradient Descent-based algorithms and some Incremental Expectation Maximization algorithms to composite optimization and to the case the forward operator can not be computed in closed form. We also provide an explicit control of convergence in expectation of 3P-SPIDER, and study its complexity in order to satisfy the epsilon-approximate stationary condition. Our results are the first to combine the composite non-convex optimization setting, a variance reduction technique to tackle the finite sum setting by using a minibatch strategy and, to allow deterministic or random approximations of the preconditioned forward operator. Finally, through an application to inference in a logistic regression model with random effects, we numerically compare 3P-SPIDER to other stochastic forward-backward algorithms and discuss the role of some design parameters of 3P-SPIDER., Comment: Statistics and Computing, In press

As AI systems become more capable, we would like to enlist their help to supervise other AIs. We experiment with methods for training a harmless AI assistant through self-improvement, without any human labels identifying harmful outputs. The only human oversight is provided through a list of rules or principles, and so we refer to the method as 'Constitutional AI'. The process involves both a supervised learning and a reinforcement learning phase. In the supervised phase we sample from an initial model, then generate self-critiques and revisions, and then finetune the original model on revised responses. In the RL phase, we sample from the finetuned model, use a model to evaluate which of the two samples is better, and then train a preference model from this dataset of AI preferences. We then train with RL using the preference model as the reward signal, i.e. we use 'RL from AI Feedback' (RLAIF). As a result we are able to train a harmless but non-evasive AI assistant that engages with harmful queries by explaining its objections to them. Both the SL and RL methods can leverage chain-of-thought style reasoning to improve the human-judged performance and transparency of AI decision making. These methods make it possible to control AI behavior more precisely and with far fewer human labels.

We present an exploratory population dynamics approach, described by Lotka-Volterra (LV) generalized equations, to explain/predict the dynamics and competition between land use/land cover (LULC) classes over vulnerable regions in Uruguay. We use the Mapbiomas-Pampa dataset composed by 20 annual LULC maps from 2000-2019. From these LULC maps we extract the main LULC classes, their spatial distribution and the time series of areas covered for each class. The interaction coefficients between species are inferred through the pairwise maximum entropy (PME) method from the spatial covariance matrices for different training periods. The main finding is that this LVPME method globally outperforms the more traditional Markov chains approach at predicting the trajectories of areas of LULC classes., Comment: 21 pages, 5 figures, 4 tables

The Microchannel X-ray Telescope (MXT) will be the first focusing X-ray telescope based on a "Lobster-Eye" optical design to be flown on Sino-French mission SVOM. SVOM will be dedicated to the study of Gamma-Ray Bursts and more generally time-domain astrophysics. The MXT telescope is a compact (focal length ~ 1.15 m) and light (< 42 kg) instrument, sensitive in the 0.2--10 keV energy range. It is composed of an optical system, based on micro-pore optics (MPOs) of 40 micron pore size, coupled to a low-noise pnCDD X-ray detector. In this paper we describe the expected scientific performance of the MXT telescope, based on the End-to-End calibration campaign performed in fall 2021, before the integration of the SVOM payload on the satellite., Comment: 22 pages, 12 figures, accepted for publication in Experimental Astronomy

MAGAL lays the foundations for a future constellation of small satellites carrying radar altimeters aiming to improve the understanding of ocean circulation variability at local, regional, and global scales. All necessary tools will be developed, including a new small, low-power altimeter payload and a miniaturized satellite platform, grounded on the Space 4.0 industry, to be manufactured inseries, minimizing production, operational and launch costs. To implement a collaborative constellation, and better tackle the gaps of large radar altimeter programmes, MAGAL will use a Data Analysis Centre, based on cloud services, for storage and process of data, based on known and improved algorithms, including overlay of layers from multiple sources (e.g. meteorology and opensource data). As a constellation of six satellites, MAGAL increases the density of sea surface topography measurements, enabling more data for altimetry products, when used in synergy with other missions, in coastal areas and over mesoscale features. This results in scientific and commercial information aggregated into a single platform, displayed in various graphical interfaces, allowing overlaid correlations. MAGAL is aligned with the insights from the EU agenda for sustainable development, adding value, alongside the underlying technology development, bringing together the sea's economy and its sustainable growth., Comment: 15 pages, 7 figures, Manuscript presented at the 4S Symposium 2022, Vilamoura, Portugal, 16 - 20 May 2022

Developing safe and useful general-purpose AI systems will require us to make progress on scalable oversight: the problem of supervising systems that potentially outperform us on most skills relevant to the task at hand. Empirical work on this problem is not straightforward, since we do not yet have systems that broadly exceed our abilities. This paper discusses one of the major ways we think about this problem, with a focus on ways it can be studied empirically. We first present an experimental design centered on tasks for which human specialists succeed but unaided humans and current general AI systems fail. We then present a proof-of-concept experiment meant to demonstrate a key feature of this experimental design and show its viability with two question-answering tasks: MMLU and time-limited QuALITY. On these tasks, we find that human participants who interact with an unreliable large-language-model dialog assistant through chat -- a trivial baseline strategy for scalable oversight -- substantially outperform both the model alone and their own unaided performance. These results are an encouraging sign that scalable oversight will be tractable to study with present models and bolster recent findings that large language models can productively assist humans with difficult tasks., Comment: v2 fixes a few typos from v1

Deep neural network classifiers partition input space into high confidence regions for each class. The geometry of these class manifolds (CMs) is widely studied and intimately related to model performance; for example, the margin depends on CM boundaries. We exploit the notions of Gaussian width and Gordon's escape theorem to tractably estimate the effective dimension of CMs and their boundaries through tomographic intersections with random affine subspaces of varying dimension. We show several connections between the dimension of CMs, generalization, and robustness. In particular we investigate how CM dimension depends on 1) the dataset, 2) architecture (including ResNet, WideResNet \& Vision Transformer), 3) initialization, 4) stage of training, 5) class, 6) network width, 7) ensemble size, 8) label randomization, 9) training set size, and 10) robustness to data corruption. Together a picture emerges that higher performing and more robust models have higher dimensional CMs. Moreover, we offer a new perspective on ensembling via intersections of CMs. Our code is at https://github.com/stanislavfort/slice-dice-optimize/, Comment: An extended version of /Slice, Dice, and Optimize: Measuring the Dimension of Neural Network Class Manifolds/

Structured illumination in Single Molecule Localization Microscopy provides new information on the position of molecules and thus improves the localization precision compared to standard localization methods. Here, we used a time-shifted sinusoidal excitation pattern to modulate the fluorescence signal of the molecules whose position information is carried by the phase and recovered by synchronous demodulation. We designed two flexible fast demodulation systems located upstream of the camera, allowing us to overcome the limiting camera acquisition frequency and thus to maximize the collection of photons in the demodulation process. The temporally modulated fluorescence signal was then sampled synchronously on the same image, repeatedly during acquisition. This microscopy, called ModLoc, allows to experimentally improve the localization precision by a factor of 2.4 in one direction, compared to classical Gaussian fitting methods. A temporal study and an experimental demonstration both show that the short lifetimes of the molecules in blinking regimes impose a modulation frequency in the kilohertz range, which is beyond the reach of current cameras. A demodulation system operating at these frequencies would thus be necessary to take full advantage of this new localization approach.

We describe our early efforts to red team language models in order to simultaneously discover, measure, and attempt to reduce their potentially harmful outputs. We make three main contributions. First, we investigate scaling behaviors for red teaming across 3 model sizes (2.7B, 13B, and 52B parameters) and 4 model types: a plain language model (LM); an LM prompted to be helpful, honest, and harmless; an LM with rejection sampling; and a model trained to be helpful and harmless using reinforcement learning from human feedback (RLHF). We find that the RLHF models are increasingly difficult to red team as they scale, and we find a flat trend with scale for the other model types. Second, we release our dataset of 38,961 red team attacks for others to analyze and learn from. We provide our own analysis of the data and find a variety of harmful outputs, which range from offensive language to more subtly harmful non-violent unethical outputs. Third, we exhaustively describe our instructions, processes, statistical methodologies, and uncertainty about red teaming. We hope that this transparency accelerates our ability to work together as a community in order to develop shared norms, practices, and technical standards for how to red team language models.

We present a general discussion of the dipole modes of a heteronuclear bosonic mixture in a harmonic trap by comparing the prediction of the sum-rule approach with full Gross-Pitaevskii (GP) calculations. Yet in the range of interaction in which the mixture is miscible and stable at the mean field level, $g_{12}^2

We unravel the coupled dipole dynamics of a two-species Bose-Einstein condensate with tunable interspecies interaction. We produce a degenerate mixture of $^{41}$K-$^{87}$Rb in an optical trap and we study the dipole oscillations of both atomic species in the linear response regime. Varying the interspecies interaction from the weakly to the strongly attractive side, we measure the frequencies and the composition of the two dipole eigenmodes. For enough strong interactions, even beyond the mean-field collapse, we find that the frequency of the low-energy eigenmode is determined only by the bare trap frequencies and the species population imbalance. The experimental results are well reproduced by numerical simulations based on two coupled Gross-Pitaevskii equations. Our findings provide a detailed picture of the dipole excitations in asymmetric bosonic mixtures., Comment: 8 pages, 4 figures

We present MDEAW, a multimodal database consisting of Electrodermal Activity (EDA) and Photoplethysmography (PPG) signals recorded during the exams for the course taught by the teacher at Eurecat Academy, Sabadell, Barcelona in order to elicit the emotional reactions to the students in a classroom scenario. Signals from 10 students were recorded along with the students' self-assessment of their affective state after each stimulus, in terms of 6 basic emotion states. All the signals were captured using portable, wearable, wireless, low-cost, and off-the-shelf equipment that has the potential to allow the use of affective computing methods in everyday applications. A baseline for student-wise affect recognition using EDA and PPG-based features, as well as their fusion, was established through ReMECS, Fed-ReMECS, and Fed-ReMECS-U. These results indicate the prospects of using low-cost devices for affective state recognition applications. The proposed database will be made publicly available in order to allow researchers to achieve a more thorough evaluation of the suitability of these capturing devices for emotion state recognition applications.

We study whether language models can evaluate the validity of their own claims and predict which questions they will be able to answer correctly. We first show that larger models are well-calibrated on diverse multiple choice and true/false questions when they are provided in the right format. Thus we can approach self-evaluation on open-ended sampling tasks by asking models to first propose answers, and then to evaluate the probability "P(True)" that their answers are correct. We find encouraging performance, calibration, and scaling for P(True) on a diverse array of tasks. Performance at self-evaluation further improves when we allow models to consider many of their own samples before predicting the validity of one specific possibility. Next, we investigate whether models can be trained to predict "P(IK)", the probability that "I know" the answer to a question, without reference to any particular proposed answer. Models perform well at predicting P(IK) and partially generalize across tasks, though they struggle with calibration of P(IK) on new tasks. The predicted P(IK) probabilities also increase appropriately in the presence of relevant source materials in the context, and in the presence of hints towards the solution of mathematical word problems. We hope these observations lay the groundwork for training more honest models, and for investigating how honesty generalizes to cases where models are trained on objectives other than the imitation of human writing., Comment: 23+17 pages; refs added, typos fixed

We have analyzed by means of scanning tunneling microscopy and spectroscopy the atomic and electronic structure of monolayers of 1T-VxPt1-xSe2 alloys grown by molecular beam epitaxy on epitaxial graphene substrates. We have focused on the composition range (0.1

Vibrations can dynamically stabilize otherwise unstable liquid interfaces and produce new dynamic equilibria, called vibro-equilibria. Typically, the vibrations are homogeneous in the liquid and the liquid interface remains approximately flat. Here, we produce controlled vertical vibration gradients by taking advantage of the resonant oscillations of sinking submerged bubbles. The locally increased amplitude of the vibrations induces a local elevation of the liquid interface that can be controlled and engineered. The mean elevation of the interface can be linked theoretically with the local vibration amplitude by a simple formula that is tested experimentally. In addition, the transport of a floating body at the interface can be induced by secondary flows triggered by the amplitude gradients of the liquid vibrations.

We apply preference modeling and reinforcement learning from human feedback (RLHF) to finetune language models to act as helpful and harmless assistants. We find this alignment training improves performance on almost all NLP evaluations, and is fully compatible with training for specialized skills such as python coding and summarization. We explore an iterated online mode of training, where preference models and RL policies are updated on a weekly cadence with fresh human feedback data, efficiently improving our datasets and models. Finally, we investigate the robustness of RLHF training, and identify a roughly linear relation between the RL reward and the square root of the KL divergence between the policy and its initialization. Alongside our main results, we perform peripheral analyses on calibration, competing objectives, and the use of OOD detection, compare our models with human writers, and provide samples from our models using prompts appearing in recent related work., Comment: Data available at https://github.com/anthropics/hh-rlhf

Monitoring the Covid19 pandemic constitutes a critical societal stake that received considerable research efforts. The intensity of the pandemic on a given territory is efficiently measured by the reproduction number, quantifying the rate of growth of daily new infections. Recently, estimates for the time evolution of the reproduction number were produced using an inverse problem formulation with a nonsmooth functional minimization. While it was designed to be robust to the limited quality of the Covid19 data (outliers, missing counts), the procedure lacks the ability to output credibility interval based estimates. This remains a severe limitation for practical use in actual pandemic monitoring by epidemiologists that the present work aims to overcome by use of Monte Carlo sampling. After interpretation of the nonsmooth functional into a Bayesian framework, several sampling schemes are tailored to adjust the nonsmooth nature of the resulting posterior distribution. The originality of the devised algorithms stems from combining a Langevin Monte Carlo sampling scheme with Proximal operators. Performance of the new algorithms in producing relevant credibility intervals for the reproduction number estimates and denoised counts are compared. Assessment is conducted on real daily new infection counts made available by the Johns Hopkins University. The interest of the devised monitoring tools are illustrated on Covid19 data from several different countries.

Electrospray ion-beam deposition (ES-IBD) is a versatile tool to study structure and reactivity of molecules from small metal clusters to large protein assemblies. It brings molecules gently into the gas phase where they can be accurately manipulated and purified, followed by controlled deposition onto various substrates. In combination with imaging techniques, direct structural information of well-defined molecules can be obtained, which is essential to test and interpret results from indirect mass spectrometry techniques. To date, ion-beam deposition experiments are limited to a small number of custom instruments worldwide, and there are no commercial alternatives. Here we present a module that adds ion-beam deposition capabilities to a popular commercial MS platform (Thermo Scientific$^{\mathrm{TM}}$ Q Exactive$^{\mathrm{TM}}$ UHMR). This combination significantly reduces the overhead associated with custom instruments, while benefiting from established high performance and reliability. We present current performance characteristics including beam intensity, landing-energy control, and deposition spot size for a broad range of molecules. In combination with atomic force microscopy (AFM) and transmission electron microscopy (TEM), we distinguish near-native from unfolded proteins and show retention of native shape of protein assemblies after dehydration and deposition. Further, we use an enzymatic assay to quantify activity of an non-covalent protein complex after deposition an a dry surface. Together, these results indicate a great potential of ES-IBD for applications in structural biology, but also outline the challenges that need to be solved for it to reach its full potential.

In this article, we propose a reliable and low-latency Long Range Wide Area Network (LoRaWAN) solution for environmental monitoring in factories at major accident risk (FMAR). In particular, a low power wearable device for sensing the toxic inflammable gases inside an industrial plant is designed with the purpose of avoiding peculiar risks and unwanted accidents to occur. Moreover, the detected data have to be urgently and reliably delivered to remote server to trigger preventive immediate actions so as to improve the machine operation. In these settings, LoRaWAN has been identified as the most proper communications technology to the needs owing to the availability of off the shelf devices and software. Hence, we assess the technological limits of LoRaWAN in terms of latency and reliability and we propose a fully LoRaWAN compliant solution to overcome these limits. The proposed solution envisages coordinated end device (ED) transmissions through the use of Downlink Control Packets (DCPs). Experimental results validate the proposed method in terms of service requirements for the considered FMAR scenario.

Large-scale pre-training has recently emerged as a technique for creating capable, general purpose, generative models such as GPT-3, Megatron-Turing NLG, Gopher, and many others. In this paper, we highlight a counterintuitive property of such models and discuss the policy implications of this property. Namely, these generative models have an unusual combination of predictable loss on a broad training distribution (as embodied in their "scaling laws"), and unpredictable specific capabilities, inputs, and outputs. We believe that the high-level predictability and appearance of useful capabilities drives rapid development of such models, while the unpredictable qualities make it difficult to anticipate the consequences of model deployment. We go through examples of how this combination can lead to socially harmful behavior with examples from the literature and real world observations, and we also perform two novel experiments to illustrate our point about harms from unpredictability. Furthermore, we analyze how these conflicting properties combine to give model developers various motivations for deploying these models, and challenges that can hinder deployment. We conclude with a list of possible interventions the AI community may take to increase the chance of these models having a beneficial impact. We intend this paper to be useful to policymakers who want to understand and regulate AI systems, technologists who care about the potential policy impact of their work, and academics who want to analyze, critique, and potentially develop large generative models., Comment: Updated to reflect the version submitted (and accepted) to ACM FAccT '22. This update incorporates feedback from peer-review and fixes minor typos. See open access FAccT conference version at: https://dl.acm.org/doi/abs/10.1145/3531146.3533229

Monitoring the evolution of the Covid19 pandemic constitutes a critical step in sanitary policy design. Yet, the assessment of the pandemic intensity within the pandemic period remains a challenging task because of the limited quality of data made available by public health authorities (missing data, outliers and pseudoseasonalities, notably), that calls for cumbersome and ad-hoc preprocessing (denoising) prior to estimation. Recently, the estimation of the reproduction number, a measure of the pandemic intensity, was formulated as an inverse problem, combining data-model fidelity and space-time regularity constraints, solved by nonsmooth convex proximal minimizations. Though promising, that formulation lacks robustness against the limited quality of the Covid19 data and confidence assessment. The present work aims to address both limitations: First, it discusses solutions to produce a robust assessment of the pandemic intensity by accounting for the low quality of the data directly within the inverse problem formulation. Second, exploiting a Bayesian interpretation of the inverse problem formulation, it devises a Monte Carlo sampling strategy, tailored to a nonsmooth log-concave a posteriori distribution, to produce relevant credibility intervalbased estimates for the Covid19 reproduction number. Clinical relevance Applied to daily counts of new infections made publicly available by the Health Authorities for around 200 countries, the proposed procedures permit robust assessments of the time evolution of the Covid19 pandemic intensity, updated automatically and on a daily basis.

A remarkable result from integral geometry is Cauchy's formula, which relates the mean path length of ballistic trajectories randomly crossing a convex 2D domain, to the ratio between the region area and its perimeter. This theorem has been generalized for non-convex domains and extended to the case of Brownian motion to find many applications in various fields including biological locomotion and wave physics. Here, we generalize the theorem to arbitrary closed trajectories exploring arbitrary domains. We demonstrate that, regardless of the complexity of the trajectory, the mean arc length still satisfies Cauchy's formula provided that no trajectory is entirely contained in the domain. Below this threshold, the mean arc length decreases with the size of the trajectory. In this case, an approximate analytical formula can still be given for convex trajectories intersecting convex domains provided they are small in comparison. To validate our analysis, we performed numerical simulations of different types of trajectories exploring arbitrary 2D domains. Our results could be applied to retrieve geometric information of bounded domains from the mean first entrance-exit length., Comment: 14 pages, 5 figures

There has been a significant progress in detecting out-of-distribution (OOD) inputs in neural networks recently, primarily due to the use of large models pretrained on large datasets, and an emerging use of multi-modality. We show a severe adversarial vulnerability of even the strongest current OOD detection techniques. With a small, targeted perturbation to the input pixels, we can change the image assignment from an in-distribution to an out-distribution, and vice versa, easily. In particular, we demonstrate severe adversarial vulnerability on the challenging near OOD CIFAR-100 vs CIFAR-10 task, as well as on the far OOD CIFAR-100 vs SVHN. We study the adversarial robustness of several post-processing techniques, including the simple baseline of Maximum of Softmax Probabilities (MSP), the Mahalanobis distance, and the newly proposed \textit{Relative} Mahalanobis distance. By comparing the loss of OOD detection performance at various perturbation strengths, we demonstrate the beneficial effect of using ensembles of OOD detectors, and the use of the \textit{Relative} Mahalanobis distance over other post-processing methods. In addition, we show that even strong zero-shot OOD detection using CLIP and multi-modality suffers from a severe lack of adversarial robustness as well. Our code is available at https://github.com/stanislavfort/adversaries_to_OOD_detection, Comment: 8 pages

The results of a study on the evolution of HiPco single-walled carbon nanotubes during the oxidizing action of H2SO4 and HNO3 are presented. The process conditions used have been chosen so as to avoid any significant damage to the nanotube structure. The type and level of functionalization, the location of the grafted functions on the surface of the nanotube and the changes in morphological characteristics of the samples were examined by using a wide and complementary range of analytical techniques. We propose an explanation for the differences in the oxidizing action of sulfuric and nitric acids. The combined results allow us to suggest possible reaction mechanisms that occur on the surface of the nanotube., Comment: {\'E}quipe 104 : Nanomat{\'e}riaux

The Expectation Maximization (EM) algorithm is the default algorithm for inference in latent variable models. As in any other field of machine learning, applications of latent variable models to very large datasets make the use of advanced parallel and distributed architectures mandatory. This paper introduces FedEM, which is the first extension of the EM algorithm to the federated learning context. FedEM is a new communication efficient method, which handles partial participation of local devices, and is robust to heterogeneous distributions of the datasets. To alleviate the communication bottleneck, FedEM compresses appropriately defined complete data sufficient statistics. We also develop and analyze an extension of FedEM to further incorporate a variance reduction scheme. In all cases, we derive finite-time complexity bounds for smooth non-convex problems. Numerical results are presented to support our theoretical findings, as well as an application to federated missing values imputation for biodiversity monitoring.

The time reversal symmetry of the wave equation allows wave refocusing back at the source. However, this symmetry does not hold in lossy media. We present a new strategy to compensate wave amplitude losses due to attenuation. The strategy leverages the instantaneous time mirror (ITM) which generates reversed waves by a sudden disruption of the medium properties. We create a heterogeneous ITM whose disruption is unequal throughout the space to create waves of different amplitude. The time-reversed waves can then cope with different attenuation paths as typically seen in heterogeneous and lossy environments. We consider an environment with biological tissues and apply the strategy to a two-dimensional digital human phantom from the abdomen. A stronger disruption is introduced where the forward waves suffer a history of higher attenuation, with a weaker disruption elsewhere. Computer simulations show heterogeneous ITM is a promising technique to improve time reversal refocusing in heterogeneous, lossy, and dispersive spaces., Comment: 11 pages, 11 figures

A variety of recent works, spanning pruning, lottery tickets, and training within random subspaces, have shown that deep neural networks can be trained using far fewer degrees of freedom than the total number of parameters. We analyze this phenomenon for random subspaces by first examining the success probability of hitting a training loss sub-level set when training within a random subspace of a given training dimensionality. We find a sharp phase transition in the success probability from $0$ to $1$ as the training dimension surpasses a threshold. This threshold training dimension increases as the desired final loss decreases, but decreases as the initial loss decreases. We then theoretically explain the origin of this phase transition, and its dependence on initialization and final desired loss, in terms of properties of the high-dimensional geometry of the loss landscape. In particular, we show via Gordon's escape theorem, that the training dimension plus the Gaussian width of the desired loss sub-level set, projected onto a unit sphere surrounding the initialization, must exceed the total number of parameters for the success probability to be large. In several architectures and datasets, we measure the threshold training dimension as a function of initialization and demonstrate that it is a small fraction of the total parameters, implying by our theory that successful training with so few dimensions is possible precisely because the Gaussian width of low loss sub-level sets is very large. Moreover, we compare this threshold training dimension to more sophisticated ways of reducing training degrees of freedom, including lottery tickets as well as a new, analogous method: lottery subspaces. Code is available at https://github.com/ganguli-lab/degrees-of-freedom., Comment: ICLR 2022

Mahalanobis distance (MD) is a simple and popular post-processing method for detecting out-of-distribution (OOD) inputs in neural networks. We analyze its failure modes for near-OOD detection and propose a simple fix called relative Mahalanobis distance (RMD) which improves performance and is more robust to hyperparameter choice. On a wide selection of challenging vision, language, and biology OOD benchmarks (CIFAR-100 vs CIFAR-10, CLINC OOD intent detection, Genomics OOD), we show that RMD meaningfully improves upon MD performance (by up to 15% AUROC on genomics OOD).

We experimentally demonstrate a multi-mode interferometer comprising a Bose-Einstein condensate of $^{39}$K atoms trapped in a harmonic potential, where the interatomic interaction can be cancelled exploiting Feshbach resonances. Kapitza-Dirac diffraction from an optical lattice coherently splits the BEC in multiple momentum components equally spaced that form different interferometric paths, closed by the trapping harmonic potential. We investigate two different interferometric schemes, where the recombination pulse is applied after a full or half oscillation in the confining potential. We find that the relative amplitudes of the momentum components at the interferometer output are sensitive to external forces, through the induced displacement of the harmonic potential with respect to the optical lattice. We show how to calibrate the interferometer, fully characterize its output and discuss perspective improvements., Comment: 7 pages, 6 figures

Near out-of-distribution detection (OOD) is a major challenge for deep neural networks. We demonstrate that large-scale pre-trained transformers can significantly improve the state-of-the-art (SOTA) on a range of near OOD tasks across different data modalities. For instance, on CIFAR-100 vs CIFAR-10 OOD detection, we improve the AUROC from 85% (current SOTA) to more than 96% using Vision Transformers pre-trained on ImageNet-21k. On a challenging genomics OOD detection benchmark, we improve the AUROC from 66% to 77% using transformers and unsupervised pre-training. To further improve performance, we explore the few-shot outlier exposure setting where a few examples from outlier classes may be available; we show that pre-trained transformers are particularly well-suited for outlier exposure, and that the AUROC of OOD detection on CIFAR-100 vs CIFAR-10 can be improved to 98.7% with just 1 image per OOD class, and 99.46% with 10 images per OOD class. For multi-modal image-text pre-trained transformers such as CLIP, we explore a new way of using just the names of outlier classes as a sole source of information without any accompanying images, and show that this outperforms previous SOTA on standard vision OOD benchmark tasks., Comment: S.F. and J.R. contributed equally