Search

Machine learning's widespread adoption in decision-making processes raises concerns about fairness, particularly regarding the treatment of sensitive features and potential discrimination against minorities. The software engineering community has responded by developing fairness-oriented metrics, empirical studies, and approaches. However, there remains a gap in understanding and categorizing practices for engineering fairness throughout the machine learning lifecycle. This paper presents a novel catalog of practices for addressing fairness in machine learning derived from a systematic mapping study. The study identifies and categorizes 28 practices from existing literature, mapping them onto different stages of the machine learning lifecycle. From this catalog, the authors extract actionable items and implications for both researchers and practitioners in software engineering. This work aims to provide a comprehensive resource for integrating fairness considerations into the development and deployment of machine learning systems, enhancing their reliability, accountability, and credibility.

Humans are capable of strategically deceptive behavior: behaving helpfully in most situations, but then behaving very differently in order to pursue alternative objectives when given the opportunity. If an AI system learned such a deceptive strategy, could we detect it and remove it using current state-of-the-art safety training techniques? To study this question, we construct proof-of-concept examples of deceptive behavior in large language models (LLMs). For example, we train models that write secure code when the prompt states that the year is 2023, but insert exploitable code when the stated year is 2024. We find that such backdoor behavior can be made persistent, so that it is not removed by standard safety training techniques, including supervised fine-tuning, reinforcement learning, and adversarial training (eliciting unsafe behavior and then training to remove it). The backdoor behavior is most persistent in the largest models and in models trained to produce chain-of-thought reasoning about deceiving the training process, with the persistence remaining even when the chain-of-thought is distilled away. Furthermore, rather than removing backdoors, we find that adversarial training can teach models to better recognize their backdoor triggers, effectively hiding the unsafe behavior. Our results suggest that, once a model exhibits deceptive behavior, standard techniques could fail to remove such deception and create a false impression of safety., Comment: updated to add missing acknowledgements

Foundation models could eventually introduce several pathways for undermining state security: accidents, inadvertent escalation, unintentional conflict, the proliferation of weapons, and the interference with human diplomacy are just a few on a long list. The Confidence-Building Measures for Artificial Intelligence workshop hosted by the Geopolitics Team at OpenAI and the Berkeley Risk and Security Lab at the University of California brought together a multistakeholder group to think through the tools and strategies to mitigate the potential risks introduced by foundation models to international security. Originating in the Cold War, confidence-building measures (CBMs) are actions that reduce hostility, prevent conflict escalation, and improve trust between parties. The flexibility of CBMs make them a key instrument for navigating the rapid changes in the foundation model landscape. Participants identified the following CBMs that directly apply to foundation models and which are further explained in this conference proceedings: 1. crisis hotlines 2. incident sharing 3. model, transparency, and system cards 4. content provenance and watermarks 5. collaborative red teaming and table-top exercises and 6. dataset and evaluation sharing. Because most foundation model developers are non-government entities, many CBMs will need to involve a wider stakeholder community. These measures can be implemented either by AI labs or by relevant government actors.

We test the hypothesis that language models trained with reinforcement learning from human feedback (RLHF) have the capability to "morally self-correct" -- to avoid producing harmful outputs -- if instructed to do so. We find strong evidence in support of this hypothesis across three different experiments, each of which reveal different facets of moral self-correction. We find that the capability for moral self-correction emerges at 22B model parameters, and typically improves with increasing model size and RLHF training. We believe that at this level of scale, language models obtain two capabilities that they can use for moral self-correction: (1) they can follow instructions and (2) they can learn complex normative concepts of harm like stereotyping, bias, and discrimination. As such, they can follow instructions to avoid certain kinds of morally harmful outputs. We believe our results are cause for cautious optimism regarding the ability to train language models to abide by ethical principles.

As language models (LMs) scale, they develop many novel behaviors, good and bad, exacerbating the need to evaluate how they behave. Prior work creates evaluations with crowdwork (which is time-consuming and expensive) or existing data sources (which are not always available). Here, we automatically generate evaluations with LMs. We explore approaches with varying amounts of human effort, from instructing LMs to write yes/no questions to making complex Winogender schemas with multiple stages of LM-based generation and filtering. Crowdworkers rate the examples as highly relevant and agree with 90-100% of labels, sometimes more so than corresponding human-written datasets. We generate 154 datasets and discover new cases of inverse scaling where LMs get worse with size. Larger LMs repeat back a dialog user's preferred answer ("sycophancy") and express greater desire to pursue concerning goals like resource acquisition and goal preservation. We also find some of the first examples of inverse scaling in RL from Human Feedback (RLHF), where more RLHF makes LMs worse. For example, RLHF makes LMs express stronger political views (on gun rights and immigration) and a greater desire to avoid shut down. Overall, LM-written evaluations are high-quality and let us quickly discover many novel LM behaviors., Comment: for associated data visualizations, see https://www.evals.anthropic.com/model-written/ for full datasets, see https://github.com/anthropics/evals

As AI systems become more capable, we would like to enlist their help to supervise other AIs. We experiment with methods for training a harmless AI assistant through self-improvement, without any human labels identifying harmful outputs. The only human oversight is provided through a list of rules or principles, and so we refer to the method as 'Constitutional AI'. The process involves both a supervised learning and a reinforcement learning phase. In the supervised phase we sample from an initial model, then generate self-critiques and revisions, and then finetune the original model on revised responses. In the RL phase, we sample from the finetuned model, use a model to evaluate which of the two samples is better, and then train a preference model from this dataset of AI preferences. We then train with RL using the preference model as the reward signal, i.e. we use 'RL from AI Feedback' (RLAIF). As a result we are able to train a harmless but non-evasive AI assistant that engages with harmful queries by explaining its objections to them. Both the SL and RL methods can leverage chain-of-thought style reasoning to improve the human-judged performance and transparency of AI decision making. These methods make it possible to control AI behavior more precisely and with far fewer human labels.

Welcome to the fifth edition of the AI Index Report! The latest edition includes data from a broad set of academic, private, and nonprofit organizations as well as more self-collected data and original analysis than any previous editions, including an expanded technical performance chapter, a new survey of robotics researchers around the world, data on global AI legislation records in 25 countries, and a new chapter with an in-depth analysis of technical AI ethics metrics. The AI Index Report tracks, collates, distills, and visualizes data related to artificial intelligence. Its mission is to provide unbiased, rigorously vetted, and globally sourced data for policymakers, researchers, executives, journalists, and the general public to develop a more thorough and nuanced understanding of the complex field of AI. The report aims to be the world's most credible and authoritative source for data and insights about AI.

Welcome to the fourth edition of the AI Index Report. This year we significantly expanded the amount of data available in the report, worked with a broader set of external organizations to calibrate our data, and deepened our connections with the Stanford Institute for Human-Centered Artificial Intelligence (HAI). The AI Index Report tracks, collates, distills, and visualizes data related to artificial intelligence. Its mission is to provide unbiased, rigorously vetted, and globally sourced data for policymakers, researchers, executives, journalists, and the general public to develop intuitions about the complex field of AI. The report aims to be the most credible and authoritative source for data and insights about AI in the world.

The Australian bushfires around the turn of the year 2020 generated an unprecedented perturbation of stratospheric composition, dynamical circulation and radiative balance. Here we show from satellite observations that the resulting planetary-scale blocking of solar radiation by the smoke is larger than any previously documented wildfires and of the same order as the radiative forcing produced by moderate volcanic eruptions. A striking effect of the solar heating of an intense smoke patch was the generation of a self-maintained anticyclonic vortex measuring 1000 km in diameter and featuring its own ozone hole. The highly stable vortex persisted in the stratosphere for over 13 weeks, travelled 66,000 km and lifted a confined bubble of smoke and moisture to 35 km altitude. Its evolution was tracked by several satellite-based sensors and was successfully resolved by the European Centre for Medium-Range Weather Forecasts operational system, primarily based on satellite data. Because wildfires are expected to increase in frequency and strength in a changing climate, we suggest that extraordinary events of this type may contribute significantly to the global stratospheric composition in the coming decades., Comment: to appear in Communications Earth & Environment

I show that cooperative exclusion processes with selective kinetic constraints exhibit fluctuation-induced forces that can be attractive or repulsive, depending on the density of boundary reservoirs, when their density-dependent diffusion coefficient exhibits a minimum. A mean-field analysis based on a nonlinear diffusion equation provides an estimation of the magnitude and sign of such a tunable Casimir-like force and suggests its occurrence in interacting particle systems with a diffusivity anomaly.

We introduce and study a family of cooperative exclusion processes whose microscopic dynamics is governed by selective kinetic constraints. They display, in sharp contrast to the simple symmetric exclusion process, density profiles that can be concave, convex or both, depending on the density of boundary particle reservoirs. A mean-field analysis based on a diffusion equation with a density-dependent diffusion coefficient qualitatively reproduces this behaviour, and suggests its occurrence in liquids with a diffusivity anomaly.

A new class of bootstrap percolation models in which particle culling occurs only for certain numbers of nearest neighbours is introduced and studied on a Bethe lattice. Upon increasing the density of initial configuration they undergo multiple hybrid (or mixed-order) phase transitions, showing that such intriguing phase behaviours may also appear in fully homogeneous situations/environments, provided that culling is selective rather than cumulative. The idea immediately extends to facilitation dynamics, suggesting a simple way to construct one-component models of multiple glasses and glass-glass transitions as well as more general coarse-grained models of complex cooperative dynamics.

We study the adsorption-desorption of fluid molecules on a solid substrate by introducing a schematic model in which the adsorption/desorption transition probabilities are given by irreversible kinetic constraints with a tunable violation of local detailed balance condition. Numerical simulations show that in one spatial dimension the model undergoes a continuous nonequilibrium phase transition whose location depends on the irreversibility strength. We show that the hierarchy of equations obeyed by multi-point correlation functions can be closed to the second order by means of a simple decoupling approximation, and that the approximated solution for the steady state yields a very good description of the overall phase diagram.

Packing spheres efficiently in large dimension $d$ is a particularly difficult optimization problem. In this paper we add an isotropic interaction potential to the pure hard-core repulsion, and show that one can tune it in order to maximize a lower bound on packing density. Our results suggest that exponentially many (in the number of particles) distinct disordered sphere packings can be effectively constructed by this method, up to a packing fraction close to $7\, d\, 2^{-d}$. The latter is determined by solving the inverse problem of maximizing the dynamical glass transition over the space of the interaction potentials. Our method crucially exploits a recent exact formulation of the thermodynamics and the dynamics of simple liquids in infinite dimension., Comment: 28 pages, 5 figures, 2 tables. Submission to SciPost. Added clarifications and references

$\beta$ and $\alpha$ relaxation processes are dynamical scaling regimes of glassy systems occurring on two separate time scales which both diverge as the glass state is approached. We study here the crossover scaling from $\beta$- to $\alpha$- relaxation in the cooperative facilitation scenario (CFS) and show that it is quantitatively described, with no adjustable parameter, by the leading order asymptotic formulas for scaling predicted by the mode-coupling theory (MCT). These results establish: (i) the mutual universality of the MCT and CFS, and (ii) the existence of a purely dynamic realization of MCT which is distinct from the well established random-first order transition scenario for disordered systems. Some implications of the emerging kinetic-static duality are discussed.

Bootstrap percolation provides an emblematic instance of phase behavior characterised by an abrupt transition with diverging critical fluctuations. This unusual hybrid situation generally occurs in particle systems in which the occupation probability of a site depends on the state of its neighbours through a certain threshold parameter. In this paper we investigate the phase behavior of the bootstrap percolation on the regular random graph in the limit in which the threshold parameter and lattice connectivity become both increasingly large while their ratio $\alpha$ is held constant. We find that the mixed phase behavior is preserved in this limit, and that multiple transitions and higher-order bifurcation singularities occur when $\alpha$ becomes a random variable., Comment: 6 pages, 4 figures