Search

Kin selection theory has proven to be a popular and widely accepted account of how altruistic behaviour can evolve under natural selection. Hamilton's rule, first published in 1964, has since been experimentally validated across a range of different species and social behaviours. In contrast to this large body of work in natural populations, however, there has been relatively little study of kin selection \emph{in silico}. In the current work, we offer what is to our knowledge the first demonstration of kin selection emerging naturally within a population of agents undergoing continuous neuroevolution. Specifically, we find that zero-sum transfer of resources from parents to their infant offspring evolves through kin selection in environments where it is hard for offspring to survive alone. In an additional experiment, we show that kin selection in our simulations relies on a combination of kin recognition and population viscosity. We believe that our work may contribute to the understanding of kin selection in minimal evolutionary systems, without explicit notions of genes and fitness maximisation., Comment: 13 pages, 5 figures

We present the first study of the Public Register of Licensed Persons and Registered Institutions maintained by the Hong Kong Securities and Futures Commission (SFC) through the lens of complex network analysis. This dataset, spanning 21 years with daily granularity, provides a unique view of the evolving social network between licensed professionals and their affiliated firms in Hong Kong's financial sector. Leveraging large language models, we classify firms (e.g., asset managers, banks) and infer the likely nationality and gender of employees based on their names. This application enhances the dataset by adding rich demographic and organizational context, enabling more precise network analysis. Our preliminary findings reveal key structural features, offering new insights into the dynamics of Hong Kong's financial landscape. We release the structured dataset to enable further research, establishing a foundation for future studies that may inform recruitment strategies, policy-making, and risk management in the financial industry., Comment: Complex Networks 2024

Enriching datasets with demographic information, such as gender, race, and age from names, is a critical task in fields like healthcare, public policy, and social sciences. Such demographic insights allow for more precise and effective engagement with target populations. Despite previous efforts employing hidden Markov models and recurrent neural networks to predict demographics from names, significant limitations persist: the lack of large-scale, well-curated, unbiased, publicly available datasets, and the lack of an approach robust across datasets. This scarcity has hindered the development of traditional supervised learning approaches. In this paper, we demonstrate that the zero-shot capabilities of Large Language Models (LLMs) can perform as well as, if not better than, bespoke models trained on specialized data. We apply these LLMs to a variety of datasets, including a real-life, unlabelled dataset of licensed financial professionals in Hong Kong, and critically assess the inherent demographic biases in these models. Our work not only advances the state-of-the-art in demographic enrichment but also opens avenues for future research in mitigating biases in LLMs., Comment: 8 pages, 7 Tables, 5 Figures

Vision Language Models (VLMs) are typically evaluated with Visual Question Answering (VQA) tasks which assess a model's understanding of scenes. Good VQA performance is taken as evidence that the model will perform well on a broader range of tasks that require both visual and language inputs. However, scene-aware VQA does not fully capture input biases or assess hallucinations caused by a misalignment between modalities. To address this, we propose a Cross-modal Alignment Similarity Test (CAST) to probe VLMs for self-consistency across modalities. This test involves asking the models to identify similarities between two scenes through text-only, image-only, or both and then assess the truthfulness of the similarities they generate. Since there is no ground-truth to compare against, this evaluation does not focus on objective accuracy but rather on whether VLMs are internally consistent in their outputs. We argue that while not all self-consistent models are capable or accurate, all capable VLMs must be self-consistent.

Quadratic Voting (QV) is a social choice mechanism that addresses the "tyranny of the majority" of one-person-one-vote mechanisms. Agents express not only their preference ordering but also their preference intensity by purchasing $x$ votes at a cost of $x^2$. Although this pricing rule maximizes utilitarian social welfare and is robust against strategic manipulation, it has not yet found many real-life applications. One key reason is that the original QV mechanism does not limit voter budgets. Two variations have since been proposed: a (no-budget) multiple-issue generalization and a fixed-budget version that allocates a constant number of credits to agents for use in multiple binary elections. While some analysis has been undertaken with respect to the multiple-issue variation, the fixed-budget version has not yet been rigorously studied. In this work, we formally propose a novel fixed-budget multiple-issue QV mechanism. This integrates the advantages of both the aforementioned variations, laying the theoretical foundations for practical use cases of QV, such as multi-agent resource allocation. We analyse our fixed-budget multiple-issue QV by comparing it with traditional voting systems, exploring potential collusion strategies, and showing that checking whether strategy profiles form a Nash equilibrium is tractable., Comment: This preprint is currently under conference peer-review

Retrieval-Augmented Generation (RAG) has emerged as a common paradigm to use Large Language Models (LLMs) alongside private and up-to-date knowledge bases. In this work, we address the challenges of using LLM-as-a-Judge when evaluating grounded answers generated by RAG systems. To assess the calibration and discrimination capabilities of judge models, we identify 7 generator failure modes and introduce GroUSE (Grounded QA Unitary Scoring of Evaluators), a meta-evaluation benchmark of 144 unit tests. This benchmark reveals that existing automated RAG evaluation frameworks often overlook important failure modes, even when using GPT-4 as a judge. To improve on the current design of automated RAG evaluation frameworks, we propose a novel pipeline and find that while closed models perform well on GroUSE, state-of-the-art open-source judges do not generalize to our proposed criteria, despite strong correlation with GPT-4's judgement. Our findings suggest that correlation with GPT-4 is an incomplete proxy for the practical performance of judge models and should be supplemented with evaluations on unit tests for precise failure mode detection. We further show that finetuning Llama-3 on GPT-4's reasoning traces significantly boosts its evaluation capabilities, improving upon both correlation with GPT-4's evaluations and calibration on reference situations.

Surface porosity has been found to be an important property for small bodies. Some asteroids and comets can exhibit an extremely high surface porosity in the first millimeter layer. This layer may be produced by various processes and maintained by the lack of an atmosphere. However, the influence of porosity on the spectro-photometric properties of small body surfaces is not yet fully understood. In this study, we looked into the effect of porosity on the spectro-photometric properties of Phobos regolith spectroscopic simulants; created by mixing the simulants with ultra-pure water, producing ice-dust particles, and then sublimating the water. The reflectance spectroscopic properties in the visible and near-infrared (0.5-4.2 $\mu$m) show no strong variations between the porous and compact samples. However, one simulant exhibits a bluing of the slope after increasing porosity, providing possible insights into the differences between the blue and red units observed on Phobos. In the mid-infrared range, a contrast increase of the 10-$\mu$m emissivity plateau due to silicates is observed. Photometry reveals a modification in the phase reddening behavior between the compact powder and the sublimation residue for both simulants. However, the observed behavior is different between the simulants, suggesting that the phase reddening may be dependent on the composition of the simulants. The phase curve also appears to be modified by the addition of porosity, with a higher contribution of forward scattering observed for the sublimation residue. The derivation of the Hapke parameters indicates an increase in roughness for the porous sample, but no significant modification of the opposition effect. This study aims to provide new insights into the understanding of porosity by using two Phobos simulants in the context of the upcoming JAXA/Martian Moons eXploration mission., Comment: Submitted to Icarus

Laser-plasma accelerators represent a promising technology for future compact accelerating systems, enabling the acceleration of tens of pC to above $1\,$GeV over just a few centimeters. Nonetheless, these devices currently lack the stability, beam quality and average current of conventional systems. While many efforts have focused on improving acceleration stability and quality, little progress has been made in increasing the beam's average current, which is essential for future laser-plasma-based applications. In this paper, we investigate a laser-plasma acceleration regime aimed at increasing the beam average current with energies up to few-MeVs, efficiently enhancing the beam charge. We present experimental results on configurations that allow reaching charges of $5-30\,$nC and a maximum conversion efficiency of around $14\,$%. Through comprehensive Particle-In-Cell simulations, we interpret the experimental results and present a detailed study on electron dynamics. From our analysis, we show that most electrons are not trapped in a plasma wave; rather, they experience ponderomotive acceleration. Thus, we prove the laser pulse as the main driver of the particles' energy gain process.

This paper proposes a novel approach towards image authentication and tampering detection by using watermarking as a communication channel for semantic information. We modify the HiDDeN deep-learning watermarking architecture to embed and extract high-dimensional real vectors representing image captions. Our method improves significantly robustness on both malign and benign edits. We also introduce a local confidence metric correlated with Message Recovery Rate, enhancing the method's practical applicability. This approach bridges the gap between traditional watermarking and passive forensic methods, offering a robust solution for image integrity verification., Comment: Code will be released

Semiconductor nanowires are believed to play a crucial role for future applications in electronics, spintronics and quantum technologies. A potential candidate is HgTe but its sensitivity to nanofabrication processes restrain its development. A way to circumvent this obstacle is the selective area growth technique. Here, in-plane HgTe nanostructures are grown thanks to selective area molecular beam epitaxy on a semi-insulating CdTe substrate covered with a patterned SiO$_{\mathrm{2}}$ mask. The shape of these nanostructures is defined by the in-plane orientation of the mask aperture along the <$110$>, <$1\bar{\mathrm{1}}0$>, or <$100$> direction, the deposited thickness, and the growth temperature. Several micron long in-plane nanowires can be achieved as well as more complex nanostructures such as networks, diamond structures or rings. A good selectivity is achieved with very little parasitic growth on the mask even for a growth temperature as low as $140${\deg}C and growth rate up to $0.5$ ML/s. For <$110$> oriented nanowires, the center of the nanostructure exhibits a trapezoidal shape with {$111$}B facets and two grains on the sides, while <$1\bar{\mathrm{1}}0$> oriented nanowires show {$111$}A facets with adatoms accumulation on the sides of the top surface. Transmission electron microscopy observations reveal a continuous epitaxial relation between the CdTe substrate and the HgTe nanowire. Measurements of the resistance with fourpoint scanning tunneling microscopy indicates a good electrical homogeneity along the main NW axis and a thermally activated transport. This growth method paves the way toward the fabrication of complex HgTe-based nanostructures for electronic transport measurements., Comment: 18 pages and 8 figures. Submitted to Nanotechnology

This article undertakes a comprehensive examination of two distinct robot morphologies: the PUMA-type arm (Programmable Universal Machine for Assembly) and the UR-type robot (Universal Robots). The primary aim of this comparative analysis is to assess their respective performances within the specialized domain of welding, focusing on predefined industrial application scenarios. These scenarios encompass a range of geometrical components earmarked for welding, along with specified welding paths, spatial constraints, and welding methodologies reflective of real-world scenarios encountered by manual welders. The case studies presented in this research serve as illustrative examples of Weez-U Welding practices, providing insights into the practical implications of employing different robot morphologies. Moreover, this study distinguishes between various base positions for the robot, thereby aiding welders in selecting the optimal base placement aligned with their specific welding objectives. By offering such insights, this research facilitates the selection of the most suitable architecture for this particular range of trajectories, thus optimizing welding efficiency and effectiveness. A departure from conventional methodologies, this study goes beyond merely considering singularities and also delves into the analysis of collisions between the robot and its environment, contingent upon the robot's posture. This holistic approach offers a more nuanced understanding of the challenges and considerations inherent in deploying robotic welding systems, providing valuable insights for practitioners and researchers alike in the field of robotic welding technology., Comment: International Design Engineering Technical Conferences & Computers and Information in Engineering Conference, ASME, Aug 2024, Washignton DC, United States

Circular Rydberg atoms (CRAs), i.e., Rydberg atoms with maximal orbital momentum, ideally combine long coherence times and strong interactions, a key property of quantum systems, in particular for the development of quantum technologies. However, the dipole-dipole interaction between CRAs has not been observed so far. We report the measurement and characterization of the resonant dipole-dipole interaction between two CRAs, individually trapped in optical tweezers, and find excellent agreement with theoretical predictions. We demonstrate a dynamic control over the strength of the interaction by tuning the orientation of an electric field. We use the interaction between the CRAs as a meter for the interatomic distance, and record the relative motion between two atoms in their traps. This motion, that we induce through the interaction between Rydberg levels with permanent electric dipoles, transiently populated during the preparation of the circular states, is a signature of spin-motion coupling.

The simplest Weyl semimetal with broken time-reversal symmetry consists of a pair of Weyl nodes located at wave vectors $\mathbf{K}_{\tau }=\tau \mathbf{b}$ in momentum space with $\tau =\pm 1$ the node index and chirality. The electronic dispersion near each node is linear. In a magnetic field $\mathbf{B}$ along $\mathbf{b}$, this dispersion is modified into a series of positive and negative energy Landau levels $n=\pm 1,\pm 2,\ldots ,$which disperse along the direction of the magnetic field, and a chiral Landau level, $n=0$, with a linear dispersion given by $e_{\tau ,n=0}\left( k_{z}\right) =-\tau \hslash v_{F}k_{z},$ where $k_{z}$ is the component of the wave vector $\mathbf{k}$ along the magnetic field and $v_{F}$ is the Fermi velocity. In the extreme quantum limit, the Fermi level is in the chiral levels near the Dirac point. When Coulomb interaction is considered, a Weyl semimetal may be unstable towards the formation of a condensate of internodal electron-hole pairs. In this article we use the full long-range Coulomb interaction and the self-consistent Hartree-Fock approximation to generate the condensed state. We study its stability with respect to a change in the Fermi velocity, doping and strength of the Coulomb interaction and also consider Weyl nodes with higher Chern number $C=2,3$. We derive the response functions of the excitonic state in the generalized random-phase approximation (GRPA). We show that, in the mean-field gap induced by the internodal coherence, there is, in the GRPA excitonic response, a series of bound electron-hole states with a binding energy that decreases until the Hartree-Fock energy gap is reached. In addition, there is a collective mode gapped at exactly the plasmon frequency: the gapless mode present in the proper excitonic response function is pushed to the plasmon frequency by the long-range Coulomb interaction., Comment: 23 pages, 11 figures

Documents are visually rich structures that convey information through text, as well as tables, figures, page layouts, or fonts. While modern document retrieval systems exhibit strong performance on query-to-text matching, they struggle to exploit visual cues efficiently, hindering their performance on practical document retrieval applications such as Retrieval Augmented Generation. To benchmark current systems on visually rich document retrieval, we introduce the Visual Document Retrieval Benchmark ViDoRe, composed of various page-level retrieving tasks spanning multiple domains, languages, and settings. The inherent shortcomings of modern systems motivate the introduction of a new retrieval model architecture, ColPali, which leverages the document understanding capabilities of recent Vision Language Models to produce high-quality contextualized embeddings solely from images of document pages. Combined with a late interaction matching mechanism, ColPali largely outperforms modern document retrieval pipelines while being drastically faster and end-to-end trainable., Comment: Under Review

We propose a multibody model of a freewing UAV. This model allows obtaining simulations of the UAV's behaviour and, in the future, to design a control law stabilising the entire flight envelope (hovering and forward flight). We also describe the realisation of a prototype and a comparison of possible methods for estimating the UAV's states. With this prototype, we report on experimental hovering flights with a non-linear incremental dynamic inversion controller to stabilise the wing and a proportional derivative controller for the fuselage stabilization.

Gravity measurements provide valuable information on the mass distribution below the earth surface relevant to various areas of geosciences such as hydrology, geodesy, geophysics, volcanology, and natural resources management. During the past decades, the needs for sensitivity, robustness, compactness, and transportability of instruments measuring the gravitational acceleration have constantly increased. Today, applications typically call for 1 $\mu \mathrm{Gal} = 10 \mathrm{nm.s}^{-2}$ ~ $10^{-9}$ g resolution on time scales ranging from minutes to years. Absolute Quantum Gravimeters (AQGs) based on matter-wave interferometry with laser-cooled atoms address all these challenges at once, even in uncontrolled environments. Furthermore, to date, quantum gravimeters are the only technology capable of providing continuous absolute gravity data over long measurement durations (~1 day to months or more). In this paper we recall the AQG working principle and present the reproducible high performance at the {\mu}Gal level on all 16 units fabricated so far. We also describe recent progress on the Differential Quantum Gravimeter (DQG) which measures simultaneously the mean gravitational acceleration and its vertical gradient at the level of $10 \mathrm{nm.s}^{-2}$ and $1$ E (1 E[eotvos] = $10^{-9}$ s$^{-2}$) respectively., Comment: 10 pages, 5 figures

Previous observations of Phobos and Deimos, the moons of Mars, have improved our understanding of these small bodies. However, their formation and composition remain poorly constrained. Physical and spectral properties suggest that Phobos may be a weakly thermal-altered captured asteroid but the dynamical properties of the martian system suggest a formation by giant collision similar to the Earth moon. In 2027, the JAXA's MMX mission aims to address these outstanding questions. We undertook measurements with a new simulant called OPPS (Observatory of Paris Phobos Simulant) which closely matches Phobos spectra in the visible to the mid-infrared range. The simulant was synthesized using a mixture of olivine, saponite, anthracite, and coal. Since observation geometry is a crucial aspect of planetary surface remote sensing exploration, we evaluated the parameters obtained by modeling the phase curves -- obtained through laboratory measurements -- of two different Phobos simulants (UTPS-TB and OPPS) using Hapke IMSA model. Our results show that the photometric properties of Phobos simulants are not fully consistent with those of Tagish Lake, Allende, or the NWA 4766 shergottite. We also investigated the detection of volatiles/organic compounds and hydrated minerals, as the presence of such components is expected on Phobos in the hypothesis of a captured primitive asteroid. The results indicate that a significant amount of organic compounds is required for the detection of C-H bands at 3.4 $\mu$m. In contrast, the 2.7 $\mu$m absorption band, due to hydrated minerals, is much deeper and easier to detect than C-H organic features at the same concentration levels. Posing limits on detectability of some possible key components of Phobos surface will be pivotal to prepare and interpret future observations of the MIRS spectrometer onboard MMX mission., Comment: 58 pages, 19 figures, submitted to Icarus

We investigate the sedimentation of chiral particles in viscous fluid flow. We identify helical ribbons as simple particles with strong translation-rotation coupling whose symmetry ensures that the centers of mass, buoyancy, resistance, and mobility coincide. Experimental measurements of both relevant mobility tensors show excellent agreement with simulations of ribbons made of interacting spheres. We observe quasi-periodic angular dynamics causing complex spatial trajectories. In tilt-spin phase space, orbits are closed due to time-reversal and reflection symmetry. Changing the helical ribbon length reveals a bifurcation at which the stable sedimentation orientations switch., Comment: 7 pages, 6 figures

Hyperspectral Imagery (HSI) has been used in many applications to non-destructively determine the material and/or chemical compositions of samples. There is growing interest in creating 3D hyperspectral reconstructions, which could provide both spatial and spectral information while also mitigating common HSI challenges such as non-Lambertian surfaces and translucent objects. However, traditional 3D reconstruction with HSI is difficult due to technological limitations of hyperspectral cameras. In recent years, Neural Radiance Fields (NeRFs) have seen widespread success in creating high quality volumetric 3D representations of scenes captured by a variety of camera models. Leveraging recent advances in NeRFs, we propose computing a hyperspectral 3D reconstruction in which every point in space and view direction is characterized by wavelength-dependent radiance and transmittance spectra. To evaluate our approach, a dataset containing nearly 2000 hyperspectral images across 8 scenes and 2 cameras was collected. We perform comparisons against traditional RGB NeRF baselines and apply ablation testing with alternative spectra representations. Finally, we demonstrate the potential of hyperspectral NeRFs for hyperspectral super-resolution and imaging sensor simulation. We show that our hyperspectral NeRF approach enables creating fast, accurate volumetric 3D hyperspectral scenes and enables several new applications and areas for future study., Comment: Main paper: 15 pages + 2 pages references. Supplemental/Appendix: 6 pages

We present a framework that combines the adjoint state method together with reverse-time back-propagation to solve prohibitively large open-system quantum control problems. Our approach enables the optimization of arbitrary cost functions with fully general controls applied on large open quantum systems described by a Lindblad master equation. It is scalable, computationally efficient, and has a low memory footprint. We apply this framework to optimize two inherently dissipative operations in superconducting qubits which lag behind in terms of fidelity and duration compared to other unitary operations: the dispersive readout and all-microwave reset of a transmon qubit. Our results show that, given a fixed set of system parameters, shaping the control pulses can yield 2x improvements in the fidelity and duration for both of these operations compared to standard strategies. Our approach can readily be applied to optimize quantum controls in a vast range of applications such as reservoir engineering, autonomous quantum error correction, and leakage-reduction units., Comment: 5 pages, 3 figures

This study aims to analyze Phobos' photometric properties using Mars Express mission observations to support the Martian Moons eXploration mission. We analyzed resolved images of Phobos acquired between 2004 and 2022 by the HRSC and the SRC cameras on board the Mars Express spacecraft. We performed photometric analysis using the Hapke model for both integrated and disk-resolved data. The Phobos phase function has a strong opposition effect due to shadow hiding, with an amplitude and a half-width of the opposition surge of 2.28$\pm$0.03 and 0.0573$\pm$0.0001, respectively. Overall, the surface of Phobos is dark, with a geometric albedo of 6.8 % in the green filter and backscattering. We also found a surface porosity of 87\%, indicating the presence of a thick dust mantle or of fractal aggregates on the top surface. The SSA maps revealed high reflectance variability, with the blue unit area in the northeast Stickney rim being up to 65\% brighter than average, while the Stickney floor is among the darkest regions, with reflectance 10 to 20% lower than average. Photometric modeling of the regions of interest selected in the red and blue units indicates that red unit terrains have a stronger opposition effect and a smaller SSA value than the blue ones, but they have similar porosity and backscattering properties. The HRSC data provide a unique investigation of the Phobos phase function and opposition surge, which is valuable information for the MMX observational planning. The Phobos opposition surge, surface porosity, phase integral, and spectral slope are very similar to the values observed for the comet 67P and for Jupiter family comets in general. Based on these similarities, we formulate a hypothesis that the Mars satellites might be the results of a binary or bilobated comet captured by Mars., Comment: 30 pages, 16 figures, accepted for publication in Astron. & Astroph

We successfully demonstrated a fully-managed, field-deployed, three-node QKD ring network with L1-OTNsec encryption, that employs a hybrid scheme of QKD and classical yet quantum-safe centrally-generated symmetric keys to support point-to-point and relay consumers.

In environments like offices, the duration of a robot's navigation between two locations may vary over time. For instance, reaching a kitchen may take more time during lunchtime since the corridors are crowded with people heading the same way. In this work, we address the problem of routing in such environments with tasks expressed in Metric Interval Temporal Logic (MITL) - a rich robot task specification language that allows us to capture explicit time requirements. Our objective is to find a strategy that maximizes the temporal robustness of the robot's MITL task. As the first step towards a solution, we define a Mixed-integer linear programming approach to solving the task planning problem over a Varying Weighted Transition System, where navigation durations are deterministic but vary depending on the time of day. Then, we apply this planner to optimize for MITL temporal robustness in Markov Decision Processes, where the navigation durations between physical locations are uncertain, but the time-dependent distribution over possible delays is known. Finally, we develop a receding horizon planner for Markov Decision Processes that preserves guarantees over MITL temporal robustness. We show the scalability of our planning algorithms in simulations of robotic tasks., Comment: ICRA 2024

The research field of Artificial Life studies how life-like phenomena such as autopoiesis, agency, or self-regulation can self-organize in computer simulations. In cellular automata (CA), a key open-question has been whether it it is possible to find environment rules that self-organize robust "individuals" from an initial state with no prior existence of things like "bodies", "brain", "perception" or "action". In this paper, we leverage recent advances in machine learning, combining algorithms for diversity search, curriculum learning and gradient descent, to automate the search of such "individuals", i.e. localized structures that move around with the ability to react in a coherent manner to external obstacles and maintain their integrity, hence primitive forms of sensorimotor agency. We show that this approach enables to find systematically environmental conditions in CA leading to self-organization of such basic forms of agency. Through multiple experiments, we show that the discovered agents have surprisingly robust capabilities to move, maintain their body integrity and navigate among various obstacles. They also show strong generalization abilities, with robustness to changes of scale, random updates or perturbations from the environment not seen during training. We discuss how this approach opens new perspectives in AI and synthetic bioengineering.

Tokenization is an understudied and often neglected component of modern LLMs. Most published works use a single tokenizer for all experiments, often borrowed from another model, without performing ablations or analysis to optimize tokenization. Moreover, the tokenizer is generally kept unchanged when fine-tuning a base model. In this paper, we show that the size, pre-tokenization regular expression, and training data of a tokenizer can significantly impact the model's generation speed, effective context size, memory usage, and downstream performance. We train specialized Byte-Pair Encoding code tokenizers, and conduct extensive ablations on the impact of tokenizer design on the performance of LLMs for code generation tasks such as HumanEval and MBPP, and provide recommendations for tokenizer hyper-parameters selection and switching the tokenizer in a pre-trained LLM. We perform our experiments on models trained from scratch and from pre-trained models, verifying their applicability to a wide range of use-cases. We find that when fine-tuning on more than 50 billion tokens, we can specialize the tokenizer of a pre-trained LLM to obtain large gains in generation speed and effective context size.

We introduce CroissantLLM, a 1.3B language model pretrained on a set of 3T English and French tokens, to bring to the research and industrial community a high-performance, fully open-sourced bilingual model that runs swiftly on consumer-grade local hardware. To that end, we pioneer the approach of training an intrinsically bilingual model with a 1:1 English-to-French pretraining data ratio, a custom tokenizer, and bilingual finetuning datasets. We release the training dataset, notably containing a French split with manually curated, high-quality, and varied data sources. To assess performance outside of English, we craft a novel benchmark, FrenchBench, consisting of an array of classification and generation tasks, covering various orthogonal aspects of model performance in the French Language. Additionally, rooted in transparency and to foster further Large Language Model research, we release codebases, and dozens of checkpoints across various model sizes, training data distributions, and training steps, as well as fine-tuned Chat models, and strong translation models. We evaluate our model through the FMTI framework, and validate 81 % of the transparency criteria, far beyond the scores of even most open initiatives. This work enriches the NLP landscape, breaking away from previous English-centric work in order to strengthen our understanding of multilinguality in language models.

It is challenging to select the right privacy-preserving mechanism for federated query processing over multiple private data silos. There exist numerous privacy-preserving mechanisms, such as secure multi-party computing (SMC), approximate query processing with differential privacy (DP), combined SMC and DP, DP-based data obfuscation, and federated learning. These mechanisms make different trade-offs among accuracy, privacy, execution efficiency, and storage efficiency. In this work, we first introduce a new privacy-preserving technique that uses a deep learning model trained using the Differentially-Private Stochastic Gradient Descent (DP-SGD) algorithm to replace portions of actual data to answer a query. We then demonstrate a novel declarative privacy-preserving workflow that allows users to specify "what private information to protect" rather than "how to protect". Under the hood, the system relies on a cost model to automatically choose privacy-preserving mechanisms as well as hyper-parameters. At the same time, the proposed workflow also allows human experts to review and tune the selected privacy-preserving mechanism for audit/compliance, and optimization purposes.