Search

Membership inference attacks (MIAs) are widely used to empirically assess the privacy risks of samples used to train a target machine learning model. State-of-the-art methods however require training hundreds of shadow models, with the same size and architecture of the target model, solely to evaluate the privacy risk. While one might be able to afford this for small models, the cost often becomes prohibitive for medium and large models. We here instead propose a novel approach to identify the at-risk samples using only artifacts available during training, with little to no additional computational overhead. Our method analyzes individual per-sample loss traces and uses them to identify the vulnerable data samples. We demonstrate the effectiveness of our artifact-based approach through experiments on the CIFAR10 dataset, showing high precision in identifying vulnerable samples as determined by a SOTA shadow model-based MIA (LiRA). Impressively, our method reaches the same precision as another SOTA MIA when measured against LiRA, despite it being orders of magnitude cheaper. We then show LT-IQR to outperform alternative loss aggregation methods, perform ablation studies on hyperparameters, and validate the robustness of our method to the target metric. Finally, we study the evolution of the vulnerability score distribution throughout training as a metric for model-level risk assessment.

Whether LLMs memorize their training data and what this means, from privacy leakage to detecting copyright violations -- has become a rapidly growing area of research over the last two years. In recent months, more than 10 new methods have been proposed to perform Membership Inference Attacks (MIAs) against LLMs. Contrary to traditional MIAs which rely on fixed -- but randomized -- records or models, these methods are mostly evaluated on datasets collected post-hoc. Sets of members and non-members, used to evaluate the MIA, are constructed using informed guesses after the release of a model. This lack of randomization raises concerns of a distribution shift between members and non-members. In the first part, we review the literature on MIAs against LLMs. While most work focuses on sequence-level MIAs evaluated in post-hoc setups, we show that a range of target models, motivations and units of interest have been considered in the literature. We then quantify distribution shifts present in the 6 datasets used in the literature, ranging from books to papers, using a bag of word classifier. Our analysis reveals that all of them suffer from severe distribution shifts. This challenges the validity of using such setups to measure LLM memorization and may undermine the benchmarking of recently proposed methods. Yet, all hope might not be lost. In the second part, we introduce important considerations to properly evaluate MIAs against LLMs and discuss potential ways forward: randomized test splits, injections of randomized (unique) sequences, randomized finetuning, and post-hoc control methods. While each option comes with its advantages and limitations, we believe they collectively provide solid grounds to guide the development of MIA methods and study LLM memorization. We conclude by proposing comprehensive, easy-to-use benchmarks for sequence- and document-level MIAs against LLMs.

Multilayer van der Waals (vdW) heterostructures have become an important platform in which to study novel fundamental effects emerging at the nanoscale. Standard nanopatterning techniques relying on electron-beam lithography and reactive ion etching, widely applied to pattern such heterostructures, however, impose some limitations on the edge accuracy and resolution, as revealed through numerous experiments with vdW quantum dots and point contacts. Here we present an alternative approach for electrode-free nanopatterning of thick multilayer vdW heterostructures based on atomic force microscopy (AFM). By applying an AC voltage of a relatively small frequency (1-10 kHz) between the sharp platinum tip and the substrate, we realize high-resolution ($\lesssim 100$ nm) etching of thick multicomponent heterostructures if the latter are deposited onto graphite slabs. Importantly, unlike more conventional electrode-free local anodic oxidation, our method does not require a special environment with excess humidity, can be applied at ambient conditions, and enables the patterning of multilayer heterostructures composed of graphene, graphite, hexagonal boron nitride (hBN), NbSe$_{2}$, WSe$_{2}$, and more., Comment: 6 pages; 3 figures

Differential privacy upper-bounds the information leakage of machine learning models, yet providing meaningful privacy guarantees has proven to be challenging in practice. The private prediction setting where model outputs are privatized is being investigated as an alternate way to provide formal guarantees at prediction time. Most current private prediction algorithms, however, rely on global sensitivity for noise calibration, which often results in large amounts of noise being added to the predictions. Data-specific noise calibration, such as smooth sensitivity, could significantly reduce the amount of noise added, but were so far infeasible to compute exactly for modern machine learning models. In this work we provide a novel and practical approach based on convex relaxation and bound propagation to compute a provable upper-bound for the local and smooth sensitivity of a prediction. This bound allows us to reduce the magnitude of noise added or improve privacy accounting in the private prediction setting. We validate our framework on datasets from financial services, medical image classification, and natural language processing and across models and find our approach to reduce the noise added by up to order of magnitude., Comment: 16 pages, 11 figures

The immense datasets used to develop Large Language Models (LLMs) often include copyright-protected content, typically without the content creator's consent. Copyright traps have been proposed to be injected into the original content, improving content detectability in newly released LLMs. Traps, however, rely on the exact duplication of a unique text sequence, leaving them vulnerable to commonly deployed data deduplication techniques. We here propose the generation of fuzzy copyright traps, featuring slight modifications across duplication. When injected in the fine-tuning data of a 1.3B LLM, we show fuzzy trap sequences to be memorized nearly as well as exact duplicates. Specifically, the Membership Inference Attack (MIA) ROC AUC only drops from 0.90 to 0.87 when 4 tokens are replaced across the fuzzy duplicates. We also find that selecting replacement positions to minimize the exact overlap between fuzzy duplicates leads to similar memorization, while making fuzzy duplicates highly unlikely to be removed by any deduplication process. Lastly, we argue that the fact that LLMs memorize across fuzzy duplicates challenges the study of LLM memorization relying on naturally occurring duplicates. Indeed, we find that the commonly used training dataset, The Pile, contains significant amounts of fuzzy duplicates. This introduces a previously unexplored confounding factor in post-hoc studies of LLM memorization, and questions the effectiveness of (exact) data deduplication as a privacy protection technique.

Light incident upon materials can induce changes in their electrical conductivity, a phenomenon referred to as photoresistance. In semiconductors, the photoresistance is negative, as light-induced promotion of electrons across the band gap enhances the number of charge carriers participating in transport. In superconductors, the photoresistance is positive because of the destruction of the superconducting state, whereas in normal metals it is vanishing. Here we report a qualitative deviation from the standard behavior in metallic graphene. We show that Dirac electrons exposed to continuous wave (CW) terahertz (THz) radiation can be thermally decoupled from the lattice by 50~K which activates hydrodynamic electron transport. In this regime, the resistance of graphene constrictions experiences a decrease caused by the THz-driven superballistic flow of correlated electrons. We analyze the dependencies of the negative photoresistance on the carrier density, and the radiation power and show that our superballistic devices operate as sensitive phonon-cooled bolometers and can thus offer a picosecond-scale response time. Beyond their fundamental implications, our findings underscore the practicality of electron hydrodynamics in designing ultra-fast THz sensors and electron thermometers., Comment: 7 pages, 3 figures

Questions of fair use of copyright-protected content to train Large Language Models (LLMs) are being actively debated. Document-level inference has been proposed as a new task: inferring from black-box access to the trained model whether a piece of content has been seen during training. SOTA methods however rely on naturally occurring memorization of (part of) the content. While very effective against models that memorize significantly, we hypothesize--and later confirm--that they will not work against models that do not naturally memorize, e.g. medium-size 1B models. We here propose to use copyright traps, the inclusion of fictitious entries in original content, to detect the use of copyrighted materials in LLMs with a focus on models where memorization does not naturally occur. We carefully design a randomized controlled experimental setup, inserting traps into original content (books) and train a 1.3B LLM from scratch. We first validate that the use of content in our target model would be undetectable using existing methods. We then show, contrary to intuition, that even medium-length trap sentences repeated a significant number of times (100) are not detectable using existing methods. However, we show that longer sequences repeated a large number of times can be reliably detected (AUC=0.75) and used as copyright traps. Beyond copyright applications, our findings contribute to the study of LLM memorization: the randomized controlled setup enables us to draw causal relationships between memorization and certain sequence properties such as repetition in model training data and perplexity., Comment: 41st International Conference on Machine Learning (ICML 2024)

Hafnia-based nanostructures and other high-k dielectrics are promising wide-gap materials for developing new opto- and nanoelectronics devices. They possess a unique combination of physical and chemical properties such as insensitivity to electrical and optical degradation, radiation damage stability, a high specific surface area, and an increased concentration of the appropriate active electron-hole centers. The present paper aims to investigate the structural, optical, and luminescent properties of anodized non-stoichiometric $HfO_2$ nanotubes. As-grown amorphous hafnia nanotubes and nanotubes annealed at 700{\deg}C with a monoclinic crystal lattice served as samples. It has been shown that the bandgap $E_g$ for direct allowed transitions amounts to $5.65\pm0.05$ eV for amorphous and $5.51\pm0.05$ eV for monoclinic nanotubes. For the first time, we have studied the features of the intrinsic cathodoluminescence and photoluminescence of the obtained nanotubular $HfO_2$ structures with an atomic deficiency in the anion sublattice at temperatures of 10 and 300 K. A broad emission band with a maximum of 2.3-2.4 eV has been revealed. We have also conducted an analysis of the kinetic dependencies of the observed photoluminescence for synthesized $HfO_2$ samples in the millisecond range at room temperature. It showed that there are several types of optically active capture and emission centers based on vacancy states in the $O_{3f}$ and $O_{4f}$ positions with different coordination numbers and a varied number of localized charge carriers ($V^0$, $V^-$, and $V^{2-}$). The uncovered regularities can be used to optimize the functional characteristics of developed-surface luminescent media based on nanotubular and nanoporous modifications of hafnia., Comment: 15 pages, 6 figures, 3 tables, 50 references

Twist-controlled moire superlattices (MS) have emerged as a versatile platform in which to realize artificial systems with complex electronic spectra. Bernal-stacked bilayer graphene (BLG) and hexagonal boron nitride (hBN) form an interesting example of the MS that has recently featured a set of unexpected behaviors, such as unconventional ferroelectricity and electronic ratchet effect. Yet, the understanding of the BLG/hBN MS electronic properties has, at present, remained fairly limited. Here we develop a multi-messenger approach that combines standard magnetotransport techniques with low-energy sub-THz excitation to get insights into the properties of this MS. We show that BLG/hBN lattice alignment results in the emergence of compensated semimetals at some integer fillings of the moire bands separated by van Hove singularities where Lifshitz transition occurs. A particularly pronounced semimetal develops when 8 electrons reside in the moire unit cell, where coexisting high-mobility electron and hole systems feature a strong magnetoresistance reaching 2350 % already at B=0.25 T. Next, by measuring the THz-driven Nernst effect in remote bands, we observe valley splitting, pointing to an orbital magnetization characterized by a strongly enhanced effective g-factor of 340. Last, using THz photoresistance measurements, we show that the high-temperature conductivity of the BLG/hBN MS is limited by electron-electron umklapp processes. Our multi-facet analysis introduces THz-driven magnetotransport as a convenient tool to probe the band structure and interaction effects in vdW materials and provides a comprehension of the BLG/hBN MS., Comment: 9 pages, 4 figures

Elegant Objects (EO) is a variation of the object-oriented programming paradigm that favors pure objects and decoration. EO programming language is based on these ideas and has been suggested by Bugayenko as an intermediate representation for object-oriented programs. This paper provides plausible representations in EO of some class-based constructions from Java, C++, and Python. We then reformulate the classical fragile base class problem in the context of these representations. Finally, we discuss an algorithm for detecting a subset of fragile base class patterns in EO programs. We show that using EO as an intermediate language is plausible and discuss possible improvements to the language to assist in richer static analysis.