Search

Dynamic critical fluctuations in magnetic materials encode important information about magnetic ordering in the associated critical exponents. Using nitrogen-vacancy centers in diamond, we implement $T_2$ (spin-decoherence) noise magnetometry to study critical dynamics in a 2D Van der Waals magnet CrSBr. By analyzing NV decoherence on time scales approaching the characteristic correlation time $\tau_c$ of critical fluctuations, we extract the critical exponent $\nu$ for the correlation length. Our result deviates from the Ising prediction and highlights the role of long-range dipolar interactions in 2D CrSBr. Furthermore, analyzing the divergence of the correlation length suggests the possibility of 2D-XY criticality in CrSBr in a temperature window near $T_C$ where static magnetic domains are absent. Our work provides a first demonstration of $T_2$ noise magnetometry to quantitatively analyze critical scaling behavior in 2D materials., Comment: 19 pages main text, 4 main text figures, 26 pages Supplementary Materials, 13 Supplementary figures

The microscopic mechanism of superconductivity in the magic-angle twisted graphene family, including magic-angle twisted trilayer graphene (MATTG), is poorly understood. Properties of MATTG, like Pauli limit violation, suggest unconventional superconductivity. Theoretical studies propose proximal magnetic states in the phase diagram, but direct experimental evidence is lacking. We show direct evidence for an in-plane magnetic order proximal to the superconducting state using two complementary electrical transport measurements. First, we probe the superconducting phase by using statistically significant switching events from superconducting to the dissipative state of MATTG. The system behaves like a network of Josephson junctions due to lattice relaxation-induced moir\'e inhomogeneity in the system. We observe non-monotonic and hysteretic responses in the switching distributions as a function of temperature and in-plane magnetic field. Second, in normal regions doped slightly away from the superconducting regime, we observe hysteresis in magnetoresistance with an in-plane magnetic field; showing evidence for in-plane magnetic order that vanishes $\sim$900 mK. Additionally, we show a broadened Berezinskii-Kosterlitz-Thouless transition due to relaxation-induced moir\'e inhomogeneity. We find superfluid stiffness $J_{\mathrm{s}}$$\sim$0.15 K with strong temperature dependence. Theoretically, the magnetic and superconducting order arising from the magnetic order's fluctuations have been proposed - we show direct evidence for both. Our observation that the hysteretic magnetoresistance is sensitive to the in-plane field may constrain possible intervalley-coherent magnetic orders and the resulting superconductivity that arises from its fluctuations., Comment: 22 pages, 8 figures, and supplementary information

Moir\'e transition metal dichalcogenide (TMD) systems provide a tunable platform for studying electron-correlation driven quantum phases. Such phases have so far been found at rational fillings of the moir\'e superlattice, and it is believed that lattice commensurability plays a key role in their stability. In this work, we show via magnetotransport measurements on twisted WSe2 that new correlated electronic phases can exist away from commensurability. The first phase is an antiferromagnetic metal that is driven by proximity to the van Hove singularity. The second is a re-entrant magnetic field-driven insulator. This insulator is formed from a small and equal density of electrons and holes with opposite spin projections - an Ising excitonic insulator.

The increasing demand for computational photography and imaging on mobile platforms has led to the widespread development and integration of advanced image sensors with novel algorithms in camera systems. However, the scarcity of high-quality data for research and the rare opportunity for in-depth exchange of views from industry and academia constrain the development of mobile intelligent photography and imaging (MIPI). Building on the achievements of the previous MIPI Workshops held at ECCV 2022 and CVPR 2023, we introduce our third MIPI challenge including three tracks focusing on novel image sensors and imaging algorithms. In this paper, we summarize and review the Nighttime Flare Removal track on MIPI 2024. In total, 170 participants were successfully registered, and 14 teams submitted results in the final testing phase. The developed solutions in this challenge achieved state-of-the-art performance on Nighttime Flare Removal. More details of this challenge and the link to the dataset can be found at https://mipi-challenge.org/MIPI2024/., Comment: CVPR 2024 Mobile Intelligent Photography and Imaging (MIPI) Workshop--Nighttime Flare Removal Challenge Report. Website: https://mipi-challenge.org/MIPI2024/

The BigCode project, an open-scientific collaboration focused on the responsible development of Large Language Models for Code (Code LLMs), introduces StarCoder2. In partnership with Software Heritage (SWH), we build The Stack v2 on top of the digital commons of their source code archive. Alongside the SWH repositories spanning 619 programming languages, we carefully select other high-quality data sources, such as GitHub pull requests, Kaggle notebooks, and code documentation. This results in a training set that is 4x larger than the first StarCoder dataset. We train StarCoder2 models with 3B, 7B, and 15B parameters on 3.3 to 4.3 trillion tokens and thoroughly evaluate them on a comprehensive set of Code LLM benchmarks. We find that our small model, StarCoder2-3B, outperforms other Code LLMs of similar size on most benchmarks, and also outperforms StarCoderBase-15B. Our large model, StarCoder2- 15B, significantly outperforms other models of comparable size. In addition, it matches or outperforms CodeLlama-34B, a model more than twice its size. Although DeepSeekCoder- 33B is the best-performing model at code completion for high-resource languages, we find that StarCoder2-15B outperforms it on math and code reasoning benchmarks, as well as several low-resource languages. We make the model weights available under an OpenRAIL license and ensure full transparency regarding the training data by releasing the SoftWare Heritage persistent IDentifiers (SWHIDs) of the source code data.

Large language models (LLMs) excel in many tasks in NLP and beyond, but most open models have very limited coverage of smaller languages and LLM work tends to focus on languages where nearly unlimited data is available for pretraining. In this work, we study the challenges of creating LLMs for Finnish, a language spoken by less than 0.1% of the world population. We compile an extensive dataset of Finnish combining web crawls, news, social media and eBooks. We pursue two approaches to pretrain models: 1) we train seven monolingual models from scratch (186M to 13B parameters) dubbed FinGPT, 2) we continue the pretraining of the multilingual BLOOM model on a mix of its original training data and Finnish, resulting in a 176 billion parameter model we call BLUUMI. For model evaluation, we introduce FIN-bench, a version of BIG-bench with Finnish tasks. We also assess other model qualities such as toxicity and bias. Our models and tools are openly available at https://turkunlp.org/gpt3-finnish., Comment: 17 pages (10 main), 7 figures, 5 tables

Standard superposition theorem has been the basis in the last decades for many power system problems decomposition involving changes in nodal injections, from productions and loads. Its application scope has however been limited to fixed grid topology and breaks as soon as a topology change happens in the grid. For instance, it cannot be applied to compute N-2 power flows simply from N-1 security analysis. Topological changes also become a flexibility used more and more frequently for congestion management. Studying the effect of combinatorial topological changes is hence of interest, but so far very computation intensive. In this paper, we propose an extension of the superposition theorem to varying grid topologies. We demonstrate it under the DC approximation for all topological changes, namely line disconnection and reconnection, bus splitting and merging. We finally apply it to two use cases related to the above mentioned, effectively extending its scope of application.

Gaussian Processes (GPs) offer an attractive method for regression over small, structured and correlated datasets. However, their deployment is hindered by computational costs and limited guidelines on how to apply GPs beyond simple low-dimensional datasets. We propose a framework to identify the suitability of GPs to a given problem and how to set up a robust and well-specified GP model. The guidelines formalise the decisions of experienced GP practitioners, with an emphasis on kernel design and options for computational scalability. The framework is then applied to a case study of glacier elevation change yielding more accurate results at test time., Comment: Accepted at the ICML Workshop on Structured Probabilistic Inference and Generative Modelling (2023)

The COVID-19 pandemic has significantly transformed the healthcare sector, with telehealth services being among the most prominent changes. The adoption of telehealth services, however, has raised new challenges, particularly in the areas of security and privacy. To better comprehend the telehealth needs and concerns of medical professionals, particularly those in private practice, we conducted a study comprised of 20 semi-structured interviews with telehealth practitioners in audiology and speech therapy. Our findings indicate that private telehealth practitioners encounter difficult choices when it comes to balancing security, privacy, usability, and accessibility, particularly while caring for vulnerable populations. Additionally, the study revealed that practitioners face challenges in ensuring HIPAA compliance due to inadequate resources and a lack of technological comprehension. Policymakers and healthcare providers should take proactive measures to address these challenges, including offering resources and training to ensure HIPAA compliance and enhancing technology infrastructure to support secure and accessible telehealth.

Sensitive information is intrinsically tied to interactions in healthcare, and its protection is of paramount importance for achieving high-quality patient outcomes. Research in healthcare privacy and security is predominantly focused on understanding the factors that increase the susceptibility of users to privacy and security breaches. To understand further, we systematically review 26 research papers in this domain to explore the existing user studies in healthcare privacy and security. Following the review, we conducted a card-sorting exercise, allowing us to identify 12 themes integral to this subject such as "Data Sharing," "Risk Awareness," and "Privacy." Further to the identification of these themes, we performed an in-depth analysis of the 26 research papers report on the insights into the discourse within the research community about healthcare privacy and security, particularly from the user perspective.

The current trend of scaling language models involves increasing both parameter count and training dataset size. Extrapolating this trend suggests that training dataset size may soon be limited by the amount of text data available on the internet. Motivated by this limit, we investigate scaling language models in data-constrained regimes. Specifically, we run a large set of experiments varying the extent of data repetition and compute budget, ranging up to 900 billion training tokens and 9 billion parameter models. We find that with constrained data for a fixed compute budget, training with up to 4 epochs of repeated data yields negligible changes to loss compared to having unique data. However, with more repetition, the value of adding compute eventually decays to zero. We propose and empirically validate a scaling law for compute optimality that accounts for the decreasing value of repeated tokens and excess parameters. Finally, we experiment with approaches mitigating data scarcity, including augmenting the training dataset with code data or removing commonly used filters. Models and datasets from our 400 training runs are freely available at https://github.com/huggingface/datablations., Comment: 50 pages (9 main), 39 figures, 15 tables

The recent developments of quantum computing present potential novel pathways for quantum chemistry, as the increased computational power of quantum computers could be harnessed to naturally encode and solve electronic structure problems. Theoretically exact quantum algorithms for chemistry have been proposed (e.g. Quantum Phase Estimation) but the limited capabilities of current noisy intermediate scale quantum devices (NISQ) motivated the development of less demanding hybrid algorithms. In this context, the Variational Quantum Eigensolver (VQE) algorithm was successfully introduced as an effective method to compute the ground state energy of small molecules. The current study investigates the Folded Spectrum (FS) method as an extension to the VQE algorithm for the computation of molecular excited states. It provides the possibility of directly computing excited states around a selected target energy, using the same ansatz as for the ground state calculation. Inspired by the variance-based methods from the Quantum Monte Carlo literature, the FS method minimizes the energy variance, thus requiring a computationally expensive squared Hamiltonian. We alleviate this potentially poor scaling by employing a Pauli grouping procedure, identifying sets of commuting Pauli strings that can be evaluated simultaneously. This allows for a significant reduction of the computational cost. We apply the FS-VQE method to small molecules (H$_2$,LiH), obtaining all electronic excited states with chemical accuracy on ideal quantum simulators.

The online privacy and security of the disabled community is a complex field that has implications for every user who navigates web services. While many disciplines have separately researched the disabled population and their online privacy and security concerns, the overlap between the two is very high but under-researched. Moreover, a complex relationship exists between the disabled population and web services where the interaction depends on several web service developmental factors, including usability and accessibility. To this aid, we explored this intersection of privacy and security of web services as perceived by the disabled community through previous studies by conducting a detailed systematic literature review and analysis of 63 articles. Our findings encompassed several topics, including how the disabled population navigates around authentication interfaces, online privacy concerns, universal design practices, and how security methods such as CAPTCHAs can be improved to become more accessible and usable for people of all needs and abilities. We further discuss the gap in the current research, including solutions such as the universal implementation of inclusive privacy and security tools and protocols.

Pyrocumulonimbus (pyroCb) clouds are storm clouds generated by extreme wildfires. PyroCbs are associated with unpredictable, and therefore dangerous, wildfire spread. They can also inject smoke particles and trace gases into the upper troposphere and lower stratosphere, affecting the Earth's climate. As global temperatures increase, these previously rare events are becoming more common. Being able to predict which fires are likely to generate pyroCb is therefore key to climate adaptation in wildfire-prone areas. This paper introduces Pyrocast, a pipeline for pyroCb analysis and forecasting. The pipeline's first two components, a pyroCb database and a pyroCb forecast model, are presented. The database brings together geostationary imagery and environmental data for over 148 pyroCb events across North America, Australia, and Russia between 2018 and 2022. Random Forests, Convolutional Neural Networks (CNNs), and CNNs pretrained with Auto-Encoders were tested to predict the generation of pyroCb for a given fire six hours in advance. The best model predicted pyroCb with an AUC of $0.90 \pm 0.04$., Comment: 5 pages, 2 figures, Tackling Climate Change with Machine Learning: workshop at NeurIPS 2022

A first causal discovery analysis from observational data of pyroCb (storm clouds generated from extreme wildfires) is presented. Invariant Causal Prediction was used to develop tools to understand the causal drivers of pyroCb formation. This includes a conditional independence test for testing $Y$ conditionally independent of $E$ given $X$ for binary variable $Y$ and multivariate, continuous variables $X$ and $E$, and a greedy-ICP search algorithm that relies on fewer conditional independence tests to obtain a smaller more manageable set of causal predictors. With these tools, we identified a subset of seven causal predictors which are plausible when contrasted with domain knowledge: surface sensible heat flux, relative humidity at $850$ hPa, a component of wind at $250$ hPa, $13.3$ micro-meters, thermal emissions, convective available potential energy, and altitude., Comment: 14 pages 9 figures. To be published in the 2022 NeurIPS Workshop on Causal Machine Learning for Real-World Impact

Disentangling facial movements from other facial characteristics, particularly from facial identity, remains a challenging task, as facial movements display great variation between individuals. In this paper, we aim to characterize individual-specific facial movements. We present a novel training approach to learn facial movements independently of other facial characteristics, focusing on each individual separately. We propose self-supervised Person-Specific Models (PSMs), in which one model per individual can learn to extract an embedding of the facial movements independently of the person's identity and other structural facial characteristics from unlabeled facial video. These models are trained using encoder-decoder-like architectures. We provide quantitative and qualitative evidence that a PSM learns a meaningful facial embedding that discovers fine-grained movements otherwise not characterized by a General Model (GM), which is trained across individuals and characterizes general patterns of facial movements. We present quantitative and qualitative evidence that this approach is easily scalable and generalizable for new individuals: facial movements knowledge learned on a person can quickly and effectively be transferred to a new person. Lastly, we propose a novel PSM using curriculum temporal learning to leverage the temporal contiguity between video frames. Our code, analysis details, and all pretrained models are available in Github and Supplementary Materials., Comment: 9 pages for main text with 6 figures + 5 supplementary figures

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

Text embeddings are commonly evaluated on a small set of datasets from a single task not covering their possible applications to other tasks. It is unclear whether state-of-the-art embeddings on semantic textual similarity (STS) can be equally well applied to other tasks like clustering or reranking. This makes progress in the field difficult to track, as various models are constantly being proposed without proper evaluation. To solve this problem, we introduce the Massive Text Embedding Benchmark (MTEB). MTEB spans 8 embedding tasks covering a total of 58 datasets and 112 languages. Through the benchmarking of 33 models on MTEB, we establish the most comprehensive benchmark of text embeddings to date. We find that no particular text embedding method dominates across all tasks. This suggests that the field has yet to converge on a universal text embedding method and scale it up sufficiently to provide state-of-the-art results on all embedding tasks. MTEB comes with open-source code and a public leaderboard at https://github.com/embeddings-benchmark/mteb., Comment: 24 pages, 14 tables, 6 figures

The Upper Indus Basin, Himalayas provides water for 270 million people and countless ecosystems. However, precipitation, a key component to hydrological modelling, is poorly understood in this area. A key challenge surrounding this uncertainty comes from the complex spatial-temporal distribution of precipitation across the basin. In this work we propose Gaussian processes with structured non-stationary kernels to model precipitation patterns in the UIB. Previous attempts to quantify or model precipitation in the Hindu Kush Karakoram Himalayan region have often been qualitative or include crude assumptions and simplifications which cannot be resolved at lower resolutions. This body of research also provides little to no error propagation. We account for the spatial variation in precipitation with a non-stationary Gibbs kernel parameterised with an input dependent lengthscale. This allows the posterior function samples to adapt to the varying precipitation patterns inherent in the distinct underlying topography of the Indus region. The input dependent lengthscale is governed by a latent Gaussian process with a stationary squared-exponential kernel to allow the function level hyperparameters to vary smoothly. In ablation experiments we motivate each component of the proposed kernel by demonstrating its ability to model the spatial covariance, temporal structure and joint spatio-temporal reconstruction. We benchmark our model with a stationary Gaussian process and a Deep Gaussian processes., Comment: 16th Bayesian Modelling Applications Workshop at UAI, 2022 (Eindhoven, Netherlands)

Masader (Alyafeai et al., 2021) created a metadata structure to be used for cataloguing Arabic NLP datasets. However, developing an easy way to explore such a catalogue is a challenging task. In order to give the optimal experience for users and researchers exploring the catalogue, several design and user experience challenges must be resolved. Furthermore, user interactions with the website may provide an easy approach to improve the catalogue. In this paper, we introduce Masader Plus, a web interface for users to browse Masader. We demonstrate data exploration, filtration, and a simple API that allows users to examine datasets from the backend. Masader Plus can be explored using this link https://arbml.github.io/masader. A video recording explaining the interface can be found here https://www.youtube.com/watch?v=SEtdlSeqchk.