Search

Large language models (LLMs) have emerged as a new paradigm for Text-to-SQL task. However, the absence of a systematical benchmark inhibits the development of designing effective, efficient and economic LLM-based Text-to-SQL solutions. To address this challenge, in this paper, we first conduct a systematical and extensive comparison over existing prompt engineering methods, including question representation, example selection and example organization, and with these experimental results, we elaborate their pros and cons. Based on these findings, we propose a new integrated solution, named DAIL-SQL, which refreshes the Spider leaderboard with 86.6% execution accuracy and sets a new bar. To explore the potential of open-source LLM, we investigate them in various scenarios, and further enhance their performance with supervised fine-tuning. Our explorations highlight open-source LLMs' potential in Text-to-SQL, as well as the advantages and disadvantages of the supervised fine-tuning. Additionally, towards an efficient and economic LLM-based Text-to-SQL solution, we emphasize the token efficiency in prompt engineering and compare the prior studies under this metric. We hope that our work provides a deeper understanding of Text-to-SQL with LLMs, and inspires further investigations and broad applications., Comment: We have released code on https://github.com/BeachWang/DAIL-SQL

With the ever-growing need to reduce energy consumption, building materials that passively heat or cool are gaining importance. However, many buildings require both heating and cooling, even within the same day. To date, few technologies can automatically switch between passive heating and cooling, and those that can require a large temperature range to cycle states (>15o C), making them ineffective for daily switching. We present a passively adaptive radiative switch that leverages the expansion in phase-change energy storage materials to actuate the motion of louvers and can cycle states in less than 3o C. The black selective-absorber louvers induce high heat gain when closed, yet when open, expose a white, emissive surface for low heat gain. During an outdoor test in which temperature was held steady, our device reduced the energetic cost of cooling by 3.1x and heating by 2.6x compared to non-switching devices. Our concept opens the door for passively adaptive thermoregulating building materials., Comment: 32 pages with supplementary information included

Moving average processes driven by exponential-tailed L\'evy noise are important extensions of their Gaussian counterparts in order to capture deviations from Gaussianity, more flexible dependence structures, and sample paths with jumps. Popular examples include non-Gaussian Ornstein--Uhlenbeck processes and type G Mat\'ern stochastic partial differential equation random fields. This paper is concerned with the open problem of determining their extremal dependence structure. We leverage the fact that such processes admit approximations on grids or triangulations that are used in practice for efficient simulations and inference. These approximations can be expressed as special cases of a class of linear transformations of independent, exponential-tailed random variables, that bridge asymptotic dependence and independence in a novel, tractable way. This result is of independent interest since models that can capture both extremal dependence regimes are scarce and the construction of such flexible models is an active area of research. This new fundamental result allows us to show that the integral approximation of general moving average processes with exponential-tailed L\'evy noise is asymptotically independent when the mesh is fine enough. Under mild assumptions on the kernel function we also derive the limiting residual tail dependence function. For the popular exponential-tailed Ornstein--Uhlenbeck process we prove that it is asymptotically independent, but with a different residual tail dependence function than its Gaussian counterpart. Our results are illustrated through simulation studies.

Model checking is essential to evaluate the adequacy of statistical models and the validity of inferences drawn from them. Particularly, hierarchical models such as latent Gaussian models (LGMs) pose unique challenges as it is difficult to check assumptions about the distribution of the latent parameters. Discrepancy measures are often used to quantify the degree to which a model fit deviates from the observed data. We construct discrepancy measures by (a) defining an alternative model with relaxed assumptions and (b) deriving the discrepancy measure most sensitive to discrepancies induced by this alternative model. We also promote a workflow for model criticism that combines model checking with subsequent robustness analysis. As a result, we obtain a general recipe to check assumptions in LGMs and the impact of these assumptions on the results. We demonstrate the ideas by assessing the latent Gaussianity assumption, a crucial but often overlooked assumption in LGMs. We illustrate the methods via examples utilising Stan and provide functions for easy usage of the methods for general models fitted through R-INLA., Comment: 40 pages, 21 figures

Despite the superior performance, Large Language Models~(LLMs) require significant computational resources for deployment and use. To overcome this issue, quantization methods have been widely applied to reduce the memory footprint of LLMs as well as increasing the inference rate. However, a major challenge is that low-bit quantization methods often lead to performance degradation. It is important to understand how quantization impacts the capacity of LLMs. Different from previous studies focused on overall performance, this work aims to investigate the impact of quantization on \emph{emergent abilities}, which are important characteristics that distinguish LLMs from small language models. Specially, we examine the abilities of in-context learning, chain-of-thought reasoning, and instruction-following in quantized LLMs. Our empirical experiments show that these emergent abilities still exist in 4-bit quantization models, while 2-bit models encounter severe performance degradation on the test of these abilities. To improve the performance of low-bit models, we conduct two special experiments: (1) fine-gained impact analysis that studies which components (or substructures) are more sensitive to quantization, and (2) performance compensation through model fine-tuning. Our work derives a series of important findings to understand the impact of quantization on emergent abilities, and sheds lights on the possibilities of extremely low-bit quantization for LLMs., Comment: 15 pages, 4 figures

The mechanisms of lithium electrodeposition, which overwhelmingly affect lithium metal battery performance and safety, remain insufficiently understood due to its electrochemical complexity. Novel, non-destructive and in situ techniques to probe electrochemical interfaces during lithium electrodeposition are highly desirable. In this work, we demonstrate the capability of transient grating spectroscopy to monitor lithium electrodeposition at the micrometer scale by generating and detecting surface acoustic waves that sensitively interact with the deposited lithium. Specifically, we show that the evolution of the frequency, velocity and damping rate of the surface acoustic waves strongly correlate with the lithium nucleation and growth process. Our work illustrates the sensitivity of high-frequency surface acoustic waves to micrometer scale changes in electrochemical cells and establishes transient grating spectroscopy as a versatile platform for future in situ investigation of electrochemical inte

As the prevalence of data analysis grows, safeguarding data privacy has become a paramount concern. Consequently, there has been an upsurge in the development of mechanisms aimed at privacy-preserving data analyses. However, these approaches are task-specific; designing algorithms for new tasks is a cumbersome process. As an alternative, one can create synthetic data that is (ideally) devoid of private information. This paper focuses on privacy-preserving data synthesis (PPDS) by providing a comprehensive overview, analysis, and discussion of the field. Specifically, we put forth a master recipe that unifies two prominent strands of research in PPDS: statistical methods and deep learning (DL)-based methods. Under the master recipe, we further dissect the statistical methods into choices of modeling and representation, and investigate the DL-based methods by different generative modeling principles. To consolidate our findings, we provide comprehensive reference tables, distill key takeaways, and identify open problems in the existing literature. In doing so, we aim to answer the following questions: What are the design principles behind different PPDS methods? How can we categorize these methods, and what are the advantages and disadvantages associated with each category? Can we provide guidelines for method selection in different real-world scenarios? We proceed to benchmark several prominent DL-based methods on the task of private image synthesis and conclude that DP-MERF is an all-purpose approach. Finally, upon systematizing the work over the past decade, we identify future directions and call for actions from researchers., Comment: Accepted at IEEE S&P (Oakland) 2024

The detection and accurate astrometry of fast-moving near-Earth objects (NEOs) has been a challenge for the follow-up community. Their fast apparent motion results in streaks in sidereal images, thus affecting the telescope's limiting magnitude and astrometric accuracy. A widely adopted technique to mitigate trailing losses is non-sidereal tracking, which transfers the streaking to background reference stars. However, no existing publicly available astrometry software is configured to detect such elongated stars. We present Astreaks, a streaking source detection algorithm, to obtain accurate astrometry of NEOs in non-sidereal data. We validate the astrometric accuracy of Astreaks on 371 non-sidereally tracked images for 115 NEOs with two instrument set-ups of the GROWTH-India Telescope. The observed NEOs had V-band magnitude in the range [15, 22] with proper motion up to 140$^{\prime\prime}$/min, thus resulting in stellar streaks as high as 6.5$^\prime$ (582 pixels) in our data. Our method obtained astrometric solutions for all images with 100% success rate. The standard deviation in Observed-minus-Computed (O-C) residuals is 0.52$^{\prime\prime}$, with O-C residuals <2$^{\prime\prime}$(<1$^{\prime\prime}$) for 98.4% (84.4%) of our measurements. These are appreciable, given the pixel scale of $\sim$0.3$^{\prime\prime}$ and $\sim$0.7$^{\prime\prime}$ of our two instrument set-ups. This demonstrates that our modular and fully-automated algorithm helps improve the telescope system's limiting magnitude without compromising astrometric accuracy by enabling non-sidereal tracking on the target. This will help the NEO follow-up community cope with the accelerated discovery rates and improved sensitivity of the next-generation NEO surveys. Astreaks has been made available to the community under an open-source license., Comment: 10 pages, 7 figures

Statistical analysis of extremes can be used to predict the probability of future extreme events, such as large rainfalls or devastating windstorms. The quality of these forecasts can be measured through scoring rules. Locally scale invariant scoring rules give equal importance to the forecasts at different locations regardless of differences in the prediction uncertainty. This is a useful feature when computing average scores but can be an unnecessarily strict requirement when mostly concerned with extremes. We propose the concept of local weight-scale invariance, describing scoring rules fulfilling local scale invariance in a certain region of interest, and as a special case local tail-scale invariance, for large events. Moreover, a new version of the weighted Continuous Ranked Probability score (wCRPS) called the scaled wCRPS (swCRPS) that possesses this property is developed and studied. The score is a suitable alternative for scoring extreme value models over areas with varying scale of extreme events, and we derive explicit formulas of the score for the Generalised Extreme Value distribution. The scoring rules are compared through simulation, and their usage is illustrated in modelling of extreme water levels, annual maximum rainfalls, and in an application to non-extreme forecast for the prediction of air pollution., Comment: 32 pages, 10 figures

Near Earth Objects (NEOs) are a transient population of small bodies with orbits near or in the terrestrial planet region. They represent a mid-stage in the dynamical cycle of asteroids and comets, which starts with their removal from the respective source regions -- the main belt and trans-Neptunian scattered disk -- and ends as bodies impact planets, disintegrate near the Sun, or are ejected from the Solar System. Here we develop a new orbital model of NEOs by numerically integrating asteroid orbits from main belt sources and calibrating the results on observations of the Catalina Sky Survey. The results imply a size-dependent sampling of the main belt with the $\nu_6$ and 3:1 resonances producing $\simeq 30$\% of NEOs with absolute magnitudes $H = 15$ and $\simeq 80$\% of NEOs with $H = 25$. Hence, the large and small NEOs have different orbital distributions. The inferred flux of $H<18$ bodies into the 3:1 resonance can be sustained only if the main-belt asteroids near the resonance drift toward the resonance at the maximal Yarkovsky rate ($\simeq 2 \times 10^{-4}$ au Myr$^{-1}$ for diameter $D=1$ km and semimajor axis $a=2.5$~au). This implies obliquities $\theta \simeq 0^\circ$ for $a<2.5$~au and $\theta \simeq 180^\circ$ for $a>2.5$~au, both in the immediate neighborhood of the resonance (the same applies to other resonances as well). We confirm the size-dependent disruption of asteroids near the Sun found in previous studies. An interested researcher can use the publicly available NEOMOD Simulator to generate user-defined samples of NEOs from our model., Comment: AJ

Recent studies have demonstrated the great power of Transformer models for time series forecasting. One of the key elements that lead to the transformer's success is the channel-independent (CI) strategy to improve the training robustness. However, the ignorance of the correlation among different channels in CI would limit the model's forecasting capacity. In this work, we design a special Transformer, i.e., Channel Aligned Robust Blend Transformer (CARD for short), that addresses key shortcomings of CI type Transformer in time series forecasting. First, CARD introduces a channel-aligned attention structure that allows it to capture both temporal correlations among signals and dynamical dependence among multiple variables over time. Second, in order to efficiently utilize the multi-scale knowledge, we design a token blend module to generate tokens with different resolutions. Third, we introduce a robust loss function for time series forecasting to alleviate the potential overfitting issue. This new loss function weights the importance of forecasting over a finite horizon based on prediction uncertainties. Our evaluation of multiple long-term and short-term forecasting datasets demonstrates that CARD significantly outperforms state-of-the-art time series forecasting methods. The code is available at the following repository:https://github.com/wxie9/CARD, Comment: ICLR 2024

Generative LLMs have been shown to effectively power AI-based code authoring tools that can suggest entire statements or blocks of code during code authoring. In this paper we present CodeCompose, an AI-assisted code authoring tool developed and deployed at Meta internally. CodeCompose is based on the InCoder LLM that merges generative capabilities with bi-directionality. We have scaled up CodeCompose to serve tens of thousands of developers at Meta, across 9 programming languages and several coding surfaces. We present our experience in making design decisions about the model and system architecture for CodeCompose that addresses these challenges. To release a LLM model at this scale, we needed to first ensure that it is sufficiently accurate. In a random sample of 20K source code files, depending on the language, we are able to reproduce hidden lines between 40% and 58% of the time, an improvement of 1.4x and 4.1x over a model trained only on public data. We gradually rolled CodeCompose out to developers. At the time of this writing, 16K developers have used it with 8% of their code coming directly from CodeCompose. To triangulate our numerical findings, we conduct a thematic analysis on the feedback from 70 developers. We find that 91.5% of the feedback is positive, with the most common themes being discovering APIs, dealing with boilerplate code, and accelerating coding. Meta continues to integrate this feedback into CodeCompose.