Search

Data selection is of great significance in pre-training large language models, given the variation in quality within the large-scale available training corpora. To achieve this, researchers are currently investigating the use of data influence to measure the importance of data instances, $i.e.,$ a high influence score indicates that incorporating this instance to the training set is likely to enhance the model performance. Consequently, they select the top-$k$ instances with the highest scores. However, this approach has several limitations. (1) Computing the influence of all available data is time-consuming. (2) The selected data instances are not diverse enough, which may hinder the pre-trained model's ability to generalize effectively to various downstream tasks. In this paper, we introduce \texttt{Quad}, a data selection approach that considers both quality and diversity by using data influence to achieve state-of-the-art pre-training results. In particular, noting that attention layers capture extensive semantic details, we have adapted the accelerated $iHVP$ computation methods for attention layers, enhancing our ability to evaluate the influence of data, $i.e.,$ its quality. For the diversity, \texttt{Quad} clusters the dataset into similar data instances within each cluster and diverse instances across different clusters. For each cluster, if we opt to select data from it, we take some samples to evaluate the influence to prevent processing all instances. To determine which clusters to select, we utilize the classic Multi-Armed Bandit method, treating each cluster as an arm. This approach favors clusters with highly influential instances (ensuring high quality) or clusters that have been selected less frequently (ensuring diversity), thereby well balancing between quality and diversity.

Machine learning (ML) models are increasingly deployed to production, calling for efficient inference serving systems. Efficient inference serving is complicated by two challenges: (i) ML models incur high computational costs, and (ii) the request arrival rates of practical applications have frequent, high, and sudden variations which make it hard to correctly provision hardware. Model cascades are positioned to tackle both of these challenges, as they (i) save work while maintaining accuracy, and (ii) expose a high-resolution trade-off between work and accuracy, allowing for fine-grained adjustments to request arrival rates. Despite their potential, model cascades haven't been used inside an online serving system. This comes with its own set of challenges, including workload adaption, model replication onto hardware, inference scheduling, request batching, and more. In this work, we propose CascadeServe, which automates and optimizes end-to-end inference serving with cascades. CascadeServe operates in an offline and online phase. In the offline phase, the system pre-computes a gear plan that specifies how to serve inferences online. In the online phase, the gear plan allows the system to serve inferences while making near-optimal adaptations to the query load at negligible decision overheads. We find that CascadeServe saves 2-3x in cost across a wide spectrum of the latency-accuracy space when compared to state-of-the-art baselines on different workloads., Comment: 17 pages, 13 figures

A long-standing goal of data management systems has been to build systems which can compute quantitative insights over large corpora of unstructured data in a cost-effective manner. Until recently, it was difficult and expensive to extract facts from company documents, data from scientific papers, or metrics from image and video corpora. Today's models can accomplish these tasks with high accuracy. However, a programmer who wants to answer a substantive AI-powered query must orchestrate large numbers of models, prompts, and data operations. For even a single query, the programmer has to make a vast number of decisions such as the choice of model, the right inference method, the most cost-effective inference hardware, the ideal prompt design, and so on. The optimal set of decisions can change as the query changes and as the rapidly-evolving technical landscape shifts. In this paper we present Palimpzest, a system that enables anyone to process AI-powered analytical queries simply by defining them in a declarative language. The system uses its cost optimization framework to implement the query plan with the best trade-offs between runtime, financial cost, and output data quality. We describe the workload of AI-powered analytics tasks, the optimization methods that Palimpzest uses, and the prototype system itself. We evaluate Palimpzest on tasks in Legal Discovery, Real Estate Search, and Medical Schema Matching. We show that even our simple prototype offers a range of appealing plans, including one that is 3.3x faster and 2.9x cheaper than the baseline method, while also offering better data quality. With parallelism enabled, Palimpzest can produce plans with up to a 90.3x speedup at 9.1x lower cost relative to a single-threaded GPT-4 baseline, while obtaining an F1-score within 83.5% of the baseline. These require no additional work by the user., Comment: 29 pages, 9 figures

Polarized attosecond $\gamma-$rays may offer excitation and hyperfine tracking of reactions relevant to nuclear physics, astrophysics, high-energy physics, etc. However, unfortunately, generation of a feasible and easy-to-deploy source is still a great challenge. Here, we put forward a novel method for producing ultra-collimated high-brilliance polarized attosecond $\gamma-$rays via the interaction of an unpolarized electron beam with a solid-density plasma. As a relativistic electron beam enters a solid-density plasma, it can be modulated into high-density clusters via the self-modulation instability of itself and further into attosecond slices due to its own hosing instability. This is accompanied by the generation of similar pulse-width $\gamma-$slices via nonlinear Compton scattering. The severe hosing instability breaks the symmetry of the excited electromagnetic fields, resulting in net linear polarization of $\gamma-$slices, which challenges the conventional perception that the interaction of an axially symmetric unpolarized electron beam with a uniform plasma cannot generate polarized radiation. In addition, we also obtain high-quality electron microbunches which may serve as an alternative source for prebunched free-electron lasers.

In this note, a brief introduction to the physical and mathematical background of the two-component Ginzburg-Landau theory is given. From this theory we derive a boundary value problem whose solution can be obtained in part by solving a minimization problem using the technique of variational method, except that two of the eight boundary conditions cannot be satisfied. To overcome the difficulty of recovering the full set of boundary conditions, we employ a variety of methods, including the uniform estimation method and the bounded monotonic theorem, which may be applied to other complicated vortex problems in gauge field theories. The twisted vortex solutions are obtained as energy-minimizing cylindrically symmetric field configurations. We also give the sharp asymptotic estimates for the twisted vortex solutions at the origin and infinity.

In the context of multi-view clustering, graph learning is recognized as a crucial technique, which generally involves constructing an adaptive neighbor graph based on probabilistic neighbors, and then learning a consensus graph to for clustering. However, they are confronted with two limitations. Firstly, they often rely on Euclidean distance to measure similarity when constructing the adaptive neighbor graph, which proves inadequate in capturing the intrinsic structure among data points in practice. Secondly, most of these methods focus solely on consensus graph, ignoring unique information from each view. Although a few graph-based studies have considered using specific information as well, the modelling approach employed does not exclude the noise impact from the specific component. To this end, we propose a novel tensor-based multi-view graph learning framework that simultaneously considers consistency and specificity, while effectively eliminating the influence of noise. Specifically, we calculate similarity distance on the Stiefel manifold to preserve the intrinsic properties of data. By making an assumption that the learned neighbor graph of each view comprises a consistent part, a specific part, and a noise part, we formulate a new tensor-based target graph learning paradigm for noise-free graph fusion. Owing to the benefits of tensor singular value decomposition (t-SVD) in uncovering high-order correlations, this model is capable of achieving a complete understanding of the target graph. Furthermore, we derive an algorithm to address the optimization problem. Experiments on six datasets have demonstrated the superiority of our method. We have released the source code on https://github.com/lshi91/CSTGL-Code.

Nonlinear subspace clustering based on a feed-forward neural network has been demonstrated to provide better clustering accuracy than some advanced subspace clustering algorithms. While this approach demonstrates impressive outcomes, it involves a balance between effectiveness and computational cost. In this study, we employ a functional link neural network to transform data samples into a nonlinear domain. Subsequently, we acquire a self-representation matrix through a learning mechanism that builds upon the mapped samples. As the functional link neural network is a single-layer neural network, our proposed method achieves high computational efficiency while ensuring desirable clustering performance. By incorporating the local similarity regularization to enhance the grouping effect, our proposed method further improves the quality of the clustering results. Additionally, we introduce a convex combination subspace clustering scheme, which combining a linear subspace clustering method with the functional link neural network subspace clustering approach. This combination approach allows for a dynamic balance between linear and nonlinear representations. Extensive experiments confirm the advancement of our methods. The source code will be released on https://lshi91.github.io/ soon.

In this paper, we consider the self-dual equation arising from Abelian Chern-Simons-Higgs theory coupled to the Einstein equations over the plane $\mathbb{R}^2$ and a compact surface $S$. We prove the existence of symmetric topological solutions and non-topological solutions on the plane by using the fixed-point theorem and a shooting method, respectively. A necessary and sufficient condition related to the string number $N$, the Euler characteristic $\chi(S)$ of $S$, and the gravitational coupling factor $G$ is given to show the existence of $N$-string solutions over a compact surface.

Latent multi-view subspace clustering has been demonstrated to have desirable clustering performance. However, the original latent representation method vertically concatenates the data matrices from multiple views into a single matrix along the direction of dimensionality to recover the latent representation matrix, which may result in an incomplete information recovery. To fully recover the latent space representation, we in this paper propose an Enhanced Latent Multi-view Subspace Clustering (ELMSC) method. The ELMSC method involves constructing an augmented data matrix that enhances the representation of multi-view data. Specifically, we stack the data matrices from various views into the block-diagonal locations of the augmented matrix to exploit the complementary information. Meanwhile, the non-block-diagonal entries are composed based on the similarity between different views to capture the consistent information. In addition, we enforce a sparse regularization for the non-diagonal blocks of the augmented self-representation matrix to avoid redundant calculations of consistency information. Finally, a novel iterative algorithm based on the framework of Alternating Direction Method of Multipliers (ADMM) is developed to solve the optimization problem for ELMSC. Extensive experiments on real-world datasets demonstrate that our proposed ELMSC is able to achieve higher clustering performance than some state-of-art multi-view clustering methods.

Designing terahertz sensors with high sensitivity to detect nanoscale thin films and single biomolecule presents a significant challenge, and addressing these obstacles is crucial for unlocking their full potential in scientific research and advanced applications. This work presents a strategy for the design optimization of metamaterial sensors employed in the detection of small amounts of dielectric materials. The sensors usually utilize the shift of the resonance frequency as an indicator of the presence of the analyte. The amount of shifting depends on intrinsic properties (electric field distribution, quality factor, and mode volume) of the bare cavity, as well as the overlap volume of its high-electric-field zone(s) and the analyte. Guided by the simplified dielectric perturbation theory, interdigitated electric split-ring resonators (ID-eSRR) are devised to significantly enhance the detection sensitivity for thin-film analytes compared to eSRRs without interdigitated fingers in the SRR gap region. The fingers of the ID-eSRR metamaterial sensor redistribute the electric field, creating strongly localized field enhancements that substantially boost the interaction with the analyte. Additionally, the periodic change of the orientation of the inherent anti-phase electric field in the interdigitated structure reduces radiation loss, leading to a higher Q-factor. Experiments with e-beam-fabricated ID-eSRR sensors operating at around 300 GHz demonstrate a remarkable frequency shift of 33.5 GHz upon deposition of a SiO2 layer with a thickness of 150 nm as an analyte simulant. The figure of merit (FOM) improves by over 50 times compared to structures without interdigitated fingers. This rational design option opens a promising avenue for highly sensitive detection of thin films and trace biomolecules., Comment: 16pages, 6 figures

Distributed detection primarily centers around two approaches: Unquantized Distributed Detection (UDD), where each sensor reports its complete observation to the fusion center (FC), and quantized-and-Coded DD (CDD), where each sensor first partitions the observation space and then reports to the FC a codeword. In this paper, we introduce Quantized-but-uncoded DD (QDD), where each sensor, after quantization, transmits a summarized value, instead of a codeword, to the FC. We show that QDD well adapts to the constraint of transmission power when compared to CDD, albeit with increased complexity in parameter selection. Moreover, we establish that, in the presence of independent observations, QDD upholds a necessary condition inherent in CDD. Specifically, the optimal sensor decision rules are the likelihood ratio quantizers (LRQ), irrelevant to the channel conditions. In the context of a single-sensor scenario involving binary decision at the sensor, we find that the optimal sensor rule in QDD is in general no longer ``channel blind", a feature presented in CDD. In addition, we compare these systems numerically under the same transmission power and bandwidth, while assuming additive white Gaussian noise (AWGN) in both sensing and reporting stages. Finally, we present some potential directions for future research., Comment: 11 pages, 8 figure, submitted to IEEE T-IT

Social media, self-driving cars, and traffic cameras produce video streams at large scales and cheap cost. However, storing and querying video at such scales is prohibitively expensive. We propose to treat large-scale video analytics as a data warehousing problem: Video is a format that is easy to produce but needs to be transformed into an application-specific format that is easy to query. Analogously, we define the problem of Video Extract-Transform-Load (V-ETL). V-ETL systems need to reduce the cost of running a user-defined V-ETL job while also giving throughput guarantees to keep up with the rate at which data is produced. We find that no current system sufficiently fulfills both needs and therefore propose Skyscraper, a system tailored to V-ETL. Skyscraper can execute arbitrary video ingestion pipelines and adaptively tunes them to reduce cost at minimal or no quality degradation, e.g., by adjusting sampling rates and resolutions to the ingested content. Skyscraper can hereby be provisioned with cheap on-premises compute and uses a combination of buffering and cloud bursting to deal with peaks in workload caused by expensive processing configurations. In our experiments, we find that Skyscraper significantly reduces the cost of V-ETL ingestion compared to adaptions of current SOTA systems, while at the same time giving robustness guarantees that these systems are lacking., Comment: 26 pages, 23 figures

Data curation tasks that prepare data for analytics are critical for turning data into actionable insights. However, due to the diverse requirements of applications in different domains, generic off-the-shelf tools are typically insufficient. As a result, data scientists often have to develop domain-specific solutions tailored to both the dataset and the task, e.g. writing domain-specific code or training machine learning models on a sufficient number of annotated examples. This process is notoriously difficult and time-consuming. We present SEED, an LLM-as-compiler approach that automatically generates domain-specific data curation solutions via Large Language Models (LLMs). Once the user describes a task, input data, and expected output, the SEED compiler produces a hybrid pipeline that combines LLM querying with more cost-effective alternatives, such as vector-based caching, LLM-generated code, and small models trained on LLM-annotated data. SEED features an optimizer that automatically selects from the four LLM-assisted modules and forms a hybrid execution pipeline that best fits the task at hand. To validate this new, revolutionary approach, we conducted experiments on $9$ datasets spanning over $5$ data curation tasks. In comparison to solutions that use the LLM on every data record, SEED achieves state-of-the-art or comparable few-shot performance, while significantly reducing the number of LLM calls., Comment: preprint, 20 pages, 4 figures

In this note we construct self--dual vortices and cosmic strings from the generalized Abelian Higgs theory. A special model of the theory is of focused interest in which the Higgs potential is a polynomial depending on $m$. When $|m|>0$, we obtain sharp existence theorems for vortices and strings correspond to gravity absence and appearance, respectively over the full plane. In particular, the vortex solution is unique if $m<0$. In order to over the difficulties posed by gravity, we introduce the regularization method when there are at least two distinct points among the set of centers of strings. When all these points coincide, the fixed point theorem works well. A series properties regarding vortices and strings for $|m|>0$ are also established.

In this note we construct self-dual cosmic strings from a gauge field theory with a generalized linear formation of potential energy density. By integrating the Einstein equation, we obtain a nonlinear elliptic equation which is equal with the sources. We prove the existence of a solution in the broken symmetry category on the full plane and the multiple string solutions are valid under a sufficient condition imposed only on the total string number N. The technique of upper-lower solutions and the method of regularization are employed to show the existence of a solution when there are at least two distant string centers. When all the string centers are identical, fixed point theorem are used to study the properties of the nonlinear elliptic equation. Finally, We give the sharp asymptotic estimate for the solution at infinity.

A blocker of $123$-avoiding permutation matrices refers to the set of zeros contained within an $n\times n$ $123$-forcing matrix. Recently, Brualdi and Cao provided a characterization of all minimal blockers, which are blockers with a cardinality of $n$. Building upon their work, a new type of blocker, flag-shaped blockers, which can be seen as a generalization of the $L$-shaped blockers defined by Brualdi and Cao, are introduced. It is demonstrated that all flag-shaped blockers are minimum blockers. The possible cardinalities of flag-shaped blockers are also determined, and the dimensions of subpolytopes that are defined by flag-shaped blockers are examined., Comment: 19 pages

Existing approaches to automatic data transformation are insufficient to meet the requirements in many real-world scenarios, such as the building sector. First, there is no convenient interface for domain experts to provide domain knowledge easily. Second, they require significant training data collection overheads. Third, the accuracy suffers from complicated schema changes. To bridge this gap, we present a novel approach that leverages the unique capabilities of large language models (LLMs) in coding, complex reasoning, and zero-shot learning to generate SQL code that transforms the source datasets into the target datasets. We demonstrate the viability of this approach by designing an LLM-based framework, termed SQLMorpher, which comprises a prompt generator that integrates the initial prompt with optional domain knowledge and historical patterns in external databases. It also implements an iterative prompt optimization mechanism that automatically improves the prompt based on flaw detection. The key contributions of this work include (1) pioneering an end-to-end LLM-based solution for data transformation, (2) developing a benchmark dataset of 105 real-world building energy data transformation problems, and (3) conducting an extensive empirical evaluation where our approach achieved 96% accuracy in all 105 problems. SQLMorpher demonstrates the effectiveness of utilizing LLMs in complex, domain-specific challenges, highlighting the potential of their potential to drive sustainable solutions., Comment: 10 pages, 7 figures

A novel intense beam-driven scheme for high yield of the tri-alpha reaction 11B(p,{\alpha})2{\alpha} was investigated. We used a foam target made of cellulose triacetate (TAC, C_9H_{16}O_8) doped with boron. It was then heated volumetrically by soft X-ray radiation from a laser heated hohlraum and turned into a homogenous, and long living plasma. We employed a picosecond laser pulse to generate a high-intensity energetic proton beam via the well-known Target Normal Sheath Acceleration (TNSA) mechanism. We observed up to 10^{10}/sr {\alpha} particles per laser shot. This constitutes presently the highest yield value normalized to the laser energy on target. The measured fusion yield per proton exceeds the classical expectation of beam-target reactions by up to four orders of magnitude under high proton intensities. This enhancement is attributed to the strong electric fields and nonequilibrium thermonuclear fusion reactions as a result of the new method. Our approach shows opportunities to pursue ignition of aneutronic fusion.

In this note we research the Abelian Higgs model subject to the Born-Infeld theory of electrodynamics for which the BPS equations can be reduced into a quasi-linear differential equation. We show that the equation exists a unique solution under two interesting boundary conditions which realize the corresponding phase transition. We construct the solution through a dynamical shooting method for which the correct shooting slope is unique. We also obtain the sharp asymptotic estimate for the solution at infinity., Comment: 10 pages, 0 figure

Generative AI has made significant strides, yet concerns about the accuracy and reliability of its outputs continue to grow. Such inaccuracies can have serious consequences such as inaccurate decision-making, the spread of false information, privacy violations, legal liabilities, and more. Although efforts to address these risks are underway, including explainable AI and responsible AI practices such as transparency, privacy protection, bias mitigation, and social and environmental responsibility, misinformation caused by generative AI will remain a significant challenge. We propose that verifying the outputs of generative AI from a data management perspective is an emerging issue for generative AI. This involves analyzing the underlying data from multi-modal data lakes, including text files, tables, and knowledge graphs, and assessing its quality and consistency. By doing so, we can establish a stronger foundation for evaluating the outputs of generative AI models. Such an approach can ensure the correctness of generative AI, promote transparency, and enable decision-making with greater confidence. Our vision is to promote the development of verifiable generative AI and contribute to a more trustworthy and responsible use of AI., Comment: 8 pages, 4 figures

Dark modes represent a class of forbidden transitions or transitions with weak dipole moments between energy states. Due to their low transition probability, it is difficult to realize their interaction with light, let alone achieve the strong interaction of the modes with the photons in a cavity. However, by mutual coupling with a bright mode, the strong interaction of dark modes with photons is possible. This type of mediated interaction is widely investigated in the metamaterials community and is known under the term electromagnetically induced transparency (EIT). Here, we report strong coupling between a plasmonic dark mode of an EIT-like metamaterial with the photons of a 1D photonic crystal cavity in the terahertz frequency range. The coupling between the dark mode and the cavity photons is mediated by a plasmonic bright mode, which is proven by the observation of a frequency splitting which depends on the strength of the inductive interaction between the plasmon bright and dark modes of the EIT-like metamaterial. In addition, since the plasmonic dark mode strongly couples with the cavity dark mode, we observes four polariton modes. The frequency splitting by interaction of the four modes (plasmonic bright and dark mode and the two eigenmodes of the photonic cavity) can be reproduced in the framework of a model of four coupled harmonic oscillators.

Data curation is a wide-ranging area which contains many critical but time-consuming data processing tasks. However, the diversity of such tasks makes it challenging to develop a general-purpose data curation system. To address this issue, we present Lingua Manga, a user-friendly and versatile system that utilizes pre-trained large language models. Lingua Manga offers automatic optimization for achieving high performance and label efficiency while facilitating flexible and rapid development. Through three example applications with distinct objectives and users of varying levels of technical proficiency, we demonstrate that Lingua Manga can effectively assist both skilled programmers and low-code or even no-code users in addressing data curation challenges., Comment: 4 pages, 6 figures, VLDB 2023 Demo paper

We propose RoTaR, a row-based table representation learning method, to address the efficiency and scalability issues faced by existing table representation learning methods. The key idea of RoTaR is to generate query-agnostic row representations that could be re-used via query-specific aggregation. In addition to the row-based architecture, we introduce several techniques: cell-aware position embedding, teacher-student training paradigm, and selective backward to improve the performance of RoTaR model., Comment: 6 pages, 3 figures, NeurIPS 2022 Table Representation Learning workshop

Zero-shot NL2SQL is crucial in achieving natural language to SQL that is adaptive to new environments (e.g., new databases, new linguistic phenomena or SQL structures) with zero annotated NL2SQL samples from such environments. Existing approaches either fine-tune pre-trained language models (PLMs) based on annotated data or use prompts to guide fixed large language models (LLMs) such as ChatGPT. PLMs can perform well in schema alignment but struggle to achieve complex reasoning, while LLMs is superior in complex reasoning tasks but cannot achieve precise schema alignment. In this paper, we propose a ZeroNL2SQL framework that combines the complementary advantages of PLMs and LLMs for supporting zero-shot NL2SQL. ZeroNL2SQL first uses PLMs to generate an SQL sketch via schema alignment, then uses LLMs to fill the missing information via complex reasoning. Moreover, in order to better align the generated SQL queries with values in the given database instances, we design a predicate calibration method to guide the LLM in completing the SQL sketches based on the database instances and select the optimal SQL query via an execution-based strategy. Comprehensive experiments show that ZeroNL2SQL can achieve the best zero-shot NL2SQL performance on real-world benchmarks. Specifically, ZeroNL2SQL outperforms the state-of-the-art PLM-based methods by 3.2% to 13% and exceeds LLM-based methods by 10% to 20% on execution accuracy., Comment: Working in progress

Timeseries analytics is of great importance in many real-world applications. Recently, the Transformer model, popular in natural language processing, has been leveraged to learn high quality feature embeddings from timeseries, core to the performance of various timeseries analytics tasks. However, the quadratic time and space complexities limit Transformers' scalability, especially for long timeseries. To address these issues, we develop a timeseries analytics tool, RITA, which uses a novel attention mechanism, named group attention, to address this scalability issue. Group attention dynamically clusters the objects based on their similarity into a small number of groups and approximately computes the attention at the coarse group granularity. It thus significantly reduces the time and space complexity, yet provides a theoretical guarantee on the quality of the computed attention. The dynamic scheduler of RITA continuously adapts the number of groups and the batch size in the training process, ensuring group attention always uses the fewest groups needed to meet the approximation quality requirement. Extensive experiments on various timeseries datasets and analytics tasks demonstrate that RITA outperforms the state-of-the-art in accuracy and is significantly faster -- with speedups of up to 63X.