Search

Software testing ensures the quality and reliability of software products, but manual test case creation is labor-intensive. With the rise of large language models (LLMs), there is growing interest in unit test creation with LLMs. However, effective assessment of LLM-generated test cases is limited by the lack of standardized benchmarks that comprehensively cover diverse programming scenarios. To address the assessment of LLM's test case generation ability and lacking dataset for evaluation, we propose the Generated Benchmark from Control-Flow Structure and Variable Usage Composition (GBCV) approach, which systematically generates programs used for evaluating LLMs' test generation capabilities. By leveraging basic control-flow structures and variable usage, GBCV provides a flexible framework to create a spectrum of programs ranging from simple to complex. Because GPT-4o and GPT-3-Turbo are publicly accessible models, to present real-world regular user's use case, we use GBCV to assess LLM performance on them. Our findings indicate that GPT-4o performs better on complex program structures, while all models effectively detect boundary values in simple conditions but face challenges with arithmetic computations. This study highlights the strengths and limitations of LLMs in test generation, provides a benchmark framework, and suggests directions for future improvement., Comment: 17 pages, 9 figures

We obtain several new bounds on exponents of interest in analytic number theory, including four new exponent pairs, new zero density estimates for the Riemann zeta-function, and new estimates for the additive energy of zeroes of the Riemann zeta-function. These results were obtained by creating the Analytic Number Theory Exponent Database (ANTEDB) to collect results and relationships between these exponents, and then systematically optimising these relationships to obtain the new bounds. We welcome further contributions to the database, which aims to allow easy conversion of new bounds on these exponents into optimised bounds on other related exponents of interest., Comment: 44 pages

This paper presents a systematic method for transforming states of polarization (SoPs) into any arbitrary target SoPs, whether linear or elliptical, by determining the precise waveplate rotation angles required for the transformation. Unlike the method based on TRIAD algorithm, which is restricted to specific target SoPs, this study generalizes the approach to enable transformations to arbitrary target states. By representing these transformations as rigid body rotations in Stokes space and computing the corresponding Mueller matrices, we establish a robust framework for accurate and versatile polarization control., Comment: 16 pages, 2 figures

We present a systematic study of QCD axion production in environments with finite baryon density and temperature, implying significant changes to axion phenomenology. Within heavy baryon chiral perturbation theory, we derive the effective Lagrangian describing axion interactions with nucleons and mesons up to next-to-leading-order in the chiral expansion. We focus on corrections to the axion-nucleon couplings from higher orders and finite density. These couplings are modified by up to an order of magnitude near nuclear saturation density, significantly impacting axion production in supernovae and neutron stars. Density-dependent corrections enhance the axion luminosity in supernovae by an order of magnitude, strengthening current best bounds by a factor of three. We stress the importance of including all axion production channels up to a given chiral order for a consistent luminosity calculation and classify the missing contributions up to the third chiral order. The modified axion-nucleon couplings also affect neutron star cooling rates via axion emission. A re-evaluation of existing neutron star cooling bounds, constrained to regions where perturbative control is reliable, weakens these bounds by a factor of four. Lastly, our results have implications for terrestrial axion searches that rely on precise knowledge of axion-nucleon couplings., Comment: 76 pages, 25 figures, data available at https://github.com/michael-stadlbauer/Axion-Couplings

Explainable AI (XAI) is a rapidly growing domain with a myriad of proposed methods as well as metrics aiming to evaluate their efficacy. However, current studies are often of limited scope, examining only a handful of XAI methods and ignoring underlying design parameters for performance, such as the model architecture or the nature of input data. Moreover, they often rely on one or a few metrics and neglect thorough validation, increasing the risk of selection bias and ignoring discrepancies among metrics. These shortcomings leave practitioners confused about which method to choose for their problem. In response, we introduce LATEC, a large-scale benchmark that critically evaluates 17 prominent XAI methods using 20 distinct metrics. We systematically incorporate vital design parameters like varied architectures and diverse input modalities, resulting in 7,560 examined combinations. Through LATEC, we showcase the high risk of conflicting metrics leading to unreliable rankings and consequently propose a more robust evaluation scheme. Further, we comprehensively evaluate various XAI methods to assist practitioners in selecting appropriate methods aligning with their needs. Curiously, the emerging top-performing method, Expected Gradients, is not examined in any relevant related study. LATEC reinforces its role in future XAI research by publicly releasing all 326k saliency maps and 378k metric scores as a (meta-)evaluation dataset. The benchmark is hosted at: https://github.com/IML-DKFZ/latec., Comment: Accepted at NeurIPS 2024

Determining the crossing numbers of Cartesian products of small graphs with arbitrarily large paths has been an ongoing topic of research since the 1970s. Doing so requires the establishment of coincident upper and lower bounds; the former is usually demonstrated by providing a suitable drawing procedure, while the latter often requires substantial theoretical arguments. Many such papers have been published, which typically focus on just one or two small graphs at a time, and use ad hoc arguments specific to those graphs. We propose a general approach which, when successful, establishes the required lower bound. This approach can be applied to the Cartesian product of any graph with arbitrarily large paths, and in each case involves solving a modified version of the crossing number problem on a finite number (typically only two or three) of small graphs. We demonstrate the potency of this approach by applying it to Cartesian products involving all 133 graphs $G$ of orders five or six, and show that it is successful in 128 cases. This includes 60 cases which a recent survey listed as either undetermined, or determined only in journals without adequate peer review.

Context: Modern open-source operating systems consist of numerous independent packages crafted by countless developers worldwide. To effectively manage this diverse array of software originating from various entities, Linux distributions have devised package management tools to streamline the process. Despite offering convenience in software installation, systems like Ubuntu's apt may obscure the freshness of its constituent packages when compared to the upstream projects. Objective: The focus of this research is to develop a method to systematically identify packages within a Linux distribution that show low development activity between versions of the OSS projects included in a release. The packages within a Linux distribution utilize a heterogeneous mix of versioning strategies in their upstream projects and these versions are passed through to the package manager, often with distribution specific version information appended, making this work both interesting and non-trivial. Method: We use regular expressions to extract the epoch and upstream project major, minor, and patch versions for more than 6000 packages in the Ubuntu distribution, documenting our process for assigning these values for projects that do not follow the semantic versioning standard. Using the libyears metric for the CHAOS project, we calculate the freshness of a subset of the packages within a distribution against the latest upstream project release. This led directly to the development of Package Version Activity Classifier (PVAC), a novel method for systematically assessing the staleness of packages across multiple distribution releases., Comment: 24 pages, Software quality in Linux Packages

This study presents a systematic approach to optimize the scheduling of exoplanet observations using the James Webb Space Telescope (JWST), focusing on targets discovered by the Transiting Exoplanet Survey Satellite (TESS). We developed a methodology to refine transit timing predictions for JWST's Cycle 3 Guest Observer program, specifically for the NIRISS/SOSS and NIRSpec/BOTS observation modes. Our process involved data collection from JWST proposal documents, cross-matching with TESS data, and applying the Transit Least Squares (TLS) algorithm for transit detection and characterization. We created comprehensive timelines for instrument usage and individual proposals, providing a visual representation of the observation schedule from July 2024 to September 2025. This approach demonstrates the potential for improved efficiency in JWST time allocation and sets a foundation for future refinements in astronomical observation planning., Comment: 15 pages

IoT context-sharing platforms are an essential component of today's interconnected IoT deployments with their security affecting the entire deployment and the critical infrastructure adopting IoT. We report on a lack of systematic approach to the security of IoT context-sharing platforms and propose the need for a methodological and systematic alternative to evaluate the existing solutions and develop `secure-by-design' solutions. We have identified the key components of a generic IoT context-sharing platform and propose using MITRE ATT&CK for threat modelling of such platforms., Comment: Accepted to 21st Annual International Conference on Privacy, Security, and Trust (PST2024)

As Large Language Models (LLMs) become increasingly sophisticated and ubiquitous in natural language processing (NLP) applications, ensuring their robustness, trustworthiness, and alignment with human values has become a critical challenge. This paper presents a novel framework for contextual grounding in textual models, with a particular emphasis on the Context Representation stage. Our approach aims to enhance the reliability and ethical alignment of these models through a comprehensive, context-aware methodology. By explicitly capturing and representing relevant situational, cultural, and ethical contexts in a machine-readable format, we lay the foundation for anchoring a model's behavior within these contexts. Our approach leverages techniques from knowledge representation and reasoning, such as ontologies, semantic web technologies, and logic-based formalisms. We evaluate our framework on real-world textual datasets, demonstrating its effectiveness in improving model performance, fairness, and alignment with human expectations, while maintaining high accuracy. Furthermore, we discuss the other key components of the framework, including context-aware encoding, context-aware learning, interpretability and explainability, and continuous monitoring and adaptation. This research contributes to the growing body of work on responsible AI, offering a practical approach to developing more reliable, trustworthy, and ethically-aligned language models. Our findings have significant implications for the deployment of LLMs in sensitive domains such as healthcare, legal systems, and social services, where contextual understanding is paramount., Comment: 14 pages

High-definition road maps play a crucial role in the functionality and verification of highly automated driving functions. These contain precise information about the road network, geometry, condition, as well as traffic signs. Despite their importance for the development and evaluation of driving functions, the generation of high-definition maps is still an ongoing research topic. While previous work in this area has primarily focused on the accuracy of road geometry, we present a novel approach for automated large-scale map generation for use in industrial applications. Our proposed method leverages a minimal number of external information about the road to process LiDAR data in segments. These segments are subsequently combined, enabling a flexible and scalable process that achieves high-definition accuracy. Additionally, we showcase the use of the resulting OpenDRIVE in driving function simulation., Comment: 8 pages, 5 figures, 1 table. arXiv admin note: text overlap with arXiv:2405.07544

In recent years, there has been a growing focus on scrutinizing the security of cellular networks, often attributing security vulnerabilities to issues in the underlying protocol design descriptions. These protocol design specifications, typically extensive documents that are thousands of pages long, can harbor inaccuracies, underspecifications, implicit assumptions, and internal inconsistencies. In light of the evolving landscape, we introduce CellularLint--a semi-automatic framework for inconsistency detection within the standards of 4G and 5G, capitalizing on a suite of natural language processing techniques. Our proposed method uses a revamped few-shot learning mechanism on domain-adapted large language models. Pre-trained on a vast corpus of cellular network protocols, this method enables CellularLint to simultaneously detect inconsistencies at various levels of semantics and practical use cases. In doing so, CellularLint significantly advances the automated analysis of protocol specifications in a scalable fashion. In our investigation, we focused on the Non-Access Stratum (NAS) and the security specifications of 4G and 5G networks, ultimately uncovering 157 inconsistencies with 82.67% accuracy. After verification of these inconsistencies on open-source implementations and 17 commercial devices, we confirm that they indeed have a substantial impact on design decisions, potentially leading to concerns related to privacy, integrity, availability, and interoperability., Comment: Accepted at USENIX Security 24

A systematic study, based on the third-moment structure function, of Sgr A*'s variability finds an exponential rise time $\tau_{1,\rm{obs}}=14.8^{+0.4}_{-1.5}~\mathrm{minutes}$ and decay time $\tau_{2,\rm{obs}}=13.1^{+1.3}_{-1.4}~\mathrm{minutes}$. This symmetry of the flux-density variability is consistent with earlier work, and we interpret it as caused by the dominance of Doppler boosting, as opposed to gravitational lensing, in Sgr~A*'s light curve. A relativistic, semi-physical model of Sgr~A* confirms an inclination angle $i<45$ degrees. The model also shows that the emission of the intrinsic radiative process can have some asymmetry even though the observed emission does not. The third-moment structure function, which is a measure of the skewness of the light-curve increments, may be a useful summary statistic in other contexts of astronomy because it senses only temporal asymmetry, i.e., it averages to zero for any temporally symmetric signal., Comment: Accepted for publication in A&A letters

We consider separable nonconvex optimization problems under affine constraints. For these problems, the Shapley-Folkman theorem provides an upper bound on the duality gap as a function of the nonconvexity of the objective functions, but does not provide a systematic way to construct primal solutions satisfying that bound. In this work, we develop a two-stage approach to do so. The first stage approximates the optimal dual value with a large set of primal feasible solutions. In the second stage, this set is trimmed down to a primal solution by computing (approximate) Caratheodory representations. The main computational requirement of our method is tractability of the Fenchel conjugates of the component functions and their (sub)gradients. When the function domains are convex, the method recovers the classical duality gap bounds obtained via Shapley-Folkman. When the function domains are nonconvex, the method also recovers classical duality gap bounds from the literature, based on a more general notion of nonconvexity.

Intrusion detection systems (IDS) are crucial security measures nowadays to enforce network security. Their task is to detect anomalies in network communication and identify, if not thwart, possibly malicious behavior. Recently, machine learning has been deployed to construct intelligent IDS. This approach, however, is quite challenging particularly in distributed, highly dynamic, yet resource-constrained systems like Edge setups. In this paper, we tackle this issue from multiple angles by analyzing the concept of intelligent IDS (I-IDS) while addressing the specific requirements of Edge devices with a special focus on reconfigurability. Then, we introduce a systematic approach to constructing the I-IDS on reconfigurable Edge hardware. For this, we implemented our proposed IDS on state-of-the-art Field Programmable Gate Arrays (FPGAs) technology as (1) a purely FPGA-based dataflow processor (DFP) and (2) a co-designed approach featuring RISC-V soft-core as FPGA-based soft-core processor (SCP). We complete our paper with a comparison of the state of the art (SoA) in this domain. The results show that DFP and SCP are both suitable for Edge applications from hardware resource and energy efficiency perspectives. Our proposed DFP solution clearly outperforms the SoA and demonstrates that required high performance can be achieved without prohibitively high hardware costs. This makes our proposed DFP suitable for Edge-based high-speed applications like modern communication technology., Comment: 15 pages, 4 figures, conference (ARC24)

Monograph "B. Grechuk, Polynomial Diophantine equations. A systematic approach" suggests solving Diophantine equations systematically in certain order. Many hundreds of the equations are left to the reader. Here, we provide complete solutions to all these equations. The difficulties of solved equations range from elementary to research level. In the last section, we present a summary table of all solved equations ordered by their size, which makes them easy to find. As a result, this document may be used as a database of solved Diophantine equations of small size, to which more difficult equations can be reduced.

We propose a systematic approach to celestial holography in massless theories beginning by studying the implications of properly incorporating field configurations built using the eigenstates of central interest: massless conformal primary wavefunctions that diagonalize the dilatation generator. Due to their singular behaviour on the locus $k\cdot x=0$, they do not belong to the space of Fourier decomposable functions, and incorporating them in the path integral domain requires careful manipulations. In this paper, we include these singular field configurations by a splitting procedure using large pure gauge/diffeomorphism transformations on the action functional. We demonstrate that doing so splits the action into an integrand supported on the singular locus $k\cdot x=0$ and an integrand on the rest of the space. Mellin transforms single out the scalings/conformal dimension in $x$, geometrically, we treat this as a proper non-compact scaling reduction, where we are able to further isolate the dynamics of the large pure diffeomorphism transformations. This takes the form of 2d chiral CFT on a 2d sphere on the singular locus $k\cdot x=0$ - the celestial sphere where the null cone of the origin cuts $\mathscr{I}$. Using this framework, we study Einstein gravity perturbatively around its self-dual sector, where the resulting microscopic 2d CFT couples to bulk scattering states. We are able to obtain an explicit representation of the $\mathcal{L} w_{1+\infty}$ algebra and leading soft splitting functions. With further marginal deformations, we also write down effective interaction vertices which provide form factors of tree-level graviton scattering in Minkowski space., Comment: 31+12 pages, 3 figures, comments welcome!

This paper collects polynomial Diophantine equations that are amazingly simple to write down but are apparently difficult to solve.

In this paper, we present our approach to systematically measure numerous performance parameters of MCP-PMTs. The experimental setups, the analyses and selected results are discussed. Although the techniques used may be different in other locations, the document is intended as a guide for comparable measurements with other types of MCP-PMTs. Measurements are shown for the following performance parameters: spectral and spatial quantum efficiency, collection efficiency, gain as a function of voltage, position and magnetic field, time resolution, rate capability and lifetime. By using a dedicated 3-axis stepper and an FPGA-based DAQ system, also inner PMT parameters are measured as a function of the active area, such as relative detection efficiency, dark count rate, time resolution, recoil electron and afterpulse distributions, as well as charge sharing and electronic crosstalk. In addition, some of the parameters are investigated inside a strong magnetic field. For many of these measurements, the change of most setup parameters and the subsequent analysis can be controlled semi-automatically by software scripts.

Driven by the rapid ascent of artificial intelligence (AI), organizations are at the epicenter of a seismic shift, facing a crucial question: How can AI be successfully integrated into existing operations? To help answer it, manage expectations and mitigate frustration, this article introduces Computational Management, a systematic approach to task automation for enhancing the ability of organizations to harness AI's potential within existing workflows. Computational Management acts as a bridge between the strategic insights of management science with the analytical rigor of computational thinking. The article offers three easy step-by-step procedures to begin the process of implementing AI within a workflow. Such procedures focus on task (re)formulation, on the assessment of the automation potential of tasks, on the completion of task specification templates for AI selection and adaptation. Included in the article there are manual and automated methods, with prompt suggestions for publicly available LLMs, to complete these three procedures. The first procedure, task (re)formulation, focuses on breaking down work activities into basic units, so they can be completed by one agent, involve a single well-defined action, and produce a distinct outcome. The second, allows the assessment of the granular task and its suitability for automation, using the Task Automation Index to rank tasks based on whether they have standardized input, well-defined rules, repetitiveness, data dependency, and objective outputs. The third, focuses on a task specification template which details information on 16 critical components of tasks, and can be used as a checklist to select or adapt the most suitable AI solution for integration into existing workflows. Computational Management provides a roadmap and a toolkit for humans and AI to thrive together, while enhancing organizational efficiency and innovation., Comment: 29 pages, 3 appendices

Ensemble learning has been a focal point of machine learning research due to its potential to improve predictive performance. This study revisits the foundational work on ensemble error decomposition, historically confined to bias-variance-covariance analysis for regression problems since the 1990s. Recent advancements introduced a "unified theory of diversity," which proposes an innovative bias-variance-diversity decomposition framework. Leveraging this contemporary understanding, our research systematically explores the application of this decomposition to guide the creation of new ensemble learning algorithms. Focusing on regression tasks, we employ neural networks as base learners to investigate the practical implications of this theoretical framework. This approach used 7 simple ensemble methods, we name them strategies, for neural networks that were used to generate 21 new ensemble algorithms. Among these, most of the methods aggregated with the snapshot strategy, one of the 7 strategies used, showcase superior predictive performance across diverse datasets w.r.t. the Friedman rank test with the Conover post-hoc test. Our systematic design approach contributes a suite of effective new algorithms and establishes a structured pathway for future ensemble learning algorithm development.

The time evolution of primordial fluctuations conceals a wealth of insights into the high-energy physics at play during the earliest moments of our Universe, which is ultimately encoded in late-time spatial correlation functions. However, the conventional procedure to compute them is technically challenging, and a complete dictionary mapping the landscape of inflationary theories and the corresponding observable signatures is not yet available. In this paper, we develop a framework to compute tree-level cosmological correlators based on following their time evolution from their origin as quantum zero-point fluctuations to the end of inflation. From first principles, the structure of the bulk time evolution imposes a set of universal differential equations in time satisfied by equal-time correlators. We automatise the process of systematically solving these equations. This allows us to accurately capture all physical effects and obtain exact results in theories formulated at the level of inflationary fluctuations that include any number of degrees of freedom with arbitrary dispersion relations and masses, coupled through any time-dependent interactions. We then illustrate the power of this formalism by exploring the phenomenology of cosmological correlators emerging from the interaction with a massive scalar field. We study both the size and the shape dependence of non-Gaussianities in the entire parameter space, including the strong mixing regime. We present novel characteristics of cosmological collider signals in (would be) single-, double-, and triple-exchange three-point correlators. In the presence of primordial features, we show that soft limits of cosmological correlators offer a new possibility to probe the inflationary landscape. Finally, we provide templates to search for in future cosmological surveys., Comment: 77 pages + appendices, 14 figures