Search

Benchmarks are critical for measuring progress of math reasoning abilities of Large Language Models (LLMs). However, existing widely-used benchmarks such as GSM8K have been rendered less useful as multiple cutting-edge LLMs achieve over 94% accuracy. While harder benchmarks have been proposed, their creation is often manual and expensive. We present Scheherazade, an automated approach for producing challenging mathematical reasoning benchmarks by logically chaining mathematical reasoning problems. We propose two different chaining methods, forward chaining and backward chaining, which require reasoning forward and backward through the chain respectively. We apply Scheherazade on GSM8K to create GSM8K-Scheherazade and evaluate 3 frontier LLMs and OpenAI's o1-preview on it. We show that while frontier models' performance declines precipitously at only a few questions chained, a preliminary evaluation suggests o1-preview performance persists up to 5 questions chained backwards. In addition, while all other models perform worse when problems are chained backwards, o1-preview performs better on backward-chained benchmarks. We will release the dataset and code publicly.

This work provides a vision on how to tackle the underutilization of compute resources in FPGA SoC devices used across 5G and edge computing infrastructures. A first step towards this end is the implementation of a resource management layer able to migrate and scale functions in such devices, based on context events. This layer sets the basis to design a hierarchical data-driven micro-orchestrator in charge of providing the lifecycle management of functions in FPGA SoC devices. In the O-RAN context, the micro-orchestrator is foreseen to take the form of an xApp/rApp tandem trained with RAN traffic and context data., Comment: Article accepted in the IEEE International Conference on 6G Networking (6GNet 2024)

This is the 2nd part of the dissertation for my master degree and compared the power consumption using the Comma-Separated-Values (CSV) and parquet dataset format with the default floating point (32bit) and Nvidia mixed precision (16bit and 32bit) while training a regression ML model. The same custom PC as per the 1st part, which was dedicated to the classification testing and analysis, was built to perform the experiments, and different ML hyper-parameters, such as batch size, neurons, and epochs, were chosen to build Deep Neural Networks (DNN). A benchmarking test with default hyper-parameter values for the DNN was used as a reference, while the experiments used a combination of different settings. The results were recorded in Excel, and descriptive statistics were chosen to calculate the mean between the groups and compare them using graphs and tables. The outcome was positive when using mixed precision combined with specific hyper-parameters. Compared to the benchmarking, optimising the regression models reduced the power consumption between 7 and 11 Watts. The regression results show that while mixed precision can help improve power consumption, we must carefully consider the hyper-parameters. A high number of batch sizes and neurons will negatively affect power consumption. However, this research required inferential statistics, specifically ANOVA and T-test, to compare the relationship between the means. The results reported no statistical significance between the means in the regression tests and accepted H0. Therefore, choosing different ML techniques and the Parquet dataset format will not improve the computational power consumption and the overall ML carbon footprint. However, a more extensive implementation with a cluster of GPUs can increase the sample size significantly, as it is an essential factor and can change the outcome of the statistical analysis., Comment: 35 pages, 16 tables, 19 figures. arXiv admin note: substantial text overlap with arXiv:2409.07853

This is the 1st part of the dissertation for my master degree and compares the power consumption using the default floating point (32bit) and Nvidia mixed precision (16bit and 32bit) while training a classification ML model. A custom PC with specific hardware was built to perform the experiments, and different ML hyper-parameters, such as batch size, neurons, and epochs, were chosen to build Deep Neural Networks (DNN). Additionally, various software was used during the experiments to collect the power consumption data in Watts from the Graphics Processing Unit (GPU), Central Processing Unit (CPU), Random Access Memory (RAM) and manually from a wattmeter connected to the wall. A benchmarking test with default hyper parameter values for the DNN was used as a reference, while the experiments used a combination of different settings. The results were recorded in Excel, and descriptive statistics were chosen to calculate the mean between the groups and compare them using graphs and tables. The outcome was positive when using mixed precision combined with specific hyper-parameters. Compared to the benchmarking, the optimisation for the classification reduced the power consumption between 7 and 11 Watts. Similarly, the carbon footprint is reduced because the calculation uses the same power consumption data. Still, a consideration is required when configuring hyper-parameters because it can negatively affect hardware performance. However, this research required inferential statistics, specifically ANOVA and T-test, to compare the relationship between the means. Furthermore, tests indicated no statistical significance of the relationship between the benchmarking and experiments. However, a more extensive implementation with a cluster of GPUs can increase the sample size significantly, as it is an essential factor and can change the outcome of the statistical analysis., Comment: 27 pages, 14 figures, 9 tables

The need for intelligent and efficient resource provisioning for the productive management of resources in real-world scenarios is growing with the evolution of telecommunications towards the 6G era. Technologies such as Open Radio Access Network (O-RAN) can help to build interoperable solutions for the management of complex systems. Probabilistic forecasting, in contrast to deterministic single-point estimators, can offer a different approach to resource allocation by quantifying the uncertainty of the generated predictions. This paper examines the cloud-native aspects of O-RAN together with the radio App (rApp) deployment options. The integration of probabilistic forecasting techniques as a rApp in O-RAN is also emphasized, along with case studies of real-world applications. Through a comparative analysis of forecasting models using the error metric, we show the advantages of Deep Autoregressive Recurrent network (DeepAR) over other deterministic probabilistic estimators. Furthermore, the simplicity of Simple-Feed-Forward (SFF) leads to a fast runtime but does not capture the temporal dependencies of the input data. Finally, we present some aspects related to the practical applicability of cloud-native O-RAN with probabilistic forecasting.

This work presents a novel, black-box software-based countermeasure against physical attacks including power side-channel and fault-injection attacks. The approach uses the concept of random self-reducibility and self-correctness to add randomness and redundancy in the execution for protection. Our approach is at the operation level, is not algorithm-specific, and thus, can be applied for protecting a wide range of algorithms. The countermeasure is empirically evaluated against attacks over operations like modular exponentiation, modular multiplication, polynomial multiplication, and number theoretic transforms. An end-to-end implementation of this countermeasure is demonstrated for RSA-CRT signature algorithm and Kyber Key Generation public key cryptosystems. The countermeasure reduced the power side-channel leakage by two orders of magnitude, to an acceptably secure level in TVLA analysis. For fault injection, the countermeasure reduces the number of faults to 95.4% in average.

Considered herein is a class of Boussinesq systems of Bona-Smith type that describe water waves in bounded two-dimensional domains with slip-wall boundary conditions and variable bottom topography. Such boundary conditions are necessary in situations involving water waves in channels, ports, and generally in basins with solid boundaries. We prove that, given appropriate initial conditions, the corresponding initial-boundary value problems have unique solutions locally in time, which is a fundamental property of deterministic mathematical modeling. Moreover, we demonstrate that the systems under consideration adhere to three basic conservation laws for water waves: mass, vorticity, and energy conservation. The theoretical analysis of these specific Boussinesq systems leads to a conservative mixed finite element formulation. Using explicit, relaxation Runge-Kutta methods for the discretization in time, we devise a fully discrete scheme for the numerical solution of initial-boundary value problems with slip-wall conditions, preserving mass, vorticity, and energy. Finally, we present a series of challenging numerical experiments to assess the applicability of the new numerical model.

Despite long-term research and development of modern shock absorbers, the effect of variations of several crucial material and model parameters still remains dubious. The goal of this work is therefore a study of the changes of shock absorber dynamics with respect to typical parameter ranges in a realistic model. We study the impact of shim properties, as well as geometric features such as discharge coefficients and bleed orifice cross section. We derive cause-effect relationships by nonlinear parameter fitting of the differential equations of the model and show digressive and progressive quadratic damping curves for shim number and thickness, sharp exponential curves for discharge coefficients, and leakage width, as well as a linear decrease of damping properties with bleed orifice area. Temperature increase affecting material properties, such as density and viscosity of the mineral oil, is found to have a mostly linear relationship with damping and pressure losses. Our results are not only significant for the general understanding of shock absorber dynamics, but also serve as a guidance for the development of specific models by following the proposed methodology.

Principled accountability for autonomous decision-making in uncertain environments requires distinguishing intentional outcomes from negligent designs from actual accidents. We propose analyzing the behavior of autonomous agents through a quantitative measure of the evidence of intentional behavior. We model an uncertain environment as a Markov Decision Process (MDP). For a given scenario, we rely on probabilistic model checking to compute the ability of the agent to influence reaching a certain event. We call this the scope of agency. We say that there is evidence of intentional behavior if the scope of agency is high and the decisions of the agent are close to being optimal for reaching the event. Our method applies counterfactual reasoning to automatically generate relevant scenarios that can be analyzed to increase the confidence of our assessment. In a case study, we show how our method can distinguish between 'intentional' and 'accidental' traffic collisions., Comment: 10 pages. Accepted for publication at IJCAI 2023 (Main Track)

For many decades, advances in static verification have focused on linear integer arithmetic (LIA) programs. Many real-world programs are, however, written with non-linear integer arithmetic (NLA) expressions, such as programs that model physical events, control systems, or nonlinear activation functions in neural networks. While there are some approaches to reasoning about such NLA programs, still many verification tools fall short when trying to analyze them. To expand the scope of existing tools, we introduce a new method of converting programs with NLA expressions into semantically equivalent LIA programs via a technique we call dual rewriting. Dual rewriting discovers a linear replacement for an NLA Boolean expression (e.g. as found in conditional branching), simultaneously exploring both the positive and negative side of the condition, and using a combination of static validation and dynamic generalization of counterexamples. While perhaps surprising at first, this is often possible because the truth value of a Boolean NLA expression can be characterized in terms of a Boolean combination of linearly-described regions/intervals where the expression is true and those where it is false. The upshot is that rewriting NLA expressions to LIA expressions beforehand enables off-the-shelf LIA tools to be applied to the wider class of NLA programs. We built a new tool DrNLA and show it can discover LIA replacements for a variety of NLA programs. We then applied our work to branching-time verification of NLA programs, creating the first set of such benchmarks (92 in total) and showing that DrNLA's rewriting enable tools such as FuncTion and T2 to verify CTL properties of 42 programs that previously could not be verified. We also show a potential use of DrNLA assisting Frama-C in program slicing, and report that execution speed is not impacted much by rewriting.

Hydraulic valves, for the purpose of targeted pressure relief and damping, are a ubiquitous part of modern suspension systems. This paper deals with the analytical computation of fixed points of the dynamical system of a single-stage shock absorber valve, which can be applied for the direct computation of its system variables at equilibrium without brute-force numerical integration. The obtained analytical expressions are given for the original dimensional version of the model derived from continuity and motion equations for a realistic valve setup. Furthermore, a large part of the study is devoted to a systematic sensitivity analysis of the model towards crucial parameter changes. Numerical investigation of a potential loss of stability and following nonlinear oscillations is performed in multi-dimensional parameter spaces, which reveals sustained valve vibrations at increased valve mass and applied pretension force. The dynamical behaviour is analysed by phase space orbits, as well as Fourier-transformed valve displacement data to identify dominant frequencies. Chaotic indicators, such as Lyapunov exponents and the Smaller Alignment Index (SALI), are utilized to understand the nature of the oscillatory motion and to distinguish between order and chaos.

Principled accountability in the aftermath of harms is essential to the trustworthy design and governance of algorithmic decision making. Legal theory offers a paramount method for assessing culpability: putting the agent 'on the stand' to subject their actions and intentions to cross-examination. We show that under minimal assumptions automated reasoning can rigorously interrogate algorithmic behaviors as in the adversarial process of legal fact finding. We model accountability processes, such as trials or review boards, as Counterfactual-Guided Logic Exploration and Abstraction Refinement (CLEAR) loops. We use the formal methods of symbolic execution and satisfiability modulo theories (SMT) solving to discharge queries about agent behavior in factual and counterfactual scenarios, as adaptively formulated by a human investigator. In order to do so, for a decision algorithm $\mathcal{A}$ we use symbolic execution to represent its logic as a statement $\Pi$ in the decidable theory $\texttt{QF_FPBV}$. We implement our framework and demonstrate its utility on an illustrative car crash scenario.

In this paper, we consider the classic Fermi-Pasta-Ulam-Tsingou system as a model of interacting particles connected by harmonic springs with a quadratic nonlinear term (first system) and a set of second-order ordinary differential equations with variability (second system) that resembles Hamilton's equations of motion of the Fermi-Pasta-Ulam-Tsingou system. In the absence of variability, the second system becomes Hamilton's equations of motion of the Fermi-Pasta-Ulam-Tsingou system (first system). Variability is introduced to Hamilton's equations of motion of the Fermi-Pasta-Ulam-Tsingou system to take into account inherent variations (for example, due to manufacturing processes), giving rise to heterogeneity in its parameters. We demonstrate that a percentage of variability smaller than a threshold can break the well-known energy recurrence phenomenon and induce localization in the energy normal-mode space. However, percentage of variability larger than the threshold may make the trajectories of the second system blow up in finite time. Using a multiple-scale expansion, we derive analytically a two normal-mode approximation that explains the mechanism for energy localization and blow up in the second system. We also investigate the chaotic behavior of the two systems as the percentage of variability is increased, utilising the maximum Lyapunov exponent and Smaller Alignment Index. Our analysis shows that when there is almost energy localization in the second system, it is more probable to observe chaos, as the number of particles increases.

The applications concerning vehicular networks benefit from the vision of beyond 5G and 6G technologies such as ultra-dense network topologies, low latency, and high data rates. Vehicular networks have always faced data privacy preservation concerns, which lead to the advent of distributed learning techniques such as federated learning. Although federated learning has solved data privacy preservation issues to some extent, the technique is quite vulnerable to model inversion and model poisoning attacks. We assume that the design of defense mechanism and attacks are two sides of the same coin. Designing a method to reduce vulnerability requires the attack to be effective and challenging with real-world implications. In this work, we propose simulated poisoning and inversion network (SPIN) that leverages the optimization approach for reconstructing data from a differential model trained by a vehicular node and intercepted when transmitted to roadside unit (RSU). We then train a generative adversarial network (GAN) to improve the generation of data with each passing round and global update from the RSU, accordingly. Evaluation results show the qualitative and quantitative effectiveness of the proposed approach. The attack initiated by SPIN can reduce up to 22% accuracy on publicly available datasets while just using a single attacker. We assume that revealing the simulation of such attacks would help us find its defense mechanism in an effective manner., Comment: 6 pages, 4 figures

In this paper, we present a method that combines information-theoretical and statistical approaches to infer connectivity in complex networks using time-series data. The method is based on estimations of the Mutual Information Rate for pairs of time-series and on statistical significance tests for connectivity acceptance using the false discovery rate method for multiple hypothesis testing. We provide the mathematical background on Mutual Information Rate, discuss the statistical significance tests and the false discovery rate. Further on, we present results for correlated normal-variates data, coupled circle and coupled logistic maps, coupled Lorenz systems and coupled stochastic Kuramoto phase oscillators. Following up, we study the effect of noise on the presented methodology in networks of coupled stochastic Kuramoto phase oscillators and of coupling heterogeneity degree on networks of coupled circle maps. We show that the method can infer the correct number and pairs of connected nodes, by means of receiver operating characteristic curves. In the more realistic case of stochastic data, we demonstrate its ability to infer the structure of the initial connectivity matrices. The method is also shown to recover the initial connectivity matrices for dynamics on the nodes of Erd\H{o}s-R\'enyi and small-world networks with varying coupling heterogeneity in their connections. The highlight of the proposed methodology is its ability to infer the underlying network connectivity based solely on the recorded datasets., Comment: 25 pages, 7 figures

In this paper, we report on the generation and propagation of traveling pulses in a homogeneous network of diffusively coupled, excitable, slow-fast dynamical neurons. The spatially extended system is modelled using the nearest neighbor coupling theory, in which the diffusion part measures the spatial distribution of the coupling topology. We derive analytically the conditions for traveling wave profiles that allow the construction of the shape of traveling nerve impulses. The analytical and numerical results are used to explore the nature of the propagating pulses. The symmetric or asymmetric nature of the traveling pulses is characterized and the wave velocity is derived as a function of system parameters. Moreover, we present our results for an extended excitable medium by considering a slow-fast biophysical model with a homogeneous, diffusive coupling that can exhibit various traveling pulses. The appearance of series of pulses is an interesting phenomenon from biophysical and dynamical perspective. Varying the perturbation and coupling parameters, we observe the propagation of activities with various amplitude modulations and transition phases of different wave profiles that affect the speed of the pulses in certain parameter regimes. We observe different types of traveling pulses, such as envelope solitons and multi-bump solutions and show how system parameters and the coupling play a major role in the formation of different traveling pulses. Finally, we obtain the conditions for stable and unstable plane waves., Comment: 17 pages, 7 figures