Search

Reinforcement learning (RL) plays a major role in solving complex sequential decision-making tasks. Hierarchical and goal-conditioned RL are promising methods for dealing with two major problems in RL, namely sample inefficiency and difficulties in reward shaping. These methods tackle the mentioned problems by decomposing a task into simpler subtasks and temporally abstracting a task in the action space. One of the key components for task decomposition of these methods is subgoal discovery. We can use the subgoal states to define hierarchies of actions and also use them in decomposing complex tasks. Under the assumption that subgoal states are more unpredictable, we propose a free energy paradigm to discover them. This is achieved by using free energy to select between two spaces, the main space and an aggregation space. The $model \; changes$ from neighboring states to a given state shows the unpredictability of a given state, and therefore it is used in this paper for subgoal discovery. Our empirical results on navigation tasks like grid-world environments show that our proposed method can be applied for subgoal discovery without prior knowledge of the task. Our proposed method is also robust to the stochasticity of environments.

Control barrier functions (CBFs) have emerged as a popular topic in safety critical control due to their ability to provide formal safety guarantees for dynamical systems. Despite their powerful capabilities, the determination of feasible CBFs for input-constrained systems is still a formidable task and a challenging research issue. The present work aims to tackle this problem by focusing on an alternative approach towards a generalization of some ideas introduced in the existing CBF literature. The approach provides a rigorous yet straightforward method to define and implement prediction-based control barrier functions for complex dynamical systems to ensure safety with bounded inputs. This is accomplished by introducing a prediction-based term into the CBF that allows for the required margin needed to null the CBF rate of change given the specified input constraints. Having established the theoretical groundwork, certain remarks are subsequently presented with regards to the scheme's implementation. Finally, the proposed prediction-based control barrier function (PB-CBF) scheme is implemented for two numerical examples. In particular, the second example is related to aircraft stall prevention, which is meant to demonstrate the functionality and capability of the PB-CBFs in handling complex nonlinear dynamical systems via simulations. In both examples, the performance of the PB-CBF is compared with that of a non-prediction based basic CBF.

This paper introduces Opus, a novel framework for generating and optimizing Workflows tailored to complex Business Process Outsourcing (BPO) use cases, focusing on cost reduction and quality enhancement while adhering to established industry processes and operational constraints. Our approach generates executable Workflows from Intention, defined as the alignment of Client Input, Client Output, and Process Context. These Workflows are represented as Directed Acyclic Graphs (DAGs), with nodes as Tasks consisting of sequences of executable Instructions, including tools and human expert reviews. We adopt a two-phase methodology: Workflow Generation and Workflow Optimization. In the Generation phase, Workflows are generated using a Large Work Model (LWM) informed by a Work Knowledge Graph (WKG) that encodes domain-specific procedural and operational knowledge. In the Optimization phase, Workflows are transformed into Workflow Graphs (WFGs), where optimal Workflows are determined through path optimization. Our experiments demonstrate that state-of-the-art Large Language Models (LLMs) face challenges in reliably retrieving detailed process data as well as generating industry-compliant workflows. The key contributions of this paper include integrating a Work Knowledge Graph (WKG) into a Large Work Model (LWM) to enable the generation of context-aware, semantically aligned, structured and auditable Workflows. It further introduces a two-phase approach that combines Workflow Generation from Intention with graph-based Workflow Optimization. Finally, we present Opus Alpha 1 Large and Opus Alpha 1 Small that outperform state-of-the-art LLMs by 38% and 29% respectively in Workflow Generation for a Medical Coding use case., Comment: 25 pages, 9 figures

For the last three years, the AutoPET competition gathered the medical imaging community around a hot topic: lesion segmentation on Positron Emitting Tomography (PET) scans. Each year a different aspect of the problem is presented; in 2024 the multiplicity of existing and used tracers was at the core of the challenge. Specifically, this year's edition aims to develop a fully automatic algorithm capable of performing lesion segmentation on a PET/CT scan, without knowing the tracer, which can either be a FDG or PSMA-based tracer. In this paper we describe how we used the nnUNetv2 framework to train two sets of 6 fold ensembles of models to perform fully automatic PET/CT lesion segmentation as well as a MIP-CNN to choose which set of models to use for segmentation.

End-to-end web testing is challenging due to the need to explore diverse web application functionalities. Current state-of-the-art methods, such as WebCanvas, are not designed for broad functionality exploration; they rely on specific, detailed task descriptions, limiting their adaptability in dynamic web environments. We introduce NaviQAte, which frames web application exploration as a question-and-answer task, generating action sequences for functionalities without requiring detailed parameters. Our three-phase approach utilizes advanced large language models like GPT-4o for complex decision-making and cost-effective models, such as GPT-4o mini, for simpler tasks. NaviQAte focuses on functionality-guided web application navigation, integrating multi-modal inputs such as text and images to enhance contextual understanding. Evaluations on the Mind2Web-Live and Mind2Web-Live-Abstracted datasets show that NaviQAte achieves a 44.23% success rate in user task navigation and a 38.46% success rate in functionality navigation, representing a 15% and 33% improvement over WebCanvas. These results underscore the effectiveness of our approach in advancing automated web application testing.

Deep neural networks are increasingly used as an effective way to represent control policies in a wide-range of learning-based control methods. For continuous-time optimal control problems (OCPs), which are central to many decision-making tasks, control policy learning can be cast as a neural ordinary differential equation (NODE) problem wherein state and control constraints are naturally accommodated. This paper presents a Lyapunov-NODE control (L-NODEC) approach to solving continuous-time OCPs for the case of stabilizing a known constrained nonlinear system around a terminal equilibrium point. We propose a Lyapunov loss formulation that incorporates a control-theoretic Lyapunov condition into the problem of learning a state-feedback neural control policy. We establish that L-NODEC ensures exponential stability of the controlled system, as well as its adversarial robustness to uncertain initial conditions. The performance of L-NODEC is illustrated on a benchmark double integrator problem and for optimal control of thermal dose delivery using a cold atmospheric plasma biomedical system. L-NODEC can substantially reduce the inference time necessary to reach the equilibrium state.

The Multi-Criteria Test Suite Minimization (MCTSM) problem aims to refine test suites by removing redundant test cases, guided by adequacy criteria such as code coverage or fault detection capability. However, current techniques either exhibit a high loss of fault detection ability or face scalability challenges due to the NP-hard nature of the problem, which limits their practical utility. We propose TripRL, a novel technique that integrates traditional criteria such as statement coverage and fault detection ability with test coverage similarity into an Integer Linear Program (ILP), to produce a diverse reduced test suite with high test effectiveness. TripRL leverages bipartite graph representation and its embedding for concise ILP formulation and combines ILP with effective reinforcement learning (RL) training. This combination renders large-scale test suite minimization more scalable and enhances test effectiveness. Our empirical evaluations demonstrate that TripRL's runtime scales linearly with the magnitude of the MCTSM problem. Notably, for large test suites where existing approaches fail to provide solutions within a reasonable time frame, our technique consistently delivers solutions in less than 47 minutes. The reduced test suites produced by TripRL also maintain the original statement coverage and fault detection ability while having a higher potential to detect unknown faults.

Dockerfile flakiness, characterized by inconsistent build behavior without Dockerfile or project source code changes, poses significant challenges in Continuous Integration and Delivery (CI/CD) pipelines. This issue can lead to unreliable deployments and increased debugging efforts, yet it remains underexplored in current research. We conduct a systematic analysis of Dockerfile flakiness, presenting a comprehensive taxonomy of common flakiness categories, including dependency-related errors and server connectivity issues. Furthermore, we introduce FlakiDock, a tool leveraging large language models and retrieval-augmented generation techniques with dynamic analysis and an iterative feedback loop to automatically repair flaky Dockerfiles. Our evaluation shows that FlakiDock achieves a 73.55% repair accuracy, outperforming existing tools such as PARFUM by 12,581% and GPT-4-based prompting by 94.63%. These results underscore the effectiveness of FlakiDock in addressing Dockerfile flakiness and improving build reliability.

End-to-end (E2E) testing is essential for ensuring web application quality. However, manual test creation is time-consuming and current test generation techniques produce random tests. In this paper, we present AUTOE2E, a novel approach that leverages Large Language Models (LLMs) to automate the generation of semantically meaningful feature-driven E2E test cases for web applications. AUTOE2E intelligently infers potential features within a web application and translates them into executable test scenarios. Furthermore, we address a critical gap in the research community by introducing E2EBENCH, a new benchmark for automatically assessing the feature coverage of E2E test suites. Our evaluation on E2EBENCH demonstrates that AUTOE2E achieves an average feature coverage of 79%, outperforming the best baseline by 558%, highlighting its effectiveness in generating high-quality, comprehensive test cases.

Let $h^{+}$ and $h^{-}$ be two complete, conformal metrics on the disc $\mathbb{D}$. Assume moreover that the derivatives of the conformal factors of the metrics $h^{+}$ and $h^{-}$ are bounded at any order with respect to the hyperbolic metric, and that the metrics have curvatures in the interval $\left(-\frac{1}{\epsilon}, -1 - \epsilon\right)$, for some $\epsilon > 0$. Let $f$ be a quasi-symmetric map. We show the existence of a globally hyperbolic convex subset $\Omega$ (see Definition 3.1) of the three-dimensional anti-de Sitter space, such that $\Omega$ has $h^{+}$ (respectively $h^{-}$) as the induced metric on its future boundary (respectively on its past boundary) and has a gluing map $\Phi_{\Omega}$ (see Definition 4.5) equal to $f$., Comment: 15 pages, comments welcom. arXiv admin note: text overlap with arXiv:1902.04027 by other authors

We study the asymptotic behavior of Moncrief lines on $2+1$ maximal globally hyperbolic spatially compact space-time $M$ of non-negative constant curvature. We show that when the unique geodesic lamination associated with $M$ is either maximal uniquely ergodic or simplicial, the Moncrief line converges, as time goes to zero, to a unique point in the Thurston boundary of the Teichm\"uller space., Comment: Comments are welcome!

Large language models have made substantial progress in addressing diverse code-related tasks. However, their adoption is hindered by inconsistencies in generating output due to the lack of real-world, domain-specific information, such as for intra-repository API calls for unseen software projects. We introduce a novel technique to mitigate hallucinations by leveraging global and local contextual information within a code repository for API completion tasks. Our approach is tailored to refine code completion tasks, with a focus on optimizing local API completions. We examine relevant import statements during API completion to derive insights into local APIs, drawing from their method signatures. For API token completion, we analyze the inline variables and correlate them with the appropriate imported modules, thereby allowing our approach to rank the most contextually relevant suggestions from the available local APIs. Further, for conversational API completion, we gather APIs that are most relevant to the developer query with a retrieval-based search across the project. We employ our tool, LANCE, within the framework of our proposed benchmark, APIEval, encompassing two different programming languages. Our evaluation yields an average accuracy of 82.6% for API token completion and 76.9% for conversational API completion tasks. On average, LANCE surpasses Copilot by 143% and 142% for API token completion and conversational API completion, respectively. The implications of our findings are substantial for developers, suggesting that our lightweight context analysis can be applied to multilingual environments without language-specific training or fine-tuning, allowing for efficient implementation with minimal examples and effort.

Industrial applications of plasma have significantly increased beyond semiconductor manufacturing in recent years. This necessitates training a skilled workforce in plasma science and technology. However, an essential challenge to this end stems from the high cost of plasma devices and diagnostics. The limited access to plasma devices has hindered plasma education, particularly in the least developed countries. To this end, this paper demonstrates how low-cost sensors and circuits can be developed to enable inexpensive plasma experiments in laboratory environments. In particular, we show how to measure high voltage, current, and power from a cold-atmospheric plasma discharge. Additionally, we develop a low-cost illuminance sensor and demonstrate how it can be used to estimate the corresponding plasma power. The low-cost sensors and electronics presented in this paper can aid educators in characterizing plasma power versus plasma illuminance.

Designing predictive controllers towards optimal closed-loop performance while maintaining safety and stability is challenging. This work explores closed-loop learning for predictive control parameters under imperfect information while considering closed-loop stability. We employ constrained Bayesian optimization to learn a model predictive controller's (MPC) cost function parametrized as a feedforward neural network, optimizing closed-loop behavior as well as minimizing model-plant mismatch. Doing so offers a high degree of freedom and, thus, the opportunity for efficient and global optimization towards the desired and optimal closed-loop behavior. We extend this framework by stability constraints on the learned controller parameters, exploiting the optimal value function of the underlying MPC as a Lyapunov candidate. The effectiveness of the proposed approach is underlined in simulations, highlighting its performance and safety capabilities., Comment: 7 pages, 3 figures, accepted for NMPC 2024

For large software applications, running the whole test suite after each code change is time- and resource-intensive. Regression test selection techniques aim at reducing test execution time by selecting only the tests that are affected by code changes. However, existing techniques select test entities at coarse granularity levels such as test class, which causes imprecise test selection and executing unaffected tests. We propose a novel approach that increases the selection precision by analyzing test code at statement level and treating test assertions as the unit for selection. We implement our fine-grained test selection approach in a tool called SELERTION and evaluate it by comparing against two state-of-the-art test selection techniques using 11 open-source subjects. Our results show that SELERTION increases selection precision for all the subjects. Our test selection reduces, on average, 63% of the overall test time, making regression testing up to 23% faster than the other techniques. Our results also indicate that subjects with longer test execution time benefit more by our fine-grained selection technique.

Automated test generation for web forms has been a longstanding challenge, exacerbated by the intrinsic human-centric design of forms and their complex, device-agnostic structures. We introduce an innovative approach, called FormNexus, for automated web form test generation, which emphasizes deriving semantic insights from individual form elements and relations among them, utilizing textual content, DOM tree structures, and visual proximity. The insights gathered are transformed into a new conceptual graph, the Form Entity Relation Graph (FERG), which offers machine-friendly semantic information extraction. Leveraging LLMs, FormNexus adopts a feedback-driven mechanism for generating and refining input constraints based on real-time form submission responses. The culmination of this approach is a robust set of test cases, each produced by methodically invalidating constraints, ensuring comprehensive testing scenarios for web forms. This work bridges the existing gap in automated web form testing by intertwining the capabilities of LLMs with advanced semantic inference methods. Our evaluation demonstrates that FormNexus combined with GPT-4 achieves 89% coverage in form submission states. This outcome significantly outstrips the performance of the best baseline model by a margin of 25%.

Airway-related quantitative imaging biomarkers are crucial for examination, diagnosis, and prognosis in pulmonary diseases. However, the manual delineation of airway trees remains prohibitively time-consuming. While significant efforts have been made towards enhancing airway modelling, current public-available datasets concentrate on lung diseases with moderate morphological variations. The intricate honeycombing patterns present in the lung tissues of fibrotic lung disease patients exacerbate the challenges, often leading to various prediction errors. To address this issue, the 'Airway-Informed Quantitative CT Imaging Biomarker for Fibrotic Lung Disease 2023' (AIIB23) competition was organized in conjunction with the official 2023 International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI). The airway structures were meticulously annotated by three experienced radiologists. Competitors were encouraged to develop automatic airway segmentation models with high robustness and generalization abilities, followed by exploring the most correlated QIB of mortality prediction. A training set of 120 high-resolution computerised tomography (HRCT) scans were publicly released with expert annotations and mortality status. The online validation set incorporated 52 HRCT scans from patients with fibrotic lung disease and the offline test set included 140 cases from fibrosis and COVID-19 patients. The results have shown that the capacity of extracting airway trees from patients with fibrotic lung disease could be enhanced by introducing voxel-wise weighted general union loss and continuity loss. In addition to the competitive image biomarkers for prognosis, a strong airway-derived biomarker (Hazard ratio>1.5, p<0.0001) was revealed for survival prognostication compared with existing clinical measurements, clinician assessment and AI-based biomarkers., Comment: 19 pages

We show that the minimum effort control of colloidal self-assembly can be naturally formulated in the order-parameter space as a generalized Schr\"{o}dinger bridge problem -- a class of fixed-horizon stochastic optimal control problems that originated in the works of Erwin Schr\"{o}dinger in the early 1930s. In recent years, this class of problems has seen a resurgence of research activities in the control and machine learning communities. Different from the existing literature on the theory and computation for such problems, the controlled drift and diffusion coefficients for colloidal self-assembly are typically nonaffine in control, and are difficult to obtain from physics-based modeling. We deduce the conditions of optimality for such generalized problems, and show that the resulting system of equations is structurally very different from the existing results in a way that standard computational approaches no longer apply. Thus motivated, we propose a data-driven learning and control framework, named `neural Schr\"{o}dinger bridge', to solve such generalized Schr\"{o}dinger bridge problems by innovating on recent advances in neural networks. We illustrate the effectiveness of the proposed framework using a numerical case study of colloidal self-assembly. We learn the controlled drift and diffusion coefficients as two neural networks using molecular dynamics simulation data, and then use these two to train a third network with Sinkhorn losses designed for distributional endpoint constraints, specific for this class of control problems.

Let $S$ be a closed hyperbolic surface and $M = \left ( 0,1 \right )$. Suppose $h$ is a Riemannian metric on $S$ with curvature strictly greater than $-1$, $h^{*}$ is a Riemannian metric on $S$ with curvature strictly less than $1$, and every contractible closed geodesic with respect to $h^{*}$ has length strictly greater than $2\pi$. Let $L$ be a measured lamination on $S$ such that every closed leaf has weight strictly less than $\pi$. Then, we prove the existence of a convex hyperbolic metric $g$ on the interior of $M$ that induces the Riemannian metric $h$ (respectively $h^{*}$) as the first (respectively third) fundamental form on $S \times \left\{ 0\right\}$ and induces a pleated surface structure on $S \times \left\{ 1\right\}$ with bending lamination $L$. This statement remains valid even in limiting cases where the curvature of $h$ is constant and equal to $-1$. Additionally, when considering a conformal class $c$ on $S$, we show that there exists a convex hyperbolic metric $g$ on the interior of $M$ that induces $c$ on $S \times \left\{ 0\right\}$, which is viewed as one component of the ideal boundary at infinity of $(M,g)$, and induces a pleated surface structure on $S \times \left\{ 0\right\}$ with bending lamination $L$. Our proof differs from previous work by Lecuire for these two last cases. Moreover, when we consider a lamination which is small enough, in a sense that we will define, and a hyperbolic metric, we show that the metric on the interior of M that realizes these data is unique., Comment: 29 pages, 6 figures. Comments are welcome!

Modern web applications make extensive use of API calls to update the UI state in response to user events or server-side changes. For such applications, API-level testing can play an important role, in-between unit-level testing and UI-level (or end-to-end) testing. Existing API testing tools require API specifications (e.g., OpenAPI), which often may not be available or, when available, be inconsistent with the API implementation, thus limiting the applicability of automated API testing to web applications. In this paper, we present an approach that leverages UI testing to enable API-level testing for web applications. Our technique navigates the web application under test and automatically generates an API-level test suite, along with an OpenAPI specification that describes the application's server-side APIs (for REST-based web applications). A key element of our solution is a dynamic approach for inferring API endpoints with path parameters via UI navigation and directed API probing. We evaluated the technique for its accuracy in inferring API specifications and the effectiveness of the "carved" API tests. Our results on seven open-source web applications show that the technique achieves 98% precision and 56% recall in inferring endpoints. The carved API tests, when added to test suites generated by two automated REST API testing tools, increase statement coverage by 52% and 29% and branch coverage by 99% and 75%, on average. The main benefits of our technique are: (1) it enables API-level testing of web applications in cases where existing API testing tools are inapplicable and (2) it creates API-level test suites that cover server-side code efficiently while exercising APIs as they would be invoked from an application's web UI, and that can augment existing API test suites.

In this paper, we present a machine learning-based data generator framework tailored to aid researchers who utilize simulations to examine various physical systems or processes. High computational costs and the resulting limited data often pose significant challenges to gaining insights into these systems or processes. Our approach involves a two-step process: initially, we train a supervised predictive model using a limited simulated dataset to predict simulation outcomes. Subsequently, a reinforcement learning agent is trained to generate accurate, simulation-like data by leveraging the supervised model. With this framework, researchers can generate more accurate data and know the outcomes without running high computational simulations, which enables them to explore the parameter space more efficiently and gain deeper insights into physical systems or processes. We demonstrate the effectiveness of the proposed framework by applying it to two case studies, one focusing on earthquake rupture physics and the other on new material development.

Developing a new class of coating materials is necessary to meet the increasing demands of energy and defense-related technologies, aerospace engineering, and harsh environmental conditions. Functional-based coatings, such as ceramic-metal nanolaminates, have gained popularity due to their ability to be customized according to specific requirements. To design and develop advanced coatings with the necessary functionalities, it is crucial to understand the effects of various parameters on the mechanical and tribological properties of these coatings. In this study, we investigate the impact of penetration depth, individual layer thickness, and ceramic-metal ratio on the mechanical and tribological properties of ceramic-metal nanolaminates, particularly NbC/Nb. Our findings reveal that the thickness of the individual metallic and ceramic layers significantly affects the coatings' properties. However, some models exhibited punctures on the top ceramic layer, which altered the scratching behavior and reduced the impact of layer thickness on it. This is because the top ceramic layer's thickness is too low, and the indenter can easily puncture it instead of pushing the ceramic atoms. The minimum thickness required to resist indentation is called the critical thickness, which depends on the indentation size and penetration depth. In the latter part of this paper, we employed machine learning to reduce computational costs, and the model predicts the friction coefficient with an R-squared value of 0.958.