Search

Treatment selection in breast cancer is guided by molecular subtypes and clinical characteristics. Recurrence risk assessment plays a crucial role in personalizing treatment. Current methods, including genomic assays, have limited accuracy and clinical utility, leading to suboptimal decisions for many patients. We developed a test for breast cancer patient stratification based on digital pathology and clinical characteristics using novel AI methods. Specifically, we utilized a vision transformer-based pan-cancer foundation model trained with self-supervised learning to extract features from digitized H&E-stained slides. These features were integrated with clinical data to form a multi-modal AI test predicting cancer recurrence and death. The test was developed and evaluated using data from a total of 8,161 breast cancer patients across 15 cohorts originating from seven countries. Of these, 3,502 patients from five cohorts were used exclusively for evaluation, while the remaining patients were used for training. Our test accurately predicted our primary endpoint, disease-free interval, in the five external cohorts (C-index: 0.71 [0.68-0.75], HR: 3.63 [3.02-4.37, p<0.01]). In a direct comparison (N=858), the AI test was more accurate than Oncotype DX, the standard-of-care 21-gene assay, with a C-index of 0.67 [0.61-0.74] versus 0.61 [0.49-0.73], respectively. Additionally, the AI test added independent information to Oncotype DX in a multivariate analysis (HR: 3.11 [1.91-5.09, p<0.01)]). The test demonstrated robust accuracy across all major breast cancer subtypes, including TNBC (C-index: 0.71 [0.62-0.81], HR: 3.81 [2.35-6.17, p=0.02]), where no diagnostic tools are currently recommended by clinical guidelines. These results suggest that our AI test can improve accuracy, extend applicability to a wider range of patients, and enhance access to treatment selection tools.

Generative AI agents, software systems powered by Large Language Models (LLMs), are emerging as a promising approach to automate cybersecurity tasks. Among the others, penetration testing is a challenging field due to the task complexity and the diverse strategies to simulate cyber-attacks. Despite growing interest and initial studies in automating penetration testing with generative agents, there remains a significant gap in the form of a comprehensive and standard framework for their evaluation and development. This paper introduces AutoPenBench, an open benchmark for evaluating generative agents in automated penetration testing. We present a comprehensive framework that includes 33 tasks, each representing a vulnerable system that the agent has to attack. Tasks are of increasing difficulty levels, including in-vitro and real-world scenarios. We assess the agent performance with generic and specific milestones that allow us to compare results in a standardised manner and understand the limits of the agent under test. We show the benefits of AutoPenBench by testing two agent architectures: a fully autonomous and a semi-autonomous supporting human interaction. We compare their performance and limitations. For example, the fully autonomous agent performs unsatisfactorily achieving a 21% Success Rate (SR) across the benchmark, solving 27% of the simple tasks and only one real-world task. In contrast, the assisted agent demonstrates substantial improvements, with 64% of SR. AutoPenBench allows us also to observe how different LLMs like GPT-4o or OpenAI o1 impact the ability of the agents to complete the tasks. We believe that our benchmark fills the gap with a standard and flexible framework to compare penetration testing agents on a common ground. We hope to extend AutoPenBench along with the research community by making it available under https://github.com/lucagioacchini/auto-pen-bench., Comment: Codes for the benchmark: https://github.com/lucagioacchini/auto-pen-bench Codes for the paper experiments: https://github.com/lucagioacchini/genai-pentest-paper

In this study, we initially investigate the application of a hybrid classical-quantum classifier (HCQC) for sentiment analysis, comparing its performance against the classical CPLEX classifier and the Transformer architecture. Our findings indicate that while the HCQC underperforms relative to the Transformer in terms of classification accuracy, but it requires significantly less time to converge to a reasonably good approximate solution. This experiment also reveals a critical bottleneck in the HCQC, whose architecture is partially undisclosed by the D-Wave property. To address this limitation, we propose a novel algorithm based on the algebraic decomposition of QUBO models, which enhances the time the quantum processing unit can allocate to problem-solving tasks., Comment: Submitted to BigHPC 2024 - https://www.itadata.it/2024/bighpc2024

We study Schr\"odinger operators on compact finite metric graphs subject to $\delta'$-coupling conditions. Based on a novel modified local Weyl law, we derive an explicit expression for the limiting mean eigenvalue distance of two different self-adjoint realisations on a given graph. Furthermore, using this spectral comparison result, we also study the limiting mean eigenvalue distance comparing $\delta'$-coupling conditions to so-called anti-Kirchhoff conditions, showing divergence and thereby confirming a numerical observation in [arXiv:2212.12531]. ., Comment: 12 pages, 1 figure; comments are welcome!

Large Language Models (LLMs) changed the way we design and interact with software systems. Their ability to process and extract information from text has drastically improved productivity in a number of routine tasks. Developers that want to include these models in their software stack, however, face a dreadful challenge: debugging LLMs' inconsistent behavior across minor variations of the prompt. We therefore introduce two metrics for classification tasks, namely sensitivity and consistency, which are complementary to task performance. First, sensitivity measures changes of predictions across rephrasings of the prompt, and does not require access to ground truth labels. Instead, consistency measures how predictions vary across rephrasings for elements of the same class. We perform an empirical comparison of these metrics on text classification tasks, using them as guideline for understanding failure modes of the LLM. Our hope is that sensitivity and consistency will be helpful to guide prompt engineering and obtain LLMs that balance robustness with performance.

Biomedical image analysis is fundamental for biomedical discovery in cell biology, pathology, radiology, and many other biomedical domains. Holistic image analysis comprises interdependent subtasks such as segmentation, detection, and recognition of relevant objects. Here, we propose BiomedParse, a biomedical foundation model for imaging parsing that can jointly conduct segmentation, detection, and recognition for 82 object types across 9 imaging modalities. Through joint learning, we can improve accuracy for individual tasks and enable novel applications such as segmenting all relevant objects in an image through a text prompt, rather than requiring users to laboriously specify the bounding box for each object. We leveraged readily available natural-language labels or descriptions accompanying those datasets and use GPT-4 to harmonize the noisy, unstructured text information with established biomedical object ontologies. We created a large dataset comprising over six million triples of image, segmentation mask, and textual description. On image segmentation, we showed that BiomedParse is broadly applicable, outperforming state-of-the-art methods on 102,855 test image-mask-label triples across 9 imaging modalities (everything). On object detection, which aims to locate a specific object of interest, BiomedParse again attained state-of-the-art performance, especially on objects with irregular shapes (everywhere). On object recognition, which aims to identify all objects in a given image along with their semantic types, we showed that BiomedParse can simultaneously segment and label all biomedical objects in an image (all at once). In summary, BiomedParse is an all-in-one tool for biomedical image analysis by jointly solving segmentation, detection, and recognition for all major biomedical image modalities, paving the path for efficient and accurate image-based biomedical discovery., Comment: Project page: https://aka.ms/biomedparse-project

The advances made by Large Language Models (LLMs) have led to the pursuit of LLM agents that can solve intricate, multi-step reasoning tasks. As with any research pursuit, benchmarking and evaluation are key corner stones to efficient and reliable progress. However, existing benchmarks are often narrow and simply compute overall task success. To face these issues, we propose AgentQuest -- a framework where (i) both benchmarks and metrics are modular and easily extensible through well documented and easy-to-use APIs; (ii) we offer two new evaluation metrics that can reliably track LLM agent progress while solving a task. We exemplify the utility of the metrics on two use cases wherein we identify common failure points and refine the agent architecture to obtain a significant performance increase. Together with the research community, we hope to extend AgentQuest further and therefore we make it available under https://github.com/nec-research/agentquest., Comment: Accepted at the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2024)

We study the $p$-\emph{torsion function} and the corresponding $p$-\emph{torsional rigidity} associated with $p$-Laplacians and, more generally, $p$-Schr\"odinger operators, for $1

Causal discovery is becoming a key part in medical AI research. These methods can enhance healthcare by identifying causal links between biomarkers, demographics, treatments and outcomes. They can aid medical professionals in choosing more impactful treatments and strategies. In parallel, Large Language Models (LLMs) have shown great potential in identifying patterns and generating insights from text data. In this paper we investigate applying LLMs to the problem of determining the directionality of edges in causal discovery. Specifically, we test our approach on a deidentified set of Non Small Cell Lung Cancer(NSCLC) patients that have both electronic health record and genomic panel data. Graphs are validated using Bayesian Dirichlet estimators using tabular data. Our result shows that LLMs can accurately predict the directionality of edges in causal graphs, outperforming existing state-of-the-art methods. These findings suggests that LLMs can play a significant role in advancing causal discovery and help us better understand complex systems.

The rapid digitization of real-world data offers an unprecedented opportunity for optimizing healthcare delivery and accelerating biomedical discovery. In practice, however, such data is most abundantly available in unstructured forms, such as clinical notes in electronic medical records (EMRs), and it is generally plagued by confounders. In this paper, we present TRIALSCOPE, a unifying framework for distilling real-world evidence from population-level observational data. TRIALSCOPE leverages biomedical language models to structure clinical text at scale, employs advanced probabilistic modeling for denoising and imputation, and incorporates state-of-the-art causal inference techniques to combat common confounders. Using clinical trial specification as generic representation, TRIALSCOPE provides a turn-key solution to generate and reason with clinical hypotheses using observational data. In extensive experiments and analyses on a large-scale real-world dataset with over one million cancer patients from a large US healthcare network, we show that TRIALSCOPE can produce high-quality structuring of real-world data and generates comparable results to marquee cancer trials. In addition to facilitating in-silicon clinical trial design and optimization, TRIALSCOPE may be used to empower synthetic controls, pragmatic trials, post-market surveillance, as well as support fine-grained patient-like-me reasoning in precision diagnosis and treatment., Comment: 6 Figures, 22 Pages, 3 Tables

Background: Atrial fibrillation (AF) is the most common sustained cardiac arrhythmia, affecting over 1% of the population. It is usually triggered by irregular electrical impulses that cause the atria to contract irregularly and ineffectively. It increases blood stasis and the risk of thrombus formation within the left atrial appendage (LAA) and aggravates adverse atrial remodeling. Despite recent efforts, LAA flow patterns representative of AF conditions and their association with LAA stasis remain poorly characterized. Aim: To develop reduced-order data-driven models of LAA flow patterns during atrial remodeling in order to uncover flow disturbances concurrent with LAA stasis that could add granularity to clinical decision criteria. Methods: We combined a geometric data augmentation process with projection of results from 180 CFD atrial simulations on a universal LAA coordinate (ULAAC) system. The projection approach enhances data visualization and facilitates direct comparison between different anatomical and functional states. ULAAC projections were used as input for a proper orthogonal decomposition (POD) algorithm to build reduced-order models of hemodynamic metrics, extracting flow characteristics associated with AF and non-AF anatomies. Results: We verified that the ULAAC system provides an adequate representation to visualize data distributions on the LAA surface and to build POD-based reduced-order models. These models revealed significant differences in LAA flow patterns for atrial geometries that underwent adverse atrial remodeling and experienced elevated blood stasis. Together with anatomical morphing-based patient-specific data augmentation, this approach could facilitate data-driven analyses to identify flow features associated with thrombosis risk due to atrial remodeling., Comment: 21 pages, 10 figures

In this paper we establish spectral comparison results for Schr\"odinger operators on a certain class of infinite quantum graphs, using recent results obtained in the finite setting. We also show that new features do appear on infinite quantum graphs such as a modified local Weyl law. In this sense, we regard this paper as a starting point for a more thorough investigation of spectral comparison results on more general infinite metric graphs., Comment: 11 pages, 3 figures, comments welcome!

Based on the main result presented in a recent paper, we derive Ambarzumian-type theorems for Schr\"odinger operators defined on quantum graphs. We recover existing results such as the classical theorem by Ambarzumian and establish some seemingly new statements, too., Comment: 5 pages, comments welcome!

Clinical trial matching is a key process in health delivery and discovery. In practice, it is plagued by overwhelming unstructured data and unscalable manual processing. In this paper, we conduct a systematic study on scaling clinical trial matching using large language models (LLMs), with oncology as the focus area. Our study is grounded in a clinical trial matching system currently in test deployment at a large U.S. health network. Initial findings are promising: out of box, cutting-edge LLMs, such as GPT-4, can already structure elaborate eligibility criteria of clinical trials and extract complex matching logic (e.g., nested AND/OR/NOT). While still far from perfect, LLMs substantially outperform prior strong baselines and may serve as a preliminary solution to help triage patient-trial candidates with humans in the loop. Our study also reveals a few significant growth areas for applying LLMs to end-to-end clinical trial matching, such as context limitation and accuracy, especially in structuring patient information from longitudinal medical records., Comment: 24 pages, 5 figures, accepted at Machine Learning for Healthcare (MLHC) 2023

We study integral kernels of strongly continuous semigroups on Lebesgue spaces over metric measure spaces. Based on semigroup smoothing properties and abstract Morrey-type inequalities, we give sufficient conditions for H\"older or Lipschitz continuity of the kernels. We apply our results to (pseudo)differential operators on domains and quantum graphs, to Laplacians on a class of fractals including the Sierpi\'nski gasket, and to structurally damped wave equations. An extension to non-autonomous problems is also discussed.

Cyber Threat Intelligence (CTI) plays a crucial role in assessing risks and enhancing security for organizations. However, the process of extracting relevant information from unstructured text sources can be expensive and time-consuming. Our empirical experience shows that existing tools for automated structured CTI extraction have performance limitations. Furthermore, the community lacks a common benchmark to quantitatively assess their performance. We fill these gaps providing a new large open benchmark dataset and aCTIon, a structured CTI information extraction tool. The dataset includes 204 real-world publicly available reports and their corresponding structured CTI information in STIX format. Our team curated the dataset involving three independent groups of CTI analysts working over the course of several months. To the best of our knowledge, this dataset is two orders of magnitude larger than previously released open source datasets. We then design aCTIon, leveraging recently introduced large language models (GPT3.5) in the context of two custom information extraction pipelines. We compare our method with 10 solutions presented in previous work, for which we develop our own implementations when open-source implementations were lacking. Our results show that aCTIon outperforms previous work for structured CTI extraction with an improvement of the F1-score from 10%points to 50%points across all tasks.

In this note we elaborate on some notions of surface area for discrete graphs which are closely related to the inverse degree. These notions then naturally lead to associated connectivity measures of graphs and to the definition of a special class of large graphs, called social graphs, that might prove interesting for applications. In addition, we derive spectral estimates involving the surface area and, as a main result, present an upper bound on the second eigenvalue for planar graphs which in some cases improves upon existing bounds from D. A. Spielman and S.-H. Teng, Spectral partitioning works: Planar graphs and finite element meshes, and M. Pl\"umer, Upper eigenvalue bounds for the Kirchhoff Laplacian on embedded metric graphs., Comment: 15 pages; six figures; comments welcome!

We study Schr\"odinger operators on compact finite metric graphs subject to $\delta$-coupling and standard boundary conditions. We compare the $n$-th eigenvalues of those self-adjoint realizations and derive an asymptotic result for the mean value of deviations. By doing this, we generalize recent results from Rudnick et al. obtained for domains in $\mathbb{R}^2$ to the setting of quantum graphs. This also leads to a generalization of related results previously and independently obtained in [arXiv:2212.09143] and [arXiv:2212.12531] for metric graphs. In addition, based on our main result, we introduce some notions of circumference for a (quantum) graph which might prove useful in the future., Comment: 10 pages

We report the Transiting Exoplanet Survey Satellite (TESS) discovery of a three-planet system around the bright Sun-like star HD~22946(V=8.3 mag),also known as TIC~100990000, located 63 parsecs away.The system was observed by TESS in Sectors 3, 4, 30 and 31 and two planet candidates, labelled TESS Objects of Interest (TOIs) 411.01 (planet $c$) and 411.02 (planet $b$), were identified on orbits of 9.57 and 4.04 days, respectively. In this work, we validate the two planets and recover an additional single transit-like signal in the light curve, which suggests the presence of a third transiting planet with a longer period of about 46 days.We assess the veracity of the TESS transit signals and use follow-up imaging and time series photometry to rule out false positive scenarios, including unresolved binary systems, nearby eclipsing binaries or background/foreground stars contaminating the light curves. Parallax measurements from Gaia EDR3, together with broad-band photometry and spectroscopic follow-up by TFOP allowed us to constrain the stellar parameters of TOI-411, including its radius of$1.157\pm0.025R_\odot$. Adopting this value, we determined the radii for the three exoplanet candidates and found that planet $b$ is a super-Earth, with a radius of $1.72\pm0.10R_\oplus$, while planet $c$ and $d$ are sub-Neptunian planets, with radii of$2.74\pm0.14R_\oplus$ and $3.23\pm0.19R_\oplus$ respectively. By using dynamical simulations, we assessed the stability of the system and evaluated the possibility of the presence of other undetected, non-transiting planets by investigating its dynamical packing. We find that the system is dynamically stable and potentially unpacked, with enough space to host at least one more planet between $c$ and $d$.(Abridged), Comment: 21 pages, 12 figures. Accepted for publication on A&A

This paper characterizes equilibrium properties of a broad class of economic models that allow multiple heterogeneous agents to interact in heterogeneous manners across several markets. Our key contribution is a new theorem providing sufficient conditions for uniqueness and stability of equilibria in this class of models. To illustrate the applicability of our theorem, we characterize the general equilibrium properties of two commonly used quantitative trade models. Specifically, our analysis provides a first proof of uniqueness and stability of the equilibrium in multi-country trade models featuring (i) multiple sectors, or (ii) heterogeneity across countries in terms of their labor cost shares. These examples also provide a practical toolkit for future research on how our theorem can be applied to establish uniqueness and stability of equilibria in a broad set of economic models.

While monitoring system behavior to detect anomalies and failures is important, existing methods based on log-analysis can only be as good as the information contained in the logs, and other approaches that look at the OS-level software state introduce high overheads. We tackle the problem with syslrn, a system that first builds an understanding of a target system offline, and then tailors the online monitoring instrumentation based on the learned identifiers of normal behavior. While our syslrn prototype is still preliminary and lacks many features, we show in a case study for the monitoring of OpenStack failures that it can outperform state-of-the-art log-analysis systems with little overhead.

Objective: The majority of detailed patient information in real-world data (RWD) is only consistently available in free-text clinical documents. Manual curation is expensive and time-consuming. Developing natural language processing (NLP) methods for structuring RWD is thus essential for scaling real-world evidence generation. Materials and Methods: Traditional rule-based systems are vulnerable to the prevalent linguistic variations and ambiguities in clinical text, and prior applications of machine-learning methods typically require sentence-level or report-level labeled examples that are hard to produce at scale. We propose leveraging patient-level supervision from medical registries, which are often readily available and capture key patient information, for general RWD applications. To combat the lack of sentence-level or report-level annotations, we explore advanced deep-learning methods by combining domain-specific pretraining, recurrent neural networks, and hierarchical attention. Results: We conduct an extensive study on 135,107 patients from the cancer registry of a large integrated delivery network (IDN) comprising healthcare systems in five western US states. Our deep learning methods attain test AUROC of 94-99% for key tumor attributes and comparable performance on held-out data from separate health systems and states. Discussion and Conclusion: Ablation results demonstrate clear superiority of these advanced deep-learning methods over prior approaches. Error analysis shows that our NLP system sometimes even corrects errors in registrar labels. We also conduct a preliminary investigation in accelerating registry curation and general RWD structuring via assisted curation for over 1.2 million cancer patients in this healthcare network.

Programmable data plane technology enables the systematic reconfiguration of the low-level processing steps applied to network packets and is a key driver in realizing the next generation of network services and applications. This survey presents recent trends and issues in the design and implementation of programmable network devices, focusing on prominent architectures, abstractions, algorithms, and applications proposed, debated, and realized over the past years. We elaborate on the trends that led to the emergence of this technology and highlight the most important pointers from the literature, casting different taxonomies for the field and identifying avenues for future research.