Search

One of the most promising use-cases for machine learning in industrial manufacturing is the early detection of defective products using a quality control system. Such a system can save costs and reduces human errors due to the monotonous nature of visual inspections. Today, a rich body of research exists which employs machine learning methods to identify rare defective products in unbalanced visual quality control datasets. These methods typically rely on two components: A visual backbone to capture the features of the input image and an anomaly detection algorithm that decides if these features are within an expected distribution. With the rise of transformer architecture as visual backbones of choice, there exists now a great variety of different combinations of these two components, ranging all along the trade-off between detection quality and inference time. Facing this variety, practitioners in the field often have to spend a considerable amount of time on researching the right combination for their use-case at hand. Our contribution is to help practitioners with this choice by reviewing and evaluating current vision transformer models together with anomaly detection methods. For this, we chose SotA models of both disciplines, combined them and evaluated them towards the goal of having small, fast and efficient anomaly detection models suitable for industrial manufacturing. We evaluated the results of our experiments on the well-known MVTecAD and BTAD datasets. Moreover, we give guidelines for choosing a suitable model architecture for a quality control system in practice, considering given use-case and hardware constraints.

Understanding how deep learning models predict oncology patient risk can provide critical insights into disease progression, support clinical decision-making, and pave the way for trustworthy and data-driven precision medicine. Building on recent advances in the spatial modeling of the tumor microenvironment using graph neural networks, we present an explainable cell graph (xCG) approach for survival prediction. We validate our model on a public cohort of imaging mass cytometry (IMC) data for 416 cases of lung adenocarcinoma. We explain survival predictions in terms of known phenotypes on the cell level by computing risk attributions over cell graphs, for which we propose an efficient grid-based layer-wise relevance propagation (LRP) method. Our ablation studies highlight the importance of incorporating the cancer stage and model ensembling to improve the quality of risk estimates. Our xCG method, together with the IMC data, is made publicly available to support further research., Comment: Findings paper presented at Machine Learning for Health (ML4H) symposium 2024, December 15-16, 2024, Vancouver, Canada, 11 pages

Modern AI agents, driven by advances in large foundation models, promise to enhance our productivity and transform our lives by augmenting our knowledge and capabilities. To achieve this vision, AI agents must effectively plan, perform multi-step reasoning and actions, respond to novel observations, and recover from errors, to successfully complete complex tasks across a wide range of scenarios. In this work, we introduce Magentic-One, a high-performing open-source agentic system for solving such tasks. Magentic-One uses a multi-agent architecture where a lead agent, the Orchestrator, plans, tracks progress, and re-plans to recover from errors. Throughout task execution, the Orchestrator directs other specialized agents to perform tasks as needed, such as operating a web browser, navigating local files, or writing and executing Python code. We show that Magentic-One achieves statistically competitive performance to the state-of-the-art on three diverse and challenging agentic benchmarks: GAIA, AssistantBench, and WebArena. Magentic-One achieves these results without modification to core agent capabilities or to how they collaborate, demonstrating progress towards generalist agentic systems. Moreover, Magentic-One's modular design allows agents to be added or removed from the team without additional prompt tuning or training, easing development and making it extensible to future scenarios. We provide an open-source implementation of Magentic-One, and we include AutoGenBench, a standalone tool for agentic evaluation. AutoGenBench provides built-in controls for repetition and isolation to run agentic benchmarks in a rigorous and contained manner -- which is important when agents' actions have side-effects. Magentic-One, AutoGenBench and detailed empirical performance evaluations of Magentic-One, including ablations and error analysis are available at https://aka.ms/magentic-one

Language models (LMs) have demonstrated expert-level reasoning and recall abilities in medicine. However, computational costs and privacy concerns are mounting barriers to wide-scale implementation. We introduce a parsimonious adaptation of phi-3-mini, MedMobile, a 3.8 billion parameter LM capable of running on a mobile device, for medical applications. We demonstrate that MedMobile scores 75.7% on the MedQA (USMLE), surpassing the passing mark for physicians (~60%), and approaching the scores of models 100 times its size. We subsequently perform a careful set of ablations, and demonstrate that chain of thought, ensembling, and fine-tuning lead to the greatest performance gains, while unexpectedly retrieval augmented generation fails to demonstrate significant improvements, Comment: 13 pages, 5 figures (2 main, 3 supplementary)

We present and analyze a two-level restricted additive Schwarz (RAS) preconditioner for heterogeneous Helmholtz problems, based on a multiscale spectral generalized finite element method (MS-GFEM) proposed in [C. Ma, C. Alber, and R. Scheichl, SIAM. J. Numer. Anal., 61 (2023), pp. 1546--1584]. The preconditioner uses local solves with impedance boundary conditions, and a global coarse solve based on the MS-GFEM approximation space constructed from local eigenproblems. It is derived by first formulating MS-GFEM as a Richardson iterative method, and without using an oversampling technique, reduces to the preconditioner recently proposed and analyzed in [Q. Hu and Z.Li, arXiv 2402.06905]. We prove that both the Richardson iterative method and the preconditioner used within GMRES converge at a rate of $\Lambda$ under some reasonable conditions, where $\Lambda$ denotes the error of the underlying MS-GFEM \rs{approximation}. Notably, the convergence proof of GMRES does not rely on the `Elman theory'. An exponential convergence property of MS-GFEM, resulting from oversampling, ensures that only a few iterations are needed for convergence with a small coarse space. Moreover, the convergence rate $\Lambda$ is not only independent of the fine-mesh size $h$ and the number of subdomains, but decays with increasing wavenumber $k$. In particular, in the constant-coefficient case, with $h\sim k^{-1-\gamma}$ for some $\gamma\in (0,1]$, it holds that $\Lambda \sim k^{-1+\frac{\gamma}{2}}$.

While previous studies have demonstrated the potential of AI to diagnose diseases in imaging data, clinical implementation is still lagging behind. This is partly because AI models require training with large numbers of examples only available for common diseases. In clinical reality, however, only few diseases are common, whereas the majority of diseases are less frequent (long-tail distribution). Current AI models overlook or misclassify these diseases. We propose a deep anomaly detection approach that only requires training data from common diseases to detect also all less frequent diseases. We collected two large real-world datasets of gastrointestinal biopsies, which are prototypical of the problem. Herein, the ten most common findings account for approximately 90% of cases, whereas the remaining 10% contained 56 disease entities, including many cancers. 17 million histological images from 5,423 cases were used for training and evaluation. Without any specific training for the diseases, our best-performing model reliably detected a broad spectrum of infrequent ("anomalous") pathologies with 95.0% (stomach) and 91.0% (colon) AUROC and generalized across scanners and hospitals. By design, the proposed anomaly detection can be expected to detect any pathological alteration in the diagnostic tail of gastrointestinal biopsies, including rare primary or metastatic cancers. This study establishes the first effective clinical application of AI-based anomaly detection in histopathology that can flag anomalous cases, facilitate case prioritization, reduce missed diagnoses and enhance the general safety of AI models, thereby driving AI adoption and automation in routine diagnostics and beyond.

We present a multiscale mixed finite element method for solving second order elliptic equations with general $L^{\infty}$-coefficients arising from flow in highly heterogeneous porous media. Our approach is based on a multiscale spectral generalized finite element method (MS-GFEM) and exploits the superior local mass conservation properties of mixed finite elements. Following the MS-GFEM framework, optimal local approximation spaces are built for the velocity field by solving local eigenvalue problems over generalized harmonic spaces. The resulting global velocity space is then enriched suitably to ensure inf-sup stability. We develop the mixed MS-GFEM for both continuous and discrete formulations, with Raviart-Thomas based mixed finite elements underlying the discrete method. Exponential convergence with respect to local degrees of freedom is proven at both the continuous and discrete levels. Numerical results are presented to support the theory and to validate the proposed method.

Together with the molecular knowledge of genes and proteins, biological images promise to significantly enhance the scientific understanding of complex cellular systems and to advance predictive and personalized therapeutic products for human health. For this potential to be realized, quality-assured image data must be shared among labs at a global scale to be compared, pooled, and reanalyzed, thus unleashing untold potential beyond the original purpose for which the data was generated. There are two broad sets of requirements to enable image data sharing in the life sciences. One set of requirements is articulated in the companion White Paper entitled Enabling Global Image Data Sharing in the Life Sciences, which is published in parallel and addresses the need to build the cyberinfrastructure for sharing the digital array data. In this White Paper, we detail a broad set of requirements, which involves collecting, managing, presenting, and propagating contextual information essential to assess the quality, understand the content, interpret the scientific implications, and reuse image data in the context of the experimental details. We start by providing an overview of the main lessons learned to date through international community activities, which have recently made considerable progress toward generating community standard practices for imaging Quality Control (QC) and metadata. We then provide a clear set of recommendations for amplifying this work. The driving goal is to address remaining challenges and democratize access to everyday practices and tools for a spectrum of biomedical researchers, regardless of their expertise, access to resources, and geographical location., Comment: This manuscript is published with a closely related companion entitled, Enabling Global Image Data Sharing in the Life Sciences, which can be found at the following link, arXiv:2401.13023 [q-bio.OT]

Artificial intelligence has started to transform histopathology impacting clinical diagnostics and biomedical research. However, while many computational pathology approaches have been proposed, most current AI models are limited with respect to generalization, application variety, and handling rare diseases. Recent efforts introduced self-supervised foundation models to address these challenges, yet existing approaches do not leverage pathologist knowledge by design. In this study, we present a novel approach to designing foundation models for computational pathology, incorporating pathologist expertise, semi-automated data curation, and a diverse dataset from over 15 laboratories, including 58 tissue types, and encompassing 129 different histochemical and immunohistochemical staining modalities. We demonstrate that our model "RudolfV" surpasses existing state-of-the-art foundation models across different benchmarks focused on tumor microenvironment profiling, biomarker evaluation, and reference case search while exhibiting favorable robustness properties. Our study shows how domain-specific knowledge can increase the efficiency and performance of pathology foundation models and enable novel application areas.

Sufficient and necessary conditions are presented for the existence of $(N,M)$-positive operator valued measures ($(N,M)$-POVMs) valid for arbitrary-dimensional quantum systems. A sufficient condition for the existence of $(N,M)$-POVMs is presented. It yields a simple relation determining an upper bound on the continuous parameter of an arbitrary $(N,M)$-POVM, below which all its POVM elements are guaranteed to be positive semidefinite. Necessary conditions are derived for the existence of optimal $(N,M)$-POVMs. One of these necessary conditions exhibits a close connection between the existence of optimal informationally complete $(N,M)$-POVMs and the existence of isospectral, traceless, orthonormal, hermitian operator bases in cases, in which the parameter $M$ exceeds the dimension of the quantum system under consideration. Another necessary condition is derived for optimal $(N,M)$-POVMs, whose parameter $M$ is less than the dimension of the quantum system. It is shown that in these latter cases all POVM elements necessarily are projection operators of equal rank. This significantly constrains the possible parameters for constructing optimal $(N,M)$-POVMs. For the special case of $M=2$ a necessary and sufficient condition for the existence of optimal $(N,2)$-POVMs is presented.

We demonstrate and characterize a first-principles approach to modeling the mass action dynamics of metabolism. Starting from a basic definition of entropy expressed as a multinomial probability density using Boltzmann probabilities with standard chemical potentials, we derive and compare the free energy dissipation and the entropy production rates. We express the relation between the entropy production and the chemical master equation for modeling metabolism, which unifies chemical kinetics and chemical thermodynamics. Subsequent implementation of an maximum free energy dissipation model for systems of coupled reactions is accomplished by using an approximation to the Marcelin equation for mass action kinetics that maximizes the entropy production. Because prediction uncertainty with respect to parameter variability is frequently a concern with mass action models utilizing rate constants, we compare and contrast the maximum entropy production model, which has its own set of rate parameters, to a population of standard mass action models in which the rate constants are randomly chosen. We show that a maximum entropy production model is characterized by a high probability of free energy dissipation rate, and likewise entropy production rate, relative to other models. We then characterize the variability of the maximum entropy production predictions with respect to uncertainties in parameters (standard free energies of formation) and with respect to ionic strengths typically found in a cell.

Tracking of plant cells in images obtained by microscope is a challenging problem due to biological phenomena such as large number of cells, non-uniform growth of different layers of the tightly packed plant cells and cell division. Moreover, images in deeper layers of the tissue being noisy and unavoidable systemic errors inherent in the imaging process further complicates the problem. In this paper, we propose a novel learning-based method that exploits the tightly packed three-dimensional cell structure of plant cells to create a three-dimensional graph in order to perform accurate cell tracking. We further propose novel algorithms for cell division detection and effective three-dimensional registration, which improve upon the state-of-the-art algorithms. We demonstrate the efficacy of our algorithm in terms of tracking accuracy and inference-time on a benchmark dataset.

Gaze tracking devices have the potential to greatly expand interactivity, yet miscalibration remains a significant barrier to use. As devices miscalibrate, people tend to compensate by intentionally offsetting their gaze, which makes detecting miscalibration from eye signals difficult. To help address this problem, we propose a novel approach to seamless calibration based on the insight that the system's model of eye gaze can be updated during reading (user does not compensate) to improve calibration for typing (user might compensate). To explore this approach, we built an auto-calibrating gaze typing prototype called EyeO, ran a user study with 20 participants, and conducted a semi-structured interview with 6 ALS community stakeholders. Our user study results suggest that seamless autocalibration can significantly improve typing efficiency and user experience. Findings from the semi-structured interview validate the need for autocalibration, and shed light on the prototype's potential usefulness, desired algorithmic and design improvements for users.

Recently proposed correlation-matrix based sufficient conditions for bipartite steerability from Alice to Bob are applied to local informationally complete positive operator valued measures (POVMs) of the $(N,M)$-type. These POVMs allow for a unified description of a large class of local generalized measurements of current interest. It is shown that this sufficient condition exhibits a peculiar scaling property. It implies that all types of informationally complete $(N,M)$-POVMs are equally powerful in detecting bipartite steerability from Alice to Bob and, in addition, they are as powerful as local orthonormal hermitian operator bases (LOOs). In order to explore the typicality of steering numerical calculations of lower bounds on Euclidean volume ratios between steerable bipartite quantum states from Alice to Bob and all quantum states are determined with the help of a hit-and-run Monte-Carlo algorithm. These results demonstrate that with the single exception of two qubits this correlation-matrix based sufficient condition significantly underestimates these volume ratios. These results are also compared with a recently proposed method which reduces the determination of bipartite steerability from Alice's qubit to Bob's arbitrary dimensional quantum system to the determination of bipartite entanglement. It is demonstrated that in general this method is significantly more effective in detecting typical steerability provided entanglement detection methods are used which transcend local measurements.

Entanglement detection by local measurements, which can possibly be performed by far distant observers, are of particular interest for applications in quantum key distribution and quantum communication. In this paper sufficient conditions for arbitrary dimensional bipartite entanglement detection based on correlation matrices and joint probability distributions of such local measurements are investigated. In particular, their dependence on the nature of the local measurements is explored for typical bipartite quantum states and for measurements involving local orthonormal hermitian operators bases (LOOs) or generalized measurements based on informationally complete positive operator valued measures of the recently introduced $(N,M)$-type ($(N,M)$-POVMs) \cite{NMPOVM}. It is shown that symmetry properties of $(N,M)$-POVMs imply that sufficient conditions for bipartite entanglement detection exhibit peculiar scaling properties relating different equally efficient local entanglement detection scenarios. For correlation-matrix based bipartite local entanglement detection, for example, this has the consequence that LOOs and all informationally complete $(N,M)$-POVMs are equally powerful. With the help of a hit-and-run Monte-Carlo algorithm the effectiveness of local entanglement detection of typical bipartite quantum states is explored numerically. For this purpose Euclidean volume ratios between locally detectable entangled states and all bipartite quantum states are determined.

With the recent availability of tissue-specific gene expression data, e.g., provided by the GTEx Consortium, there is interest in comparing gene co-expression patterns across tissues. One promising approach to this problem is to use a multilayer network analysis framework and perform multilayer community detection. Communities in gene co-expression networks reveal groups of genes similarly expressed across individuals, potentially involved in related biological processes responding to specific environmental stimuli or sharing common regulatory variations. We construct a multilayer network in which each of the four layers is an exocrine gland tissue-specific gene co-expression network. We develop methods for multilayer community detection with correlation matrix input and an appropriate null model. Our correlation matrix input method identifies five groups of genes that are similarly co-expressed in multiple tissues (a community that spans multiple layers, which we call a generalist community) and two groups of genes that are co-expressed in just one tissue (a community that lies primarily within just one layer, which we call a specialist community). We further found gene co-expression communities where the genes physically cluster across the genome significantly more than expected by chance (on chromosomes 1 and 11). This clustering hints at underlying regulatory elements determining similar expression patterns across individuals and cell types. We suggest that KRTAP3-1, KRTAP3-3, and KRTAP3-5 share regulatory elements in skin and pancreas. Furthermore, we find that CELA3A and CELA3B share associated expression quantitative trait loci in the pancreas. The results indicate that our multilayer community detection method for correlation matrix input extracts biologically interesting communities of genes.

In this paper, we present a deep learning segmentation approach to classify and quantify the two most prevalent primary liver cancers - hepatocellular carcinoma and intrahepatic cholangiocarcinoma - from hematoxylin and eosin (H&E) stained whole slide images. While semantic segmentation of medical images typically requires costly pixel-level annotations by domain experts, there often exists additional information which is routinely obtained in clinical diagnostics but rarely utilized for model training. We propose to leverage such weak information from patient diagnoses by deriving complementary labels that indicate to which class a sample cannot belong to. To integrate these labels, we formulate a complementary loss for segmentation. Motivated by the medical application, we demonstrate for general segmentation tasks that including additional patches with solely weak complementary labels during model training can significantly improve the predictive performance and robustness of a model. On the task of diagnostic differentiation between hepatocellular carcinoma and intrahepatic cholangiocarcinoma, we achieve a balanced accuracy of 0.91 (CI 95%: 0.86 - 0.95) at case level for 165 hold-out patients. Furthermore, we also show that leveraging complementary labels improves the robustness of segmentation and increases performance at case level.

Authentication, authorization, and trust verification are central parts of an access control system. The conditions for granting access in such a system are collected in access policies. Since access conditions are often complex, dedicated languages -- policy languages -- for defining policies are in use. However, current policy languages are unable to express such conditions having privacy of users in mind. With privacy-preserving technologies, users are enabled to prove information to the access system without revealing it. In this work, we present a generic design for supporting privacy-preserving technologies in policy languages. Our design prevents unnecessary disclosure of sensitive information while still allowing the formulation of expressive rules for access control. For that we make use of zero-knowledge proofs (NIZKs). We demonstrate our design by applying it to the TPL policy language, while using SNARKs. Also, we evaluate the resulting ZK-TPL language and its associated toolchain. Our evaluation shows that for regular-sized credentials communication and verification overhead is negligible., Comment: This is the full version of a paper which appears in 21th IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom 2022)