Search

Self-supervised pre-training of deep learning models with contrastive learning is a widely used technique in image analysis. Current findings indicate a strong potential for contrastive pre-training on medical images. However, further research is necessary to incorporate the particular characteristics of these images. We hypothesize that the similarity of medical images hinders the success of contrastive learning in the medical imaging domain. To this end, we investigate different strategies based on deep embedding, information theory, and hashing in order to identify and reduce redundancy in medical pre-training datasets. The effect of these different reduction strategies on contrastive learning is evaluated on two pre-training datasets and several downstream classification tasks. In all of our experiments, dataset reduction leads to a considerable performance gain in downstream tasks, e.g., an AUC score improvement from 0.78 to 0.83 for the COVID CT Classification Grand Challenge, 0.97 to 0.98 for the OrganSMNIST Classification Challenge and 0.73 to 0.83 for a brain hemorrhage classification task. Furthermore, pre-training is up to nine times faster due to the dataset reduction. In conclusion, the proposed approach highlights the importance of dataset quality and provides a transferable approach to improve contrastive pre-training for classification downstream tasks on medical images., Comment: Published in Computers in Biology and Medicine

Numerical models of geothermal reservoirs typically depend on hundreds or thousands of unknown parameters, which must be estimated using sparse, noisy data. However, these models capture complex physical processes, which frequently results in long run-times and simulation failures, making the process of estimating the unknown parameters a challenging task. Conventional techniques for parameter estimation and uncertainty quantification, such as Markov chain Monte Carlo (MCMC), can require tens of thousands of simulations to provide accurate results and are therefore challenging to apply in this context. In this paper, we study the ensemble Kalman inversion (EKI) algorithm as an alternative technique for approximate parameter estimation and uncertainty quantification for geothermal reservoir models. EKI possesses several characteristics that make it well-suited to a geothermal setting; it is derivative-free, parallelisable, robust to simulation failures, and requires far fewer simulations than conventional uncertainty quantification techniques such as MCMC. We illustrate the use of EKI in a reservoir modelling context using a combination of synthetic and real-world case studies. Through these case studies, we also demonstrate how EKI can be paired with flexible parametrisation techniques capable of accurately representing prior knowledge of the characteristics of a reservoir and adhering to geological constraints, and how the algorithm can be made robust to simulation failures. Our results demonstrate that EKI provides a reliable and efficient means of obtaining accurate parameter estimates for large-scale, two-phase geothermal reservoir models, with appropriate characterisation of uncertainty.

Detecting arbitrarily shaped clusters in high-dimensional noisy data is challenging for current clustering methods. We introduce SHADE (Structure-preserving High-dimensional Analysis with Density-based Exploration), the first deep clustering algorithm that incorporates density-connectivity into its loss function. Similar to existing deep clustering algorithms, SHADE supports high-dimensional and large data sets with the expressive power of a deep autoencoder. In contrast to most existing deep clustering methods that rely on a centroid-based clustering objective, SHADE incorporates a novel loss function that captures density-connectivity. SHADE thereby learns a representation that enhances the separation of density-connected clusters. SHADE detects a stable clustering and noise points fully automatically without any user input. It outperforms existing methods in clustering quality, especially on data that contain non-Gaussian clusters, such as video data. Moreover, the embedded space of SHADE is suitable for visualization and interpretation of the clustering results as the individual shapes of the clusters are preserved., Comment: Short version accepted at ICDM 2024

Observational studies showed that galaxy disks are already in place in the first few billion years of the universe. The early disks detected so far, with typical half-light radii of 3 kiloparsecs at stellar masses around 10^11 M_sun for redshift z~3, are significantly smaller than today's disks with similar masses, in agreement with expectations from current galaxy models. Here, we report observations of a giant disk at z=3.25, when the universe was only 2 billion years old, with a half-light radius of 9.6 kiloparsecs and stellar mass of 3.7^+2.6_-2.2x10^11 M_sun. This galaxy is larger than any other kinematically-confirmed disks at similar epochs and surprisingly similar to today's largest disks regarding size and mass. JWST imaging and spectroscopy reveal its spiral morphology and a rotational velocity consistent with local Tully-Fisher relation. Multi-wavelength observations show that it lies in an exceptionally dense environment, where the galaxy number density is over ten times higher than the cosmic average and mergers are frequent. The discovery of such a giant disk suggests the presence of favorable physical conditions for large-disk formation in dense environments in the early universe, which may include efficient accretion of gas carrying coherent angular momentum and non-destructive mergers between exceptionally gas-rich progenitor galaxies., Comment: 22 pages, 11 figures; submitted

Large language models (LLMs), such as ChatGPT and Copilot, are transforming software development by automating code generation and, arguably, enable rapid prototyping, support education, and boost productivity. Therefore, correctness and quality of the generated code should be on par with manually written code. To assess the current state of LLMs in generating correct code of high quality, we conducted controlled experiments with ChatGPT and Copilot: we let the LLMs generate simple algorithms in Java and Python along with the corresponding unit tests and assessed the correctness and the quality (coverage) of the generated (test) codes. We observed significant differences between the LLMs, between the languages, between algorithm and test codes, and over time. The present paper reports these results together with the experimental methods allowing repeated and comparable assessments for more algorithms, languages, and LLMs over time.

In the era of big data, machine learning (ML) has become a powerful tool in various fields, notably impacting structural dynamics. ML algorithms offer advantages by modeling physical phenomena based on data, even in the absence of underlying mechanisms. However, uncertainties such as measurement noise and modeling errors can compromise the reliability of ML predictions, highlighting the need for effective uncertainty awareness to enhance prediction robustness. This paper presents a comprehensive review on navigating uncertainties in ML, categorizing uncertainty-aware approaches into probabilistic methods (including Bayesian and frequentist perspectives) and non-probabilistic methods (such as interval learning and fuzzy learning). Bayesian neural networks, known for their uncertainty quantification and nonlinear mapping capabilities, are emphasized for their superior performance and potential. The review covers various techniques and methodologies for addressing uncertainties in ML, discussing fundamentals and implementation procedures of each method. While providing a concise overview of fundamental concepts, the paper refrains from in-depth critical explanations. Strengths and limitations of each approach are examined, along with their applications in structural dynamic forward problems like response prediction, sensitivity assessment, and reliability analysis, and inverse problems like system identification, model updating, and damage identification. Additionally, the review identifies research gaps and suggests future directions for investigations, aiming to provide comprehensive insights to the research community. By offering an extensive overview of both probabilistic and non-probabilistic approaches, this review aims to assist researchers and practitioners in making informed decisions when utilizing ML techniques to address uncertainties in structural dynamic problems., Comment: 114 pages, 27 figures, 6 tables, references added

The European Central Bank is preparing for the potential issuance of a central bank digital currency (CBDC), called the digital euro. A recent regulatory proposal by the European Commission defines several requirements for the digital euro, such as support for both online and offline payments. Offline payments are expected to enable cash-like privacy, local payment settlement, and the enforcement of holding limits. While other central banks have expressed similar desired functionality, achieving such offline payments poses a novel technical challenge. We observe that none of the existing research solutions, including offline E-cash schemes, are fully compliant. Proposed solutions based on secure elements offer no guarantees in case of compromise and can therefore lead to significant payment fraud. The main contribution of this paper is PayOff, a novel CBDC design motivated by the digital euro regulation, which focuses on offline payments. We analyze the security implications of local payment settlement and identify new security objectives. PayOff protects user privacy, supports complex regulations such as holding limits, and implements safeguards to increase robustness against secure element failure. Our analysis shows that PayOff provides strong privacy and identifies residual leakages that may arise in real-world deployments. Our evaluation shows that offline payments can be fast and that the central bank can handle high payment loads with moderate computing resources. However, the main limitation of PayOff is that offline payment messages and storage requirements grow in the number of payments that the sender makes or receives without going online in between.

We introduce MOSCITO (MOlecular Dynamics Subspace Clustering with Temporal Observance), a subspace clustering for molecular dynamics data. MOSCITO groups those timesteps of a molecular dynamics trajectory together into clusters in which the molecule has similar conformations. In contrast to state-of-the-art methods, MOSCITO takes advantage of sequential relationships found in time series data. Unlike existing work, MOSCITO does not need a two-step procedure with tedious post-processing, but directly models essential properties of the data. Interpreting clusters as Markov states allows us to evaluate the clustering performance based on the resulting Markov state models. In experiments on 60 trajectories and 4 different proteins, we show that the performance of MOSCITO achieves state-of-the-art performance in a novel single-step method. Moreover, by modeling temporal aspects, MOSCITO obtains better segmentation of trajectories, especially for small numbers of clusters., Comment: Accepted as a research paper at BIOKDD 2024

The ability to make accurate predictions with quantified uncertainty provides a crucial foundation for the successful management of a geothermal reservoir. Conventional approaches for making predictions using geothermal reservoir models involve estimating unknown model parameters using field data, then propagating the uncertainty in these estimates through to the predictive quantities of interest. However, the unknown parameters are not always of direct interest; instead, the predictions are of primary importance. Data space inversion (DSI) is an alternative methodology that allows for the efficient estimation of predictive quantities of interest, with quantified uncertainty, that avoids the need to estimate model parameters entirely. In this paper, we evaluate the applicability of DSI to geothermal reservoir modelling. We first review the processes of model calibration, prediction and uncertainty quantification from a Bayesian perspective, and introduce data space inversion as a simple, efficient technique for approximating the posterior predictive distribution. We then apply the DSI framework to two model problems in geothermal reservoir modelling. We evaluate the accuracy and efficiency of DSI relative to other common methods for uncertainty quantification, study how the number of reservoir model simulations affects the resulting approximation to the posterior predictive distribution, and demonstrate how the framework can be enhanced through the use of suitable reparametrisations. Our results support the idea that data space inversion is a simple, robust and efficient technique for making predictions with quantified uncertainty using geothermal reservoir models, providing a useful alternative to more conventional approaches.

Network reliability analysis remains a challenge due to the increasing size and complexity of networks. This paper presents a novel sampling method and an active learning method for efficient and accurate network reliability estimation under node failure and edge failure scenarios. The proposed sampling method adopts Monte Carlo technique to sample component lifetimes and the K-terminal spanning tree algorithm to accelerate structure function computation. Unlike existing methods that compute only one structure function value per sample, our method generates multiple component state vectors and corresponding structure function values from each sample. Network reliability is estimated based on survival signatures derived from these values. A transformation technique extends this method to handle both node failure and edge failure. To enhance efficiency of proposed sampling method and achieve adaptability to network topology changes, we introduce an active learning method utilizing a random forest (RF) classifier. This classifier directly predicts structure function values, integrates network behaviors across diverse topologies, and undergoes iterative refinement to enhance predictive accuracy. Importantly, the trained RF classifier can directly predict reliability for variant networks, a capability beyond the sampling method alone. Through investigating several network examples and two practical applications, the effectiveness of both proposed methods is demonstrated.

Many dynamical systems are subjected to stochastic influences, such as random excitations, noise, and unmodeled behavior. Tracking the system's state and parameters based on a physical model is a common task for which filtering algorithms, such as Kalman filters and their non-linear extensions, are typically used. However, many of these filters use assumptions on the transition probabilities or the covariance model, which can lead to inaccuracies in non-linear systems. We will show the application of a stochastic coupling filter that can approximate arbitrary transition densities under non-Gaussian noise. The filter is based on transport maps, which couple the approximation densities to a user-chosen reference density, allowing for straightforward sampling and evaluation of probabilities., Comment: Published in Advances in Reliability, Safety and Security ESREL 2024 Contributions, https://esrel2024.com/wp-content/uploads/articles/part9/transport-map-coupling-filter-for-state-parameter-estimation.pdf

We present the study of millisecond-resolved polymer brush swelling dynamics using infrared spectroscopy with a custom-built quantum cascade laser-based infrared spectrometer at a 1 kHz sampling rate after averaging. By cycling the humidity of the environment of the polymer brush, we are able to measure the swelling dynamics sequentially at different wavenumbers. The high sampling rate provides us with information on the reconformation of the brush at a higher temporal resolution than previously reported. Using spectroscopic ellipsometry, we study the brush swelling dynamics as a reference experiment and to correct artefacts of the infrared measurement approach. This technique informs on the changes in the brush thickness and refractive index. Our results indicate that the swelling dynamics of the polymer brush are poorly described by Fickian diffusion and the Berens-Hopfenberg formalism, pointing toward more complicated underlying transport.

Due to the sensitive nature of medicine, it is particularly important and highly demanded that AI methods are explainable. This need has been recognised and there is great research interest in xAI solutions with medical applications. However, there is a lack of user-centred evaluation regarding the actual impact of the explanations. We evaluate attribute- and prototype-based explanations with the Proto-Caps model. This xAI model reasons the target classification with human-defined visual features of the target object in the form of scores and attribute-specific prototypes. The model thus provides a multimodal explanation that is intuitively understandable to humans thanks to predefined attributes. A user study involving six radiologists shows that the explanations are subjectivly perceived as helpful, as they reflect their decision-making process. The results of the model are considered a second opinion that radiologists can discuss using the model's explanations. However, it was shown that the inclusion and increased magnitude of model explanations objectively can increase confidence in the model's predictions when the model is incorrect. We can conclude that attribute scores and visual prototypes enhance confidence in the model. However, additional development and repeated user studies are needed to tailor the explanation to the respective use case., Comment: Accepted at The 2nd World Conference on eXplainable Artificial Intelligence

We conducted measurements of $K^- + {^3{\rm He}} \to \pi \!Y \!N + N'$ reactions using a $1~{\rm GeV}/c$ $K^-$-beam, with the objective of understanding the broad decay width of $\bar{K} \!N \!N$ (approximately twice as broad as that of $\Lambda(1405)$ considered to be the $\bar{K} \!N$ quasi-bound state). We successfully reproduced distributions of the $\pi \! Y \! N$ invariant mass and momentum transfer for $\pi \! Y \! N$ using model fitting functions for $\bar{K} \!N \!N$ formation and quasi-free $\bar{K}$ absorption (${\rm QF}_{\bar{K}-{\rm abs}}$) processes. The model can describe the experimental data quite well, and four $\bar{K} \! N \! N \to \pi \! Y \! N $ cross-sections were obtained. The results indicate that mesonic decay is the dominant decay branch of $\bar{K} \! N \! N$. The results also suggest that $\Gamma_{\pi \Lambda N} \sim \Gamma_{\pi \Sigma N}$, which indicates that the $I_{\bar{K} \! N}=1$ absorption channel, in addition to the $I_{\bar{K} \! N}=0$ absorption channel, substantially contribute to the $\bar{K} \! N \! N$ decay, making the $\bar{K} \! N \! N$ state approximately twice as unstable as $\Lambda$(1405).

Poor generalization performance caused by distribution shifts in unseen domains often hinders the trustworthy deployment of deep neural networks. Many domain generalization techniques address this problem by adding a domain invariant regularization loss terms during training. However, there is a lack of modular software that allows users to combine the advantages of different methods with minimal effort for reproducibility. DomainLab is a modular Python package for training user specified neural networks with composable regularization loss terms. Its decoupled design allows the separation of neural networks from regularization loss construction. Hierarchical combinations of neural networks, different domain generalization methods, and associated hyperparameters, can all be specified together with other experimental setup in a single configuration file. Hierarchical combinations of neural networks, different domain generalization methods, and associated hyperparameters, can all be specified together with other experimental setup in a single configuration file. In addition, DomainLab offers powerful benchmarking functionality to evaluate the generalization performance of neural networks in out-of-distribution data. The package supports running the specified benchmark on an HPC cluster or on a standalone machine. The package is well tested with over 95 percent coverage and well documented. From the user perspective, it is closed to modification but open to extension. The package is under the MIT license, and its source code, tutorial and documentation can be found at https://github.com/marrlab/DomainLab.

We address the online combinatorial choice of weight multipliers for multi-objective optimization of many loss terms parameterized by neural works via a probabilistic graphical model (PGM) for the joint model parameter and multiplier evolution process, with a hypervolume based likelihood promoting multi-objective descent. The corresponding parameter and multiplier estimation as a sequential decision process is then cast into an optimal control problem, where the multi-objective descent goal is dispatched hierarchically into a series of constraint optimization sub-problems. The subproblem constraint automatically adapts itself according to Pareto dominance and serves as the setpoint for the low level multiplier controller to schedule loss landscapes via output feedback of each loss term. Our method is multiplier-free and operates at the timescale of epochs, thus saves tremendous computational resources compared to full training cycle multiplier tuning. It also circumvents the excessive memory requirements and heavy computational burden of existing multi-objective deep learning methods. We applied it to domain invariant variational auto-encoding with 6 loss terms on the PACS domain generalization task, and observed robust performance across a range of controller hyperparameters, as well as different multiplier initial conditions, outperforming other multiplier scheduling methods. We offered modular implementation of our method, admitting extension to custom definition of many loss terms.

In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February version on the great majority of capabilities and benchmarks; (2) Gemini 1.5 Flash, a more lightweight variant designed for efficiency with minimal regression in quality. Gemini 1.5 models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra's state-of-the-art performance across a broad set of benchmarks. Studying the limits of Gemini 1.5's long-context ability, we find continued improvement in next-token prediction and near-perfect retrieval (>99%) up to at least 10M tokens, a generational leap over existing models such as Claude 3.0 (200k) and GPT-4 Turbo (128k). Finally, we highlight real-world use cases, such as Gemini 1.5 collaborating with professionals on completing their tasks achieving 26 to 75% time savings across 10 different job categories, as well as surprising new capabilities of large language models at the frontier; when given a grammar manual for Kalamang, a language with fewer than 200 speakers worldwide, the model learns to translate English to Kalamang at a similar level to a person who learned from the same content.

We present the development of multifunctional blue-emission organic light-emitting diodes (OLEDs) using TADF-exciplex materials. These OLEDs exhibit sensitivity to external stimuli and achieve a maximum external quantum efficiency (EQE) of 11.6 % through partly liquid processing. This technique allows for large-scale production on arbitrary geometries. The potential multifunctionality of the devices arises from their response to low external magnetic fields (up to 100 mT) with an efficiency up to 2.5 % for magnetoconductance, while maximum magneto-electroluminescence effects of 4.1 % were detected. We investigated novel aspects, including the utilization of two organic materials without further doping and the investigation of the impact of 2,2',2''-(1,3,5-Benzinetriyl)-tris(1phenyl-1-H-benzimidazole) (TPBi) processing in liquid and vapor form. The insights gained provide a fundamental understanding regarding the applicability of exciplex (EX) materials for fully solution-processed OLEDs through a deliberate omission of doping. Our work represents a significant advancement on the path towards multifunctional OLED technology, with potential applications in cost-efficient, scalable organic full-color displays and advanced sensing system, Comment: 25 pages manuscript, 3 pages supplementary information

We study the reentrant condensation of polyelectrolytes in dilute solutions of small multivalent salts, whose phase-transition mechanism remains poorly understood. Motivated by recent full atomic simulation results reported by the Caltech group on phase behaviors of polyelectrolytes in presence of multivalent salts (DOIs: 10.1021/acs.macromol.3c02437 and 10.1021/acs.langmuir.3c03640), in this work we construct a simple but effective mean-field model which can rationalize the essential features of the reentrant condensation including the phase diagram of polyelectrolyte. The model unveils that the strong adsorption between the ionic monomers and multivalent ions can be at the origin of the peculiar phenomenon that rather low concentrations of multivalent salts trigger both collapse and re-entry transitions. For the first time, the analytical solution of the model indicates that a minimum of coupling energy due to sharing multivalent salt ions between ionic monomers is essential for a phase transition to occur, which can explain the enigmatic observation that polyelectrolytes can only show phase transition in a dilute solution of salts with selective multivalency. Our analytical calculations also show that the incompatibility of the uncharged moieties of the polyelectrolytes with water is critical to regulate phase behaviors of polyelectrolytes in aqueous solutions. This is in agreement with recent experimental investigations on solution properties of amphiphilic proteins. The obtained results will contribute to the understanding of liquid-liquid phase separation in biological systems where multivalent ions bound to bio-polyelectrolytes play an essential role., Comment: 37 pages, 11 figures

Sub-mm surveys toward overdense regions in the early Universe are essential to uncover the obscured star-formation and the cold gas content of assembling galaxies within massive dark matter halos. In this work, we present deep ALMA mosaic observations covering an area of $\sim 2'\times2'$ around MQN01 (MUSE Quasar Nebula 01), one of the largest and brightest Ly-$\alpha$ emitting nebulae discovered thus far surrounding a radio-quiet quasar at $z\simeq3.25$. Our observations target the 1.2- and the 3-mm dust continuum, as well as the carbon monoxide CO(4-3) transition in galaxies in the vicinity of the quasar. We identify a robust sample of eleven CO line-emitting galaxies (including a closely-separated quasar companion) which lie within $\pm 4000\,{\rm km\,s^{-1}}$ relatively to the quasar systemic redshift. A fraction of these objects are missed in previous deep rest-frame optical/UV surveys thus highlighting the critical role of (sub-)mm imaging. We also detect a total of eleven sources revealed in their 1.2-mm dust continuum with six of them having either high-fidelity spectroscopic redshift information from rest-frame UV metal absorptions, or CO line which place them in the same narrow redshift range. A comparison of the CO luminosity function (LF) and 1.2-mm number count density with that of the general fields points to a galaxy overdensity of $\delta > 10$. We find evidence of a systematic flattening at the bright-end of the CO LF with respect to the trend measured in blank fields. Our findings reveal that galaxies in dense regions at $z\sim3$ are more massive and significantly richer in molecular gas than galaxies in fields, hence enabling a faster and accelerated assembly. This is the first of a series of studies to characterize one of the densest regions of the Universe found so far at $z > 3$., Comment: 25 pages, 14 figures. Accepted for publication in A&A

Reddiment is a web-based dashboard that links sentiment analysis of subreddit texts with share prices. The system consists of a backend, frontend and various services. The backend, in Node.js, manages the data and communicates with crawlers that collect Reddit comments and stock market data. Sentiment is analyzed with the help of Vader and TextBlob. The frontend, based on SvelteKit, provides users with a dashboard for visualization. The distribution is carried out via Docker containers and Docker Compose. The project offers expansion options, e.g. the integration of cryptocurrency rates. Reddiment enables the analysis of sentiment and share prices from subreddit data., Comment: Ostbayerische Technische Hochschule Amberg-Weiden, CyberLytics-Lab, Technical Reports, CL-2022-06, July 2022, in German language