Search

Selective Classification, wherein models can reject low-confidence predictions, promises reliable translation of machine-learning based classification systems to real-world scenarios such as clinical diagnostics. While current evaluation of these systems typically assumes fixed working points based on pre-defined rejection thresholds, methodological progress requires benchmarking the general performance of systems akin to the $\mathrm{AUROC}$ in standard classification. In this work, we define 5 requirements for multi-threshold metrics in selective classification regarding task alignment, interpretability, and flexibility, and show how current approaches fail to meet them. We propose the Area under the Generalized Risk Coverage curve ($\mathrm{AUGRC}$), which meets all requirements and can be directly interpreted as the average risk of undetected failures. We empirically demonstrate the relevance of $\mathrm{AUGRC}$ on a comprehensive benchmark spanning 6 data sets and 13 confidence scoring functions. We find that the proposed metric substantially changes metric rankings on 5 out of the 6 data sets.

High-quality, stable child care and early education (CCEE) can have lasting, positive impacts on children. However, there are ongoing challenges in recruiting, supporting, and retaining a qualified, stable CCEE workforce that provides high-quality care. CCEE educators typically have low levels of compensation; limited opportunities for education, training, and professional development; inconsistent working conditions; and high levels of stress and burnout. There are also high rates of job turnover, which can diminish the quality of care by straining remaining educators. The COVID-19 pandemic has exacerbated these issues. While several states and localities are taking steps to build and stabilize their CCEE workforce, important questions remain about how to best strengthen the workforce to meet the needs of children and families, particularly those who experience poverty or who are racially, ethnically, and linguistically diverse. The "Building and Sustaining the Child Care and Early Education Workforce" (BASE) project team completed a literature review and an environmental scan that were designed to identify and document existing knowledge about the CCEE workforce and develop strategies to strengthen it. The literature review synthesized research on factors that shape CCEE workforce dynamics, defined here as how educators enter, advance in, and exit out of different roles, settings, and types of care. The literature review predominantly focused on studies conducted between 2017 and 2021 and examined the effectiveness of strategies that were implemented to build a sustained, qualified workforce. The environmental scan identified and reviewed the range of strategies currently being used across the country to build, advance, and sustain the CCEE workforce. The team also completed a data scan that assessed the strengths and weaknesses of several potential data sources that could be used to examine workforce dynamics and address key gaps in existing research.

Child care and early education (CCEE) are essential resources for families and children across the country. Yet qualified educators are choosing to leave their positions, and young professionals are choosing not to enter the field. This instability has been a long-lasting issue and is thought to be driven largely by structural and systemic forces that shape the everyday experiences of the CCEE workforce. These issues have been exacerbated by the COVID-19 pandemic. This report lays out a conceptual framework to serve as a guide for researchers, policymakers, and administrators who are conducting research on the CCEE workforce and who are designing and implementing strategies to improve entry, retention, turnover, and advancement of the workforce. This report is also intended to inform how the field can work to build and sustain this critical workforce. The conceptual framework consists of three main components that are intended to provide a clear understanding of factors that can disrupt the stability of the CCEE workforce pipeline. By articulating the main components, the conceptual framework encourages researchers, policymakers, and administrators to consider a range of multilevel influences on workforce dynamics when designing strategies and when evaluating whether strategies are working as intended. This framework also aims to underscore that the individuals who make up the workforce--many of whom have been marginalized due to race, ethnicity, social class, and gender--interact with broader social, political, and cultural contexts that shape economic and educational policies, structures, and opportunities.

Facial expression perception in humans inherently relies on prior knowledge and contextual cues, contributing to efficient and flexible processing. For instance, multi-modal emotional context (such as voice color, affective text, body pose, etc.) can prompt people to perceive emotional expressions in objectively neutral faces. Drawing inspiration from this, we introduce a novel approach for facial expression classification that goes beyond simple classification tasks. Our model accurately classifies a perceived face and synthesizes the corresponding mental representation perceived by a human when observing a face in context. With this, our model offers visual insights into its internal decision-making process. We achieve this by learning two independent representations of content and context using a VAE-GAN architecture. Subsequently, we propose a novel attention mechanism for context-dependent feature adaptation. The adapted representation is used for classification and to generate a context-augmented expression. We evaluate synthesized expressions in a human study, showing that our model effectively produces approximations of human mental representations. We achieve State-of-the-Art classification accuracies of 81.01% on the RAVDESS dataset and 79.34% on the MEAD dataset. We make our code publicly available., Comment: GCPR 2024

In non-relativistic physics the concepts of geometry and topology are usually applied to characterise spatial structures, or structures in momentum space. We introduce the concept of temporal geometry, which encompasses the geometric and topological properties of temporal shapes, i.e. trajectories traced by a tip of a time-dependent vector in vector space. We apply it to vectors of ultrafast electron current or induced polarization in chiral molecules. The central concepts of temporal geometry - curvature and connection - emerge as ubiquitous features of photoexcited, non-equilibrium, chiral electron dynamics. We demonstrate that curvature and connection (i) rely on the interplay of molecular chirality and the polarization properties of light pulses, (ii) can be introduced for multiphoton processes, and (iii) control enantio-sensitive geometric observables via non-equilibrium electronic dynamics excited by tailored laser fields. Our findings may open a way to ultrafast, topologically non-trivial, and enantio-sensitive chemical dynamics., Comment: 32 pages, 5 figures

The imitation of cursive handwriting is mainly limited to generating handwritten words or lines. Multiple synthetic outputs must be stitched together to create paragraphs or whole pages, whereby consistency and layout information are lost. To close this gap, we propose a method for imitating handwriting at the paragraph level that also works for unseen writing styles. Therefore, we introduce a modified latent diffusion model that enriches the encoder-decoder mechanism with specialized loss functions that explicitly preserve the style and content. We enhance the attention mechanism of the diffusion model with adaptive 2D positional encoding and the conditioning mechanism to work with two modalities simultaneously: a style image and the target text. This significantly improves the realism of the generated handwriting. Our approach sets a new benchmark in our comprehensive evaluation. It outperforms all existing imitation methods at both line and paragraph levels, considering combined style and content preservation.

Ultrathin metasurface holograms, with thicknesses comparable to the operating wavelength, leverage multiple degrees of freedom of light to address independent image channels, thereby significantly enhancing information capacity. Although the wavelength of light can be used to encode holographic image channels, high-capacity wavelength-multiplexing holography has traditionally been achieved only through 3D volume holograms based on Bragg diffraction. We demonstrate ultranarrow-linewidth wavelength-vortex multiplexing holography in ultrathin metasurface holograms. By applying dispersion engineering to the elementary grating functions of a multiplexing hologram, we develop a sparse k-vector-filtering aperture array in momentum space that achieves sharp wavelength selectivity in conjunction with orbital angular momentum selectivity. Further leveraging transformer neural networks for the design of phase-only multiplexing holograms, we reconstruct up to 118 independent image channels from a single metasurface hologram, achieving an ultranarrow linewidth of 2 nm in the visible range. Finally, we apply the developed wavelength-vortex multiplexing metasurface holograms for holographic visual cryptography, achieving unprecedented security with an information rate more than 2500 times higher than that of traditional visual cryptography schemes. Our results open exciting avenues for the use of metasurface holograms in various applications, including 3D displays, holographic encryption, beam shaping, LiDAR, microscopy, data storage, and optical artificial intelligence.

Single-photon cameras present a promising avenue for high-resolution 3D imaging. They have ultra-high sensitivity -- down to individual photons -- and can record photon arrival times with extremely high (sub-nanosecond) resolution. Single-photon 3D cameras estimate the round-trip time of a laser pulse by forming equi-width (EW) histograms of detected photon timestamps. Acquiring and transferring such EW histograms requires high bandwidth and in-pixel memory, making SPCs less attractive in resource-constrained settings such as mobile devices and AR/VR headsets. In this work we propose a 3D sensing technique based on equi-depth (ED) histograms. ED histograms compress timestamp data more efficiently than EW histograms, reducing the bandwidth requirement. Moreover, to reduce the in-pixel memory requirement, we propose a lightweight algorithm to estimate ED histograms in an online fashion without explicitly storing the photon timestamps. This algorithm is amenable to future in-pixel implementations. We propose algorithms that process ED histograms to perform 3D computer-vision tasks of estimating scene distance maps and performing visual odometry under challenging conditions such as high ambient light. Our work paves the way towards lower bandwidth and reduced in-pixel memory requirements for SPCs, making them attractive for resource-constrained 3D vision applications. Project page: $\href{https://www.computational.camera/pedh}{https://www.computational.camera/pedh}$

Medical image classification in radiology faces significant challenges, particularly in generalizing to unseen pathologies. In contrast, CLIP offers a promising solution by leveraging multimodal learning to improve zero-shot classification performance. However, in the medical domain, lesions can be small and might not be well represented in the embedding space. Therefore, in this paper, we explore the potential of visual prompt engineering to enhance the capabilities of Vision Language Models (VLMs) in radiology. Leveraging BiomedCLIP, trained on extensive biomedical image-text pairs, we investigate the impact of embedding visual markers directly within radiological images to guide the model's attention to critical regions. Our evaluation on the JSRT dataset, focusing on lung nodule malignancy classification, demonstrates that incorporating visual prompts $\unicode{x2013}$ such as arrows, circles, and contours $\unicode{x2013}$ significantly improves classification metrics including AUROC, AUPRC, F1 score, and accuracy. Moreover, the study provides attention maps, showcasing enhanced model interpretability and focus on clinically relevant areas. These findings underscore the efficacy of visual prompt engineering as a straightforward yet powerful approach to advance VLM performance in medical image analysis., Comment: Accepted at ECCV 2024 Workshop on Emergent Visual Abilities and Limits of Foundation Models

Robust semantic segmentation of intraoperative image data holds promise for enabling automatic surgical scene understanding and autonomous robotic surgery. While model development and validation are primarily conducted on idealistic scenes, geometric domain shifts, such as occlusions of the situs, are common in real-world open surgeries. To close this gap, we (1) present the first analysis of state-of-the-art (SOA) semantic segmentation models when faced with geometric out-of-distribution (OOD) data, and (2) propose an augmentation technique called "Organ Transplantation", to enhance generalizability. Our comprehensive validation on six different OOD datasets, comprising 600 RGB and hyperspectral imaging (HSI) cubes from 33 pigs, each annotated with 19 classes, reveals a large performance drop in SOA organ segmentation models on geometric OOD data. This performance decline is observed not only in conventional RGB data (with a dice similarity coefficient (DSC) drop of 46 %) but also in HSI data (with a DSC drop of 45 %), despite the richer spectral information content. The performance decline increases with the spatial granularity of the input data. Our augmentation technique improves SOA model performance by up to 67 % for RGB data and 90 % for HSI data, achieving performance at the level of in-distribution performance on real OOD test data. Given the simplicity and effectiveness of our augmentation method, it is a valuable tool for addressing geometric domain shifts in surgical scene segmentation, regardless of the underlying model. Our code and pre-trained models are publicly available at https://github.com/IMSY-DKFZ/htc., Comment: Silvia Seidlitz and Jan Sellner contributed equally

Analyzing the cone photoreceptor pattern in images obtained from the living human retina using quantitative methods can be crucial for the early detection and management of various eye conditions. Confocal adaptive optics scanning light ophthalmoscope (AOSLO) imaging enables visualization of the cones from reflections of waveguiding cone photoreceptors. While there have been significant improvements in automated algorithms for segmenting cones in confocal AOSLO images, the process of labelling data remains labor-intensive and manual. This paper introduces a method based on deep learning (DL) for detecting and segmenting cones in AOSLO images. The models were trained on a semi-automatically labelled dataset of 20 AOSLO batches of images of 18 participants for 0$^{\circ}$, 1$^{\circ}$, and 2$^{\circ}$ from the foveal center. F1 scores were 0.968, 0.958, and 0.954 for 0$^{\circ}$, 1$^{\circ}$, and 2$^{\circ}$, respectively, which is better than previously reported DL approaches. Our method minimizes the need for labelled data by only necessitating a fraction of labelled cones, which is especially beneficial in the field of ophthalmology, where labelled data can often be limited.

The precise control of phonon polaritons(PhPs) is essential for advancements in nanophotonic applications like on-chip optical communication and quantum information processing. Ghost hyperbolic phonon polaritons (g-HPs), which have been recently discovered, feature in-plane hyperbolic dispersion and oblique wavefronts, enabling long-range propagation. Despite their potential, controlling the directionality and selective excitation of g-HPs remains challenging. Our research demonstrates that modifying the shape of the launching micro/nano antenna can achieve this control. Using an asymmetric triangular gold antenna on a calcite crystal surface, we achieve highly directional g-HP excitation by selectively targeting specific polariton modes. Additionally, the mode of g-HPs can be adjusted by changing the excitation wavelength or rotating the antenna. Remarkably, our near-field imaging experiments show g-HP propagation over distances exceeding 35 micrometers, more than twice the length reported in previous studies. This work merges g-HP theory with structural engineering, enhancing the control over g-HPs and paving the way for innovative applications in mid-IR optoelectronics.

Generating physician letters is a time-consuming task in daily clinical practice. This study investigates local fine-tuning of large language models (LLMs), specifically LLaMA models, for physician letter generation in a privacy-preserving manner within the field of radiation oncology. Our findings demonstrate that base LLaMA models, without fine-tuning, are inadequate for effectively generating physician letters. The QLoRA algorithm provides an efficient method for local intra-institutional fine-tuning of LLMs with limited computational resources (i.e., a single 48 GB GPU workstation within the hospital). The fine-tuned LLM successfully learns radiation oncology-specific information and generates physician letters in an institution-specific style. ROUGE scores of the generated summary reports highlight the superiority of the 8B LLaMA-3 model over the 13B LLaMA-2 model. Further multidimensional physician evaluations of 10 cases reveal that, although the fine-tuned LLaMA-3 model has limited capacity to generate content beyond the provided input data, it successfully generates salutations, diagnoses and treatment histories, recommendations for further treatment, and planned schedules. Overall, clinical benefit was rated highly by the clinical experts (average score of 3.44 on a 4-point scale). With careful physician review and correction, automated LLM-based physician letter generation has significant practical value.

Accurate assessment of lymph node size in 3D CT scans is crucial for cancer staging, therapeutic management, and monitoring treatment response. Existing state-of-the-art segmentation frameworks in medical imaging often rely on fully annotated datasets. However, for lymph node segmentation, these datasets are typically small due to the extensive time and expertise required to annotate the numerous lymph nodes in 3D CT scans. Weakly-supervised learning, which leverages incomplete or noisy annotations, has recently gained interest in the medical imaging community as a potential solution. Despite the variety of weakly-supervised techniques proposed, most have been validated only on private datasets or small publicly available datasets. To address this limitation, the Mediastinal Lymph Node Quantification (LNQ) challenge was organized in conjunction with the 26th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2023). This challenge aimed to advance weakly-supervised segmentation methods by providing a new, partially annotated dataset and a robust evaluation framework. A total of 16 teams from 5 countries submitted predictions to the validation leaderboard, and 6 teams from 3 countries participated in the evaluation phase. The results highlighted both the potential and the current limitations of weakly-supervised approaches. On one hand, weakly-supervised approaches obtained relatively good performance with a median Dice score of $61.0\%$. On the other hand, top-ranked teams, with a median Dice score exceeding $70\%$, boosted their performance by leveraging smaller but fully annotated datasets to combine weak supervision and full supervision. This highlights both the promise of weakly-supervised methods and the ongoing need for high-quality, fully annotated data to achieve higher segmentation performance., Comment: Submitted to MELBA

With sepsis remaining a leading cause of mortality, early identification of septic patients and those at high risk of death is a challenge of high socioeconomic importance. The driving hypothesis of this study was that hyperspectral imaging (HSI) could provide novel biomarkers for sepsis diagnosis and treatment management due to its potential to monitor microcirculatory alterations. We conducted a comprehensive study involving HSI data of the palm and fingers from more than 480 patients on the day of their intensive care unit (ICU) admission. The findings demonstrate that HSI measurements can predict sepsis with an area under the receiver operating characteristic curve (AUROC) of 0.80 (95 % confidence interval (CI) [0.76; 0.84]) and mortality with an AUROC of 0.72 (95 % CI [0.65; 0.79]). The predictive performance improves substantially when additional clinical data is incorporated, leading to an AUROC of up to 0.94 (95 % CI [0.92; 0.96]) for sepsis and 0.84 (95 % CI [0.78; 0.89]) for mortality. We conclude that HSI presents novel imaging biomarkers for the rapid, non-invasive prediction of sepsis and mortality, suggesting its potential as an important modality for guiding diagnosis and treatment., Comment: Markus A. Weigand, Lena Maier-Hein and Maximilian Dietrich contributed equally

Motivated by the recent rapid progress in high-$T_c$ nickelate superconductors, we comprehensively study the physical properties of the alternating bilayer trilayer stacking nickelate La$_7$Ni$_5$O$_{17}$. The high-symmetry phase of this material, without the tilting of oxygen octahedra, is not stable at ambient conditions but becomes stable under high pressure, where a small hole pocket $\gamma_0$, composed of the $d_{3z^2-r^2}$ states in the trilayer sublattice, appears. This pocket was identified in our previous work for trilayer La$_4$Ni$_3$O$_{10}$ as important to develop superconductivity. Moreover, using random-phase approximation calculations, we find a leading $s^\pm$ pairing state for the high-symmetry phase under pressure with similar pairing strength as that obtained previously for the bilayer La$_3$Ni$_2$O$_7$ compound, suggesting a similar or higher superconducting transition temperature $T_c$. In addition, we find that the dominant magnetic fluctuations in the system driving this pairing state have antiferromagnetic structure both in-plane and between the planes of the top and bottom trilayer and bilayer sublattices, while the middle trilayer is magnetically decoupled.

Motivated by the recently discovered high-$T_c$ bilayer nickelate superconductor La$_3$Ni$_2$O$_7$, we comprehensively research a bilayer $2\times2\times2$ cluster for different electronic densities $n$ by using the Lanczos method. We also employ the random-phase approximation to quantify the first magnetic instability with increasing Hubbard coupling strength, also varying $n$. Based on the spin structure factor $S(q)$, we have obtained a rich magnetic phase diagram in the plane defined by $n$ and $U/W$, at fixed Hund coupling. We have observed numerous states, such as A-AFM, Stripes, G-AFM, and C-AFM. For half-filling $n=2$ (two electrons per Ni site, corresponding to $N$ = 16 electrons), the canonical superexchange interaction leads to a robust G-AFM state $(\pi,\pi,\pi)$ with antiferromagnetic couplings in plane and between layers. By increasing or decreasing electronic densities, ferromagnetic tendencies emerge from the ``half-empty'' and ``half-full'' mechanisms, leading to many other interesting magnetic tendencies. In addition, the spin-spin correlations become weaker both in the hole or electron doping regions compared with half-filling. At $n = 1.5$ (or $N=12$), density corresponding to La$_3$Ni$_2$O$_7$, we obtained the ``Stripe 2'' ground state (antiferromagnetic coupling in one in-plane direction, ferromagnetic coupling in the other, and antiferromagnetic coupling along the $z$-axis) in the $2\times2\times2$ cluster. In addition, we obtained a much stronger AFM coupling along the $z$-axis than the magnetic coupling in the $xy$ plane. The random-phase approximation calculations with varying $n$ give very similar results as Lanczos. Meanwhile, a state with $q/\pi = (0.6, 0.6, 1)$ close to the E-phase wavevector is found in our RPA calculations by slightly reducing the filling to $n=1.25$, possibly responsible for the E-phase SDW recently observed in experiments., Comment: 12 pages, 9 figures

In specific motifs of three recurrently connected neurons with probabilistic response, the spontaneous information flux, defined as the mutual information between subsequent states, has been shown to increase by adding ongoing white noise of some optimal strength to each of the neurons \cite{krauss2019recurrence}. However, the precise conditions for and mechanisms of this phenomenon called 'recurrence resonance' (RR) remain largely unexplored. Using Boltzmann machines of different sizes and with various types of weight matrices, we show that RR can generally occur when a system has multiple dynamical attractors, but is trapped in one or a few of them. In probabilistic networks, the phenomenon is bound to a suitable observation time scale, as the system could autonomously access its entire attractor landscape even without the help of external noise, given enough time. Yet, even in large systems, where time scales for observing RR in the full network become too long, the resonance can still be detected in small subsets of neurons. Finally, we show that short noise pulses can be used to transfer recurrent neural networks, both probabilistic and deterministic, between their dynamical attractors. Our results are relevant to the fields of reservoir computing and neuroscience, where controlled noise may turn out a key factor for efficient information processing leading to more robust and adaptable systems.

Automated Machine Learning (AutoML) significantly simplifies the deployment of machine learning models by automating tasks from data preprocessing to model selection to ensembling. AutoML systems for tabular data often employ post hoc ensembling, where multiple models are combined to improve predictive accuracy. This typically results in longer inference times, a major limitation in practical deployments. Addressing this, we introduce a hardware-aware ensemble selection approach that integrates inference time into post hoc ensembling. By leveraging an existing framework for ensemble selection with quality diversity optimization, our method evaluates ensemble candidates for their predictive accuracy and hardware efficiency. This dual focus allows for a balanced consideration of accuracy and operational efficiency. Thus, our approach enables practitioners to choose from a Pareto front of accurate and efficient ensembles. Our evaluation using 83 classification datasets shows that our approach sustains competitive accuracy and can significantly improve ensembles' operational efficiency. The results of this study provide a foundation for extending these principles to additional hardware constraints, setting the stage for the development of more resource-efficient AutoML systems., Comment: Accepted at Third International Conference on Automated Machine Learning (AutoML 2024), Workshop Track; for code, see https://github.com/Atraxus/HA-ES

Measuring the similarity of different representations of neural architectures is a fundamental task and an open research challenge for the machine learning community. This paper presents the first comprehensive benchmark for evaluating representational similarity measures based on well-defined groundings of similarity. The representational similarity (ReSi) benchmark consists of (i) six carefully designed tests for similarity measures, (ii) 23 similarity measures, (iii) eleven neural network architectures, and (iv) six datasets, spanning over the graph, language, and vision domains. The benchmark opens up several important avenues of research on representational similarity that enable novel explorations and applications of neural architectures. We demonstrate the utility of the ReSi benchmark by conducting experiments on various neural network architectures, real world datasets and similarity measures. All components of the benchmark are publicly available and thereby facilitate systematic reproduction and production of research results. The benchmark is extensible, future research can build on and further expand it. We believe that the ReSi benchmark can serve as a sound platform catalyzing future research that aims to systematically evaluate existing and explore novel ways of comparing representations of neural architectures., Comment: Feedback welcome! Code and data at https://github.com/mklabunde/resi

Controlled long-range transport of micro- and nano-scale objects is a key requirement in lab-on-a-chip and microfluidic applications, enabling the efficient capture, concentration, manipulation, and detection of analytes. Traditional methods such as microfluidic pumps and optical trapping face challenges including high power consumption and limited range of action. This study introduces a plasmonic Brownian ratchet designed for the directed transport of dielectric nanometer-sized particles at low optical powers. Through numerical simulations, the ratchet geometry was optimized to enhance electric fields, optical forces, and trapping potentials. Experimentally, the plasmonic ratchet demonstrated the ability to rectify random thermal motion of 40 nm polysterene spheres over extended distances in a specific direction, achieving velocities up to 2.4 $\mu$m/s at excitation powers as low as 0.785 kW/cm^2. This plasmonic ratchet offers a robust and efficient solution for the targeted delivery and concentration of nanoscale analytes on chips, with significant implications for advancing applications in the life sciences., Comment: to be submitted to ACS Nano

Most complex diseases, including cancer and non-malignant diseases like asthma, have distinct molecular subtypes that require distinct clinical approaches. However, existing computational patient stratification methods have been benchmarked almost exclusively on cancer omics data and only perform well when mutually exclusive subtypes can be characterized by many biomarkers. Here, we contribute with a massive evaluation attempt, quantitatively exploring the power of 22 unsupervised patient stratification methods using both, simulated and real transcriptome data. From this experience, we developed UnPaSt (https://apps.cosy.bio/unpast/) optimizing unsupervised patient stratification, working even with only a limited number of subtype-predictive biomarkers. We evaluated all 23 methods on real-world breast cancer and asthma transcriptomics data. Although many methods reliably detected major breast cancer subtypes, only few identified Th2-high asthma, and UnPaSt significantly outperformed its closest competitors in both test datasets. Essentially, we showed that UnPaSt can detect many biologically insightful and reproducible patterns in omic datasets., Comment: The first two authors listed are joint first authors. The last two authors listed are joint last authors

The application of machine learning in medicine and healthcare has led to the creation of numerous diagnostic and prognostic models. However, despite their success, current approaches generally issue predictions using data from a single modality. This stands in stark contrast with clinician decision-making which employs diverse information from multiple sources. While several multimodal machine learning approaches exist, significant challenges in developing multimodal systems remain that are hindering clinical adoption. In this paper, we introduce a multimodal framework, AutoPrognosis-M, that enables the integration of structured clinical (tabular) data and medical imaging using automated machine learning. AutoPrognosis-M incorporates 17 imaging models, including convolutional neural networks and vision transformers, and three distinct multimodal fusion strategies. In an illustrative application using a multimodal skin lesion dataset, we highlight the importance of multimodal machine learning and the power of combining multiple fusion strategies using ensemble learning. We have open-sourced our framework as a tool for the community and hope it will accelerate the uptake of multimodal machine learning in healthcare and spur further innovation.

This paper does not describe a novel method. Instead, it studies an essential foundation for reliable benchmarking and ultimately real-world application of AI-based image analysis: generating high-quality reference annotations. Previous research has focused on crowdsourcing as a means of outsourcing annotations. However, little attention has so far been given to annotation companies, specifically regarding their internal quality assurance (QA) processes. Therefore, our aim is to evaluate the influence of QA employed by annotation companies on annotation quality and devise methodologies for maximizing data annotation efficacy. Based on a total of 57,648 instance segmented images obtained from a total of 924 annotators and 34 QA workers from four annotation companies and Amazon Mechanical Turk (MTurk), we derived the following insights: (1) Annotation companies perform better both in terms of quantity and quality compared to the widely used platform MTurk. (2) Annotation companies' internal QA only provides marginal improvements, if any. However, improving labeling instructions instead of investing in QA can substantially boost annotation performance. (3) The benefit of internal QA depends on specific image characteristics. Our work could enable researchers to derive substantially more value from a fixed annotation budget and change the way annotation companies conduct internal QA., Comment: Accepted at ECCV 2024, preprint, Computer Vision, Data Annotation

In this paper we obtain a new solution of Einstein field equations which describes a boosted Kerr-Newman black hole relative to a Lorentz frame at future null infinity. To simplify our analysis we consider a particular configuration in which the boost is aligned with the black hole angular momentum. The boosted Kerr-Newman black hole is obtained considering the complete asymptotic Lorentz transformations of Robinson-Trautman coordinates to Bondi-Sachs, including the perturbation term of the boosted Robinson-Trautman metric. To verify that the final form of the metric is indeed a solution of Einstein field equations, we evaluate the corresponding energy-momentum tensor the boosted Kerr-Newman solution. To this end, we consider the electromagnetic energy-momentum tensor built with the Kerr boosted metric together with its timelike killing vector. We show that the Papapetrou field thus obtained engender an energy-momentum tensor which satisfies Einstein field equations up to 4th order for the Kerr-Newman metric. To proceed, we examine the causal structure of the boosted Kerr-Newman black hole in Bondi-Sachs coordinates as in a preferred timelike foliation. We show that the ultimate effect of a nonvanishing charge is to shrink the overall size of the event horizon and ergosphere areas when compared to the neutral boosted Kerr black holes. Considering the preferred timelike foliation we obtain the electromagnetic fields for a proper nonrotating frame of reference. We show that while the electric field displays a pure radial behaviour, the magnetic counterpart develops an involved structure with two intense lobes of the magnetic field observed in the direction opposite to the boost.

Understanding the nature and identity of dark matter is a key goal in the physics community. In the case that TeV-scale dark matter particles decay or annihilate into standard model particles, very-high-energy (VHE) gamma rays (greater than 100 GeV) will be present in the final state. The Very Energetic Radiation Imaging Telescope Array System (VERITAS) is an imaging atmospheric Cherenkov telescope array that can indirectly detect VHE gamma rays in an energy range of 100 GeV to > 30 TeV. Dwarf spheroidal galaxies (dSphs) are ideal candidates in the search for dark matter due to their high dark matter content, high mass-to-light ratios, and their low gamma-ray fluxes from astrophysical processes. This study uses a legacy data set of 638 hours collected on 17 dSphs, built over 11 years with an observing strategy optimized according to the dark matter content of the targets. The study addresses a broad dark matter particle mass range, extending from 200 GeV to 30 PeV. In the absence of a detection, we set the upper limits on the dark matter velocity-weighted annihilation cross section., Comment: 18 pages, 7 figures, 3 tables, accepted in PRD

Photonics has been revolutionized by breakthroughs in optical metasurfaces and layered two-dimensional materials. Yet, integrating these two fields in a singular system has remained challenging. Here, we introduce the concept of van der Waals (vdW) heterostructure metasurfaces, where ultrathin multilayer vdW material stacks are shaped into precisely engineered resonant nanostructures for boosting light-matter interactions. By leveraging quasi-bound states in the continuum to create intrinsic cavities from WS$_2$ monolayers encapsulated in hexagonal boron nitride, we observe room-temperature strong coupling and polaritonic luminescence, which further unveils a saturation of the strong-coupling regime at ultralow fluences <1 nJ/cm2, more than three orders of magnitude smaller than in previous 2D-cavity systems. Our approach, seamlessly merging metasurfaces and vdW materials, unlocks new avenues for ultrathin optical devices with atomic-scale precision and control., Comment: main + supplementary (28 pages, 4 main figures, 9 supplementary figures)

We report on recent progress on five-loop calculations in perturbative QCD. We discuss the computation of the perturbative quark condensate, decoupling in QCD, and the precision determination of charm and bottom quark masses. For the latter, we give some first results from a gauge-invariant subset of five-loop diagrams., Comment: 12 pages, 2 figures. Contribution to proceedings of the Loops and Legs in Quantum Field Theory conference (LL2024)

In February 2017, the blazar OJ~287 underwent a period of intense multiwavelength activity. It reached a new historic peak in the soft X-ray (0.3-10 keV) band, as measured by Swift-XRT. This event coincides with a very-high-energy (VHE) $\gamma$-ray outburst that led VERITAS to detect emission above 100 GeV, with a detection significance of $10\sigma$ (from 2016 December 9 to 2017 March 31). The time-averaged VHE $\gamma$-ray spectrum was consistent with a soft power law ($\Gamma = -3.81 \pm 0.26$) and an integral flux corresponding to $\sim2.4\%$ that of the Crab Nebula above the same energy. Contemporaneous data from multiple instruments across the electromagnetic spectrum reveal complex flaring behavior, primarily in the soft X-ray and VHE bands. To investigate the possible origin of such an event, our study focuses on three distinct activity states: before, during, and after the February 2017 peak. The spectral energy distributions during these periods suggest the presence of at least two non-thermal emission zones, with the more compact one responsible for the observed flare. Broadband modeling results and observations of a new radio knot in the jet of OJ~287 in 2017 are consistent with a flare originating from a strong recollimation shock outside the radio core., Comment: 21 pages, 7 figures, accepted for publication in The Astrophysical Journal

Olfaction, often overlooked in cultural heritage studies, holds profound significance in shaping human experiences and identities. Examining historical depictions of olfactory scenes can offer valuable insights into the role of smells in history. We show that a transfer-learning approach using weakly labeled training data can remarkably improve the classification of fragrant spaces and, more generally, artistic scene depictions. We fine-tune Places365-pre-trained models by querying two cultural heritage data sources and using the search terms as supervision signal. The models are evaluated on two manually corrected test splits. This work lays a foundation for further exploration of fragrant spaces recognition and artistic scene classification. All images and labels are released as the ArtPlaces dataset at https://zenodo.org/doi/10.5281/zenodo.11584328.

The interaction of light with superluminally moving matter entails unconventional phenomena, from Fresnel drag to Hawking radiation and to light amplification. While relativity makes these effects inaccessible using objects in motion, synthetic motion - enabled via space-time modulated internal degrees of freedom - is free from these constraints. Here we observe synthetic velocity of a reflectivity modulation travelling on an Indium-Tin-Oxide (ITO) interface, generated by ultrafast laser illumination at multiple positions and times. The interaction of the moving reflectivity modulation with a probe light beam acts as a non-separable spatio-temporal transformation that diffracts the light, changing its frequency and momentum content. The recorded frequency-momentum diffraction pattern is defined by the velocity of the diffracted probe wave relative to the modulation. Our experiments open a path towards mimicking relativistic mechanics and developing programmable spatio-temporal transformations of light., Comment: 9 pages, 3 figures

The electroretinogram (ERG) is a clinical test that records the retina's electrical response to light. The ERG is a promising way to study different neurodevelopmental and neurodegenerative disorders, including autism spectrum disorder (ASD) - a neurodevelopmental condition that impacts language, communication, and reciprocal social interactions. However, in heterogeneous populations, such as ASD, where the ability to collect large datasets is limited, the application of artificial intelligence (AI) is complicated. Synthetic ERG signals generated from real ERG recordings carry similar information as natural ERGs and, therefore, could be used as an extension for natural data to increase datasets so that AI applications can be fully utilized. As proof of principle, this study presents a Generative Adversarial Network capable of generating synthetic ERG signals of children with ASD and typically developing control individuals. We applied a Time Series Transformer and Visual Transformer with Continuous Wavelet Transform to enhance classification results on the extended synthetic signals dataset. This approach may support classification models in related psychiatric conditions where the ERG may help classify disorders.

In the case of single-digit picosecond bunch length, synchrotron light sources produce intense coherent radiation up to the THz range. The reduction of the bunch length by lowering the momentum compaction factor (low-$\alpha$) gives rise to the micro-bunching instability, which is on one hand a crucial roadblock in the X-ray range during to the resulting effective bunch lengthening but on the other hand also an opportunity for the generation of intense THz radiation if it can be controlled appropriately. In the KIT storage ring KARA (Karlsruhe Research Accelerator), two parallel plates with periodic rectangular corrugations are planned to be installed in an electron storage ring. These plates create an additional longitudinal impedance based on their geometry, which can affect the beam dynamics. The resulting impedance manipulation will be used to study and control the longitudinal electron beam dynamics and the emitted coherent synchrotron radiation (CSR). This paper presents the results of systematic studies in simulation of the impact of additional corrugated plate impedances on the longitudinal beam dynamics using the example of the KARA storage ring. If the periodicity of the wake function of the corrugated plates matches the size of the substructures in the longitudinal bunch profile, the instability threshold can be effectively manipulated. This extends intense THz radiation to different beam current regimes., Comment: 11 pages, 13 figures, to be published in Physical Review Accelerators and Beams

We studied the frequency and quality factor of mechanical plasmonic nanoresonators as a function of temperature, ranging from ambient to 4 K. Our investigation focused on individual gold nanorods and nanodisks of various sizes. We observed that oscillation frequencies increase linearly as temperature decreases until saturation is reached at cryogenic temperatures. This behavior is explained by the temperature dependence of the elastic modulus, with a Debye temperature compatible with reported bulk values for gold. To describe the behavior of the quality factor, we developed a model considering the nanostructures as anelastic solids, identifying a dissipation peak around 150 K due to a thermally activated process, likely of the Niblett-Wilks mechanism type. Importantly, our findings suggest that external dissipation factors are more critical to improving quality factors than internal friction, which can be increased by modifying the nanoresonator's environment. Our results enable the design of structures with high vibration frequencies and quality factors by effectively controlling external losses., Comment: Main text and supporting information, 18 pages, 6 figures

Emotions and smell are underrepresented in digital art history. In this exploratory work, we show that recognising emotions from smell-related artworks is technically feasible but has room for improvement. Using style transfer and hyperparameter optimization we achieve a minor performance boost and open up the field for future extensions., Comment: 5 pages, 3 figures

We introduce the SmoQyDEAC.jl package, a Julia implementation of the Differential Evolution Analytic Continuation (DEAC) algorithm [N. S. Nichols et al., Phys. Rev. E 106, 025312 (2022)] for analytically continuing noisy imaginary time correlation functions to the real frequency axis. Our implementation supports fermionic and bosonic correlation functions on either the imaginary time or Matsubara frequency axes, and treatment of the covariance error in the input data. This paper presents an overview of the DEAC algorithm and the features implemented in the SmoQyDEAC.jl. It also provides detailed benchmarks of the package's output against the popular maximum entropy and stochastic analytic continuation methods. The code for this package can be downloaded from our GitHub repository at https://github.com/SmoQySuite/SmoQyDEAC.jl or installed using the Julia package manager. The online documentation, including examples, can be accessed at https://smoqysuite.github.io/SmoQyDEAC.jl/stable/.

This paper introduces a novel combination of two tasks, previously treated separately: acoustic-to-articulatory speech inversion (AAI) and phoneme-to-articulatory (PTA) motion estimation. We refer to this joint task as acoustic phoneme-to-articulatory speech inversion (APTAI) and explore two different approaches, both working speaker- and text-independently during inference. We use a multi-task learning setup, with the end-to-end goal of taking raw speech as input and estimating the corresponding articulatory movements, phoneme sequence, and phoneme alignment. While both proposed approaches share these same requirements, they differ in their way of achieving phoneme-related predictions: one is based on frame classification, the other on a two-staged training procedure and forced alignment. We reach competitive performance of 0.73 mean correlation for the AAI task and achieve up to approximately 87% frame overlap compared to a state-of-the-art text-dependent phoneme force aligner., Comment: to be published in Interspeech 2024 proceedings

Modern Neural Network (NN) architectures heavily rely on vast numbers of multiply-accumulate arithmetic operations, constituting the predominant computational cost. Therefore, this paper proposes a high-throughput, scalable and energy efficient non-element-wise matrix multiplication unit on FPGAs as a basic component of the NNs. We firstly streamline inter-layer and intra-layer redundancies of MADDNESS algorithm, a LUT-based approximate matrix multiplication, to design a fast, efficient scalable approximate matrix multiplication module termed "Approximate Multiplication Unit (AMU)". The AMU optimizes LUT-based matrix multiplications further through dedicated memory management and access design, decoupling computational overhead from input resolution and boosting FPGA-based NN accelerator efficiency significantly. The experimental results show that using our AMU achieves up to 9x higher throughput and 112x higher energy efficiency over the state-of-the-art solutions for the FPGA-based Quantised Neural Network (QNN) accelerators.

Head Start uses a whole-family approach to support the well-being of children and families with low incomes. As a two-generation model, Head Start provides comprehensive services for families and children from birth through age 5, and family support services for parents. Intentional alignment and coordination across these services is expected to lead to positive outcomes for families and their children. This brief introduces a theory of change focused on one distinct and understudied aspect of the coordinated services provided by Head Start's model: family support services. Through family support services, Head Start programs aim to help families identify and reach "their goals and dreams" by directly providing them with or referring them to services that support those goals and build on individual strengths. These support services address needs regarding education and employment, financial capability, housing and food assistance, emergency and crisis intervention, substance use treatment, physical health, and mental health, among others. Determining families' strengths and needs, identifying relevant support services, communicating with service providers, and helping families access services--that is, the "coordination of family support services"--in combination with the child-focused services, is expected to promote families' safety, health, and economic security.

We present a novel approach to achieve spatial variations in the degree of non-covalent functionalization of twisted bilayer graphene (tBLG). The tBLG with twist angles varying between ~ 5{\deg} and 7{\deg} was non-covalently functionalized with 1,4,5,8,9,11-hexaazatriphenylenehexacarbonitrile (HATCN) molecules. Our results show a correlation between the degree of functionalization and the twist angle of tBLG. This correlation was determined through Raman spectroscopy, where areas with larger twist angles exhibited a lower HATCN peak intensity compared to areas with smaller twist angles. We suggest that the HATCN adsorption follows the moir\'e pattern of tBLG by avoiding AA-stacked areas and attach predominantly to areas with a local AB-stacking order of tBLG, forming an overall ABA-stacking configuration. This is supported by density functional theory (DFT) calculations. Our work highlights the role of the moir\'e lattice in controlling the non-covalent functionalization of tBLG. Our approach can be generalized for designing nanoscale patterns on two-dimensional (2D) materials using moir\'e structures as a template.

Mathematical models are vital to the field of metrology, playing a key role in the derivation of measurement results and the calculation of uncertainties from measurement data, informed by an understanding of the measurement process. These models generally represent the correlation between the quantity being measured and all other pertinent quantities. Such relationships are used to construct measurement systems that can interpret measurement data to generate conclusions and predictions about the measurement system itself. Classic models are typically analytical, built on fundamental physical principles. However, the rise of digital technology, expansive sensor networks, and high-performance computing hardware have led to a growing shift towards data-driven methodologies. This trend is especially prominent when dealing with large, intricate networked sensor systems in situations where there is limited expert understanding of the frequently changing real-world contexts. Here, we demonstrate the variety of opportunities that data-driven modeling presents, and how they have been already implemented in various real-world applications., Comment: 31 pages, Preprint

In this paper, different techniques of few-shot, zero-shot, and regular object detection have been investigated. The need for few-shot learning and zero-shot learning techniques is crucial and arises from the limitations and challenges in traditional machine learning, deep learning, and computer vision methods where they require large amounts of data, plus the poor generalization of those traditional methods. Those techniques can give us prominent results by using only a few training sets reducing the required amounts of data and improving the generalization. This survey will highlight the recent papers of the last three years that introduce the usage of few-shot learning and zero-shot learning techniques in addressing the challenges mentioned earlier. In this paper we reviewed the Zero-shot, few-shot and regular object detection methods and categorized them in an understandable manner. Based on the comparison made within each category. It been found that the approaches are quite impressive. This integrated review of diverse papers on few-shot, zero-shot, and regular object detection reveals a shared focus on advancing the field through novel frameworks and techniques. A noteworthy observation is the scarcity of detailed discussions regarding the difficulties encountered during the development phase. Contributions include the introduction of innovative models, such as ZSD-YOLO and GTNet, often showcasing improvements with various metrics such as mean average precision (mAP),Recall@100 (RE@100), the area under the receiver operating characteristic curve (AUROC) and precision. These findings underscore a collective move towards leveraging vision-language models for versatile applications, with potential areas for future research including a more thorough exploration of limitations and domain-specific adaptations.

For a network of discrete states with a periodically driven Markovian dynamics, we develop an inference scheme for an external observer who has access to some transitions. Based on waiting-time distributions between these transitions, the periodic probabilities of states connected by these observed transitions and their time-dependent transition rates can be inferred. Moreover, the smallest number of hidden transitions between accessible ones and some of their transition rates can be extracted. We prove and conjecture lower bounds on the total entropy production for such periodic stationary states. Even though our techniques are based on generalizations of known methods for steady states, we obtain original results for those as well.

Processing histopathological Whole Slide Images (WSI) leads to massive storage requirements for clinics worldwide. Even after lossy image compression during image acquisition, additional lossy compression is frequently possible without substantially affecting the performance of deep learning-based (DL) downstream tasks. In this paper, we show that the commonly used JPEG algorithm is not best suited for further compression and we propose Stain Quantized Latent Compression (SQLC ), a novel DL based histopathology data compression approach. SQLC compresses staining and RGB channels before passing it through a compression autoencoder (CAE ) in order to obtain quantized latent representations for maximizing the compression. We show that our approach yields superior performance in a classification downstream task, compared to traditional approaches like JPEG, while image quality metrics like the Multi-Scale Structural Similarity Index (MS-SSIM) is largely preserved. Our method is online available.

To develop intelligent speech assistants and integrate them seamlessly with intra-operative decision-support frameworks, accurate and efficient surgical phase recognition is a prerequisite. In this study, we propose a multimodal framework based on Gated Multimodal Units (GMU) and Multi-Stage Temporal Convolutional Networks (MS-TCN) to recognize surgical phases of port-catheter placement operations. Our method merges speech and image models and uses them separately in different surgical phases. Based on the evaluation of 28 operations, we report a frame-wise accuracy of 92.65 $\pm$ 3.52% and an F1-score of 92.30 $\pm$ 3.82%. Our results show approximately 10% improvement in both metrics over previous work and validate the effectiveness of integrating multimodal data for the surgical phase recognition task. We further investigate the contribution of individual data channels by comparing mono-modal models with multimodal models., Comment: 5 Pages, Interspeech 2024

Interpretability is crucial for machine learning algorithms in high-stakes medical applications. However, high-performing neural networks typically cannot explain their predictions. Post-hoc explanation methods provide a way to understand neural networks but have been shown to suffer from conceptual problems. Moreover, current research largely focuses on providing local explanations for individual samples rather than global explanations for the model itself. In this paper, we propose Attri-Net, an inherently interpretable model for multi-label classification that provides local and global explanations. Attri-Net first counterfactually generates class-specific attribution maps to highlight the disease evidence, then performs classification with logistic regression classifiers based solely on the attribution maps. Local explanations for each prediction can be obtained by interpreting the attribution maps weighted by the classifiers' weights. Global explanation of whole model can be obtained by jointly considering learned average representations of the attribution maps for each class (called the class centers) and the weights of the linear classifiers. To ensure the model is ``right for the right reason", we further introduce a mechanism to guide the model's explanations to align with human knowledge. Our comprehensive evaluations show that Attri-Net can generate high-quality explanations consistent with clinical knowledge while not sacrificing classification performance., Comment: Extension of paper: Inherently Interpretable Multi-Label Classification Using Class-Specific Counterfactuals (Sun et al., MIDL 2023)

Semantic segmentation is an essential component of medical image analysis research, with recent deep learning algorithms offering out-of-the-box applicability across diverse datasets. Despite these advancements, segmentation failures remain a significant concern for real-world clinical applications, necessitating reliable detection mechanisms. This paper introduces a comprehensive benchmarking framework aimed at evaluating failure detection methodologies within medical image segmentation. Through our analysis, we identify the strengths and limitations of current failure detection metrics, advocating for the risk-coverage analysis as a holistic evaluation approach. Utilizing a collective dataset comprising five public 3D medical image collections, we assess the efficacy of various failure detection strategies under realistic test-time distribution shifts. Our findings highlight the importance of pixel confidence aggregation and we observe superior performance of the pairwise Dice score (Roy et al., 2019) between ensemble predictions, positioning it as a simple and robust baseline for failure detection in medical image segmentation. To promote ongoing research, we make the benchmarking framework available to the community., Comment: This work has been submitted for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Diffusion-weighted MRI (DW-MRI) is used to quantitatively characterize the microscopic structure of soft tissue due to the anisotropic diffusion of water in muscle. Applications such as fiber tractography or modeling of tumor spread in soft tissue require precise detection of muscle fiber orientation, which is derived from the principal eigenvector of the diffusion tensor. For clinical applications, high image quality and high signal-to-noise ratio (SNR) of DW-MRI for fiber orientation must be balanced with an appropriate scan duration. Muscles with known structural heterogeneity, e.g. bipennate muscles such as the thigh rectus femoris, provide a natural quality benchmark to determine fiber orientation at different scan parameters. Here, we analyze DW-MR images of the thigh of a healthy volunteer at different SNRs and use PCA to identify subsets of voxels with different directions of diffusion tensor eigenvectors. We propose to use the mixing index of spatial co-localization of the clustered eigenvectors as a quality metric for fiber orientation detection. Comparing acquisitions at different SNRs, we find that high SNR results in a low mixing index, reflecting a clear separation of the two compartments of the bipennate muscle on either side of the central tendon. Because the mixing index allows joint estimation of spatial and directional noise in DW-MRI as a single parameter, it will allow future quantitative optimization of DW-MRI protocols for soft tissue.

Understanding the biophysical and biochemical properties of molecular nanocarriers under physiological conditions and with minimal interference is crucial for advancing nanomedicine, photopharmacology, drug delivery, nanotheranostics and synthetic biology. Yet, analytical methods struggle to combine precise chemical imaging and measurements without perturbative labeling. This challenge is exemplified for azobenzene-based photoswitchable lipids, which are intriguing reagents for controlling nanocarrier properties on fast timescales, enabling, e.g., precise light-induced drug release processes. Here, we leverage the chemical recognition and high spatio-temporal resolution of scattering-type scanning near-field optical microscopy (s-SNOM) to demonstrate non-destructive, label-free mid-infrared (MIR) imaging and spectroscopy of photoswitchable liposomes below the diffraction limit and the tracking of their dynamics down to 50 ms resolution. The vesicles are adsorbed on an ultrathin 10-nm SiN membrane, which separates the sample space from the tip space for stable and hour-long observations. By implementing a transient nanoscopy approach, we accurately resolve, for the first time, photoinduced changes in both the shape and the MIR spectral signature of individual vesicles and reveal abrupt change dynamics of the underlying photoisomerization process. Our findings highlight the methods potential for future studies on the complex dynamics of unlabeled nanoscale soft matter, as well as, in a broader context, for host-guest systems, energy materials or drugs., Comment: 4 figures, 10 supplementary figures