Search

The increased adoption of diffusion models in text-to-image generation has triggered concerns on their reliability. Such models are now closely scrutinized under the lens of various metrics, notably calibration, fairness, or compute efficiency. We focus in this work on two issues that arise when deploying these models: a lack of diversity when prompting images, and a tendency to recreate images from the training set. To solve both problems, we propose a method that coaxes the sampled trajectories of pretrained diffusion models to land on images that fall outside of a reference set. We achieve this by adding repellency terms to the diffusion SDE throughout the generation trajectory, which are triggered whenever the path is expected to land too closely to an image in the shielded reference set. Our method is sparse in the sense that these repellency terms are zero and inactive most of the time, and even more so towards the end of the generation trajectory. Our method, named SPELL for sparse repellency, can be used either with a static reference set that contains protected images, or dynamically, by updating the set at each timestep with the expected images concurrently generated within a batch. We show that adding SPELL to popular diffusion models improves their diversity while impacting their FID only marginally, and performs comparatively better than other recent training-free diversity methods. We also demonstrate how SPELL can ensure a shielded generation away from a very large set of protected images by considering all 1.2M images from ImageNet as the protected set.

Uncertainty quantification is a key pillar of trustworthy machine learning. It enables safe reactions under unsafe inputs, like predicting only when the machine learning model detects sufficient evidence, discarding anomalous data, or emitting warnings when an error is likely to be inbound. This is particularly crucial in safety-critical areas like medical image classification or self-driving cars. Despite the plethora of proposed uncertainty quantification methods achieving increasingly higher scores on performance benchmarks, uncertainty estimates are often shied away from in practice. Many machine learning projects start from pretrained latent representations that come without uncertainty estimates. Uncertainties would need to be trained by practitioners on their own, which is notoriously difficult and resource-intense. This thesis makes uncertainty estimates easily accessible by adding them to the latent representation vectors of pretrained computer vision models. Besides proposing approaches rooted in probability and decision theory, such as Monte-Carlo InfoNCE (MCInfoNCE) and loss prediction, we delve into both theoretical and empirical questions. We show that these unobservable uncertainties about unobservable latent representations are indeed provably correct. We also provide an uncertainty-aware representation learning (URL) benchmark to compare these unobservables against observable ground-truths. Finally, we compile our findings to pretrain lightweight representation uncertainties on large-scale computer vision models that transfer to unseen datasets in a zero-shot manner. Our findings do not only advance the current theoretical understanding of uncertainties over latent variables, but also facilitate the access to uncertainty quantification for future researchers inside and outside the field, enabling straightforward but trustworthy machine learning., Comment: Doctoral thesis

Uncertainty quantification, once a singular task, has evolved into a spectrum of tasks, including abstained prediction, out-of-distribution detection, and aleatoric uncertainty quantification. The latest goal is disentanglement: the construction of multiple estimators that are each tailored to one and only one source of uncertainty. This paper presents the first benchmark of uncertainty disentanglement. We reimplement and evaluate a comprehensive range of uncertainty estimators, from Bayesian over evidential to deterministic ones, across a diverse range of uncertainty tasks on ImageNet. We find that, despite recent theoretical endeavors, no existing approach provides pairs of disentangled uncertainty estimators in practice. We further find that specialized uncertainty tasks are harder than predictive uncertainty tasks, where we observe saturating performance. Our results provide both practical advice for which uncertainty estimators to use for which specific task, and reveal opportunities for future research toward task-centric and disentangled uncertainties. All our reimplementations and Weights & Biases logs are available at https://github.com/bmucsanyi/untangle., Comment: 68 pages

Accurate uncertainty estimation is vital to trustworthy machine learning, yet uncertainties typically have to be learned for each task anew. This work introduces the first pretrained uncertainty modules for vision models. Similar to standard pretraining this enables the zero-shot transfer of uncertainties learned on a large pretraining dataset to specialized downstream datasets. We enable our large-scale pretraining on ImageNet-21k by solving a gradient conflict in previous uncertainty modules and accelerating the training by up to 180x. We find that the pretrained uncertainties generalize to unseen datasets. In scrutinizing the learned uncertainties, we find that they capture aleatoric uncertainty, disentangled from epistemic components. We demonstrate that this enables safe retrieval and uncertainty-aware dataset visualization. To encourage applications to further problems and domains, we release all pretrained checkpoints and code under https://github.com/mkirchhof/url .

We explore strain-activated emission centers formed by atomic force microscopy (AFM) indentation in monolayer \ce{WSe2} on a flexible polymer substrate. In the indented areas, we observe sharp new photoluminescence (PL) peaks characterized by sublinear power dependence in the spectral regions 1.62 $-$ 1.66 eV and 1.70 $-$ 1.73 eV. After low-temperature thermal annealing ($< 120$ $^{\circ}$C), \ce{WSe2} experiences strain relaxation, leading to a blue shift of the peaks' spectral position and their ultimate disappearance. Our analysis of peaks' position vs. strain allows drawing multiple conclusions regarding the nature of these emission centers. We elucidate the roles of excitonic confinement and hybridization between free excitons and defect-related states, a process activated by the level of strain. Overall, our approach suggests that the energy of localized emitters may be controlled via strain engineering.

Momentum-indirect excitons composed of electrons and holes in different valleys define optoelectronic properties of many semiconductors, but are challenging to detect due to their weak coupling to light. The identification of an excitons' valley character is further limited by complexities associated with momentum-selective probes. Here, we study the photoluminescence of indirect excitons in controllably strained prototypical 2D semiconductors (WSe$_2$, WS$_2$) at cryogenic temperatures. We find that these excitons i) exhibit valley-specific energy shifts, enabling their valley fingerprinting, and ii) hybridize with bright excitons, becoming directly accessible to optical spectroscopy methods. This approach allows us to identify multiple previously inaccessible excitons with wavefunctions residing in K, $\Gamma$, or Q valleys in the momentum space as well as various types of defect-related excitons. Overall, our approach is well-suited to unravel and tune intervalley excitons in various semiconductors., Comment: 9 pages, 4 figures, 1 table

As machine learning technology gets applied to actual products and solutions, new challenges have emerged. Models unexpectedly fail to generalize to small changes in the distribution, tend to be confident on novel data they have never seen, or cannot communicate the rationale behind their decisions effectively with the end users. Collectively, we face a trustworthiness issue with the current machine learning technology. This textbook on Trustworthy Machine Learning (TML) covers a theoretical and technical background of four key topics in TML: Out-of-Distribution Generalization, Explainability, Uncertainty Quantification, and Evaluation of Trustworthiness. We discuss important classical and contemporary research papers of the aforementioned fields and uncover and connect their underlying intuitions. The book evolved from the homonymous course at the University of T\"ubingen, first offered in the Winter Semester of 2022/23. It is meant to be a stand-alone product accompanied by code snippets and various pointers to further sources on topics of TML. The dedicated website of the book is https://trustworthyml.io/., Comment: 373 pages, textbook at the University of T\"ubingen

Representation learning has significantly driven the field to develop pretrained models that can act as a valuable starting point when transferring to new datasets. With the rising demand for reliable machine learning and uncertainty quantification, there is a need for pretrained models that not only provide embeddings but also transferable uncertainty estimates. To guide the development of such models, we propose the Uncertainty-aware Representation Learning (URL) benchmark. Besides the transferability of the representations, it also measures the zero-shot transferability of the uncertainty estimate using a novel metric. We apply URL to evaluate eleven uncertainty quantifiers that are pretrained on ImageNet and transferred to eight downstream datasets. We find that approaches that focus on the uncertainty of the representation itself or estimate the prediction risk directly outperform those that are based on the probabilities of upstream classes. Yet, achieving transferable uncertainty quantification remains an open challenge. Our findings indicate that it is not necessarily in conflict with traditional representation learning goals. Code is provided under https://github.com/mkirchhof/url ., Comment: Accepted at the Neural Information Processing Systems Track on Datasets and Benchmarks (NeurIPS D&B 2023)

Phononic crystals (PnCs) are artificially patterned media exhibiting bands of allowed and forbidden zones for phonons. Many emerging applications of PnCs from solid-state simulators to quantum memories could benefit from the on-demand tunability of the phononic band structure. Here, we demonstrate the fabrication of suspended graphene PnCs in which the phononic band structure is controlled by mechanical tension applied electrostatically. We show signatures of a mechanically tunable phononic band gap. The experimental data supported by simulation suggest a phononic band gap at 28$-$33 MHz in equilibrium, which upshifts by 9 MHz under a mechanical tension of 3.1 Nm$^{-1}$. This is an essential step towards tunable phononics paving the way for more experiments on phononic systems based on 2D materials.

Contrastively trained encoders have recently been proven to invert the data-generating process: they encode each input, e.g., an image, into the true latent vector that generated the image (Zimmermann et al., 2021). However, real-world observations often have inherent ambiguities. For instance, images may be blurred or only show a 2D view of a 3D object, so multiple latents could have generated them. This makes the true posterior for the latent vector probabilistic with heteroscedastic uncertainty. In this setup, we extend the common InfoNCE objective and encoders to predict latent distributions instead of points. We prove that these distributions recover the correct posteriors of the data-generating process, including its level of aleatoric uncertainty, up to a rotation of the latent space. In addition to providing calibrated uncertainty estimates, these posteriors allow the computation of credible intervals in image retrieval. They comprise images with the same latent as a given query, subject to its uncertainty. Code is available at https://github.com/mkirchhof/Probabilistic_Contrastive_Learning, Comment: Accepted at ICML 2023

Nanomechanical spectroscopy (NMS) is a recently developed approach to determine optical absorption spectra of nanoscale materials via mechanical measurements. It is based on measuring changes in the resonance frequency of a membrane resonator vs. the photon energy of incoming light. This method is a direct measurement of absorption, which has practical advantages compared to common optical spectroscopy approaches. In the case of two-dimensional (2D) materials, NMS overcomes limitations inherent to conventional optical methods, such as the complications associated with measurements at high magnetic fields and low temperatures. In this work, we develop a protocol for NMS of 2D materials that yields two orders of magnitude improved sensitivity compared to previous approaches, while being simpler to use. To this end, we use electrical sample actuation, which simplifies the experiment and provides a reliable calibration for greater accuracy. Additionally, the use of low-stress silicon nitride membranes as our substrate reduces the noise-equivalent power to $NEP = 890 fW/\sqrt{Hz}$, comparable to commercial semiconductor photodetectors. We use our approach to spectroscopically characterize a two-dimensional transition metal dichalcogenide (WS$_2$), a layered magnetic semiconductor (CrPS$_4$), and a plasmonic supercrystal consisting of gold nanoparticles.

Due to their good ratio of stiffness and strength to weight, foam materials find use in lightweight engineering. Though, in many applications like structural bending or tension, the scale separation between macroscopic structure and the foam's mesostructure like cells size, is relatively weak and the mechanical properties of the foam appear to be size dependent. Positive as well as negative size effects have been observed for certain basic tests of foams, i.e., the material appears either to be more compliant or stiffer than would be expected from larger specimens. Performing tests with sufficiently small specimens is challenging as any disturbances from damage of cell walls during sample preparation or from loading devices must be avoided. Correspondingly, the number of respective data in literature is relatively low and the results are partly contradictory. In order to avoid the problems from sample preparation or bearings, the present study employs virtual tests with CT data of real medium-density ceramic foams. A number of samples of different size is "cut" from the resulting voxel data. Subsequently, the apparent elastic properties of each virtual sample are "measured" directly by a free vibrational analysis using finite cell method, thereby avoiding any disturbances from load application or bearings. The results exhibit a large scatter of the apparent moduli per sample size, but with a clear negative size effect in all investigated basic modes of deformation (bending, torsion, uniaxial). Finally, the results are compared qualitatively and quantitatively to available experimental data from literature, yielding common trends as well as open questions.

In this paper, we propose to adopt the MDE paradigm for the development of Machine Learning (ML)-enabled software systems with a focus on the Internet of Things (IoT) domain. We illustrate how two state-of-the-art open-source modeling tools, namely MontiAnna and ML-Quadrat can be used for this purpose as demonstrated through a case study. The case study illustrates using ML, in particular deep Artificial Neural Networks (ANNs), for automated image recognition of handwritten digits using the MNIST reference dataset, and integrating the machine learning components into an IoT system. Subsequently, we conduct a functional comparison of the two frameworks, setting out an analysis base to include a broad range of design considerations, such as the problem domain, methods for the ML integration into larger systems, and supported ML methods, as well as topics of recent intense interest to the ML community, such as AutoML and MLOps. Accordingly, this paper is focused on elucidating the potential of the MDE approach in the ML domain. This supports the ML engineer in developing the (ML/software) model rather than implementing the code, and additionally enforces reusability and modularity of the design through enabling the out-of-the-box integration of ML functionality as a component of the IoT or cyber-physical systems., Comment: ACM / IEEE 25th International Conference on Model Driven Engineering Languages and Systems (MODELS 2022) Companion - MDE Intelligence Workshop

Mechanical strain is a powerful tuning knob for excitons, Coulomb-bound electron-hole complexes dominating optical properties of two-dimensional semiconductors. While the strain response of bright free excitons is broadly understood, the behavior of dark free excitons (long-lived excitations that generally do not couple to light due to spin and momentum conservation) or localized excitons related to defects remains mostly unexplored. Here, we develop a technique capable of straining pristine suspended WSe2 kept at cryogenic temperatures up to 3\% to study the strain behavior of these fragile many-body states. We find that under the application of strain, dark and localized excitons in monolayer WSe2 - a prototypical 2D semiconductor - are brought into energetic resonance, forming a new hybrid state that inherits the properties of the constituent species. The characteristics of the hybridized state, including an order-of-magnitude enhanced light/matter coupling, avoided-crossing energy shifts, and strain tunability of many-body interactions, are all supported by first-principles calculations. The hybridized exciton reported here may play a critical role in the operation of single quantum emitters based on WSe2. Furthermore, the techniques we developed may be used to fingerprint unidentified excitonic states, Comment: 15 pages, 5 figures

Proxy-based Deep Metric Learning (DML) learns deep representations by embedding images close to their class representatives (proxies), commonly with respect to the angle between them. However, this disregards the embedding norm, which can carry additional beneficial context such as class- or image-intrinsic uncertainty. In addition, proxy-based DML struggles to learn class-internal structures. To address both issues at once, we introduce non-isotropic probabilistic proxy-based DML. We model images as directional von Mises-Fisher (vMF) distributions on the hypersphere that can reflect image-intrinsic uncertainties. Further, we derive non-isotropic von Mises-Fisher (nivMF) distributions for class proxies to better represent complex class-specific variances. To measure the proxy-to-image distance between these models, we develop and investigate multiple distribution-to-point and distribution-to-distribution metrics. Each framework choice is motivated by a set of ablational studies, which showcase beneficial properties of our probabilistic approach to proxy-based DML, such as uncertainty-awareness, better-behaved gradients during training, and overall improved generalization performance. The latter is especially reflected in the competitive performance on the standard DML benchmarks, where our approach compares favorably, suggesting that existing proxy-based DML can significantly benefit from a more probabilistic treatment. Code is available at github.com/ExplainableML/Probabilistic_Deep_Metric_Learning., Comment: Accepted as conference paper at ECCV 2022

Interest in understanding and factorizing learned embedding spaces through conceptual explanations is steadily growing. When no human concept labels are available, concept discovery methods search trained embedding spaces for interpretable concepts like object shape or color that can provide post-hoc explanations for decisions. Unlike previous work, we argue that concept discovery should be identifiable, meaning that a number of known concepts can be provably recovered to guarantee reliability of the explanations. As a starting point, we explicitly make the connection between concept discovery and classical methods like Principal Component Analysis and Independent Component Analysis by showing that they can recover independent concepts under non-Gaussian distributions. For dependent concepts, we propose two novel approaches that exploit functional compositionality properties of image-generating processes. Our provably identifiable concept discovery methods substantially outperform competitors on a battery of experiments including hundreds of trained models and dependent concepts, where they exhibit up to 29 % better alignment with the ground truth. Our results highlight the strict conditions under which reliable concept discovery without human labels can be guaranteed and provide a formal foundation for the domain. Our code is available online., Comment: v5: UAI2023 camera-ready including supplementary material. The first two authors contributed equally

We present a tunable phononic crystal which undergoes a phase transition from mechanically insulating to mechanically transmissive (metallic). Specifically, in our simulations for a phononic lattice under biaxial tension ($\sigma_{xx} =\sigma_{yy}$ = 0.01 N$/$m), we find a bandgap for out-of-plane phonons in the range of 48.8 - 56.4 MHz, which we can close by increasing the degree of tension uniaxiality ($\sigma_{xx} / \sigma_{yy}$) to 1.7. To manipulate the tension distribution, we design a realistic device of finite size, where $\sigma_{xx} / \sigma_{yy}$ is tuned by applying a gate voltage to a phononic crystal made from suspended graphene. We show that the phase transition can be probed via acoustic transmission measurements and that the phononic bandgap persists even after the inclusion surface contaminants and random tension variations present in realistic devices. The proposed system acts as a transistor for phonons with an on/off ratio of $10^5$ (100 dB suppression) and is thus a valuable extension for phonon logic applications. In addition, this mechanical analogue to a metal-insulator transition (mMIT) allows tunable coupling between mechanical entities (e.g. mechanical qubits)., Comment: SI at the end of the document

We introduce a nanomechanical platform for fast and sensitive measurements of the spectrally-resolved optical dielectric function of 2D materials. At the heart of our approach is a suspended 2D material integrated into a nanomechanical resonator illuminated by a wavelength-tunable laser source. From the heating-related frequency shift of the resonator as well as its optical reflection measured as a function of photon energy, we obtain the real and imaginary parts of the dielectric function. Our measurements are unaffected by substrate-related screening and do not require any assumptions on the underling optical constants. This fast ($\tau_{rise}$ $\sim$ 135 ns), sensitive (noise-equivalent power = 90 $\frac{pW}{\sqrt{Hz}}$ ), and broadband (1.2 $-$ 3.1 eV, extendable to UV-THz) method provides an attractive alternative to spectroscopic or ellipsometric characterisation techniques.

Control over spin transport in antiferromagnetic systems is essential for future spintronic applications with operational speeds extending to ultrafast time scales. Here, we study the transition from the gigahertz (GHz) to terahertz (THz) regime of spin transport and spin-to-charge current conversion (S2C) in the prototypical antiferromagnet IrMn by employing spin pumping and THz spectroscopy techniques. We reveal a factor of 4 shorter characteristic propagation lengths of the spin current at THz frequencies (~ 0.5 nm) as compared to the GHz regime (~ 2 nm) which may be attributed to the ballistic and diffusive nature of electronic spin transport, respectively. The conclusion is supported by an extraction of sub-picosecond temporal dynamics of the THz spin current. We also report on a significant impact of the S2C originating from the IrMn/non-magnetic metal interface which is much more pronounced in the THz regime and opens the door for optimization of the spin control at ultrafast time scales.

We demonstrate a new type of dual gate transistor to induce record electric fields through two-dimensional materials (2DMs). At the heart of this device is a 2DM suspended between two volumes of ionic liquid (IL) with independently controlled potentials. The potential difference between the ILs falls across an ultrathin layer consisting of the 2DM and the electrical double layers above and below it, thereby producing an intense electric field across the 2DM. We determine the field strength via i) electrical transport measurements and ii) direct measurements of electrochemical potentials of the ILs using semiconducting 2DM, WSe2. The field strength across the material reaches more than 3.5 V/nm, the largest static electric field through any electronic device to date. We demonstrate that this field is strong enough to close the bandgap of trilayer WSe2 driving a semiconductor-to-metal transition. Our approach grants access to previously-inaccessible phenomena occurring in ultrastrong electric fields.

A key task in multi-label classification is modeling the structure between the involved classes. Modeling this structure by probabilistic and interpretable means enables application in a broad variety of tasks such as zero-shot learning or learning from incomplete data. In this paper, we present the probabilistic rule stacking learner (pRSL) which uses probabilistic propositional logic rules and belief propagation to combine the predictions of several underlying classifiers. We derive algorithms for exact and approximate inference and learning, and show that pRSL reaches state-of-the-art performance on various benchmark datasets. In the process, we introduce a novel multicategorical generalization of the noisy-or gate. Additionally, we report simulation results on the quality of loopy belief propagation algorithms for approximate inference in bipartite noisy-or networks.