Search

Traditional semantic image search methods aim to retrieve images that match the meaning of the text query. However, these methods typically search for objects on the whole image, without considering the localization of objects within the image. This paper presents an extension of existing object detection models for semantic image search that considers the semantic alignment between object proposals and text queries, with a focus on searching for objects within images. The proposed model uses a single feature extractor, a pre-trained Convolutional Neural Network, and a transformer encoder to encode the text query. Proposal-text alignment is performed using contrastive learning, producing a score for each proposal that reflects its semantic alignment with the text query. The Region Proposal Network (RPN) is used to generate object proposals, and the end-to-end training process allows for an efficient and effective solution for semantic image search. The proposed model was trained end-to-end, providing a promising solution for semantic image search that retrieves images that match the meaning of the text query and generates semantically relevant object proposals.

Nanomechanical spectroscopy (NMS) is a recently developed approach to determine optical absorption spectra of nanoscale materials via mechanical measurements. It is based on measuring changes in the resonance frequency of a membrane resonator vs. the photon energy of incoming light. This method is a direct measurement of absorption, which has practical advantages compared to common optical spectroscopy approaches. In the case of two-dimensional (2D) materials, NMS overcomes limitations inherent to conventional optical methods, such as the complications associated with measurements at high magnetic fields and low temperatures. In this work, we develop a protocol for NMS of 2D materials that yields two orders of magnitude improved sensitivity compared to previous approaches, while being simpler to use. To this end, we use electrical sample actuation, which simplifies the experiment and provides a reliable calibration for greater accuracy. Additionally, the use of low-stress silicon nitride membranes as our substrate reduces the noise-equivalent power to $NEP = 890 fW/\sqrt{Hz}$, comparable to commercial semiconductor photodetectors. We use our approach to spectroscopically characterize a two-dimensional transition metal dichalcogenide (WS$_2$), a layered magnetic semiconductor (CrPS$_4$), and a plasmonic supercrystal consisting of gold nanoparticles.

With face-to-face music collaboration being severely limited during the recent pandemic, mixed reality technologies and their potential to provide musicians a feeling of "being there" with their musical partner can offer tremendous opportunities. In order to assess this potential, we conducted a laboratory study in which musicians made music together in real-time while simultaneously seeing their jamming partner's mixed reality point cloud via a head-mounted display and compared mental effects such as flow, affect, and co-presence to an audio-only baseline. In addition, we tracked the musicians' physiological signals and evaluated their features during times of self-reported flow. For users jamming in mixed reality, we observed a significant increase in co-presence. Regardless of the condition (mixed reality or audio-only), we observed an increase in positive affect after jamming remotely. Furthermore, we identified heart rate and HF/LF as promising features for classifying the flow state musicians experienced while making music together., Comment: Conditionally Accepted for CHI 2023

The acoustic contrast factor (ACF) is calculated from the relative density and compressibility differences between a fluid and an object in the fluid. To name but one application, this acoustic contrast can be exploited using acoustophoretic systems to isolate cancer cells from a liquid biopsy, such as a blood sample. Knowing the ACF of a cancer cell represents a crucial step in the design of acoustophoretic systems for this purpose, potentially allowing the isolation of circulating cancer cells without labels or contact. For biological cells the static compressibility is different from the high frequency counterpart relevant for the ACF. In this study, we started by characterizing the ACF of low vs. high metastatic cell lines with known associated differences in phenotypic static E-modulus. The change in the static E-modulus, however, was not reflected in a change of the ACF, prompting a more in depth analysis of the influences on the ACF. We demonstrate that static E-modulus increased biological cells through formaldehyde fixation have an increased ACF. Conversely static E-modulus decreased biological cells treated with actin polymerization inhibitor cytochalasin D have a decreased ACF. Complementing these mechanical tests, a numerical COMSOL model was implemented and used to parametrically explore the effects of cell density, cell density ratios, dynamic compressibility and therefore the dynamic bulk modulus. Collectively the combined laboratory and numerical experiments reveal that a change in the static E-modulus alone might, but does not automatically lead to a change of the dynamic ACF for biological cells. This highlights the need for a multiparametic view of the biophysical basis of the cellular ACF, as well as the challenges in harnessing acoustophoretic systems to isolate circulating cells based on their mechanical properties alone.

Neural Radiance Fields (NeRFs) are emerging as a ubiquitous scene representation that allows for novel view synthesis. Increasingly, NeRFs will be shareable with other people. Before sharing a NeRF, though, it might be desirable to remove personal information or unsightly objects. Such removal is not easily achieved with the current NeRF editing frameworks. We propose a framework to remove objects from a NeRF representation created from an RGB-D sequence. Our NeRF inpainting method leverages recent work in 2D image inpainting and is guided by a user-provided mask. Our algorithm is underpinned by a confidence based view selection procedure. It chooses which of the individual 2D inpainted images to use in the creation of the NeRF, so that the resulting inpainted NeRF is 3D consistent. We show that our method for NeRF editing is effective for synthesizing plausible inpaintings in a multi-view coherent manner. We validate our approach using a new and still-challenging dataset for the task of NeRF inpainting.

One of the challenges of nanoelectromechanical systems (NEMS) is the effective transduction of the tiny resonators. Vertical structures, such as nanomechanical pillar resonators, which are exploited in a wide range of fields, such as optomechanics, acoustic metamaterials, and nanomechanical sensing, are particularly challenging to transduce. Existing electromechanical transduction methods are ill-suited as they complicate the pillars' fabrication process, put constraints on the pillars' material, and do not enable a transduction of freestanding pillars. Here, we present an electromechanical transduction method for single nanomechanical pillar resonators based on surface acoustic waves (SAWs). We demonstrate the transduction of freestanding nanomechanical platinum-carbon pillars in the first-order bending and compression mode. Since the principle of the transduction method is based on resonant scattering of a SAW by a nanomechanical resonator, our transduction method is independent of the pillar's material and not limited to pillar-shaped geometries. It represents a general method to transduce vertical mechanical resonators with nanoscale lateral dimensions., Comment: 24 pages, 6 figures

Here, by making use of medium and high resolution autocorrected off-axis electron holography, we directly probe the electrostatic potential as well as in-plane and out-of-plane charge delocalization at edges and steps in multilayer hexagonal boron nitride. In combination with ab-initio calculations, the data allows to directly reveal the formation of out-of-plane covalent bonds at folded zig-zag edges and steps comprising two monolayers and the absence of which at monolayer steps. The technique paves the way for studying other charge (de)localization phenomena in 2D materials, e.g., at polar edges, topological edge states and defects.

Speech is the fundamental mode of human communication, and its synthesis has long been a core priority in human-computer interaction research. In recent years, machines have managed to master the art of generating speech that is understandable by humans. But the linguistic content of an utterance encompasses only a part of its meaning. Affect, or expressivity, has the capacity to turn speech into a medium capable of conveying intimate thoughts, feelings, and emotions -- aspects that are essential for engaging and naturalistic interpersonal communication. While the goal of imparting expressivity to synthesised utterances has so far remained elusive, following recent advances in text-to-speech synthesis, a paradigm shift is well under way in the fields of affective speech synthesis and conversion as well. Deep learning, as the technology which underlies most of the recent advances in artificial intelligence, is spearheading these efforts. In the present overview, we outline ongoing trends and summarise state-of-the-art approaches in an attempt to provide a comprehensive overview of this exciting field., Comment: Submitted to the Proceedings of IEEE

Actuator self-heating limits the achievable force and can cause unwanted structural deformations. This is especially apparent in quasi-static actuation systems that require the actuator to maintain a stable position over an extended period. As a solution, we use the concept of a Tunable Magnet. Tunable magnets rely on in-situ magnetization state tuning of AlNico to create an infinitely adjustable magnetic flux. They consist of an AlNiCo low coercivity permanent magnet together with a magnetizing coil. After tuning, the AlNiCo retains its magnetic field without further energy input, which eliminates the static heat dissipation. To enable implementation in actuation systems, the AlNiCo needs to be robustly tunable in the presence of a varying system air-gap. We achieve this by implementing a magnetization state tuning method, based on a magnetic circuit model of the actuator, measured AlNiCo BH data and air-gap flux feedback control. The proposed tuning method consists of 2 main steps. The prediction step, during which the required magnet operating point is determined, and the demagnetization step, where a feedback controller drives a demagnetization current to approach this operating point. With this method implemented for an AlNiCo 5 tunable magnet in a reluctance actuator configuration, we achieve tuning with a maximum error of 15.86 "mT" and a minimum precision of 0.67 "mT" over an air-gap range of 200 "{\mu}m". With this tuning accuracy, actuator heating during static periods is almost eliminated. Only a small bias current is needed to compensate for the tuning error.

Let $\mathfrak{g}$ be a curved $L_\infty$-algebra endowed with a complete filtration $\mathfrak{F}\mathfrak{g}$. Suppose there exists an integer $r \in \mathbb{N}_0$ for which the curvature $\mu_0$ satisfies $\mu_0 \in \mathfrak{F}_{2r+1} \mathfrak{g}$ and the spectral sequence yields $E_{r+1}^{p,q} =0$ for $p,q$ with $p+q=2$. We prove that then a Maurer-Cartan element exists. In addition, we show, as a typical application, that for $P$ a possibly inhomogeneous Koszul operad with generating set in arities 1,2 (e.g. $P$=Com,As,BV,Lie,Ger), a $P_\infty$-algebra $A$ is intrinsically formal if its twisted deformation complex $\mathrm{Def}(H(A)\stackrel{\mathrm{id}}{\to} H(A))$ is acyclic in total degree 1.

Affective speech analysis is an ongoing topic of research. A relatively new problem in this field is the analysis of vocal bursts, which are nonverbal vocalisations such as laughs or sighs. Current state-of-the-art approaches to address affective vocal burst analysis are mostly based on wav2vec2 or HuBERT features. In this paper, we investigate the use of the wav2vec successor data2vec in combination with a multitask learning pipeline to tackle different analysis problems at once. To assess the performance of our efficient multitask learning architecture, we participate in the 2022 ACII Affective Vocal Burst Challenge, showing that our approach substantially outperforms the baseline established there in three different subtasks.

Artificial Neural Networks (ANN) are already heavily involved in methods and applications for frequent tasks in the field of computational chemistry such as representation of potential energy surfaces (PES) and spectroscopic predictions. This perspective provides an overview of the foundations of neural network-based full-dimensional potential energy surfaces, their architectures, underlying concepts, their representation and applications to chemical systems. Methods for data generation and training procedures for PES construction are discussed and means for error assessment and refinement through transfer learning are presented. A selection of recent results illustrates the latest improvements regarding accuracy of PES representations and system size limitations in dynamics simulations, but also NN application enabling direct prediction of physical results without dynamics simulations. The aim is to provide an overview for the current state-of-the-art NN approaches in computational chemistry and also to point out the current challenges in enhancing reliability and applicability of NN methods on larger scale.

The interaction of highly charged ions (HCI) with solids leads to particle sputtering, which can be used for defect-mediated engineering of the properties of the material. Ions can store energy in the form of kinetic and potential energy (sum of the ionization energies) and transfer it to the solid upon impact. The interaction and sputtering mechanisms depend significantly on the projectile energies. However, the relevance of various interaction mechanisms is unknown. Here we show that for slow HCI (5 keV) the interaction mechanisms leading to particle emission by electronic excitation and transferred kinetic energy are independent from each other, which is consistent with our atomistic simulations. We have irradiated substrate supported (Au, SiO$_2$) monolayers of MoS$_2$ with highly charged xenon ions (charge state: 17$+$ - 40$+$), extracted the emitted neutral, post-ionized Mo particles into a time-of-flight mass spectrometer and determined their velocity distributions. We find two main contributions, one at high velocities and a second one at lower velocities, and assign them to kinetic and potential effects respectively. Our data suggests that the dominant mechanism for potential sputtering is related to electron-phonon coupling, while non-thermal processes play no significant role. We anticipate that our work will be a starting point for further experiments and simulations to determine whether the different processes resulting from E$_{pot}$ and E$_{kin}$ can be separated or whether synergistic effects play a role., Comment: 9 pages, 5 figures

The combination of transfer learning (TL) a low level potential energy surface (PES) to a higher level of electronic structure theory together with ring-polymer instanton (RPI) theory is explored and applied to malonaldehyde. The RPI approach provides a semiclassical approximation of the tunneling splitting and depends sensitively on the accuracy of the PES. With second order M{\o}ller-Plesset perturbation theory (MP2) as the low-level (LL) model and energies and forces from coupled cluster singles, doubles and perturbative triples (CCSD(T)) as the high-level (HL) model, it is demonstrated that CCSD(T) information from only 25 to 50 judiciously selected structures along and around the instanton path suffice to reach HL-accuracy for the tunneling splitting. In addition, the global quality of the HL-PES is demonstrated through a mean average error of 0.3 kcal/mol for energies up to 40 kcal/mol above the minimum energy structure (a factor of 2 higher than the energies employed during TL) and $< 2 $ cm$^{-1}$ for harmonic frequencies compared with computationally challenging normal mode calculations at the CCSD(T) level.

Explanation mechanisms from the field of Counterfactual Thinking are a widely-used paradigm for Explainable Artificial Intelligence (XAI), as they follow a natural way of reasoning that humans are familiar with. However, all common approaches from this field are based on communicating information about features or characteristics that are especially important for an AI's decision. We argue that in order to fully understand a decision, not only knowledge about relevant features is needed, but that the awareness of irrelevant information also highly contributes to the creation of a user's mental model of an AI system. Therefore, we introduce a new way of explaining AI systems. Our approach, which we call Alterfactual Explanations, is based on showing an alternative reality where irrelevant features of an AI's input are altered. By doing so, the user directly sees which characteristics of the input data can change arbitrarily without influencing the AI's decision. We evaluate our approach in an extensive user study, revealing that it is able to significantly contribute to the participants' understanding of an AI. We show that alterfactual explanations are suited to convey an understanding of different aspects of the AI's reasoning than established counterfactual explanation methods., Comment: Accepted at IJCAI 2022 Workshop on XAI