Search

Audio description (AD) is a crucial accessibility service provided to blind persons and persons with visual impairment, designed to convey visual information in acoustic form. Despite recent advancements in multilingual machine translation research, the lack of well-crafted and time-synchronized AD data impedes the development of audio description translation (ADT) systems that address the needs of multilingual countries such as Switzerland. Furthermore, since the majority of ADT systems rely solely on text, uncertainty exists as to whether incorporating visual information from the corresponding video clips can enhance the quality of ADT outputs. In this work, we present SwissADT, the first ADT system implemented for three main Swiss languages and English. By collecting well-crafted AD data augmented with video clips in German, French, Italian, and English, and leveraging the power of Large Language Models (LLMs), we aim to enhance information accessibility for diverse language populations in Switzerland by automatically translating AD scripts to the desired Swiss language. Our extensive experimental ADT results, composed of both automatic and human evaluations of ADT quality, demonstrate the promising capability of SwissADT for the ADT task. We believe that combining human expertise with the generation power of LLMs can further enhance the performance of ADT systems, ultimately benefiting a larger multilingual target population.

Audio descriptions (ADs) function as acoustic commentaries designed to assist blind persons and persons with visual impairments in accessing digital media content on television and in movies, among other settings. As an accessibility service typically provided by trained AD professionals, the generation of ADs demands significant human effort, making the process both time-consuming and costly. Recent advancements in natural language processing (NLP) and computer vision (CV), particularly in large language models (LLMs) and vision-language models (VLMs), have allowed for getting a step closer to automatic AD generation. This paper reviews the technologies pertinent to AD generation in the era of LLMs and VLMs: we discuss how state-of-the-art NLP and CV technologies can be applied to generate ADs and identify essential research directions for the future.

Calculating by analytical theory the deformation of finite-sized elastic bodies in response to internally applied forces is a challenge. Here, we derive explicit analytical expressions for the amplitudes of modes of surface deformation of a homogeneous, isotropic, linearly elastic sphere. The modes under consideration correspond to overall changes in volume and to relative uniaxial elongation or contraction. Both situations of an elastic sphere embedded under no-slip conditions in a linearly elastic, homogeneous, isotropic, infinitely extended background medium and of a free-standing elastic sphere are considered. For example, our analytical expressions are important as a basis for computational material optimization in the context of spherical soft actuators., Comment: 29 pages, 0 figures

We report high-resolution dilatometry studies on single crystals of the Shastry-Sutherland-lattice magnet NdB$_4$ supported by specific heat and magnetometry data. Our dilatometric studies evidence pronounced anomalies at the phase boundaries which imply strong magneto-elastic coupling. The evolution of the three zero-field phase transitions separating distinct antiferromagnetic phases at $TN=17.2$~K, $TIT=6.8$~K and $TLT=4.8$~K can thus be traced in applied magnetic fields which provides the magnetic phase diagrams for $B\parallel c$ up to 15~T and for $B\parallel [110]$ up to 35~T. New in-field phases are discovered for both field directions and already known phases are confirmed. In particular, phase boundaries between different phases are unambiguously shown by sign changes of observed anomalies and corresponding changes in uniaxial pressure effects. For $B||c$, we find a 1/4-magnetization plateau in addition to a previously reported plateau at 1/5 of the saturation magnetization. TN increases for $B\parallel c$ in fields up to 15~T implying that magnetic moments of the all-in/all-out structure in the high temperature AFM ordered phase are driven towards the $c$ axis in high magnetic fields. Uniaxial pressure dependencies ${\partial}T_{\mathrm{crit}}/{\partial}p_{\mathrm{c}}$ of the phase transition temperatures for magnetic fields and pressure applied along the $c$ axis are derived from the data.

Amongst the various fascinating types of material behavior featured by magnetic gels and elastomers are magnetostrictive effects. That is, deformations in shape or changes in volume are induced from outside by external magnetic fields. Application of the materials as soft actuators is therefore conceivable. Mostly, straight contraction or extension of the materials along a certain direction is discussed and investigated in this context. Here, we demonstrate that various further, different, higher modes of deformation can be excited. To this end, different spatial arrangements of the magnetizable particles enclosed by the soft elastic matrix, which constitute the materials, need to be controlled and realized. We address various different types of spatial configurations of the particles and evaluate resulting types of deformation using theoretical tools developed for this purpose. Examples are sheet-like arrangements of particles, circular or star-shaped arrangements of chain-like aggregates, or actual three-dimensional star-like particle configurations. We hope to stimulate with our work the development of experimental design and engineering methods so that selected spatial particle arrangements in magnetic gels and elastomers can be put to reality. Overall, we in this way wish to promote the transfer of these promising class of materials to real-world applications., Comment: 17 pages, 19 figures

Privacy-utility tradeoff remains as one of the fundamental issues of differentially private machine learning. This paper introduces a geometrically inspired kernel-based approach to mitigate the accuracy-loss issue in classification. In this approach, a representation of the affine hull of given data points is learned in Reproducing Kernel Hilbert Spaces (RKHS). This leads to a novel distance measure that hides privacy-sensitive information about individual data points and improves the privacy-utility tradeoff via significantly reducing the risk of membership inference attacks. The effectiveness of the approach is demonstrated through experiments on MNIST dataset, Freiburg groceries dataset, and a real biomedical dataset. It is verified that the approach remains computationally practical. The application of the approach to federated learning is considered and it is observed that the accuracy-loss due to data being distributed is either marginal or not significantly high.

Macroscopic elastic core-shell systems can be generated as toy models to be deformed and haptically studied by hand. On the mesoscale, colloidal core-shell particles and microgels are fabricated and investigated by different types of microscopy. We analyse, using linear elasticity theory, the response of spherical core-shell systems under the influence of a line density of force that is oriented radially and acts along the equator of the outer surface. Interestingly, deformational coupling of the shell to the core can determine the resulting overall appearance in response to the forces. We address various combinations of radii, stiffness, and Poisson ratio of core and shell and illustrate the resulting deformations. Macroscopically, the situation could be realized by wrapping a cord around the equator of a macroscopic model system and pulling it tight. On the mesoscale, colloidal microgel particles symmetrically confined to the interface between two immiscible fluids are pulled radially outward by surface tension., Comment: 18 pages, 5 figures, article

This study leverages the data representation capability of fuzzy based membership-mappings for practical secure distributed deep learning using fully homomorphic encryption. The impracticality issue of secure machine (deep) learning with fully homomorphic encrypted data, arising from large computational overhead, is addressed via applying fuzzy attributes. Fuzzy attributes are induced by globally convergent and robust variational membership-mappings based local deep models. Fuzzy attributes combine the local deep models in a robust and flexible manner such that the global model can be evaluated homomorphically in an efficient manner using a boolean circuit composed of bootstrapped binary gates. The proposed method, while preserving privacy in a distributed learning scenario, remains accurate, practical, and scalable. The method is evaluated through numerous experiments including demonstrations through MNIST dataset and Freiburg Groceries Dataset. Further, a biomedical application related to mental stress detection on individuals is considered.

Electron correlations amplify quantum fluctuations and, as such, they have been recognized as the origin of a rich landscape of quantum phases. Whether and how they lead to gapless topological states is an outstanding question, and a framework that allows for determining novel phases and identifying new materials is in pressing need. Here we advance a general approach, in which strong correlations cooperate with crystalline symmetry to drive gapless topological states. We test this materials design principle by exploring Kondo lattice models and materials whose space group symmetries may promote different kinds of electronic degeneracies, with a particular focus on square-net systems. Weyl-Kondo nodal-line semimetals -- with nodes pinned to the Fermi energy -- are identified. We describe how this approach can be applied to discover strongly correlated topological semimetals, identify three heavy fermion compounds as new candidates, provide first direct experimental evidence for our prediction in Ce$_2$Au$_3$In$_5$, and discuss how our approach may lead to many more. Our findings illustrate the potential of the proposed materials design principle to guide the search for new topological metals in a broad range of strongly correlated systems., Comment: To appear in Nature Physics; submitted version of the manuscript (the accepted version will appear in the journal); 35 pages, 4 figures main text + 8 figures supplementary information

In order to develop machine learning and deep learning models that take into account the guidelines and principles of trustworthy AI, a novel information theoretic trustworthy AI framework is introduced. A unified approach to "privacy-preserving interpretable and transferable learning" is considered for studying and optimizing the tradeoffs between privacy, interpretability, and transferability aspects. A variational membership-mapping Bayesian model is used for the analytical approximations of the defined information theoretic measures for privacy-leakage, interpretability, and transferability. The approach consists of approximating the information theoretic measures via maximizing a lower-bound using variational optimization. The study presents a unified information theoretic approach to study different aspects of trustworthy AI in a rigorous analytical manner. The approach is demonstrated through numerous experiments on benchmark datasets and a real-world biomedical application concerned with the detection of mental stress on individuals using heart rate variability analysis., Comment: arXiv admin note: text overlap with arXiv:2105.04615, arXiv:2104.07060

A fuzzy theoretic analytical approach was recently introduced that leads to efficient and robust models while addressing automatically the typical issues associated to parametric deep models. However, a formal conceptualization of the fuzzy theoretic analytical deep models is still not available. This paper introduces using measure theoretic basis the notion of \emph{membership-mapping} for representing data points through attribute values (motivated by fuzzy theory). A property of the membership-mapping, that can be exploited for data representation learning, is of providing an interpolation on the given data points in the data space. An analytical approach to the variational learning of a membership-mappings based data representation model is considered.

We report high-resolution capacitance dilatometry studies on the uniaxial length changes in a NdB$_4$ single crystal. The evolution of magnetically ordered phases below $T_{\rm N}$= 17.2~K (commensurate antiferromagnetic phase, cAFM), $T_{\rm IT}$= 6.8~K (intermediate incommensurate phase, IT), and $T_{\rm LT}$= 4.8~K (low-temperature phase, LT) is associated with pronounced anomalies in the thermal expansion coefficients. The data imply significant magneto-elastic coupling and evidence of a structural phase transition at $T_{\rm LT}$ . While both cAFM and LT favor structural anisotropy $\delta$ between in-plane and out-of-plane length changes, it competes with the IT-type of order, i.e., $\delta$ is suppressed in that phase. Notably, finite anisotropy well above $T_{\rm N}$ indicates short-range correlations which are, however, of neither cAFM, IT, nor LT-type. Gr\"uneisen analysis of the ratio of thermal expansion coefficient and specific heat enables the derivation of uniaxial as well as hydrostatic pressure dependencies. While $\alpha$/$c_{\rm p}$ evidences a single dominant energy scale in LT, our data imply precursory fluctuations of a competing phase in IT and cAFM, respectively. Our results suggest the presence of orbital degrees of freedom competing with cAFM and successive evolution of a magnetically and orbitally ordered ground state.

Machine translation (MT) has been shown to produce a number of errors that require human post-editing, but the extent to which professional human translation (HT) contains such errors has not yet been compared to MT. We compile pre-translated documents in which MT and HT are interleaved, and ask professional translators to flag errors and post-edit these documents in a blind evaluation. We find that the post-editing effort for MT segments is only higher in two out of three language pairs, and that the number of segments with wrong terminology, omissions, and typographical problems is similar in HT., Comment: EAMT 2020 (Research Track)

Soft elastic composite materials can serve as actuators when they transform changes in external fields into mechanical deformation. Here, we address the corresponding deformational behavior of magnetic gels and elastomers, consisting of magnetizable colloidal particles in a soft polymeric matrix and exposed to external magnetic fields. Since many practical realizations of such materials involve particulate inclusions of polydisperse size distributions, we concentrate on the effect that mixed particle sizes have on the overall deformational response. To perform a systematic study, our focus is on binary size distributions. We systematically vary the fraction of larger particles relative to smaller ones and characterize the resulting magnetostrictive behavior. The consequences for systems of various different spatial particle arrangements and different degrees of compressibility of the elastic matrix are evaluated. In parts, we observe a qualitative change in the overall response for selected systems of mixed particle sizes. Specifically, overall changes in volume and relative elongations or contractions in response to an induced magnetization can be reversed into the opposite types of behavior. Our results should apply to the characteristics of other soft elastic composite materials like electrorheological gels and elastomers when exposed to external electric fields as well. Overall, we hope to stimulate the further investigation on the purposeful use of mixed particle sizes as a means to design tailored requested material behavior., Comment: 14 pages, 17 figures

Soft actuators allow to transform external stimuli to mechanical deformations. Because of their deformational response to external magnetic fields, magnetic gels and elastomers represent ideal candidates for such tasks. Mostly, linear magnetostrictive deformations, that is, elongations or contractions along straight axes are discussed in this context. In contrast to that, we here suggest to realize a twist actuator that responds by torsional deformations around the axis of the applied magnetic field. For this purpose, we theoretically investigate the overall mechanical response of a basic model system containing discrete magnetizable particles in a soft elastic matrix. Two different types of discrete particle arrangements are used as starting conditions in the nonmagnetized state. These contain globally twisted anisotropic particle arrangements on the one hand, and groups of discrete helical-like particle structures positioned side by side on the other hand. Besides the resulting twist upon magnetization, we also evaluate different other modes of deformation. Our analysis supports the construction of magnetically orientable and actuatable torsional mixing devices in fluidic applications or other types of soft actuators that initiate relative rotations between different components., Comment: 13 pages, 14 figures

The putative generalization of the thermodynamic uncertainty relation (TUR) to underdamped dynamics is still an open problem. So far, bounds that have been derived for such a dynamics are not particularly transparent and they do not converge to the known TUR in the overdamped limit. Furthermore, it was found that there are restrictions for a TUR to hold such as the absence of a magnetic field. In this article we first analyze the properties of driven free diffusion in the underdamped regime and show that it inherently violates the overdamped TUR for finite times. Based on numerical evidence, we then conjecture a bound for one-dimensional driven diffusion in a potential which is based on the result for free diffusion. This bound converges to the known overdamped TUR in the corresponding limit. Moreover, the conjectured bound holds for observables that involve higher powers of the velocity as long as the observable is odd under time-reversal. Finally, we address the applicability of this bound to underdamped dynamics in higher dimensions., Comment: 10 pages, 4 figures

Magnetic gels and elastomers are promising candidates to construct reversibly excitable soft actuators, triggered from outside by magnetic fields. These magnetic fields induce or alter the magnetic interactions between discrete rigid particles embedded in a soft elastic polymeric matrix, leading to overall deformations. It is a major challenge in theory to correctly predict from the discrete particle configuration the type of deformation resulting for a finite-sized system. Considering an elastic sphere, we here present such an approach. The method is in principle exact, at least within the framework of linear elasticity theory and for large enough interparticle distances. Different particle arrangements are considered. We find, for instance, that regular simple cubic configurations show elongation of the sphere along the magnetization if oriented along a face or space diagonal of the cubic unit cell. Contrariwise, with the magnetization along the edge of the cubic unit cell, they contract. The opposite is true in this geometry for body- and face-centered configurations. Remarkably, for the latter configurations but the magnetization along a face or space diagonal of the unit cell, contraction was observed to revert to expansion with decreasing Poisson ratio of the elastic material. Randomized configurations were considered as well. They show a tendency of elongating the sphere along the magnetization, which is more pronounced for compressible systems. Our results can be tested against actual experiments for spherical samples. Moreover, our approach shall support the search of optimal particle distributions for a maximized effect of actuation., Comment: 15 pages, 10 figures

Separating and labeling each instance of a nucleus (instance-aware segmentation) is the key challenge in segmenting single cell nuclei on fluorescence microscopy images. Deep Neural Networks can learn the implicit transformation of a nuclear image into a probability map indicating the class membership of each pixel (nucleus or background), but the use of post-processing steps to turn the probability map into a labeled object mask is error-prone. This especially accounts for nuclear images of tissue sections and nuclear images across varying tissue preparations. In this work, we aim to evaluate the performance of state-of-the-art deep learning architectures to segment nuclei in fluorescence images of various tissue origins and sample preparation types without post-processing. We compare architectures that operate on pixel to pixel translation and an architecture that operates on object detection and subsequent locally applied segmentation. In addition, we propose a novel strategy to create artificial images to extend the training set. We evaluate the influence of ground truth annotation quality, image scale and segmentation complexity on segmentation performance. Results show that three out of four deep learning architectures (U-Net, U-Net with ResNet34 backbone, Mask R-CNN) can segment fluorescent nuclear images on most of the sample preparation types and tissue origins with satisfactory segmentation performance. Mask R-CNN, an architecture designed to address instance aware segmentation tasks, outperforms other architectures. Equal nuclear mean size, consistent nuclear annotations and the use of artificially generated images result in overall acceptable precision and recall across different tissues and sample preparation types., Comment: 10 pages + 3 supplementary pages

The recent success of Generative Adversarial Networks (GAN) is a result of their ability to generate high quality images from a latent vector space. An important application is the generation of images from a text description, where the text description is encoded and further used in the conditioning of the generated image. Thus the generative network has to additionally learn a mapping from the text latent vector space to a highly complex and multi-modal image data distribution, which makes the training of such models challenging. To handle the complexities of fashion image and meta data, we propose Ontology Generative Adversarial Networks (O-GANs) for fashion image synthesis that is conditioned on an hierarchical fashion ontology in order to improve the image generation fidelity. We show that the incorporation of the ontology leads to better image quality as measured by Fr\'{e}chet Inception Distance and Inception Score. Additionally, we show that the O-GAN achieves better conditioning results evaluated by implicit similarity between the text and the generated image., Comment: Under review at MLKgraphs2019: http://www.dexa.org/mlkgraphs2019

The thermodynamic uncertainty relation provides a universal lower bound on the product of entropy production and the fluctuations of any current. While proven for Markov dynamics on a discrete set of states and for overdamped Langevin dynamics, its status for underdamped dynamics is still open. We consider a two-dimensional harmonically confined charged particle in a magnetic field under the action of an external torque. We show analytically that, depending on the sign of the magnetic field, the thermodynamic uncertainty relation does not hold for the currents associated with work and heat. A strong magnetic field can effectively localize the particle with concomitant bounded fluctuations and low dissipation. Numerical results for a three-dimensional variant and for further currents suggest that the existence of such a bound depends crucially on the specific current., Comment: 8 pages, 4 figures (v2) Sec.III is revised. Fig.1 is updated

Diagnosis and risk stratification of cancer and many other diseases require the detection of genomic breakpoints as a prerequisite of calling copy number alterations (CNA). This, however, is still challenging and requires time-consuming manual curation. As deep-learning methods outperformed classical state-of-the-art algorithms in various domains and have also been successfully applied to life science problems including medicine and biology, we here propose Deep SNP, a novel Deep Neural Network to learn from genomic data. Specifically, we used a manually curated dataset from 12 genomic single nucleotide polymorphism array (SNPa) profiles as truth-set and aimed at predicting the presence or absence of genomic breakpoints, an indicator of structural chromosomal variations, in windows of 40,000 probes. We compare our results with well-known neural network models as well as Rawcopy though this tool is designed to predict breakpoints and in addition genomic segments with high sensitivity. We show, that Deep SNP is capable of successfully predicting the presence or absence of a breakpoint in large genomic windows and outperforms state-of-the-art neural network models. Qualitative examples suggest that integration of a localization unit may enable breakpoint detection and prediction of genomic segments, even if the breakpoint coordinates were not provided for network training. These results warrant further evaluation of DeepSNP for breakpoint localization and subsequent calling of genomic segments., Comment: Accepted at the Joint ICML and IJCAI 2018 Workshop on Computational Biology

Employing large deviation theory, we explore current fluctuations of underdamped Brownian motion for the paradigmatic example of a single particle in a one dimensional periodic potential. Two different approaches to the large deviation function of the particle current are presented. First, we derive an explicit expression for the large deviation functional of the empirical phase space density, which replaces the level 2.5 functional used for overdamped dynamics. Using this approach, we obtain several bounds on the large deviation function of the particle current. We compare these to bounds for overdamped dynamics that have recently been derived motivated by the thermodynamic uncertainty relation. Second, we provide a method to calculate the large deviation function via the cumulant generating function. We use this method to assess the tightness of the bounds in a numerical case study for a cosine potential., Comment: 11 pages, 4 figures

The so-called 'raspberry' model refers to the hybrid lattice-Boltzmann and Langevin molecular dynamics scheme for simulating the dynamics of suspensions of colloidal particles, originally developed by [V. Lobaskin and B. D\"unweg, New J. Phys. 6, 54 (2004)], wherein discrete surface points are used to achieve fluid-particle coupling. This technique has been used in many simulation studies on the behavior of colloids. However, there are fundamental questions with regards to the use of this model. In this paper, we examine the accuracy with which the raspberry method is able to reproduce Stokes-level hydrodynamic interactions when compared to analytic expressions for solid spheres in simple-cubic crystals. To this end, we consider the quality of numerical experiments that are traditionally used to establish these properties and we discuss their shortcomings. We show that there is a discrepancy between the translational and rotational mobility reproduced by the simple raspberry model and present a way to numerically remedy the problem by adding internal coupling points. Finally, we examine a non-convex shape, namely a colloidal dumbbell, and show that the filled raspberry model replicates the desired hydrodynamic behavior in bulk for this more complicated shape. Our investigation is continued in [J. de Graaf, et al., J. Chem. Phys. 143, 084107 (2015)], wherein we consider the raspberry model in the confining geometry of two parallel plates., Comment: 25 pages, 11 figures

The so-called 'raspberry' model refers to the hybrid lattice-Boltzmann (LB) and Langevin molecular dynamics scheme for simulating the dynamics of suspensions of colloidal particles, originally developed by [V. Lobaskin and B. D\"unweg, New J. Phys. 6, 54 (2004)], wherein discrete surface points are used to achieve fluid-particle coupling. In this paper, we present a follow up to our study of the effectiveness of the raspberry model in reproducing hydrodynamic interactions in the Stokes regime for spheres arranged in a simple-cubic crystal [L. Fischer, et al., J. Chem. Phys. 143, 084108 (2015)]. Here, we consider the accuracy with which the raspberry model is able to reproduce such interactions for particles confined between two parallel plates. To this end, we compare our LB simulation results to established theoretical expressions and finite-element calculations. We show that there is a discrepancy between the translational and rotational mobility when only surface coupling points are used, as also found in Part I of our joint publication. We demonstrate that adding internal coupling points to the raspberry, can be used to correct said discrepancy in confining geometries as well. Finally, we show that the raspberry model accurately reproduces hydrodynamic interactions between a spherical colloid and planar walls up to roughly one LB lattice spacing., Comment: 13 pages, 8 figures