Search

Diffusion models have found great success in generating high quality, natural samples of speech, but their potential for density estimation for speech has so far remained largely unexplored. In this work, we leverage an unconditional diffusion model trained only on clean speech for the assessment of speech quality. We show that the quality of a speech utterance can be assessed by estimating the likelihood of a corresponding sample in the terminating Gaussian distribution, obtained via a deterministic noising process. The resulting method is purely unsupervised, trained only on clean speech, and therefore does not rely on annotations. Our diffusion-based approach leverages clean speech priors to assess quality based on how the input relates to the learned distribution of clean data. Our proposed log-likelihoods show promising results, correlating well with intrusive speech quality metrics such as POLQA and SI-SDR.

This paper presents an unsupervised method for single-channel blind dereverberation and room impulse response (RIR) estimation, called BUDDy. The algorithm is rooted in Bayesian posterior sampling: it combines a likelihood model enforcing fidelity to the reverberant measurement, and an anechoic speech prior implemented by an unconditional diffusion model. We design a parametric filter representing the RIR, with exponential decay for each frequency subband. Room acoustics estimation and speech dereverberation are jointly carried out, as the filter parameters are iteratively estimated and the speech utterance refined along the reverse diffusion trajectory. In a blind scenario where the room impulse response is unknown, BUDDy successfully performs speech dereverberation in various acoustic scenarios, significantly outperforming other blind unsupervised baselines. Unlike supervised methods, which often struggle to generalize, BUDDy seamlessly adapts to different acoustic conditions. This paper extends our previous work by offering new experimental results and insights into the algorithm's performance and versatility. We first investigate the robustness of informed dereverberation methods to RIR estimation errors, to motivate the joint acoustic estimation and dereverberation paradigm. Then, we demonstrate the adaptability of our method to high-resolution singing voice dereverberation, study its performance in RIR estimation, and conduct subjective evaluation experiments to validate the perceptual quality of the results, among other contributions. Audio samples and code can be found online., Comment: Submitted to IEEE/ACM Transactions on Audio, Speech and Language Processing

Fiber-coupled sensors are well suited for sensing and microscopy in hard-to-reach environments such as biological or cryogenic systems. We demonstrate fiber-based magnetic imaging based on nitrogen-vacancy (NV) sensor spins at the tip of a fiber-coupled diamond nanobeam. We incorporated angled ion implantation into the nanobeam fabrication process to realize a small ensemble of NV spins at the nanobeam tip. By gluing the nanobeam to a tapered fiber, we created a robust and transportable probe with optimized optical coupling efficiency. We demonstrate the imaging capability of the fiber-coupled nanobeam by measuring the magnetic field generated by a current-carrying wire. With its robust coupling and efficient readout at the fiber-coupled interface, our probe could allow new studies of (quantum) materials and biological samples.

We release the EARS (Expressive Anechoic Recordings of Speech) dataset, a high-quality speech dataset comprising 107 speakers from diverse backgrounds, totaling in 100 hours of clean, anechoic speech data. The dataset covers a large range of different speaking styles, including emotional speech, different reading styles, non-verbal sounds, and conversational freeform speech. We benchmark various methods for speech enhancement and dereverberation on the dataset and evaluate their performance through a set of instrumental metrics. In addition, we conduct a listening test with 20 participants for the speech enhancement task, where a generative method is preferred. We introduce a blind test set that allows for automatic online evaluation of uploaded data. Dataset download links and automatic evaluation server can be found online., Comment: Accepted at Interspeech 2024

To obtain improved speech enhancement models, researchers often focus on increasing performance according to specific instrumental metrics. However, when the same metric is used in a loss function to optimize models, it may be detrimental to aspects that the given metric does not see. The goal of this paper is to illustrate the risk of overfitting a speech enhancement model to the metric used for evaluation. For this, we introduce enhancement models that exploit the widely used PESQ measure. Our "PESQetarian" model achieves 3.82 PESQ on VB-DMD while scoring very poorly in a listening experiment. While the obtained PESQ value of 3.82 would imply "state-of-the-art" PESQ-performance on the VB-DMD benchmark, our examples show that when optimizing w.r.t. a metric, an isolated evaluation on the same metric may be misleading. Instead, other metrics should be included in the evaluation and the resulting performance predictions should be confirmed by listening., Comment: Accepted at Interspeech 2024

In this paper, we present an unsupervised single-channel method for joint blind dereverberation and room impulse response estimation, based on posterior sampling with diffusion models. We parameterize the reverberation operator using a filter with exponential decay for each frequency subband, and iteratively estimate the corresponding parameters as the speech utterance gets refined along the reverse diffusion trajectory. A measurement consistency criterion enforces the fidelity of the generated speech with the reverberant measurement, while an unconditional diffusion model implements a strong prior for clean speech generation. Without any knowledge of the room impulse response nor any coupled reverberant-anechoic data, we can successfully perform dereverberation in various acoustic scenarios. Our method significantly outperforms previous blind unsupervised baselines, and we demonstrate its increased robustness to unseen acoustic conditions in comparison to blind supervised methods. Audio samples and code are available online., Comment: Submitted to IWAENC 2024

Natural climate solutions (NCS) are critical for mitigating climate change through ecosystem-based carbon removal and emissions reductions. NCS implementation can also generate biodiversity and human well-being co-benefits and trade-offs ("NCS co-impacts"), but the volume of evidence on NCS co-impacts has grown rapidly across disciplines, is poorly understood, and remains to be systematically collated and synthesized. A global evidence map of NCS co-impacts would overcome key barriers to NCS implementation by providing relevant information on co-benefits and trade-offs where carbon mitigation potential alone does not justify NCS projects. We employ large language models to assess over two million articles, finding 257,266 relevant articles on NCS co-impacts. We analyze this large and dispersed body of literature using innovative machine learning methods to extract relevant data (e.g., study location, species, and other key variables), and create a global evidence map on NCS co-impacts. Evidence on NCS co-impacts has grown approximately ten-fold in three decades, although some of the most abundant evidence is associated with pathways that have less mitigation potential. We find that studies often examine multiple NCS pathways, indicating natural NCS pathway complements, and each NCS is often associated with two or more coimpacts. Finally, NCS co-impacts evidence and priority areas for NCS are often mismatched--some countries with high mitigation potential from NCS have few published studies on the broader co-impacts of NCS implementation. Our work advances and makes available novel methods and systematic and representative data of NCS co-impacts studies, thus providing timely insights to inform NCS research and action globally., Comment: 28 pages, 5 figures

We show that the maximal shifts in the minimal free resolution of the quotients of a polynomial ring by a monomial ideal are subadditive as a function of the homological degree. This answers a question that has received some attention in recent years. To do so, we define and study a new model for the homology of posets, given by the so called synor complex. We also introduce an Eilenberg-Zilber type shuffle product on the simplicial chain complex of lattices. Combining these concepts we prove that the existence of a non-zero homology class for a lattice forces certain non-zero homology classes in lower intervals. This result then translates into properties of the minimal free resolution. In particular, it implies a generalization of the original question.

Molecular-scale diodes made from self-assembled monolayers (SAMs) could complement silicon-based technologies with smaller, cheaper, and more versatile devices. However, advancement of this emerging technology is limited by insufficient electronic performance exhibited by the molecular current rectifiers. We overcome this barrier by exploiting the charge-transfer state that results from co-assembling SAMs of molecules with strong electron donor and acceptor termini. We obtain a substantial enhancement in current rectification, which correlates with the degree of charge transfer, as confirmed by several complementary techniques. These findings provide a previously unexplored method for manipulating the properties of molecular electronic devices by exploiting donor/acceptor interactions. They also serve as a model test platform for the study of doping mechanisms in organic systems. Our devices have the potential for fast widespread adoption due to their low-cost processing and self-assembly onto silicon substrates, which could allow seamless integration with current technologies.

Let $\Delta$ be a $(d-1)$-dimensional simplicial complex and $h^ \Delta = (h_0^ \Delta ,\ldots, h_d^ \Delta)$ its $h$-vector. For a face uniform subdivision operation ${\mathcal F}$ we write $\Delta_{\mathcal F}$ for the subdivided complex and $H_{\mathcal F}$ for the matrix such that $h^ {\Delta_{\mathcal F}} = H_{\mathcal F} h^ \Delta$. In connection with the real rootedness of symmetric decompositions Athanasiadis and Tzanaki studied for strictly positive $h$-vectors the inequalities $\frac{h_0^ \Delta}{h_1^ \Delta} \leq \frac{h_1^\Delta}{h_{d-1}^ \Delta} \leq \cdots \leq \frac{h_d^ \Delta}{h_0^\Delta}$ and $\frac{h_1^\Delta}{h_{d-1}^\Delta} \geq \cdots \geq \frac{h_{d-2}^\Delta}{h_2^\Delta} \geq \frac{h_{d-1}^\Delta}{h_1^\Delta}$. In this paper we show that if the inequalities holds for a simplicial complex $\Delta$ and $H_{\mathcal F}$ is TP$_2$ (all entries and two minors are non-negative) then the inequalities hold for $\Delta_{\mathcal F}$. We prove that if ${\mathcal F}$ is the barycentric subdivision then $H_{\mathcal F}$ is TP$_2$. If ${\mathcal F}$ is the $r$\textsuperscript{th}-edgewise subdivision then work of Diaconis and Fulman shows $H_{\mathcal F}$ is TP$_2$. Indeed in this case by work of Mao and Wang $H_{\mathcal F}$ is even TP., Comment: Annals of Combinatorics to appear

Large-scale multi-task robotic manipulation systems often rely on text to specify the task. In this work, we explore whether a robot can learn by observing humans. To do so, the robot must understand a person's intent and perform the inferred task despite differences in the embodiments and environments. We introduce Vid2Robot, an end-to-end video-conditioned policy that takes human videos demonstrating manipulation tasks as input and produces robot actions. Our model is trained with a large dataset of prompt video-robot trajectory pairs to learn unified representations of human and robot actions from videos. Vid2Robot uses cross-attention transformer layers between video features and the current robot state to produce the actions and perform the same task as shown in the video. We use auxiliary contrastive losses to align the prompt and robot video representations for better policies. We evaluate Vid2Robot on real-world robots and observe over 20% improvement over BC-Z when using human prompt videos. Further, we also show cross-object motion transfer ability that enables video-conditioned policies to transfer a motion observed on one object in the prompt video to another object in the robot's own environment. Videos available at https://vid2robot.github.io, Comment: Robotics: Science & Systems (RSS) 2024. https://vid2robot.github.io/

This paper sets up an approach for shape optimization problems constrained by variational inequalities (VI) in an appropriate shape space. In contrast to classical VI, where no explicit dependence on the domain is given, VI constrained shape optimization problems are in particular highly challenging because of two main reasons: Firstly, one needs to operate in inherently non-linear, non-convex and infinite-dimensional shape spaces. Secondly, the problem cannot be solved directly without any regularization techniques in general because, e.g., one cannot expect the existence of the shape derivative for an arbitrary shape functional depending on solutions to VI. This paper introduces a specific shape manifold and presents an optimization technique to handle the non-differentiabilities on this shape manifold. In particular, we formulate an optimization system based on G\^ateaux semiderivatives and Eulerian derivatives for a shape optimization problem constrained by the Bingham flow variational inequality. Numerical results show the applicability and efficiency of the proposed approach., Comment: 34 pages, 11 figures; change(s): correction on p. 5 and p. 8

For a finite-dimensional vector space $V$, the common basis complex of $V$ is the simplicial complex whose vertices are the proper non-zero subspaces of $V$, and $\sigma$ is a simplex if and only if there exists a basis $B$ of $V$ that contains a basis of $S$ for all $S\in \sigma$. This complex was introduced by Rognes in 1992 in connection with stable buildings. In this article, we prove that the common basis complex is homotopy equivalent to the proper part of the poset of partial direct sum decompositions of $V$. Moreover, we establish this result in a more general combinatorial context, including the case of free groups, matroids, vector spaces with non-degenerate sesquilinear forms, and free modules over commutative Hermite rings, such as local rings or Dedekind domains., Comment: 17 pages; v2: improved exposition and removed typos according to a referee's suggestions; to appear in IMRN

With the development of audio playback devices and fast data transmission, the demand for high sound quality is rising for both entertainment and communications. In this quest for better sound quality, challenges emerge from distortions and interferences originating at the recording side or caused by an imperfect transmission pipeline. To address this problem, audio restoration methods aim to recover clean sound signals from the corrupted input data. We present here audio restoration algorithms based on diffusion models, with a focus on speech enhancement and music restoration tasks. Traditional approaches, often grounded in handcrafted rules and statistical heuristics, have shaped our understanding of audio signals. In the past decades, there has been a notable shift towards data-driven methods that exploit the modeling capabilities of DNNs. Deep generative models, and among them diffusion models, have emerged as powerful techniques for learning complex data distributions. However, relying solely on DNN-based learning approaches carries the risk of reducing interpretability, particularly when employing end-to-end models. Nonetheless, data-driven approaches allow more flexibility in comparison to statistical model-based frameworks, whose performance depends on distributional and statistical assumptions that can be difficult to guarantee. Here, we aim to show that diffusion models can combine the best of both worlds and offer the opportunity to design audio restoration algorithms with a good degree of interpretability and a remarkable performance in terms of sound quality. We explain the diffusion formalism and its application to the conditional generation of clean audio signals. We believe that diffusion models open an exciting field of research with the potential to spawn new audio restoration algorithms that are natural-sounding and remain robust in difficult acoustic situations., Comment: Currently in revision for IEEE Signal Processing Magazine Special Issue "Model-based and Data-Driven Audio Signal Processing"

Foundation models that incorporate language, vision, and more recently actions have revolutionized the ability to harness internet scale data to reason about useful tasks. However, one of the key challenges of training embodied foundation models is the lack of data grounded in the physical world. In this paper, we propose AutoRT, a system that leverages existing foundation models to scale up the deployment of operational robots in completely unseen scenarios with minimal human supervision. AutoRT leverages vision-language models (VLMs) for scene understanding and grounding, and further uses large language models (LLMs) for proposing diverse and novel instructions to be performed by a fleet of robots. Guiding data collection by tapping into the knowledge of foundation models enables AutoRT to effectively reason about autonomy tradeoffs and safety while significantly scaling up data collection for robot learning. We demonstrate AutoRT proposing instructions to over 20 robots across multiple buildings and collecting 77k real robot episodes via both teleoperation and autonomous robot policies. We experimentally show that such "in-the-wild" data collected by AutoRT is significantly more diverse, and that AutoRT's use of LLMs allows for instruction following data collection robots that can align to human preferences., Comment: 26 pages, 9 figures, ICRA 2024 VLMNM Workshop

Using cosmological simulations of galaxy cluster regions from The Three Hundred project we study the nature of gas in filaments feeding massive clusters. By stacking the diffuse material of filaments throughout the cluster sample, we measure average gas properties such as density, temperature, pressure, entropy and Mach number and construct one-dimensional profiles for a sample of larger, radially-oriented filaments to determine their characteristic features as cosmological objects. Despite the similarity in velocity space between the gas and dark matter accretion patterns onto filaments and their central clusters, we confirm some differences, especially concerning the more ordered radial velocity dispersion of dark matter around the cluster and the larger accretion velocity of gas relative to dark matter in filaments. We also study the distribution of shocked gas around filaments and galaxy clusters, showing that the surrounding shocks allow an efficient internal transport of material, suggesting a laminar infall. The stacked temperature profile of filaments is typically colder towards the spine, in line with the cosmological rarefaction of matter. Therefore, filaments are able to isolate their inner regions, maintaining lower gas temperatures and entropy. Finally, we study the evolution of the gas density-temperature phase diagram of our stacked filament, showing that filamentary gas does not behave fully adiabatically through time but it is subject to shocks during its evolution, establishing a characteristic z = 0, entropy-enhanced distribution at intermediate distances from the spine of about 1 - 2 $h^{-1}$ Mpc for a typical galaxy cluster in our sample., Comment: 16 pages, 13 figures. Accepted for publication in MNRAS

Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for many applications. Can such a consolidation happen in robotics? Conventionally, robotic learning methods train a separate model for every application, every robot, and even every environment. Can we instead train generalist X-robot policy that can be adapted efficiently to new robots, tasks, and environments? In this paper, we provide datasets in standardized data formats and models to make it possible to explore this possibility in the context of robotic manipulation, alongside experimental results that provide an example of effective X-robot policies. We assemble a dataset from 22 different robots collected through a collaboration between 21 institutions, demonstrating 527 skills (160266 tasks). We show that a high-capacity model trained on this data, which we call RT-X, exhibits positive transfer and improves the capabilities of multiple robots by leveraging experience from other platforms. More details can be found on the project website https://robotics-transformer-x.github.io., Comment: Project website: https://robotics-transformer-x.github.io

To explore the coupling between a growing population of microorganisms such as E. coli and a nonuniform nutrient distribution, we formulate a minimalistic model. It consists of active Brownian particles that divide and grow at a nutrient-dependent rate following the Monod equation. The nutrient concentration obeys a diffusion equation with a consumption term and a point source. In this setting the heterogeneity in the nutrient distribution can be tuned by the diffusion coefficient. In particle-based simulations, we demonstrate that passive and weakly active particles form proliferation-induced clusters when the nutrient is localized, without relying on further mechanisms such as chemotaxis or adhesion. In contrast, strongly active particles disperse in the whole system during their lifetime and no clustering is present. The steady population is unaffected by activity or nonuniform nutrient distribution and only determined by the ratio of nutrient influx and bacterial death. However, the transient dynamics strongly depends on the nutrient distribution and activity. Passive particles in almost uniform nutrient profiles display a strong population overshoot, with clusters forming all over the system. In contrast, when slowly diffusing nutrients remain centred around the source, the bacterial population quickly approaches the steady state due to its strong coupling to the nutrient. Conversely, the population overshoot of highly active particles becomes stronger when the nutrient localisation increases. We successfully map the transient population dynamics onto a uniform model where the effect of the nonuniform nutrient and bacterial distributions are rationalized by two effective areas., Comment: 13 figures

In this work, we demonstrate that the ptychographic phase problem can be solved in a live fashion during scanning, while data is still being collected. We propose a generally applicable modification of the widespread projection-based algorithms such as Error Reduction (ER) and Difference Map (DM). This novel variant of ptychographic phase retrieval enables immediate visual feedback during experiments, reconstruction of arbitrary-sized objects with a fixed amount of computational resources, and adaptive scanning. By building upon the Real-Time Iterative Spectrogram Inversion (RTISI) family of algorithms from the audio processing literature, we show that live variants of projection-based methods such as DM can be derived naturally and may even achieve higher-quality reconstructions than their classic non-live counterparts with comparable effective computational load., Comment: Submitted to ICASSP 24

Speech emotion conversion is the task of converting the expressed emotion of a spoken utterance to a target emotion while preserving the lexical content and speaker identity. While most existing works in speech emotion conversion rely on acted-out datasets and parallel data samples, in this work we specifically focus on more challenging in-the-wild scenarios and do not rely on parallel data. To this end, we propose a diffusion-based generative model for speech emotion conversion, the EmoConv-Diff, that is trained to reconstruct an input utterance while also conditioning on its emotion. Subsequently, at inference, a target emotion embedding is employed to convert the emotion of the input utterance to the given target emotion. As opposed to performing emotion conversion on categorical representations, we use a continuous arousal dimension to represent emotions while also achieving intensity control. We validate the proposed methodology on a large in-the-wild dataset, the MSP-Podcast v1.10. Our results show that the proposed diffusion model is indeed capable of synthesizing speech with a controllable target emotion. Crucially, the proposed approach shows improved performance along the extreme values of arousal and thereby addresses a common challenge in the speech emotion conversion literature., Comment: Accepted to ICASSP 2024

Several recent contributions in the field of iterative STFT phase retrieval have demonstrated that the performance of the classical Griffin-Lim method can be considerably improved upon. By using the same projection operators as Griffin-Lim, but combining them in innovative ways, these approaches achieve better results in terms of both reconstruction quality and required number of iterations, while retaining a similar computational complexity per iteration. However, like Griffin-Lim, these algorithms operate in an offline manner and thus require an entire spectrogram as input, which is an unrealistic requirement for many real-world speech communication applications. We propose to extend RTISI -- an existing online (frame-by-frame) variant of the Griffin-Lim algorithm -- into a flexible framework that enables straightforward online implementation of any algorithm based on iterative projections. We further employ this framework to implement online variants of the fast Griffin-Lim algorithm, the accelerated Griffin-Lim algorithm, and two algorithms from the optics domain. Evaluation results on speech signals show that, similarly to the offline case, these algorithms can achieve a considerable performance gain compared to RTISI., Comment: Submitted to ICASSP 24

The activity of central supermassive black holes might affect the alignment of galaxy spin axes with respect to the closest cosmic filaments. We exploit the SAMI Galaxy Survey to study possible relations between black hole activity and the spin-filament alignments of stars and ionised gas separately. To explore the impact of instantaneous black hole activity, active galaxies are selected according to emission-line diagnostics. Central stellar velocity dispersion ($\sigma_c$) is used as a proxy for black hole mass and its integrated activity. We find evidence for the gas spin-filament alignments to be influenced by AGN, with Seyfert galaxies showing a stronger perpendicular alignment at fixed bulge mass with respect to galaxies where ionisation is consequence of low-ionizaition nuclear emission-line regions (LINERs) or old stellar populations (retired galaxies). On the other hand, the greater perpendicular tendency for the stellar spin-filament alignments of high-bulge mass galaxies is dominated by retired galaxies. Stellar alignments show a stronger correlation with $\sigma_c$ compared to the gas alignments. We confirm that bulge mass ($M_{bulge}$) is the primary parameter of correlation for both stellar and gas spin-filament alignments (with no residual dependency left for $\sigma_c$), while $\sigma_c$ is the most important property for secular star formation quenching (with no residual dependency left for $M_{bulge}$). These findings indicate that $M_{bulge}$ and $\sigma_c$ are the most predictive parameters of two different galaxy evolution processes, suggesting mergers trigger spin-filament alignment flips and integrated black hole activity drives star formation quenching., Comment: 20 pages, 16 figures, accepted for publication in MNRAS

The phase-field method is based on the energy minimization principle which is a geometric method for modeling diffusive cracks that are popularly implemented with irreversibility based on Griffith's criterion. This method requires a length-scale parameter that smooths the sharp discontinuity, which influences the diffuse band and results in mesh-sensitive fracture propagation results. Recently, a novel approach based on the optimization on Riemannian shape spaces has been proposed, where the crack path is realized by techniques from shape optimization. This approach requires the shape derivative, which is derived in a continuous sense and used for a gradient-based algorithm to minimize the energy of the system. Due to the continuous derivation of the shape derivative, this approach yields mesh-independent results. In this paper, the novel approach based on shape optimization is presented, followed by an assessment of the predicted crack path in anisotropic brittle material using numerical calculations from a phase-field model.

We investigate a complex system involving multiple shapes to be optimized in a domain, taking into account geometric constraints on the shapes and uncertainty appearing in the physics. We connect the differential geometry of product shape manifolds with multi-shape calculus, which provides a novel framework for the handling of piecewise smooth shapes. This multi-shape calculus is applied to a shape optimization problem where shapes serve as obstacles in a system governed by steady state incompressible Navier-Stokes flow. Numerical experiments use our recently developed stochastic augmented Lagrangian method and we investigate the choice of algorithmic parameters using the example of this application.

Garland introduced a vanishing criterion for a characteristic zero cohomology group of a locally finite and locally connected simplicial complex. The criterion is based on the spectral gaps of the graph Laplacians of the links of faces and has turned out to be effective in a wide range of examples. In this note we extend the approach to include a range of non-simplicial (co)chain complexes associated to combinatorial structures we call Garland posets and elaborate further on the case of cubical complexes., Comment: Example 4.3 added, Example 6.2 added, inequality in Proposition 3.4(ii) sharpened to an equality

We study how vision-language models trained on Internet-scale data can be incorporated directly into end-to-end robotic control to boost generalization and enable emergent semantic reasoning. Our goal is to enable a single end-to-end trained model to both learn to map robot observations to actions and enjoy the benefits of large-scale pretraining on language and vision-language data from the web. To this end, we propose to co-fine-tune state-of-the-art vision-language models on both robotic trajectory data and Internet-scale vision-language tasks, such as visual question answering. In contrast to other approaches, we propose a simple, general recipe to achieve this goal: in order to fit both natural language responses and robotic actions into the same format, we express the actions as text tokens and incorporate them directly into the training set of the model in the same way as natural language tokens. We refer to such category of models as vision-language-action models (VLA) and instantiate an example of such a model, which we call RT-2. Our extensive evaluation (6k evaluation trials) shows that our approach leads to performant robotic policies and enables RT-2 to obtain a range of emergent capabilities from Internet-scale training. This includes significantly improved generalization to novel objects, the ability to interpret commands not present in the robot training data (such as placing an object onto a particular number or icon), and the ability to perform rudimentary reasoning in response to user commands (such as picking up the smallest or largest object, or the one closest to another object). We further show that incorporating chain of thought reasoning allows RT-2 to perform multi-stage semantic reasoning, for example figuring out which object to pick up for use as an improvised hammer (a rock), or which type of drink is best suited for someone who is tired (an energy drink)., Comment: Website: https://robotics-transformer.github.io/

We present in this paper an informed single-channel dereverberation method based on conditional generation with diffusion models. With knowledge of the room impulse response, the anechoic utterance is generated via reverse diffusion using a measurement consistency criterion coupled with a neural network that represents the clean speech prior. The proposed approach is largely more robust to measurement noise compared to a state-of-the-art informed single-channel dereverberation method, especially for non-stationary noise. Furthermore, we compare to other blind dereverberation methods using diffusion models and show superiority of the proposed approach for large reverberation times. We motivate the presented algorithm by introducing an extension for blind dereverberation allowing joint estimation of the room impulse response and anechoic speech. Audio samples and code can be found online (https://uhh.de/inf-sp-derev-dps).

We analyze cold-gas distributions in Virgo cluster galaxies using resolved CO(2-1) (tracing molecular hydrogen, H2) and HI observations from the Virgo Environment Traced In CO (VERTICO) and the VLA Imaging of Virgo in Atomic Gas (VIVA) surveys. From a theoretical perspective, it is expected that environmental processes in clusters will have a stronger influence on diffuse atomic gas compared to the relatively dense molecular gas component, and that these environmental perturbations can compress the cold interstellar medium in cluster galaxies leading to elevated star formation. In this work we observationally test these predictions for star-forming satellite galaxies within the Virgo cluster. We divide our Virgo galaxy sample into HI-normal, HI-tailed, and HI-truncated classes and show, unsurprisingly, that the HI-tailed galaxies have the largest quantitative HI asymmetries. We also compare to a control sample of non-cluster galaxies and find that Virgo galaxies, on average, have HI asymmetries that are 40 +/- 10 per cent larger than the control. There is less separation between control, HI-normal, HI-tailed, and HI-truncated galaxies in terms of H2 asymmetries, and on average, Virgo galaxies have H2 asymmetries that are only marginally (20 +/- 10 per cent) larger than the control sample. We find a weak correlation between HI and H2 asymmetries over our entire sample, but a stronger correlation for those specific galaxies being strongly impacted by environmental perturbations. Finally, we divide the discs of the HI-tailed Virgo galaxies into a leading half and trailing half according to the observed tail direction. We find evidence for excess molecular gas mass on the leading halves of the disc. This excess molecular gas on the leading half is accompanied by an excess in star formation rate such that the depletion time is, on average, unchanged., Comment: 15 pages, 8 figures, 1 table, accepted for publication in A&A

Spaces where each element describes a shape, so-called shape spaces, are of particular interest in shape optimization and its applications. Theory and algorithms in shape optimization are often based on techniques from differential geometry. Challenges arise when an application demands a non-smooth shape, which is commonly-encountered as an optimal shape for fluid-mechanical problems. In order to avoid the restriction to infinitely-smooth shapes of a commonly-used shape space, we construct a space containing shapes in $\mathbb{R}^2$ that can be identified with a Riemannian product manifold but at the same time admits piecewise-smooth curves as elements. We combine the new product manifold with an approach for optimizing multiple non-intersecting shapes. For the newly-defined shapes, adjustments are made in the known shape optimization definitions and algorithms to ensure their usability in applications. Numerical results regarding a fluid-mechanical problem constrained by the Navier-Stokes equations, where the viscous energy dissipation is minimized, show its applicability., Comment: 11 pages, 3 figures

In this paper, we present a stochastic augmented Lagrangian approach on (possibly infinite-dimensional) Riemannian manifolds to solve stochastic optimization problems with a finite number of deterministic constraints.We investigate the convergence of the method, which is based on a stochastic approximation approach with random stopping combined with an iterative procedure for updating Lagrange multipliers. The algorithm is applied to a multi-shape optimization problem with geometric constraints and demonstrated numerically.

In this paper, we present a causal speech signal improvement system that is designed to handle different types of distortions. The method is based on a generative diffusion model which has been shown to work well in scenarios with missing data and non-linear corruptions. To guarantee causal processing, we modify the network architecture of our previous work and replace global normalization with causal adaptive gain control. We generate diverse training data containing a broad range of distortions. This work was performed in the context of an "ICASSP Signal Processing Grand Challenge" and submitted to the non-real-time track of the "Speech Signal Improvement Challenge 2023", where it was ranked fifth., Comment: Accepted by ICASSP 2023

Recently, score-based generative models have been successfully employed for the task of speech enhancement. A stochastic differential equation is used to model the iterative forward process, where at each step environmental noise and white Gaussian noise are added to the clean speech signal. While in limit the mean of the forward process ends at the noisy mixture, in practice it stops earlier and thus only at an approximation of the noisy mixture. This results in a discrepancy between the terminating distribution of the forward process and the prior used for solving the reverse process at inference. In this paper, we address this discrepancy and propose a forward process based on a Brownian bridge. We show that such a process leads to a reduction of the mismatch compared to previous diffusion processes. More importantly, we show that our approach improves in objective metrics over the baseline process with only half of the iteration steps and having one hyperparameter less to tune., Comment: 5 pages, 2 figures, Accepted to Interspeech 20223