Search

In this paper, we tackle the task of musical stem retrieval. Given a musical mix, it consists in retrieving a stem that would fit with it, i.e., that would sound pleasant if played together. To do so, we introduce a new method based on Joint-Embedding Predictive Architectures, where an encoder and a predictor are jointly trained to produce latent representations of a context and predict latent representations of a target. In particular, we design our predictor to be conditioned on arbitrary instruments, enabling our model to perform zero-shot stem retrieval. In addition, we discover that pretraining the encoder using contrastive learning drastically improves the model's performance. We validate the retrieval performances of our model using the MUSDB18 and MoisesDB datasets. We show that it significantly outperforms previous baselines on both datasets, showcasing its ability to support more or less precise (and possibly unseen) conditioning. We also evaluate the learned embeddings on a beat tracking task, demonstrating that they retain temporal structure and local information., Comment: Submitted to ICASSP 2025

Predicting diverse object motions from a single static image remains challenging, as current video generation models often entangle object movement with camera motion and other scene changes. While recent methods can predict specific motions from motion arrow input, they rely on synthetic data and predefined motions, limiting their application to complex scenes. We introduce Motion Modes, a training-free approach that explores a pre-trained image-to-video generator's latent distribution to discover various distinct and plausible motions focused on selected objects in static images. We achieve this by employing a flow generator guided by energy functions designed to disentangle object and camera motion. Additionally, we use an energy inspired by particle guidance to diversify the generated motions, without requiring explicit training data. Experimental results demonstrate that Motion Modes generates realistic and varied object animations, surpassing previous methods and even human predictions regarding plausibility and diversity. Project Webpage: https://motionmodes.github.io/

With the rapid advancements in medical data acquisition and production, increasingly richer representations exist to characterize medical information. However, such large-scale data do not usually meet computing resource constraints or algorithmic complexity, and can only be processed after compression or reduction, at the potential loss of information. In this work, we consider specific Gaussian mixture models (HD-GMM), tailored to deal with high dimensional data and to limit information loss by providing component-specific lower dimensional representations. We also design an incremental algorithm to compute such representations for large data sets, overcoming hardware limitations of standard methods. Our procedure is illustrated in a magnetic resonance fingerprinting study, where it achieves a 97% dictionary compression for faster and more accurate map reconstructions.

Using log-geometry, we construct a model for the configuration category of a smooth algebraic variety. As an application, we prove the formality of certain configuration spaces., Comment: 20 pages

In this paper, we propose a novel Self-Supervised-Learning scheme to train rhythm analysis systems and instantiate it for few-shot beat tracking. Taking inspiration from the Contrastive Predictive Coding paradigm, we propose to train a Log-Mel-Spectrogram Transformer encoder to contrast observations at times separated by hypothesized beat intervals from those that are not. We do this without the knowledge of ground-truth tempo or beat positions, as we rely on the local maxima of a Predominant Local Pulse function, considered as a proxy for Tatum positions, to define candidate anchors, candidate positives (located at a distance of a power of two from the anchor) and negatives (remaining time positions). We show that a model pre-trained using this approach on the unlabeled FMA, MTT and MTG-Jamendo datasets can successfully be fine-tuned in the few-shot regime, i.e. with just a few annotated examples to get a competitive beat-tracking performance.

In this study, we present the integration of a neural network-based parameterization into the global atmospheric model ARP-GEM1, leveraging the Python interface of the OASIS coupler. This approach facilitates the exchange of fields between the Fortran-based ARP-GEM1 model and a Python component responsible for neural network inference. As a proof-of-concept experiment, we trained a neural network to emulate the deep convection parameterization of ARP-GEM1. Using the flexible Fortran/Python interface, we have successfully replaced ARP-GEM1's deep convection scheme with a neural network emulator. To assess the performance of the neural network deep convection scheme, we have run a 5-years ARP-GEM1 simulation using the neural network emulator. The evaluation of averaged fields showed good agreement with output from an ARP-GEM1 simulation using the physics-based deep convection scheme. The Python component was deployed on a separate partition from the general circulation model, using GPUs to increase inference speed of the neural network., Comment: 10 pages, 5 figures, submitted to Artificial Intelligence for the Earth Systems

Hole transport and localization through small polarons is essential to the performance of TiO$_2$ in photocatalysis applications. The existence of small hole polaron in bulk rutile TiO$_2$ has been however controversial with contradicting evidences from theory and experiments. Here, we use first principles computations and more specifically a Koopmans' compliant hybrid functional and charge correction to study small hole polarons in bulk rutile. We find that a fraction of exchange exists satisfying Koopmans' compliance for the polaron state and reproducing the band gap provided that charge correction is used. We clearly show that first principles computations indicate that the small hole polaron is unstable in bulk for rutile and stable for anatase TiO$_2$.

Understanding the dehydrogenation of transition metal oxide surfaces under electrochemical potential is critical to the control of important chemical processes such as the oxygen evolution reaction (OER). Using first principles computations, we model the thermodynamic dehydrogenation process on RuO$_2$(110) and compare the results to experimental cyclic voltammetry (CV) on single crystal. We use a cluster expansion model trained on *ab initio* energy data coupled with Monte Carlo (MC) sampling to derive the macroscopic electrochemical observables, i.e., experimental CV, from the energetics of different hydrogen coverage microstates on well-defined RuO$_2$(110). Our model reproduces the unique "two-peaks" cyclic voltammogram observed experimentally with current density peak positions and shapes in good qualitative agreement. We show that RuO$_2$(110) starts as a water-covered surface with hydrogen on bridge (BRG) and coordination-unsaturated sites (CUS) at low potential (less than 0.4 V vs. reversible hydrogen electrode, RHE). As the potential increases, the hydrogens on BRG desorb, becoming the main contributor to the first CV peak with smaller contributions from CUS. When all BRG hydrogens are desorbed (before 1.2 V vs. RHE), the remaining CUS hydrogens desorb abruptly in a very small potential window leading to the sharp second peak observed during CV. Our work shows that above 1.23 V, the OER proceeds on a fully dehydrogenated RuO$_2$(110) surface., Comment: 20 pages, 6 figures in Main Text, 3 figures in Supplementary Info

This work reports a pulsed laser-assisted synthesis, detailed structural characterization, and study of plasmonic properties of three sets of TiN/TiNO thin films with high electron density. The first two sets of TiN films were grown at 600C and 700C under a high vacuum condition. The third set of TiN film was grown in the presence of 5 mTorr of molecular oxygen at 700C. The results have shown that TiN films deposited in a high vacuum are metallic, have large optical reflectance, and have high optical and electrical conductivity. The TiN films, grown in 5 mTorr O2, were partially oxidized and semiconducting with room temperature resistivity nearly three times larger than those of the TiN films grown under high vacuum conditions. The optical conductivity of these films was analyzed using a Kramers-Kronig transformation of reflectance and a Lorentz-Drude model; the optical conductivity determined by these two different methods agreed very well. To corroborate our experimental spectral observations, we have calculated the phonon dispersions and Raman active modes of TiNO using the virtual crystal approximation. A comparative analysis of the phonon dispersions between rutile TiO2 and rocksalt TiNO has shown that the incorporation of nitrogen atoms does not significantly alter the phonon dispersions of rutile TiO2. However, it results in the emergence of new phonon modes at approximately 7.128 THz at the Gamma point, which corresponds to the experimentally observed Multi-Photon Phase-MPP. From the collateral study of experimental results and theoretical corroboration, a suitable multi-layer optical model was proposed for the TiN/TINO epitaxial thin films to extract the individual complex dielectric function from which many other optical parameters can be calculated.

In this note, we explain an operadic proof of the BTT Theorem stating that the deformation theory of Calabi-Yau varieties is unobstructed. We also provide a short new proof of the non-commutative BTT for Calabi-Yau dg-categories. Finally, we observe that our proof also produces partial results in positive characteristic., Comment: Submitted to the proceedings of the conference "Derived Lie Algebras in Geometry and Topology"

We introduce a high-fidelity portrait shadow removal model that can effectively enhance the image of a portrait by predicting its appearance under disturbing shadows and highlights. Portrait shadow removal is a highly ill-posed problem where multiple plausible solutions can be found based on a single image. While existing works have solved this problem by predicting the appearance residuals that can propagate local shadow distribution, such methods are often incomplete and lead to unnatural predictions, especially for portraits with hard shadows. We overcome the limitations of existing local propagation methods by formulating the removal problem as a generation task where a diffusion model learns to globally rebuild the human appearance from scratch as a condition of an input portrait image. For robust and natural shadow removal, we propose to train the diffusion model with a compositional repurposing framework: a pre-trained text-guided image generation model is first fine-tuned to harmonize the lighting and color of the foreground with a background scene by using a background harmonization dataset; and then the model is further fine-tuned to generate a shadow-free portrait image via a shadow-paired dataset. To overcome the limitation of losing fine details in the latent diffusion model, we propose a guided-upsampling network to restore the original high-frequency details (wrinkles and dots) from the input image. To enable our compositional training framework, we construct a high-fidelity and large-scale dataset using a lightstage capturing system and synthetic graphics simulation. Our generative framework effectively removes shadows caused by both self and external occlusions while maintaining original lighting distribution and high-frequency details. Our method also demonstrates robustness to diverse subjects captured in real environments., Comment: 17 pages, siggraph asia, TOG

The Prototypical Network (ProtoNet) has emerged as a popular choice in Few-shot Learning (FSL) scenarios due to its remarkable performance and straightforward implementation. Building upon such success, we first propose a simple (yet novel) method to fine-tune a ProtoNet on the (labeled) support set of the test episode of a C-way-K-shot test episode (without using the query set which is only used for evaluation). We then propose an algorithmic framework that combines ProtoNet with optimization-based FSL algorithms (MAML and Meta-Curvature) to work with such a fine-tuning method. Since optimization-based algorithms endow the target learner model with the ability to fast adaption to only a few samples, we utilize ProtoNet as the target model to enhance its fine-tuning performance with the help of a specifically designed episodic fine-tuning strategy. The experimental results confirm that our proposed models, MAML-Proto and MC-Proto, combined with our unique fine-tuning method, outperform regular ProtoNet by a large margin in few-shot audio classification tasks on the ESC-50 and Speech Commands v2 datasets. We note that although we have only applied our model to the audio domain, it is a general method and can be easily extended to other domains., Comment: Accepted at MLSP 2024

This is the second part of a series of two articles focused on the development and evaluation of the ARP-GEM1 global atmosphere model. The first paper introduced the model's new physics and speedup improvements. In this second part, we evaluate ARP-GEM1 through a set of 30-year prescribed sea-surface-temperature simulations at 55, 25, 12.6, and 6.3 km resolutions. The model demonstrates reliability in representing global climate metrics, comparable to the top-performing CMIP6 models, while maintaining high computational efficiency at all resolutions. These simulations further demonstrate the feasibility of O(10)-km climate simulations, and highlight their added value, particularly for capturing phenomena such as cyclones. Ultimately, these exploratory simulations should be considered an intermediate step toward the development and tuning of even higher-resolution, convection-permitting kilometer-scale configurations., Comment: 16 pages, 16 figures, submitted to Journal of Climate

This is the first part of a series of two articles describing the ARP-GEM global atmosphere model version 1 and its evaluation in simulations from 55 km to 6 km resolutions. This article provides a complete description of ARP-GEM1, focusing on its new physical parameterizations and acceleration factors aimed at improving computational efficiency. ARP-GEM1 is approximately 15 times faster than its base model version, ARPEGE-Climat v6.3, while maintaining accurate simulations and enhancing model performance, as shown in Part II.This significant acceleration results from a combination of optimizations, rather than a single factor, with each factor's contribution quantified. Additionally, a detailed decomposition of the model's speedup per component is presented. Streamlining the model reduces computational costs, enhances transparency and portability, and maximizes the benefits of future advanced computing technologies. The results presented here suggest that kilometer-scale global climate simulations should become feasible in the near future., Comment: 20 pages, 4 figures, submitted to Journal of Climate

Existing portrait relighting methods struggle with precise control over facial shadows, particularly when faced with challenges such as handling hard shadows from directional light sources or adjusting shadows while remaining in harmony with existing lighting conditions. In many situations, completely altering input lighting is undesirable for portrait retouching applications: one may want to preserve some authenticity in the captured environment. Existing shadow editing methods typically restrict their application to just the facial region and often offer limited lighting control options, such as shadow softening or rotation. In this paper, we introduce COMPOSE: a novel shadow editing pipeline for human portraits, offering precise control over shadow attributes such as shape, intensity, and position, all while preserving the original environmental illumination of the portrait. This level of disentanglement and controllability is obtained thanks to a novel decomposition of the environment map representation into ambient light and an editable gaussian dominant light source. COMPOSE is a four-stage pipeline that consists of light estimation and editing, light diffusion, shadow synthesis, and finally shadow editing. We define facial shadows as the result of a dominant light source, encoded using our novel gaussian environment map representation. Utilizing an OLAT dataset, we have trained models to: (1) predict this light source representation from images, and (2) generate realistic shadows using this representation. We also demonstrate comprehensive and intuitive shadow editing with our pipeline. Through extensive quantitative and qualitative evaluations, we have demonstrated the robust capability of our system in shadow editing., Comment: Accepted at ECCV 2024

This paper explores the automated process of determining stem compatibility by identifying audio recordings of single instruments that blend well with a given musical context. To tackle this challenge, we present Stem-JEPA, a novel Joint-Embedding Predictive Architecture (JEPA) trained on a multi-track dataset using a self-supervised learning approach. Our model comprises two networks: an encoder and a predictor, which are jointly trained to predict the embeddings of compatible stems from the embeddings of a given context, typically a mix of several instruments. Training a model in this manner allows its use in estimating stem compatibility - retrieving, aligning, or generating a stem to match a given mix - or for downstream tasks such as genre or key estimation, as the training paradigm requires the model to learn information related to timbre, harmony, and rhythm. We evaluate our model's performance on a retrieval task on the MUSDB18 dataset, testing its ability to find the missing stem from a mix and through a subjective user study. We also show that the learned embeddings capture temporal alignment information and, finally, evaluate the representations learned by our model on several downstream tasks, highlighting that they effectively capture meaningful musical features., Comment: Proceedings of the 25th International Society for Music Information Retrieval Conference, ISMIR 2024

We attack the question of E_2-formality of differential graded algebras over prime fields via obstruction theory. We are able to prove that E_2-algebras whose cohomology ring is a polynomial algebra on even degree classes are intrinsically formal. As a consequence we prove E_2-formality of the classifying space of some compact Lie group or of Davis-Januszkiewicz spaces., Comment: 15 pages

Most novel view synthesis methods such as NeRF are unable to capture the true high dynamic range (HDR) radiance of scenes since they are typically trained on photos captured with standard low dynamic range (LDR) cameras. While the traditional exposure bracketing approach which captures several images at different exposures has recently been adapted to the multi-view case, we find such methods to fall short of capturing the full dynamic range of indoor scenes, which includes very bright light sources. In this paper, we present PanDORA: a PANoramic Dual-Observer Radiance Acquisition system for the casual capture of indoor scenes in high dynamic range. Our proposed system comprises two 360{\deg} cameras rigidly attached to a portable tripod. The cameras simultaneously acquire two 360{\deg} videos: one at a regular exposure and the other at a very fast exposure, allowing a user to simply wave the apparatus casually around the scene in a matter of minutes. The resulting images are fed to a NeRF-based algorithm that reconstructs the scene's full high dynamic range. Compared to HDR baselines from previous work, our approach reconstructs the full HDR radiance of indoor scenes without sacrificing the visual quality while retaining the ease of capture from recent NeRF-like approaches., Comment: 10 pages, 8 figures

This is a survey on formality results relying on weight structures. A weight structure is a naturally occurring grading on certain differential graded algebras. If this weight satisfies a purity property, one can deduce formality. Algebraic geometry provides us with such weight structures as the cohomology of algebraic varieties tends to present additional structures including a Hodge structure or a Galois action., Comment: 40 pages

The development of tandem photovoltaics and photoelectrochemical solar cells requires new absorber materials with band gaps in the range of ~1.5-2.3 eV, for use in the top cell paired with a narrower-gap bottom cell. An outstanding challenge is finding materials with suitable optoelectronic and defect properties, good operational stability, and synthesis conditions that preserve underlying device layers. This study demonstrates the Zintl phosphide compound CaZn$_2$P$_2$ as a compelling candidate semiconductor for these applications. We prepare phase pure, 500 nm-thick CaZn$_2$P$_2$ thin films using a scalable reactive sputter deposition process at growth temperatures as low as 100 {\deg}C, which is desirable for device integration. UV-vis spectroscopy shows that CaZn$_2$P$_2$ films exhibit an optical absorptivity of ~10$^4$ cm$^-$$^1$ at ~1.95 eV direct band gap. Room-temperature photoluminescence (PL) measurements show near-band-edge optical emission, and time-resolved microwave conductivity (TRMC) measurements indicate a photoexcited carrier lifetime of ~30 ns. CaZn$_2$P$_2$ is highly stable in both ambient conditions and moisture, as evidenced by PL and TRMC measurements. Experimental data are supported by first-principles calculations, which indicate the absence of low-formation-energy, deep intrinsic defects. Overall, our study should motivate future work integrating this potential top cell absorber material into tandem solar cells.

Obstructive sleep apnea (OSA) is frequent and responsible for cardiovascular complications and excessive daytime sleepiness. It is underdiagnosed due to the difficulty to access the gold standard for diagnosis, polysomnography (PSG). Alternative methods using smartphone sensors could be useful to increase diagnosis. The objective is to assess the performances of Apneal, an application that records the sound using a smartphone's microphone and movements thanks to a smartphone's accelerometer and gyroscope, to estimate patients' AHI. In this article, we perform a monocentric proof-of-concept study with a first manual scoring step, and then an automatic detection of respiratory events from the recorded signals using a sequential deep-learning model which was released internally at Apneal at the end of 2022 (version 0.1 of Apneal automatic scoring of respiratory events), in adult patients during in-hospital polysomnography.46 patients (women 34 per cent, mean BMI 28.7 kg per m2) were included. For AHI superior to 15, sensitivity of manual scoring was 0.91, and positive predictive value (PPV) 0.89. For AHI superior to 30, sensitivity was 0.85, PPV 0.94. We obtained an AUC-ROC of 0.85 and an AUC-PR of 0.94 for the identification of AHI superior to 15, and AUC-ROC of 0.95 and AUC-PR of 0.93 for AHI superior to 30. Promising results are obtained for the automatic annotations of events.This article shows that manual scoring of smartphone-based signals is possible and accurate compared to PSG-based scorings. Automatic scoring method based on a deep learning model provides promising results. A larger multicentric validation study, involving subjects with different SAHS severity is required to confirm these results., Comment: 21 pages, 6 figures

Lead-free halide double perovskites provide a promising solution for the long-standing issues of lead-containing halide perovskites, i.e., the toxicity of Pb and the low stability under ambient conditions and high-intensity illumination. Their light-to-electricity or thermal-to-electricity conversion is strongly determined by the dynamics of the corresponding lattice vibrations. Here, we present the measurement of lattice dynamics in a prototypical lead-free halide double perovskite, i.e., Cs2NaInCl6. Our quantitative measurements and first-principles calculations show that the scatterings among lattice vibrations at room temperature are at the timescale of ~ 1 ps, which stems from the extraordinarily strong anharmonicity in Cs2NaInCl6. We further quantitatively characterize the degree of anharmonicity of all the ions in the single Cs2NaInCl6 crystal, and demonstrate that this strong anharmonicity is synergistically contributed by the bond hierarchy, the tilting of the NaCl6 and InCl6 octahedral units, and the rattling of Cs+ ions. Consequently, the crystalline Cs2NaInCl6 possesses an ultralow thermal conductivity of ~0.43 W/mK at room temperature, and a weak temperature dependence of T-0.41. Our findings here uncovered the underlying mechanisms behind the dynamics of lattice vibrations in double perovskites, which could largely benefit the design of optoelectronics and thermoelectrics based on halide double perovskites.

Fundamentally understanding the lattice dynamics and microscopic mechanisms of thermal transport in cubic hybrid organic-inorganic perovskites remains elusive, primarily due to their strong anharmonicity and frequent phase transitions. In this work, we comprehensively investigate the thermal transport behavior in cubic hybrid perovskite FAPbI3, integrating first principles-based anharmonic lattice dynamics with a linearized Wigner transport formula. The Temperature Dependent Effective Potential (TDEP) technique allows us to stabilize the negative soft modes, primarily dominated by organic cations, at finite temperatures in cubic FAPbI3. We then predict an ultra-low thermal conductivity of ~0.63 Wm^(-1) K^(-1) in cubic FAPbI3 at 300 K, with a temperature dependence of T^(-0.740), suggesting a good crystalline nature of phonon transport. Notably, the ultra-low thermal conductivity in cubic FAPbI3 is primarily attributed to the [PbI3]1- units, challenging the conventional focus on organic FA+ cations. This shift in focus is due to the presence of Pb(s)-I(p) anti-bonding sates within the [PbI3]1- units. Furthermore, thermal transport in cubic FAPbI3 is predominantly governed by the particle-like phonon propagation channel across the entire temperature range of 300-500 K, a result of diminished suppression of low-frequency phonons by FA+ cations and large inter-branch spacings. Finally, our findings underscore that the anharmonic force constants are highly temperature-sensitive, leading to underestimations of thermal conductivity when relying on 0-K anharmonic force constants. Our study not only elucidates the microscopic mechanisms of thermal transport in cubic FAPbI3 but also provides a crucial framework for the discovery, design, and understanding of hybrid organic-inorganic compounds with ultra-low thermal conductivity.

The chalcogenide perovskite BaZrS$_3$ has attracted much attention as a promising solar absorber for thin-film photovoltaics. Here, we use first-principles calculations to evaluate its carrier transport and defect properties. We find that BaZrS$_3$ has a phonon-limited electron mobility of 37 cm$^2$/Vs comparable to that in halide perovskites but lower hole mobility of 11 cm$^2$/Vs. The defect computations indicate that BaZrS$_3$ is intrinsically n-type due to shallow sulfur vacancies, but that strong compensation by sulfur vacancies will prevent attempts to make it p-type. We also establish that BaZrS$_3$ shows some degree of defect tolerance, presenting only few low formation energy, deep intrinsic defects. Among the deep defects, sulfur interstitials are the dominant nonradiative recombination centers but exhibit a moderate capture coefficient. Our work highlights the material's intrinsic limitations in carrier mobility and p-type doping and suggests focusing on suppressing the formation of sulfur interstitials to reach longer carrier lifetime.

This paper addresses the problem of self-supervised general-purpose audio representation learning. We explore the use of Joint-Embedding Predictive Architectures (JEPA) for this task, which consists of splitting an input mel-spectrogram into two parts (context and target), computing neural representations for each, and training the neural network to predict the target representations from the context representations. We investigate several design choices within this framework and study their influence through extensive experiments by evaluating our models on various audio classification benchmarks, including environmental sounds, speech and music downstream tasks. We focus notably on which part of the input data is used as context or target and show experimentally that it significantly impacts the model's quality. In particular, we notice that some effective design choices in the image domain lead to poor performance on audio, thus highlighting major differences between these two modalities., Comment: Self-supervision in Audio, Speech and Beyond workshop, IEEE International Conference on Acoustics, Speech, and Signal Processing, 2024

The silicon T centre's narrow, telecommunications-band optical emission, long spin coherence, and direct photonic integration have spurred interest in this emitter as a spin-photon interface for distributed quantum computing and networking. However, key parameters of the T centre's spin-selective optical transitions remain undetermined or ambiguous in literature. In this paper we present a Hamiltonian of the T centre TX state and determine key parameters of the optical transition from T$_0$ to TX$_0$ from a combined analysis of published results, density functional theory, and new spectroscopy. We resolve ambiguous values of the internal defect potential in the literature, and we present the first measurements of electrically tuned T centre emission. As a result, we provide a model of the T centre's optical and spin properties under strain, electric, and magnetic fields that can be utilized for realizing quantum technologies., Comment: 18 pages and 13 figures in the main manuscript with appendix included

We study the connection between mixing properties for bipartite graphs and materialization of the mutual information in one-shot settings. We show that mixing properties of a graph imply impossibility to extract the mutual information shared by the ends of an edge randomly sampled in the graph. We apply these impossibility results to some questions motivated by information-theoretic cryptography. In particular, we show that communication complexity of a secret key agreement in one-shot setting is inherently uneven: for some inputs, almost all communication complexity inevitably falls on only one party., Comment: 17 pages, 4 figures; version 2: a few typos fixed, some arguments are explained in more detail