Search

Causal sets are locally finite, partially ordered sets (posets), which are considered as discrete models of spacetimes. On the one hand, causal sets corresponding to a spacetime manifold are commonly generated with a random process called sprinkling. This process keeps only a discrete set of points of the manifold and their causal relations (loosing the spacetime symmetries in each sprinkle). On the other hand, the main conjecture of causal set theory is that given an ensemble of causal sets there is a corresponding spacetime manifold and the continuum symmetries of it are like all manifold properties "reconstructable" from the partial orders of all the causal sets in the ensemble. But most generic finite posets have very few layers ("instances of time") in contrast to sprinkles with many layers in a sufficiently large spacetime region. In a recent project, I investigated the automorphism groups of (finite) posets in order to identify and classify their symmetries systematically. The comparison of local symmetries of generic posets (including Kleitmann-Rothschild orders) with sprinkled causal sets may help us to find those posets that can serve as discrete spacetime models in causal set theory., Comment: 5 pages, 1 figure, contribution to the conference proceedings of the 17th Marcel Grossmann meeting (7-12 Juli 2024)

Lagrangian averaging is a valuable tool for the analysis and modelling of multiscale processes in fluid dynamics. The numerical computation of Lagrangian (time) averages from simulation data is challenging, however. It can be carried out by tracking a large number of particles or, following a recent approach, by solving a dedicated set of partial differential equations (PDEs). Both approaches are computationally demanding because they require an entirely new computation for each time at which the Lagrangian mean fields are desired. We overcome this drawback by developing a PDE-based method that delivers Lagrangian mean fields for all times through the single solution of evolutionary PDEs. This allows for an on-the-fly implementation, in which Lagrangian averages are computed along with the dynamical variables. This is made possible by the use of a special class of temporal filters whose kernels are sums of exponential functions. We focus on two specific kernels involving one and two exponential functions. We implement these in the rotating shallow-water model and demonstrate their effectiveness at filtering out large-amplitude Poincar\'e waves while retaining the salient features of an underlying slowly evolving turbulent flow.

Partially ordered sets (posets) have a universal appearance as an abstract structure in many areas of mathematics. Though, even their explicit enumeration remains unknown in general, and only the counts of all partial orders on sets of up to 16 unlabelled elements have been calculated to date, see sequence A000112 in the OEIS. In this work, we study automorphisms of posets in order to formulate a classification by local symmetries. These symmetries give rise to a division operation on the set of all posets and lead us to the construction of symmetry classes that are easier to characterise and enumerate. Additionally to the enumeration of symmetry classes, I derive polynomial expressions that count certain subsets of posets with a large number of layers (a large height). As an application in physics, I investigate local symmetries (or rather their lack of) in causal sets, which are discrete spacetime models used as a candidate framework for quantum gravity., Comment: 33 pages, 5 figures, 3 tables

We consider the algebra of massive fermions restricted to a diamond in two-dimensional Minkowski spacetime, and in the Minkowski vacuum state. While the massless modular Hamiltonian is known for this setting, the derivation of the massive one is an open problem. We compute the small-mass corrections to the modular Hamiltonian in a perturbative approach, finding some terms which were previously overlooked. Our approach can in principle be extended to all orders in the mass, even though it becomes computationally challenging., Comment: 37 pages, 11 figures, matches published version

We introduce the Song Describer dataset (SDD), a new crowdsourced corpus of high-quality audio-caption pairs, designed for the evaluation of music-and-language models. The dataset consists of 1.1k human-written natural language descriptions of 706 music recordings, all publicly accessible and released under Creative Common licenses. To showcase the use of our dataset, we benchmark popular models on three key music-and-language tasks (music captioning, text-to-music generation and music-language retrieval). Our experiments highlight the importance of cross-dataset evaluation and offer insights into how researchers can use SDD to gain a broader understanding of model performance., Comment: Accepted to NeurIPS 2023 Workshop on Machine Learning for Audio

This paper investigates foundation models tailored for music informatics, a domain currently challenged by the scarcity of labeled data and generalization issues. To this end, we conduct an in-depth comparative study among various foundation model variants, examining key determinants such as model architectures, tokenization methods, temporal resolution, data, and model scalability. This research aims to bridge the existing knowledge gap by elucidating how these individual factors contribute to the success of foundation models in music informatics. Employing a careful evaluation framework, we assess the performance of these models across diverse downstream tasks in music information retrieval, with a particular focus on token-level and sequence-level classification. Our results reveal that our model demonstrates robust performance, surpassing existing models in specific key metrics. These findings contribute to the understanding of self-supervised learning in music informatics and pave the way for developing more effective and versatile foundation models in the field. A pretrained version of our model is publicly available to foster reproducibility and future research., Comment: 5 pages

Recently, multi-band spectrogram-based approaches such as Band-Split RNN (BSRNN) have demonstrated promising results for music source separation. In our recent work, we introduce the BS-RoFormer model which inherits the idea of band-split scheme in BSRNN at the front-end, and then uses the hierarchical Transformer with Rotary Position Embedding (RoPE) to model the inner-band and inter-band sequences for multi-band mask estimation. This model has achieved state-of-the-art performance, but the band-split scheme is defined empirically, without analytic supports from the literature. In this paper, we propose Mel-RoFormer, which adopts the Mel-band scheme that maps the frequency bins into overlapped subbands according to the mel scale. In contract, the band-split mapping in BSRNN and BS-RoFormer is non-overlapping and designed based on heuristics. Using the MUSDB18HQ dataset for experiments, we demonstrate that Mel-RoFormer outperforms BS-RoFormer in the separation tasks of vocals, drums, and other stems., Comment: submitted as an ISMIR 2023 late-breaking and demo paper

In the era of data-driven Music Information Retrieval (MIR), the scarcity of labeled data has been one of the major concerns to the success of an MIR task. In this work, we leverage the semi-supervised teacher-student training approach to improve MIR tasks. For training, we scale up the unlabeled music data to 240k hours, which is much larger than any public MIR datasets. We iteratively create and refine the pseudo-labels in the noisy teacher-student training process. Knowledge expansion is also explored to iteratively scale up the model sizes from as small as less than 3M to almost 100M parameters. We study the performance correlation between data size and model size in the experiments. By scaling up both model size and training data, our models achieve state-of-the-art results on several MIR tasks compared to models that are either trained in a supervised manner or based on a self-supervised pretrained model. To our knowledge, this is the first attempt to study the effects of scaling up both model and training data for a variety of MIR tasks.

This paper introduces a four-dimensional (4D) geometry-based stochastic model (GBSM) for polarized multiple-input multiple-output (MIMO) systems with moving scatterers. We propose a novel motion path model with high degrees of freedom based on the Brownian Motion (BM) random process for randomly moving scatterers. This model is capable of analyzing the effect of both deterministically and randomly moving scatterers on channel properties. The mixture of Von Mises Fisher (VMF) distribution is considered for scatterers resulting in a more general and practical model. The proposed motion path model is applied to the clusters of scatterers with the mixture of VMF distribution, and a closed form formula for calculating space time correlation function (STCF) is achieved, allowing the study of the behavior of channel correlation and channel capacity in the time domain with the presence of stationary and moving scatterers. To obtain numerical results for channel capacity, we employed Monte Carlo simulation method for channel realization purpose. The impact of moving scatterers on the performance of polarized MIMO systems is evaluated using 2 by 2 MIMO configurations with various dual polarizations, i.e. V/V, V/H, and slanted 45{\deg} polarizations for different signal-to-noise (SNR) regimes. The proposed motion path model can be applied to study various dynamic systems with moving objects. The presented process and achieved formula are general and can be applied to polarized MIMO systems with any arbitrary number of antennas and polarizations., Comment: 10 pages, 10 Figures

We introduce a framework that recommends music based on the emotions of speech. In content creation and daily life, speech contains information about human emotions, which can be enhanced by music. Our framework focuses on a cross-domain retrieval system to bridge the gap between speech and music via emotion labels. We explore different speech representations and report their impact on different speech types, including acting voice and wake-up words. We also propose an emotion similarity regularization term in cross-domain retrieval tasks. By incorporating the regularization term into training, similar speech-and-music pairs in the emotion space are closer in the joint embedding space. Our comprehensive experimental results show that the proposed model is effective in textless speech-to-music retrieval., Comment: To Appear IEEE ICASSP 2023

In this paper, we introduce Jointist, an instrument-aware multi-instrument framework that is capable of transcribing, recognizing, and separating multiple musical instruments from an audio clip. Jointist consists of an instrument recognition module that conditions the other two modules: a transcription module that outputs instrument-specific piano rolls, and a source separation module that utilizes instrument information and transcription results. The joint training of the transcription and source separation modules serves to improve the performance of both tasks. The instrument module is optional and can be directly controlled by human users. This makes Jointist a flexible user-controllable framework. Our challenging problem formulation makes the model highly useful in the real world given that modern popular music typically consists of multiple instruments. Its novelty, however, necessitates a new perspective on how to evaluate such a model. In our experiments, we assess the proposed model from various aspects, providing a new evaluation perspective for multi-instrument transcription. Our subjective listening study shows that Jointist achieves state-of-the-art performance on popular music, outperforming existing multi-instrument transcription models such as MT3. We conducted experiments on several downstream tasks and found that the proposed method improved transcription by more than 1 percentage points (ppt.), source separation by 5 SDR, downbeat detection by 1.8 ppt., chord recognition by 1.4 ppt., and key estimation by 1.4 ppt., when utilizing transcription results obtained from Jointist. Demo available at \url{https://jointist.github.io/Demo}., Comment: arXiv admin note: text overlap with arXiv:2206.10805

This paper introduces effective design choices for text-to-music retrieval systems. An ideal text-based retrieval system would support various input queries such as pre-defined tags, unseen tags, and sentence-level descriptions. In reality, most previous works mainly focused on a single query type (tag or sentence) which may not generalize to another input type. Hence, we review recent text-based music retrieval systems using our proposed benchmark in two main aspects: input text representation and training objectives. Our findings enable a universal text-to-music retrieval system that achieves comparable retrieval performances in both tag- and sentence-level inputs. Furthermore, the proposed multimodal representation generalizes to 9 different downstream music classification tasks. We present the code and demo online.

Energy scenarios and transition pathways need estimates of technical wind energy potentials. However, the standard policy-side approach uses observed wind speeds, thereby neglecting the effects of kinetic energy (KE) removal by the wind turbines that depletes the regional wind resource, lowers wind speeds, and reduces capacity factors. The standard approach therefore significantly overestimates the wind resource potential relative to estimates using numerical models of the atmosphere with interactive wind farm parameterizations. Here, we test the extent to which these effects of KE removal can be accounted for by our KE Budget of the Atmosphere (KEBA) approach over Kansas in the central US, a region with a high wind energy resource. We find that KEBA reproduces the simulated estimates within 10 - 11%, which are 30 - 50% lower than estimates using the standard approach. We also evaluate important differences in the depletion of the wind resource between daytime and nighttime conditions, which are due to effects of stability. Our results indicate that the KEBA approach is a simple yet adequate approach to evaluating regional-scale wind resource potentials, and that resource depletion effects need to be accounted for at such scales in policy applications., Comment: 31 pages, 6 figures, submitted to PLOS ONE

We present a numerical approximation scheme for the Tomita-Takesaki modular operator of local subalgebras in linear quantum fields, working at one-particle level. This is applied to the local subspaces for double cones in the vacuum sector of a massive scalar free field in (1 + 1)- and (3 + 1)-dimensional Minkowski spacetime, using a discretization of time-0 data in position space. In the case of a wedge region, one component of the modular generator is well-known to be a mass-independent multiplication operator; our results strongly suggest that for the double cone, the corresponding component is still at least close to a multiplication operator, but that it is dependent on mass and angular momentum., Comment: 23 pages, 16 figures, source code as ancillary files

Geometric quantization is a natural way to construct quantum models starting from classical data. In this work, we start from a symplectic vector space with an inner product and -- using techniques of geometric quantization -- construct the quantum algebra and equip it with a distinguished state. We compare our result with the construction due to Sorkin -- which starts from the same input data -- and show that our distinguished state coincides with the Sorkin-Johnson state. Sorkin's construction was originally applied to the free scalar field over a causal set (locally finite, partially ordered set). Our perspective suggests a natural generalization to less linear examples, such as an interacting field., Comment: 34 pages, 3 figures

In this paper, we introduce Jointist, an instrument-aware multi-instrument framework that is capable of transcribing, recognizing, and separating multiple musical instruments from an audio clip. Jointist consists of the instrument recognition module that conditions the other modules: the transcription module that outputs instrument-specific piano rolls, and the source separation module that utilizes instrument information and transcription results. The instrument conditioning is designed for an explicit multi-instrument functionality while the connection between the transcription and source separation modules is for better transcription performance. Our challenging problem formulation makes the model highly useful in the real world given that modern popular music typically consists of multiple instruments. However, its novelty necessitates a new perspective on how to evaluate such a model. During the experiment, we assess the model from various aspects, providing a new evaluation perspective for multi-instrument transcription. We also argue that transcription models can be utilized as a preprocessing module for other music analysis tasks. In the experiment on several downstream tasks, the symbolic representation provided by our transcription model turned out to be helpful to spectrograms in solving downbeat detection, chord recognition, and key estimation., Comment: Submitted to ISMIR

Transformer is a successful deep neural network (DNN) architecture that has shown its versatility not only in natural language processing but also in music information retrieval (MIR). In this paper, we present a novel Transformer-based approach to tackle beat and downbeat tracking. This approach employs SpecTNT (Spectral-Temporal Transformer in Transformer), a variant of Transformer that models both spectral and temporal dimensions of a time-frequency input of music audio. A SpecTNT model uses a stack of blocks, where each consists of two levels of Transformer encoders. The lower-level (or spectral) encoder handles the spectral features and enables the model to pay attention to harmonic components of each frame. Since downbeats indicate bar boundaries and are often accompanied by harmonic changes, this step may help downbeat modeling. The upper-level (or temporal) encoder aggregates useful local spectral information to pay attention to beat/downbeat positions. We also propose an architecture that combines SpecTNT with a state-of-the-art model, Temporal Convolutional Networks (TCN), to further improve the performance. Extensive experiments demonstrate that our approach can significantly outperform TCN in downbeat tracking while maintaining comparable result in beat tracking., Comment: This paper is accepted for publication at ICASSP 2022

Content creators often use music to enhance their stories, as it can be a powerful tool to convey emotion. In this paper, our goal is to help creators find music to match the emotion of their story. We focus on text-based stories that can be auralized (e.g., books), use multiple sentences as input queries, and automatically retrieve matching music. We formalize this task as a cross-modal text-to-music retrieval problem. Both the music and text domains have existing datasets with emotion labels, but mismatched emotion vocabularies prevent us from using mood or emotion annotations directly for matching. To address this challenge, we propose and investigate several emotion embedding spaces, both manually defined (e.g., valence/arousal) and data-driven (e.g., Word2Vec and metric learning) to bridge this gap. Our experiments show that by leveraging these embedding spaces, we are able to successfully bridge the gap between modalities to facilitate cross modal retrieval. We show that our method can leverage the well established valence-arousal space, but that it can also achieve our goal via data-driven embedding spaces. By leveraging data-driven embeddings, our approach has the potential of being generalized to other retrieval tasks that require broader or completely different vocabularies., Comment: International Society for Music Information Retrieval (ISMIR) 2021, Best Student Paper

We present Music Tagging Transformer that is trained with a semi-supervised approach. The proposed model captures local acoustic characteristics in shallow convolutional layers, then temporally summarizes the sequence of the extracted features using stacked self-attention layers. Through a careful model assessment, we first show that the proposed architecture outperforms the previous state-of-the-art music tagging models that are based on convolutional neural networks under a supervised scheme. The Music Tagging Transformer is further improved by noisy student training, a semi-supervised approach that leverages both labeled and unlabeled data combined with data augmentation. To our best knowledge, this is the first attempt to utilize the entire audio of the million song dataset., Comment: International Society for Music Information Retrieval (ISMIR) 2021

Music classification is a music information retrieval (MIR) task to classify music items to labels such as genre, mood, and instruments. It is also closely related to other concepts such as music similarity and musical preference. In this tutorial, we put our focus on two directions - the recent training schemes beyond supervised learning and the successful application of music classification models. The target audience for this web book is researchers and practitioners who are interested in state-of-the-art music classification research and building real-world applications. We assume the audience is familiar with the basic machine learning concepts. In this book, we present three lectures as follows: 1. Music classification overview: Task definition, applications, existing approaches, datasets, 2. Beyond supervised learning: Semi- and self-supervised learning for music classification, 3. Towards real-world applications: Less-discussed, yet important research issues in practice., Comment: This is a web book written for a tutorial session of the 22nd International Society for Music Information Retrieval Conference, Nov 8-12, 2021. Please visit https://music-classification.github.io/tutorial/ for the original, web book format

Transformers have drawn attention in the MIR field for their remarkable performance shown in natural language processing and computer vision. However, prior works in the audio processing domain mostly use Transformer as a temporal feature aggregator that acts similar to RNNs. In this paper, we propose SpecTNT, a Transformer-based architecture to model both spectral and temporal sequences of an input time-frequency representation. Specifically, we introduce a novel variant of the Transformer-in-Transformer (TNT) architecture. In each SpecTNT block, a spectral Transformer extracts frequency-related features into the frequency class token (FCT) for each frame. Later, the FCTs are linearly projected and added to the temporal embeddings (TEs), which aggregate useful information from the FCTs. Then, a temporal Transformer processes the TEs to exchange information across the time axis. By stacking the SpecTNT blocks, we build the SpecTNT model to learn the representation for music signals. In experiments, SpecTNT demonstrates state-of-the-art performance in music tagging and vocal melody extraction, and shows competitive performance for chord recognition. The effectiveness of SpecTNT and other design choices are further examined through ablation studies., Comment: 6 pages

In this work, we present the design and fabrication of a packaged whispering gallery mode (WGM) device based on an optical nanoantenna as the coupler and a glass microsphere as the resonator. The microspheres were fabricated from SiO$_2$ fiber or Er$^{3+}$-doped fiber, the latter creating a WGM laser with a threshold of 93 $\mu$W at 1531 nm. The coupler-resonator WGM device is packaged in a glass capillary. The performance of the packaged microlaser is characterized, with lasing emission both excited in and collected from the WGM cavity via the nanoantenna. The packaged system provides isolation from environmental contamination, a small size, and unidirectional coupling while maintaining a high quality (Q-) factor ($\sim$10$^8$). It opens up new possibilities for practical applications of WGM microdevices in a variety of fields such as low threshold lasers, filters, and sensors.

We describe numerical and analytical investigations of causal sets sprinkled into spacetime manifolds. The first part of the paper is a numerical study of finite causal sets sprinkled into Alexandrov subsets of Minkowski spacetime of dimensions $1 + 1$, $1 + 2$ and $1 + 3$. In particular we consider the rank 2 past of sprinkled causet events, which is the set of events that are two links to the past. Assigning one of the rank 2 past events as `preferred past' for each event yields a `preferred past structure', which was recently proposed as the basis for a causal set d'Alembertian. We test six criteria for selecting rank 2 past subsets. One criterion performs particularly well at uniquely selecting -- with very high probability -- a preferred past satisfying desirable properties. The second part of the paper concerns (infinite) sprinkled causal sets for general spacetime manifolds. After reviewing the construction of the sprinkling process with the Poisson measure, we consider various specific applications. Among other things, we compute the probability of obtaining a sprinkled causal set of a given isomorphism class by combinatorial means, using a correspondence between causal sets in Alexandrov subsets of $1 + 1$ dimensional Minkowski spacetime and 2D-orders. These methods are also used to compute the expected size of the past infinity as a proportion of the total size of a sprinkled causal set.