Search

In this paper, we study the stability of a simple model of a Hyperloop vehicle resulting from the interaction between electromagnetic and aeroelastic forces for both constant and periodically varying coefficients (i.e., parametric excitation). For the constant coefficients, through linear stability analysis, we analytically identify three distinct regions for the physically significant equilibrium point. Further inspection reveals that the system exhibits limit-cycle vibrations in one of these regions. Using the harmonic balance method, we determine the properties of the limit cycle, thereby unravelling the frequency and amplitude that characterize the periodic oscillations of the system's variables. For the varying coefficients case, the stability is studied using Floquet analysis and Hills determinant method. The part of the stability boundary related to parametric resonance has an elliptical shape, while the remaining part remains unchanged. One of the major findings is that a linear parametric force, can suppress or amplify the parametric resonance induced by another parametric force depending on the amplitude of the former. In the context of the Hyperloop system, this means that parametric resonance caused by base excitation-in other words by the linearized parametric electromagnetic force can be suppressed by modulating the coefficient of the aeroelastic force in the same frequency. The effectiveness is highly dependent on the phase difference between the modulation and the base excitation. The origin of the suppression is attributed to the stabilizing character of the parametric aeroelastic force as revealed through energy analysis. We provide analytical expressions for the stability boundaries and for the stability's dependence on the phase shift of the modulation.

Detections of fast X-ray transients (FXTs) have been accrued over the last few decades. However, their origin has remained mysterious. There is now rapid progress thanks to timely discoveries and localisations with the Einstein Probe mission. Early results indicate that FXTs may frequently, but not always, be associated with gamma-ray bursts (GRBs). Here, we report on the multi-wavelength counterpart of FXT EP240414a, which has no reported gamma-ray counterpart. The transient is located 25.7~kpc in projection from a massive galaxy at $z=0.40$. We perform comprehensive photometric and spectroscopic follow-up. The optical light curve shows at least three distinct emission episodes with timescales of $\sim 1, 4$ and 15 days and peak absolute magnitudes of $M_R \sim -20$, $-21$, and $-19.5$, respectively. The optical spectrum at early times is extremely blue, inconsistent with afterglow emission. It may arise from the interaction of both jet and supernova shock waves with the stellar envelope and a dense circumstellar medium, as has been suggested for some Fast Blue Optical Transients (LFBOTs). At late times, the spectrum evolves to a broad-lined~Type~Ic supernova, similar to those seen in collapsar long-GRBs. This implies that the progenitor of EP240414a is a massive star creating a jet-forming supernova inside a dense envelope, resulting in an X-ray outburst with a luminosity of $\sim 10^{48}$ erg s$^{-1}$, and the complex observed optical/IR light curves. If correct, this argues for a causal link between the progenitors of long-GRBs, FXTs and LFBOTs., Comment: 36 pages, 13 figures, submitted to ApJ

Automatic speech recognition (ASR) with an encoder equipped with self-attention, whether streaming or non-streaming, takes quadratic time in the length of the speech utterance. This slows down training and decoding, increase their cost, and limit the deployment of the ASR in constrained devices. SummaryMixing is a promising linear-time complexity alternative to self-attention for non-streaming speech recognition that, for the first time, preserves or outperforms the accuracy of self-attention models. Unfortunately, the original definition of SummaryMixing is not suited to streaming speech recognition. Hence, this work extends SummaryMixing to a Conformer Transducer that works in both a streaming and an offline mode. It shows that this new linear-time complexity speech encoder outperforms self-attention in both scenarios while requiring less compute and memory during training and decoding.

Self-supervised learning (SSL) models usually require weeks of pre-training with dozens of high-end GPUs. These models typically have a multi-headed self-attention (MHSA) context encoder. However, MHSA takes quadratic time and space in the input length, contributing to the high pre-training cost. Linear-complexity alternatives to MHSA have been proposed. For instance, in supervised training, the SummaryMixing model is the first to outperform MHSA across multiple speech processing tasks. However, these cheaper alternatives have not been explored for SSL yet. This paper studies a linear-complexity context encoder for SSL for the first time. With better or equivalent performance for the downstream tasks of the MP3S benchmark, SummaryMixing reduces the pre-training time and peak VRAM of wav2vec 2.0 model by 18% and by 23%, respectively, leading to the pre-training of a 155M wav2vec 2.0 model finished within one week with 4 Tesla A100 GPUs. Code is available at https://github.com/SamsungLabs/SummaryMixing., Comment: Interspeech 2024

Federated learning (FL) allows clients to collaboratively train a global model without sharing their local data with a server. However, clients' contributions to the server can still leak sensitive information. Differential privacy (DP) addresses such leakage by providing formal privacy guarantees, with mechanisms that add randomness to the clients' contributions. The randomness makes it infeasible to train large transformer-based models, common in modern federated learning systems. In this work, we empirically evaluate the practicality of fine-tuning large scale on-device transformer-based models with differential privacy in a federated learning system. We conduct comprehensive experiments on various system properties for tasks spanning a multitude of domains: speech recognition, computer vision (CV) and natural language understanding (NLU). Our results show that full fine-tuning under differentially private federated learning (DP-FL) generally leads to huge performance degradation which can be alleviated by reducing the dimensionality of contributions through parameter-efficient fine-tuning (PEFT). Our benchmarks of existing DP-PEFT methods show that DP-Low-Rank Adaptation (DP-LoRA) consistently outperforms other methods. An even more promising approach, DyLoRA, which makes the low rank variable, when naively combined with FL would straightforwardly break differential privacy. We therefore propose an adaptation method that can be combined with differential privacy and call it DP-DyLoRA. Finally, we are able to reduce the accuracy degradation and word error rate (WER) increase due to DP to less than 2% and 7% respectively with 1 million clients and a stringent privacy budget of $\epsilon=2$., Comment: 16 pages, 10 figures, 5 tables

The nature of the minute-to-hour long Fast X-ray Transients (FXTs) localised by telescopes such as Chandra, Swift, and XMM-Newton remains mysterious, with numerous models suggested for the events. Here, we report multi-wavelength observations of EP240315a, a 1600 s long transient detected by the Einstein Probe, showing it to have a redshift of z=4.859. We measure a low column density of neutral hydrogen, indicating that the event is embedded in a low-density environment, further supported by direct detection of leaking ionising Lyman-continuum. The observed properties are consistent with EP240315a being a long-duration gamma-ray burst, and these observations support an interpretation in which a significant fraction of the FXT population are lower-luminosity examples of similar events. Such transients are detectable at high redshifts by the Einstein Probe and, in the (near) future, out to even larger distances by SVOM, THESEUS, and Athena, providing samples of events into the epoch of reionisation., Comment: 41 pages, 7 figures, submitted

Federated learning (FL) is an emerging machine learning (ML) training paradigm where clients own their data and collaborate to train a global model, without revealing any data to the server and other participants. Researchers commonly perform experiments in a simulation environment to quickly iterate on ideas. However, existing open-source tools do not offer the efficiency required to simulate FL on larger and more realistic FL datasets. We introduce pfl-research, a fast, modular, and easy-to-use Python framework for simulating FL. It supports TensorFlow, PyTorch, and non-neural network models, and is tightly integrated with state-of-the-art privacy algorithms. We study the speed of open-source FL frameworks and show that pfl-research is 7-72$\times$ faster than alternative open-source frameworks on common cross-device setups. Such speedup will significantly boost the productivity of the FL research community and enable testing hypotheses on realistic FL datasets that were previously too resource intensive. We release a suite of benchmarks that evaluates an algorithm's overall performance on a diverse set of realistic scenarios. The code is available on GitHub at https://github.com/apple/pfl-research.

Railway transition zones (RTZs) experience higher rates of degradation compared to open tracks, which leads to increased maintenance costs and reduced vailability. Despite existing literature on railway track assessment and maintenance, effective design solutions for RTZs are still limited. Therefore, a robust design criterion is required to develop effective solutions. This paper presents a two-step approach for formulation of a design criterion to delay the onset of processes leading to uneven track geometry due to operation driven permanent deformations in RTZs. Firstly, a systematic analysis of each track component in a RTZ is performed by examining spatial and temporal variations in kinematic responses, stresses and energies. Secondly, the study proposes an energy-based criterion to be assessed using a model with linear elastic material behavior, and states that an amplification in the total strain energy in the proximity of the transition interface is an indicator of increased (and thus non-uniform) degradation in RTZs compared to the open tracks. The correlation between the total strain energy (assessed in the model with linear material behaviour) and the permanent irreversible deformations is demonstrated using a model with non-linear elastoplastic material behavior of the ballast layer. In the end, it is claimed that minimising the magnitude of total strain energy will lead to reduced degradation and a uniform distribution of total strain energy in each trackbed layer along the longitudinal direction of the track will ensure uniformity in the track geometry.

The Transducer (e.g. RNN-Transducer or Conformer-Transducer) generates an output label sequence as it traverses the input sequence. It is straightforward to use in streaming mode, where it generates partial hypotheses before the complete input has been seen. This makes it popular in speech recognition. However, in streaming mode the Transducer has a mathematical flaw which, simply put, restricts the model's ability to change its mind. The fix is to replace local normalisation (e.g. a softmax) with global normalisation, but then the loss function becomes impossible to evaluate exactly. A recent paper proposes to solve this by approximating the model, severely degrading performance. Instead, this paper proposes to approximate the loss function, allowing global normalisation to apply to a state-of-the-art streaming model. Global normalisation reduces its word error rate by 9-11% relative, closing almost half the gap between streaming and lookahead mode., Comment: 9 pages plus references and appendices

Modern speech processing systems rely on self-attention. Unfortunately, token mixing with self-attention takes quadratic time in the length of the speech utterance, slowing down inference and training and increasing memory consumption. Cheaper alternatives to self-attention for ASR have been developed, but they fail to consistently reach the same level of accuracy. This paper, therefore, proposes a novel linear-time alternative to self-attention. It summarises an utterance with the mean over vectors for all time steps. This single summary is then combined with time-specific information. We call this method "SummaryMixing". Introducing SummaryMixing in state-of-the-art ASR models makes it feasible to preserve or exceed previous speech recognition performance while making training and inference up to 28% faster and reducing memory use by half., Comment: Interspeech 2024

Long-distance travel has seen little attention in the past, largely due its sporadic nature. A single long-distance trip can amount to a distance equivalent to a year's worth of commute trips, resulting in a similar, if not worse, environmental footprint. Understanding travellers' behaviour is thus just as relevant for such trips. As international travel is slowly picking up from the COVID-19 pandemic, it has been marred by an abundance of national and regional pandemic-related safety measures. While their primary goal is to protect the local population from infection, these safety may also make travellers feel safer while travelling. This perceived safety can - and likely does - differ from the true efficacy of the measures. In this research, we investigate people's perception of eight COVID-19-related safety measures related to long-distance trips and how subjective perception of safety impacts their mode choice among car, train and aircraft. We employ a Hierarchical Information Integration (HII) approach to capture subjective perceptions and then model the obtained data by means of a Latent Class Choice Model, resulting in four distinct segments. To extrapolate the segments onto the rating experiment of HII, we apply a weighted least squares (WLS) regression, to obtain segment-specific safety perception. Two segments show a relatively high value-of-time (72EUR/h and 50EUR/h), tend to be more mode-agnostic and prefer determining the level of risk by themselves (relying primarily on infection and vaccination rate). The remaining two segments have a lower value-of-time (38EUR / h and 15EUR/h) and have strong mode affinity, for the train and car respectively. Future research could look into a way that segments the sample based on both the mode choice and rating experiment, providing additional insights into the heterogeneity of individuals in their perceptions.

Visual microscopic study of diseased tissue by pathologists has been the cornerstone for cancer diagnosis and prognostication for more than a century. Recently, deep learning methods have made significant advances in the analysis and classification of tissue images. However, there has been limited work on the utility of such models in generating histopathology images. These synthetic images have several applications in pathology including utilities in education, proficiency testing, privacy, and data sharing. Recently, diffusion probabilistic models were introduced to generate high quality images. Here, for the first time, we investigate the potential use of such models along with prioritized morphology weighting and color normalization to synthesize high quality histopathology images of brain cancer. Our detailed results show that diffusion probabilistic models are capable of synthesizing a wide range of histopathology images and have superior performance compared to generative adversarial networks.

Federated Learning (FL) is a technique to train models using data distributed across devices. Differential Privacy (DP) provides a formal privacy guarantee for sensitive data. Our goal is to train a large neural network language model (NNLM) on compute-constrained devices while preserving privacy using FL and DP. However, the DP-noise introduced to the model increases as the model size grows, which often prevents convergence. We propose Partial Embedding Updates (PEU), a novel technique to decrease noise by decreasing payload size. Furthermore, we adopt Low Rank Adaptation (LoRA) and Noise Contrastive Estimation (NCE) to reduce the memory demands of large models on compute-constrained devices. This combination of techniques makes it possible to train large-vocabulary language models while preserving accuracy and privacy.

Federated learning with differential privacy, i.e. private federated learning (PFL), makes it possible to train models on private data distributed across users' devices without harming privacy. PFL is efficient for models, such as neural networks, that have a fixed number of parameters, and thus a fixed-dimensional gradient vector. Such models include neural-net language models, but not tokenizers, the topic of this work. Training a tokenizer requires frequencies of words from an unlimited vocabulary, and existing methods for finding an unlimited vocabulary need a separate privacy budget. A workaround is to train the tokenizer on publicly available data. However, in this paper we first show that a tokenizer trained on mismatched data results in worse model performance compared to a privacy-violating "oracle" tokenizer that accesses user data, with perplexity increasing by 20%. We also show that sub-word tokenizers are better suited to the federated context than word-level ones, since they can encode new words, though with more tokens per word. Second, we propose a novel method to obtain a tokenizer without using any additional privacy budget. During private federated learning of the language model, we sample from the model, train a new tokenizer on the sampled sequences, and update the model embeddings. We then continue private federated learning, and obtain performance within 1% of the "oracle" tokenizer. Since this process trains the tokenizer only indirectly on private data, we can use the "postprocessing guarantee" of differential privacy and thus use no additional privacy budget.