Search

Recent research has demonstrated that training a linear connector between speech foundation encoders and large language models (LLMs) enables this architecture to achieve strong ASR capabilities. Despite the impressive results, it remains unclear whether these simple approaches are robust enough across different scenarios and speech conditions, such as domain shifts and different speech perturbations. In this paper, we address these questions by conducting various ablation experiments using a recent and widely adopted approach called SLAM-ASR. We present novel empirical findings that offer insights on how to effectively utilize the SLAM-ASR architecture across a wide range of settings. Our main findings indicate that the SLAM-ASR exhibits poor performance in cross-domain evaluation settings. Additionally, speech perturbations within in-domain data, such as changes in speed or the presence of additive noise, can significantly impact performance. Our findings offer critical insights for fine-tuning and configuring robust LLM-based ASR models, tailored to different data characteristics and computational resources., Comment: Submitted to ICASSP 2025 SALMA Workshop

Despite the recent success of end-to-end models for automatic speech recognition, recognizing special rare and out-of-vocabulary words, as well as fast domain adaptation with text, are still challenging. It often happens that biasing to the special entities leads to a degradation in the overall performance. We propose a light on-the-fly method to improve automatic speech recognition performance by combining a bias list of named entities with a word-level n-gram language model with the shallow fusion approach based on the Aho-Corasick string matching algorithm. The Aho-Corasick algorithm has proved to be more efficient than other methods and allows fast context adaptation. An n-gram language model is introduced as a graph with fail and output arcs, where the arc weights are adapted from the n-gram probabilities. The language model is used as an additional support to keyword biasing when the language model is combined with bias entities in a single context graph to take care of the overall performance. We demonstrate our findings on 4 languages, 2 public and 1 private datasets including performance on named entities and out-of-vocabulary entities. We achieve up to 21.6% relative improvement in the general word error rate with no practical difference in the inverse real-time factor., Comment: Submitted to ICASSP2025

The training of automatic speech recognition (ASR) with little to no supervised data remains an open question. In this work, we demonstrate that streaming Transformer-Transducer (TT) models can be trained from scratch in consumer and accessible GPUs in their entirety with pseudo-labeled (PL) speech from foundational speech models (FSM). This allows training a robust ASR model just in one stage and does not require large data and computational budget compared to the two-step scenario with pre-training and fine-tuning. We perform a comprehensive ablation on different aspects of PL-based streaming TT models such as the impact of (1) shallow fusion of n-gram LMs, (2) contextual biasing with named entities, (3) chunk-wise decoding for low-latency streaming applications, and (4) TT overall performance as the function of the FSM size. Our results demonstrate that TT can be trained from scratch without supervised data, even with very noisy PLs. We validate the proposed framework on 6 languages from CommonVoice and propose multiple heuristics to filter out hallucinated PLs., Comment: Accepted to EMNLP Findings 2024

Network data are observed in various applications where the individual entities of the system interact with or are connected to each other, and often these interactions are defined by their associated strength or importance. Clustering is a common task in network analysis that involves finding groups of nodes displaying similarities in the way they interact with the rest of the network. However, most clustering methods use the strengths of connections between entities in their original form, ignoring the possible differences in the capacities of individual nodes to send or receive edges. This often leads to clustering solutions that are heavily influenced by the nodes' capacities. One way to overcome this is to analyse the strengths of connections in relative rather than absolute terms, expressing each edge weight as a proportion of the sending (or receiving) capacity of the respective node. This, however, induces additional modelling constraints that most existing clustering methods are not designed to handle. In this work we propose a stochastic block model for composition-weighted networks based on direct modelling of compositional weight vectors using a Dirichlet mixture, with the parameters determined by the cluster labels of the sender and the receiver nodes. Inference is implemented via an extension of the classification expectation-maximisation algorithm that uses a working independence assumption, expressing the complete data likelihood of each node of the network as a function of fixed cluster labels of the remaining nodes. A model selection criterion is derived to aid the choice of the number of clusters. The model is validated using simulation studies, and showcased on network data from the Erasmus exchange program and a bike sharing network for the city of London.

In traditional conversational intelligence from speech, a cascaded pipeline is used, involving tasks such as voice activity detection, diarization, transcription, and subsequent processing with different NLP models for tasks like semantic endpointing and named entity recognition (NER). Our paper introduces TokenVerse, a single Transducer-based model designed to handle multiple tasks. This is achieved by integrating task-specific tokens into the reference text during ASR model training, streamlining the inference and eliminating the need for separate NLP models. In addition to ASR, we conduct experiments on 3 different tasks: speaker change detection, endpointing, and NER. Our experiments on a public and a private dataset show that the proposed method improves ASR by up to 7.7% in relative WER while outperforming the cascaded pipeline approach in individual task performance. Our code is publicly available: https://github.com/idiap/tokenverse-unifying-speech-nlp, Comment: Accepted at EMNLP 2024 (Main Conference)

Self-supervised pretrained models exhibit competitive performance in automatic speech recognition on finetuning, even with limited in-domain supervised data. However, popular pretrained models are not suitable for streaming ASR because they are trained with full attention context. In this paper, we introduce XLSR-Transducer, where the XLSR-53 model is used as encoder in transducer setup. Our experiments on the AMI dataset reveal that the XLSR-Transducer achieves 4% absolute WER improvement over Whisper large-v2 and 8% over a Zipformer transducer model trained from scratch. To enable streaming capabilities, we investigate different attention masking patterns in the self-attention computation of transformer layers within the XLSR-53 model. We validate XLSR-Transducer on AMI and 5 languages from CommonVoice under low-resource scenarios. Finally, with the introduction of attention sinks, we reduce the left context by half while achieving a relative 12% improvement in WER., Comment: 5 pages, double column

In this paper, we delve into the pedestrian behavior understanding problem from the perspective of three different tasks: intention estimation, action prediction, and event risk assessment. We first define the tasks and discuss how these tasks are represented and annotated in two widely used pedestrian datasets, JAAD and PIE. We then propose a new benchmark based on these definitions, available annotations, and three new classes of metrics, each designed to assess different aspects of the model performance. We apply the new evaluation approach to examine four SOTA prediction models on each task and compare their performance w.r.t. metrics and input modalities. In particular, we analyze the differences between intention estimation and action prediction tasks by considering various scenarios and contextual factors. Lastly, we examine model agreement across these two tasks to show their complementary role. The proposed benchmark reveals new facts about the role of different data modalities, the tasks, and relevant data properties. We conclude by elaborating on our findings and proposing future research directions., Comment: 8 pages, 5 figures, 6 tables

As part of an ongoing worldwide effort to comprehend and monitor insect biodiversity, this paper presents the BIOSCAN-5M Insect dataset to the machine learning community and establish several benchmark tasks. BIOSCAN-5M is a comprehensive dataset containing multi-modal information for over 5 million insect specimens, and it significantly expands existing image-based biological datasets by including taxonomic labels, raw nucleotide barcode sequences, assigned barcode index numbers, and geographical information. We propose three benchmark experiments to demonstrate the impact of the multi-modal data types on the classification and clustering accuracy. First, we pretrain a masked language model on the DNA barcode sequences of the BIOSCAN-5M dataset, and demonstrate the impact of using this large reference library on species- and genus-level classification performance. Second, we propose a zero-shot transfer learning task applied to images and DNA barcodes to cluster feature embeddings obtained from self-supervised learning, to investigate whether meaningful clusters can be derived from these representation embeddings. Third, we benchmark multi-modality by performing contrastive learning on DNA barcodes, image data, and taxonomic information. This yields a general shared embedding space enabling taxonomic classification using multiple types of information and modalities. The code repository of the BIOSCAN-5M Insect dataset is available at https://github.com/zahrag/BIOSCAN-5M.

Ferroelectric materials play a crucial role in a broad range of technologies due to their unique properties that are deeply connected to the pattern and behavior of their ferroelectric (FE) domains. Chief among them, barium titanate (BaTiO$_3$; BTO) sees widespread applications such as in electronics but equally is a ferroelectric model system for fundamental research, e.g., to study the interplay of such FE domains, the domain walls (DWs), and their macroscopic properties, owed to BTO's multiple and experimentally accessible phase transitions. Here, we employ Second Harmonic Generation Microscopy (SHGM) to $\it{in-situ}$ investigate the cubic-to-tetragonal (at $\sim$126$^\circ$C) and the tetragonal-to-orthorhombic (at $\sim$5$^\circ$C) phase transition in single-crystalline BTO via 3-dimensional (3D) DW mapping. We demonstrate that SHGM imaging provides the direct visualization of FE domain switching as well as the domain dynamics in 3D, shedding light on the interplay of the domain structure and the phase transition. These results allow us to extract the different transition temperatures locally, to unveil the hysteresis behavior, and to determine the type of phase transition at play (1st/2nd order) from the recorded SHGM data. The capabilities of SHGM in uncovering these crucial phenomena can easily be applied to other ferroelectrics to provide new possibilities for $\it{in-situ}$ engineering of advanced ferroic devices., Comment: 8 pages, 3 figures, with appended supplementary material (2 pages, 2 figures)

Closed-loop neuroscience experimentation, where recorded neural activity is used to modify the experiment on-the-fly, is critical for deducing causal connections and optimizing experimental time. A critical step in creating a closed-loop experiment is real-time inference of neural activity from streaming recordings. One challenging modality for real-time processing is multi-photon calcium imaging (CI). CI enables the recording of activity in large populations of neurons however, often requires batch processing of the video data to extract single-neuron activity from the fluorescence videos. We use the recently proposed robust time-trace estimator-Sparse Emulation of Unused Dictionary Objects (SEUDO) algorithm-as a basis for a new on-line processing algorithm that simultaneously identifies neurons in the fluorescence video and infers their time traces in a way that is robust to as-yet unidentified neurons. To achieve real-time SEUDO (realSEUDO), we optimize the core estimator via both algorithmic improvements and an fast C-based implementation, and create a new cell finding loop to enable realSEUDO to also identify new cells. We demonstrate comparable performance to offline algorithms (e.g., CNMF), and improved performance over the current on-line approach (OnACID) at speeds of 120 Hz on average., Comment: 20 pages, 8 figures

Ferroelectric domain walls (DWs) are promising structures for assembling future nano-electronic circuit elements on a larger scale, since reporting domain wall currents of up to 1 mA per single DW. One key requirement hereto is their reproducible manufacturing by gaining preparative control over domain size and domain wall conductivity (DWC). To date, most works on DWC have focused on exploring the fundamental electrical properties of individual DWs within single shot experiments, with emphasis on quantifying the origins for DWC. Very few reports exist when it comes to compare the DWC properties between two separate DWs, and literally nothing exists where issues of reproducibility in DWC devices have been addressed. To fill this gap while facing the challenge of finding guidelines achieving predictable DWC performance, we report on a procedure that allows us to reproducibly prepare single hexagonal domains of a predefined diameter into uniaxial ferroelectric (FE) lithium niobate (LN) single crystals of 200 and 300 micrometers thickness, respectively. We show that the domain diameter can be controlled with an error of a few percent. As-grown DWs are then subjected to a standard procedure of current-controlled high-voltage DWC enhancement, repetitively reaching a DWC increase of 6 orders of magnitude. While all resulting DWs show significantly enhanced DWC values, subtle features in their individual current-voltage (I-V) characteristics hint towards different 3D shapes into the bulk, with variations probably reflecting local heterogeneities by defects, DW pinning, and surface-near DW inclination, which seem to have a larger impact than expected., Comment: 12 pages, 7 figures, with appended supplement (9 pages, 7 figures)

Accurate prediction of drivers' gaze is an important component of vision-based driver monitoring and assistive systems. Of particular interest are safety-critical episodes, such as performing maneuvers or crossing intersections. In such scenarios, drivers' gaze distribution changes significantly and becomes difficult to predict, especially if the task and context information is represented implicitly, as is common in many state-of-the-art models. However, explicit modeling of top-down factors affecting drivers' attention often requires additional information and annotations that may not be readily available. In this paper, we address the challenge of effective modeling of task and context with common sources of data for use in practical systems. To this end, we introduce SCOUT+, a task- and context-aware model for drivers' gaze prediction, which leverages route and map information inferred from commonly available GPS data. We evaluate our model on two datasets, DR(eye)VE and BDD-A, and demonstrate that using maps improves results compared to bottom-up models and reaches performance comparable to the top-down model SCOUT which relies on privileged ground truth information. Code is available at https://github.com/ykotseruba/SCOUT., Comment: Accepted at IEEE Intelligent Vehicles Symposium (IV), 2024

Driving is a visuomotor task, i.e., there is a connection between what drivers see and what they do. While some models of drivers' gaze account for top-down effects of drivers' actions, the majority learn only bottom-up correlations between human gaze and driving footage. The crux of the problem is lack of public data with annotations that could be used to train top-down models and evaluate how well models of any kind capture effects of task on attention. As a result, top-down models are trained and evaluated on private data and public benchmarks measure only the overall fit to human data. In this paper, we focus on data limitations by examining four large-scale public datasets, DR(eye)VE, BDD-A, MAAD, and LBW, used to train and evaluate algorithms for drivers' gaze prediction. We define a set of driving tasks (lateral and longitudinal maneuvers) and context elements (intersections and right-of-way) known to affect drivers' attention, augment the datasets with annotations based on the said definitions, and analyze the characteristics of data recording and processing pipelines w.r.t. capturing what the drivers see and do. In sum, the contributions of this work are: 1) quantifying biases of the public datasets, 2) examining performance of the SOTA bottom-up models on subsets of the data involving non-trivial drivers' actions, 3) linking shortcomings of the bottom-up models to data limitations, and 4) recommendations for future data collection and processing. The new annotations and code for reproducing the results is available at https://github.com/ykotseruba/SCOUT., Comment: Accepted at IEEE Intelligent Vehicles Symposium (IV), 2024

Virtual Reality (VR) has repeatedly proven its effectiveness in student learning. However, despite its benefits, the student equipped with a personal headset remains isolated from the real world while immersed in a virtual space and the classic student-teacher model of learning is difficult to transpose in such a situation. This study aims to bring the teacher back into the learning process when students use a VR headset. We describe the benefits of using a companion for educational purposes, taking as a test case the concept of gravity. We present an experimental setup designed to compare three different teaching contexts: with a physically present real teacher, using a live video of the teacher, and with a VR avatar of the teacher. We designed and evaluated three scenarios to teach the concept of gravity: an introduction to the concept of free fall, a parabolic trajectory workshop and a final exercise combining both approaches. Due to sanitary conditions, only pre-tests are reported. The results showed that the effectiveness of using the VR simulations for learning and the self-confidence level of the students increased as well. The interviews show that the students ranked the teaching modes in this order: VR companion mode, video communication and real teacher.

Training students in basic concepts of physics, such as the ones related to mass, volume, or density, is much more complicated than just stating the underlying definitions and laws. One of the reasons for this is that most students have deeply rooted delusions and misconceptions about the behavior of objects, sometimes close to magical thinking. Many innovative and promising technologies, in particular Virtual Reality (VR), can be used to enhance student learning. We compared the effectiveness of a serious immersive game in teaching the concept of density in various conditions: a 2D version in an embedded web browser and a 3D immersive game in VR. We also developed a specific questionnaire to assess students' knowledge improvement. Primary results have shown an increase in learning efficiency using VR. Also, most students were able to see the shortcomings of their initial theories and revise them, which means that they improved their understanding of this topic.

Most tidal disruption events (TDEs) are currently found in time-domain optical and soft X-ray surveys, both of which are prone to significant obscuration. The infrared (IR), however, is a powerful probe of dust-enshrouded environments, and hence, we recently performed a systematic search of NEOWISE mid-IR data for nearby, obscured TDEs within roughly 200 Mpc. We identified 18 TDE candidates in galactic nuclei, using difference imaging to uncover nuclear variability amongst significant host galaxy emission. These candidates were selected based on the following IR light curve properties: (1) $L_\mathrm{W2}\gtrsim10^{42}$ erg s$^{-1}$ at peak, (2) fast rise, followed by a slow, monotonic decline, (3) no significant prior variability, and (4) no evidence for AGN activity in WISE colors. The majority of these sources showed no variable optical counterpart, suggesting that optical surveys indeed miss numerous obscured TDEs. Using narrow line ionization levels and variability arguments, we identified 6 sources as possible underlying AGN, yielding a total of 12 TDEs in our gold sample. This gold sample yields a lower limit on the IR-selected TDE rate of $2.0\pm0.3\times10^{-5}$ galaxy$^{-1}$ year$^{-1}$ ($1.3\pm0.2\times10^{-7}$ Mpc$^{-3}$ year$^{-1}$), which is comparable to optical and X-ray TDE rates. The IR-selected TDE host galaxies do not show a green valley overdensity nor a preference for quiescent, Balmer strong galaxies, which are both overrepresented in optical and X-ray TDE samples. This IR-selected sample represents a new population of dusty TDEs that have historically been missed by optical and X-ray surveys and helps alleviate tensions between observed and theoretical TDE rates and the so-called missing energy problem., Comment: 28 pages, 13 figures (+ 7 pages, 4 figures in appendix), accepted for publication in ApJ

We consider the approximate properties of tight wavelet frames on Vilenkin group $G$. Let $\{G_n\}_{n\in \mathbb{Z} }$ be a main chain of subgroups, $X$ be a set of characters. We define a step function $\lambda({\chi})$ that is constant on cosets ${G}_n^\bot\setminus{G}_{n-1}^\bot$ by equalities $\lambda ({G}_n^\bot\setminus{G}_{n-1}^\bot)=\lambda_n>0$ for which $\sum\frac{1}{\lambda_n}<\infty$. We find the order of approximation of functions $f$ for which $\int_X|\lambda( {\chi})\hat{f}(\chi)|^2d\nu(\chi)<\infty$. As a corollary, we obtain an approximation error for functions from Sobolev spaces with logarithmic weight., Comment: 18 pages, 3 figures. arXiv admin note: text overlap with arXiv:2203.06352, arXiv:2307.06588

Understanding biodiversity is a global challenge, in which DNA barcodes - short snippets of DNA that cluster by species - play a pivotal role. In particular, invertebrates, a highly diverse and under-explored group, pose unique taxonomic complexities. We explore machine learning approaches, comparing supervised CNNs, fine-tuned foundation models, and a DNA barcode-specific masking strategy across datasets of varying complexity. While simpler datasets and tasks favor supervised CNNs or fine-tuned transformers, challenging species-level identification demands a paradigm shift towards self-supervised pretraining. We propose BarcodeBERT, the first self-supervised method for general biodiversity analysis, leveraging a 1.5 M invertebrate DNA barcode reference library. This work highlights how dataset specifics and coverage impact model selection, and underscores the role of self-supervised pretraining in achieving high-accuracy DNA barcode-based identification at the species and genus level. Indeed, without the fine-tuning step, BarcodeBERT pretrained on a large DNA barcode dataset outperforms DNABERT and DNABERT-2 on multiple downstream classification tasks. The code repository is available at https://github.com/Kari-Genomics-Lab/BarcodeBERT, Comment: Main text: 5 pages, Total: 9 pages, 2 figures, accepted at the 4th Workshop on Self-Supervised Learning: Theory and Practice (NeurIPS 2023)

To further advance driver monitoring and assistance systems, it is important to understand how drivers allocate their attention, in other words, where do they tend to look and why. Traditionally, factors affecting human visual attention have been divided into bottom-up (involuntary attraction to salient regions) and top-down (driven by the demands of the task being performed). Although both play a role in directing drivers' gaze, most of the existing models for drivers' gaze prediction apply techniques developed for bottom-up saliency and do not consider influences of the drivers' actions explicitly. Likewise, common driving attention benchmarks lack relevant annotations for drivers' actions and the context in which they are performed. Therefore, to enable analysis and modeling of these factors for drivers' gaze prediction, we propose the following: 1) we correct the data processing pipeline used in DR(eye)VE to reduce noise in the recorded gaze data; 2) we then add per-frame labels for driving task and context; 3) we benchmark a number of baseline and SOTA models for saliency and driver gaze prediction and use new annotations to analyze how their performance changes in scenarios involving different tasks; and, lastly, 4) we develop a novel model that modulates drivers' gaze prediction with explicit action and context information. While reducing noise in the DR(eye)VE gaze data improves results of all models, we show that using task information in our proposed model boosts performance even further compared to bottom-up models on the cleaned up data, both overall (by 24% KLD and 89% NSS) and on scenarios that involve performing safety-critical maneuvers and crossing intersections (by up to 10--30% KLD). Extended annotations and code are available at https://github.com/ykotseruba/SCOUT., Comment: Accepted at IEEE Intelligent Vehicles Symposium (IV), 2024

Amplifying radially and azimuthally polarized beams is a significant challenge due to the instability of the complex beam shape and polarization in inhomogeneous environment. In this Letter, we demonstrated experimentally an efficient approach to directly amplify cylindrical-vector beams with axially symmetric polarization and doughnut-shaped intensity profile in a picosecond MOPA system based on a double-clad ytterbium-doped tapered fiber. To prevent polarization and beam shape distortion during amplification, for the first time to the best of our knowledge, we proposed using the spun architecture of the tapered fiber. In contrast to an isotropic fiber architecture, a spun configuration possessing nearly-circular polarization eigenstates supports stable wavefront propagation. Applying this technique, we amplified the cylindrical-vector beam up to 22 W of average power with 10 ps pulses at a central wavelength of 1030 nm and a repetition rate of 15 MHz, maintaining both mode and polarization stability., Comment: 5 pages, 7 figures

In an effort to catalog insect biodiversity, we propose a new large dataset of hand-labelled insect images, the BIOSCAN-Insect Dataset. Each record is taxonomically classified by an expert, and also has associated genetic information including raw nucleotide barcode sequences and assigned barcode index numbers, which are genetically-based proxies for species classification. This paper presents a curated million-image dataset, primarily to train computer-vision models capable of providing image-based taxonomic assessment, however, the dataset also presents compelling characteristics, the study of which would be of interest to the broader machine learning community. Driven by the biological nature inherent to the dataset, a characteristic long-tailed class-imbalance distribution is exhibited. Furthermore, taxonomic labelling is a hierarchical classification scheme, presenting a highly fine-grained classification problem at lower levels. Beyond spurring interest in biodiversity research within the machine learning community, progress on creating an image-based taxonomic classifier will also further the ultimate goal of all BIOSCAN research: to lay the foundation for a comprehensive survey of global biodiversity. This paper introduces the dataset and explores the classification task through the implementation and analysis of a baseline classifier.

Ferroelectric domain wall (DW) conductivity (DWC) can be attributed to two separate mechanisms: (a) the injection/ejection of charge carriers across the Schottky barrier formed at the (metal-) electrode-DW junction and (b) the transport of those charge carriers along the DW. Current-voltage (IU) characteristics, recorded at variable temperatures from LiNbO$_3$ (LNO) DWs, are clearly able to differentiate between these two contributions. Practically, they allow us here to directly quantify the physical parameters relevant for the two mechanisms (a) and (b) mentioned above. These are, e.g., the resistance of the DW, the saturation current, the ideality factor, and the Schottky barrier height of the electrode/DW junction. Furthermore, the activation energies needed to initiate the thermally-activated electronic transport along the DWs, can be extracted. In addition, we show that electronic transport along LiNbO$_3$ DWs can be elegantly viewed and interpreted in an adapted semiconductor picture based on a double-diode/double-resistor equivalent circuit model, the R2D2 model. Finally, our R2D2 model was checked for its universality by fitting the DWC data not only to z-cut LNO bulk DWs, but equally to z-cut thin-film LNO DWs, and DWC from x-cut DWs as reported in literature., Comment: 10 pages, 7 figures; with included supporting information (6 pages, 4 figures)

GPU decoding significantly accelerates the output of ASR predictions. While GPUs are already being used for online ASR decoding, post-processing and rescoring on GPUs have not been properly investigated yet. Rescoring with available contextual information can considerably improve ASR predictions. Previous studies have proven the viability of lattice rescoring in decoding and biasing language model (LM) weights in offline and online CPU scenarios. In real-time GPU decoding, partial recognition hypotheses are produced without lattice generation, which makes the implementation of biasing more complex. The paper proposes and describes an approach to integrate contextual biasing in real-time GPU decoding while exploiting the standard Kaldi GPU decoder. Besides the biasing of partial ASR predictions, our approach also permits dynamic context switching allowing a flexible rescoring per each speech segment directly on GPU. The code is publicly released and tested with open-sourced test sets., Comment: Accepted to Interspeech 2023

How to better reduce measurement variability and bias introduced by subjectivity in crowdsourced labelling remains an open question. We introduce a theoretical framework for understanding how random error and measurement bias enter into crowdsourced annotations of subjective constructs. We then propose a pipeline that combines pairwise comparison labelling with Elo scoring, and demonstrate that it outperforms the ubiquitous majority-voting method in reducing both types of measurement error. To assess the performance of the labelling approaches, we constructed an agent-based model of crowdsourced labelling that lets us introduce different types of subjectivity into the tasks. We find that under most conditions with task subjectivity, the comparison approach produced higher $f_1$ scores. Further, the comparison approach is less susceptible to inflating bias, which majority voting tends to do. To facilitate applications, we show with simulated and real-world data that the number of required random comparisons for the same classification accuracy scales log-linearly $O(N \log N)$ with the number of labelled items. We also implemented the Elo system as an open-source Python package., Comment: Accepted for publication at ACM CSCW 2023

Positron annihilation in solid state matter can be utilized to detect and identify open-volume defects. The momentum distribution of the annihilation radiation is an important observable in positron-based measurements, and can reveal information on the chemical surroundings of the defect sites. In this work we present a variational quantum Monte Carlo method for simulation of the momentum densities of annihilating electron-positron pairs in semiconductors and insulators. We study finite-size effects, effects of lattice vibrations, and different levels of trial wave functions. Small simulation cells and simple wave function forms are found to be sufficient for accurate calculations in simulation of pristine lattices, enabling cheap accumulation of results. We compare calculated predictions of the Doppler broadening of the 511-keV 2{\gamma} annihilation line of the in aluminium nitride and silicon against experimental data measured from reference samples. Our results achieve better agreement with experiments in the these materials than conventional state-of-the-art methods, and proves that direct modeling of the electron-positron correlations is important for a supporting theory of positron annihilation sprectroscopies, Comment: 10 pages, 4 figures

Voice communication between air traffic controllers (ATCos) and pilots is critical for ensuring safe and efficient air traffic control (ATC). This task requires high levels of awareness from ATCos and can be tedious and error-prone. Recent attempts have been made to integrate artificial intelligence (AI) into ATC in order to reduce the workload of ATCos. However, the development of data-driven AI systems for ATC demands large-scale annotated datasets, which are currently lacking in the field. This paper explores the lessons learned from the ATCO2 project, a project that aimed to develop a unique platform to collect and preprocess large amounts of ATC data from airspace in real time. Audio and surveillance data were collected from publicly accessible radio frequency channels with VHF receivers owned by a community of volunteers and later uploaded to Opensky Network servers, which can be considered an "unlimited source" of data. In addition, this paper reviews previous work from ATCO2 partners, including (i) robust automatic speech recognition, (ii) natural language processing, (iii) English language identification of ATC communications, and (iv) the integration of surveillance data such as ADS-B. We believe that the pipeline developed during the ATCO2 project, along with the open-sourcing of its data, will encourage research in the ATC field. A sample of the ATCO2 corpus is available on the following website: https://www.atco2.org/data, while the full corpus can be purchased through ELDA at http://catalog.elra.info/en-us/repository/browse/ELRA-S0484. We demonstrated that ATCO2 is an appropriate dataset to develop ASR engines when little or near to no ATC in-domain data is available. For instance, with the CNN-TDNNf kaldi model, we reached the performance of as low as 17.9% and 24.9% WER on public ATC datasets which is 6.6/7.6% better than "out-of-domain" but supervised CNN-TDNNf model., Comment: Manuscript under review

In this paper we propose a novel virtual simulation-pilot engine for speeding up air traffic controller (ATCo) training by integrating different state-of-the-art artificial intelligence (AI) based tools. The virtual simulation-pilot engine receives spoken communications from ATCo trainees, and it performs automatic speech recognition and understanding. Thus, it goes beyond only transcribing the communication and can also understand its meaning. The output is subsequently sent to a response generator system, which resembles the spoken read back that pilots give to the ATCo trainees. The overall pipeline is composed of the following submodules: (i) automatic speech recognition (ASR) system that transforms audio into a sequence of words; (ii) high-level air traffic control (ATC) related entity parser that understands the transcribed voice communication; and (iii) a text-to-speech submodule that generates a spoken utterance that resembles a pilot based on the situation of the dialogue. Our system employs state-of-the-art AI-based tools such as Wav2Vec 2.0, Conformer, BERT and Tacotron models. To the best of our knowledge, this is the first work fully based on open-source ATC resources and AI tools. In addition, we have developed a robust and modular system with optional submodules that can enhance the system's performance by incorporating real-time surveillance data, metadata related to exercises (such as sectors or runways), or even introducing a deliberate read-back error to train ATCo trainees to identify them. Our ASR system can reach as low as 5.5% and 15.9% word error rates (WER) on high and low-quality ATC audio. We also demonstrate that adding surveillance data into the ASR can yield callsign detection accuracy of more than 96%., Comment: Under review