Search

Our objective is to translate continuous sign language into spoken language text. Inspired by the way human interpreters rely on context for accurate translation, we incorporate additional contextual cues together with the signing video, into a new translation framework. Specifically, besides visual sign recognition features that encode the input video, we integrate complementary textual information from (i) captions describing the background show, (ii) translation of previous sentences, as well as (iii) pseudo-glosses transcribing the signing. These are automatically extracted and inputted along with the visual features to a pre-trained large language model (LLM), which we fine-tune to generate spoken language translations in text form. Through extensive ablation studies, we show the positive contribution of each input cue to the translation performance. We train and evaluate our approach on BOBSL -- the largest British Sign Language dataset currently available. We show that our contextual approach significantly enhances the quality of the translations compared to previously reported results on BOBSL, and also to state-of-the-art methods that we implement as baselines. Furthermore, we demonstrate the generality of our approach by applying it also to How2Sign, an American Sign Language dataset, and achieve competitive results.

Malaria remains a significant global health burden, particularly in resource-limited regions where timely and accurate diagnosis is critical to effective treatment and control. Deep Learning (DL) has emerged as a transformative tool for automating malaria detection and it offers high accuracy and scalability. However, the effectiveness of these models is constrained by challenges in data quality and model generalization including imbalanced datasets, limited diversity and annotation variability. These issues reduce diagnostic reliability and hinder real-world applicability. This article provides a comprehensive analysis of these challenges and their implications for malaria detection performance. Key findings highlight the impact of data imbalances which can lead to a 20\% drop in F1-score and regional biases which significantly hinder model generalization. Proposed solutions, such as GAN-based augmentation, improved accuracy by 15-20\% by generating synthetic data to balance classes and enhance dataset diversity. Domain adaptation techniques, including transfer learning, further improved cross-domain robustness by up to 25\% in sensitivity. Additionally, the development of diverse global datasets and collaborative data-sharing frameworks is emphasized as a cornerstone for equitable and reliable malaria diagnostics. The role of explainable AI techniques in improving clinical adoption and trustworthiness is also underscored. By addressing these challenges, this work advances the field of AI-driven malaria detection and provides actionable insights for researchers and practitioners. The proposed solutions aim to support the development of accessible and accurate diagnostic tools, particularly for resource-constrained populations., Comment: 22 pages, 17 figures, 17 tables, In: Journal of Sensor Networks and Data Communications (JSNDC). ISSN: 2994-6433. DOI: 10.33140/JSNDC.04.03.09

Code review is a widespread practice to improve software quality and transfer knowledge. It is often seen as time-consuming due to the need for manual effort and potential delays. Several AI-assisted tools, such as Qodo, GitHub Copilot, and Coderabbit, provide automated reviews using large language models (LLMs). The effects of such tools in the industry are yet to be examined. This study examines the impact of LLM-based automated code review tools in an industrial setting. The study was conducted within a software development environment that adopted an AI-assisted review tool (based on open-source Qodo PR Agent). Around 238 practitioners across ten projects had access to the tool. We focused on three projects with 4,335 pull requests, 1,568 of which underwent automated reviews. Data collection comprised three sources: (1) a quantitative analysis of pull request data, including comment labels indicating whether developers acted on the automated comments, (2) surveys sent to developers regarding their experience with reviews on individual pull requests, and (3) a broader survey of 22 practitioners capturing their general opinions on automated reviews. 73.8% of automated comments were resolved. However, the average pull request closure duration increased from five hours 52 minutes to eight hours 20 minutes, with varying trends across projects. Most practitioners reported a minor improvement in code quality due to automated reviews. The LLM-based tool proved useful in software development, enhancing bug detection, increasing awareness of code quality, and promoting best practices. However, it also led to longer pull request closure times and introduced drawbacks like faulty reviews, unnecessary corrections, and irrelevant comments., Comment: To be presented at ICSE 2025 47th International Conference on Software Engineering, Software Engineering in Practice (SEIP) Track

The rapid development of 5G New Radio (NR) and millimeter-wave (mmWave) communication systems highlights the critical importance of maintaining accurate phase synchronization to ensure reliable and efficient communication. This study focuses on evaluating phase noise models and implementing Minimum Mean Square Error (MMSE) algorithms for Common Phase Error (CPE) compensation. Through extensive simulations, we demonstrate that CPE compensation significantly enhances signal quality by reducing Error Vector Magnitude (EVM) and Bit Error Rate (BER) across various Signal-to-Noise Ratio (SNR) levels and antenna configurations. Results indicate that implementing MMSE-based CPE estimation and compensation in 5G-NR mmWave systems reduced EVM from 7.4\% to 4.6\% for 64QAM and from 5.4\% to 4.3\% for 256QAM, while also decreasing BER from $5.5 \times 10^{-3}$ to $5.2 \times 10^{-5}$ for 64QAM, demonstrating significant improvements in signal quality and reliability across various SNR levels and antenna configurations. Our findings provide valuable insights for optimizing phase noise mitigation strategies in 5G-NR mmWave systems, contributing to the development of more robust and efficient next-generation wireless networks., Comment: 16 pages, 13 figures, 7 tables

Assigning a subset of labels from a fixed pool of labels to a given input text is a text classification problem with many real-world applications, such as in recommender systems. Two separate research streams address this issue. Hierarchical Text Classification (HTC) focuses on datasets with smaller label pools of hundreds of entries, accompanied by a semantic label hierarchy. In contrast, eXtreme Multi-Label Text Classification (XML) considers very large label pools with up to millions of entries, in which the labels are not arranged in any particular manner. However, in XML, a common approach is to construct an artificial hierarchy without any semantic information before or during the training process. Here, we investigate how state-of-the-art models from one domain perform when trained and tested on datasets from the other domain. The HBGL and HGLCR models from the HTC domain are trained and tested on the datasets Wiki10-31K, AmazonCat-13K, and Amazon-670K from the XML domain. On the other side, the XML models CascadeXML and XR-Transformer are trained and tested on the datasets Web of Science, The New York Times Annotated Corpus, and RCV1-V2 from the HTC domain. HTC models, on the other hand, are not equipped to handle the size of XML datasets and achieve poor transfer results. The code and numerous files that are needed to reproduce our results can be obtained from https://github.com/FloHauss/XMC_HTC

Recent advances in large language models have demonstrated promising capabilities in following simple instructions through instruction tuning. However, real-world tasks often involve complex, multi-step instructions that remain challenging for current NLP systems. Despite growing interest in this area, there lacks a comprehensive survey that systematically analyzes the landscape of complex instruction understanding and processing. Through a systematic review of the literature, we analyze available resources, representation schemes, and downstream tasks related to instructional text. Our study examines 177 papers, identifying trends, challenges, and opportunities in this emerging field. We provide AI/NLP researchers with essential background knowledge and a unified view of various approaches to complex instruction understanding, bridging gaps between different research directions and highlighting future research opportunities.

Sophisticated grammatical error detection/correction tools are available for a small set of languages such as English and Chinese. However, it is not straightforward -- if not impossible -- to adapt them to morphologically rich languages with complex writing rules like Turkish which has more than 80 million speakers. Even though several tools exist for Turkish, they primarily focus on spelling errors rather than grammatical errors and lack features such as web interfaces, error explanations and feedback mechanisms. To fill this gap, we introduce GECTurk WEB, a light, open-source, and flexible web-based system that can detect and correct the most common forms of Turkish writing errors, such as the misuse of diacritics, compound and foreign words, pronouns, light verbs along with spelling mistakes. Our system provides native speakers and second language learners an easily accessible tool to detect/correct such mistakes and also to learn from their mistakes by showing the explanation for the violated rule(s). The proposed system achieves 88,3 system usability score, and is shown to help learn/remember a grammatical rule (confirmed by 80% of the participants). The GECTurk WEB is available both as an offline tool at https://github.com/GGLAB-KU/gecturkweb or online at www.gecturk.net.

Multilingual language models often perform unevenly across different languages due to limited generalization capabilities for some languages. This issue is significant because of the growing interest in making universal language models that work well for all languages. Instruction tuning with multilingual instruction-response pairs has been used to improve model performance across various languages. However, this approach is challenged by high computational costs, a lack of quality tuning data for all languages, and the "curse of multilinguality" -- the performance drop per language after adding many languages. Recent studies have found that working with datasets with few languages and a smaller number of instances can be beneficial. Yet, there exists no systematic investigation into how choosing different languages affects multilingual instruction tuning. Our study proposes a method to select languages for instruction tuning in a linguistically informed way, aiming to boost model performance across languages and tasks. We use a simple algorithm to choose diverse languages and test their effectiveness on various benchmarks and open-ended questions. Our results show that this careful selection generally leads to better outcomes than choosing languages at random. We suggest a new and simple way of enhancing multilingual models by selecting diverse languages based on linguistic features that could help develop better multilingual systems and guide dataset creation efforts. All resources, including the code for language selection and multilingual instruction tuning, are made available in our official repository at https://github.com/GGLAB-KU/ling-informed-mit enabling reproducibility and further research in this area., Comment: 31 pages, 6 figures

Dialogue State Tracking (DST) is crucial for understanding user needs and executing appropriate system actions in task-oriented dialogues. Majority of existing DST methods are designed to work within predefined ontologies and assume the availability of gold domain labels, struggling with adapting to new slots values. While Large Language Models (LLMs)-based systems show promising zero-shot DST performance, they either require extensive computational resources or they underperform existing fully-trained systems, limiting their practicality. To address these limitations, we propose a zero-shot, open-vocabulary system that integrates domain classification and DST in a single pipeline. Our approach includes reformulating DST as a question-answering task for less capable models and employing self-refining prompts for more adaptable ones. Our system does not rely on fixed slot values defined in the ontology allowing the system to adapt dynamically. We compare our approach with existing SOTA, and show that it provides up to 20% better Joint Goal Accuracy (JGA) over previous methods on datasets like Multi-WOZ 2.1, with up to 90% fewer requests to the LLM API.

Fundamental Measure Theory (FMT) is a successful and versatile approach for describing the properties of the hard-sphere fluid and hard-sphere mixtures within the framework of classical density functional theory (DFT). Lutsko [Phys. Rev. E 102, 062137 (2020)] introduced a version of FMT containing two free parameters, to be fixed by additional physical constraints. Whereas Lutsko focused on the stability of crystalline phases, we introduce and employ two statistical mechanical sum rules pertinent for the fluid phase, that are not automatically satisfied by FMT. By minimizing the relative deviation between different routes to calculate the excess chemical potential and the isothermal compressibility we determine the two free parameters of the theory. Our results indicate that requiring consistency with these sum rules can improve the quality of predictions of FMT for properties of the hard-sphere fluid phase. We suggest that employing these (test particle) sum rules, which apply for any interparticle pair-potential, might provide a means of testing the performance and accuracy of general DFT approximations.

Postselection is an operation that allows the selection of specific measurement outcomes. It serves as a powerful theoretical tool for enhancing the performance of existing quantum algorithms. Despite recent developments such as time reversal in quantum measurements and IBM's mid-circuit measurements, postselection continues to face significant challenges, most notably poor, often exponential, scaling. This study investigates how Two-Way Quantum Computing (2WQC) offers potential solutions to these challenges. By introducing the concept of postparation and enabling dynamic quantum state control, 2WQC has the potential to mitigate scaling issues and improve the practicality of postselection, thereby fostering advancements in the field of quantum algorithms., Comment: 8 pages, 4 figures

The ever-evolving landscape of wireless communication technologies has led to the development of 5G-NR (5G New Radio) networks promising higher data rates and lower latency. However, with these advancements come challenges in managing intra-cell and inter-cell interference, particularly during the random-access procedure. This article aims to explore the impact of UE (User Equipment) and cell configuration parameters on interference and establish improved interference management strategies. To assess the effectiveness of these strategies, Key Performance Indicators (KPIs) such as Correct Detection Rate (CDR) will be considered. Additionally, the Matched Filtering (MF)-Based PRACH (Physical Random-Access CHannel) detection algorithm will be studied to evaluate the PRACH performance. The findings underscore the critical role of interference management and the configuration of User Equipment (UE)/Cell parameters in ensuring the robustness and reliability of the NR-PRACH performance in 5G NR networks. Optimizing these configurations can lead to improved network performance and enhanced communication quality., Comment: 19 pages, 13 figures, 3 tables, 4 equations, conference

In this paper, we present a novel approach to interference detection in 5G New Radio (5G-NR) networks using Convolutional Neural Networks (CNN). Interference in 5G networks challenges high-quality service due to dense user equipment deployment and increased wireless environment complexity. Our CNN-based model is designed to detect Physical Random Access Channel (PRACH) sequences amidst various interference scenarios, leveraging the spatial and temporal characteristics of PRACH signals to enhance detection accuracy and robustness. Comprehensive datasets of simulated PRACH signals under controlled interference conditions were generated to train and validate the model. Experimental results show that our CNN-based approach outperforms traditional PRACH detection methods in accuracy, precision, recall and F1-score. This study demonstrates the potential of AI/ML techniques in advancing interference management in 5G networks, providing a foundation for future research and practical applications in optimizing network performance and reliability.

The focus of this paper is on 3D motion editing. Given a 3D human motion and a textual description of the desired modification, our goal is to generate an edited motion as described by the text. The key challenges include the scarcity of training data and the need to design a model that accurately edits the source motion. In this paper, we address both challenges. We propose a methodology to semi-automatically collect a dataset of triplets comprising (i) a source motion, (ii) a target motion, and (iii) an edit text, introducing the new MotionFix dataset. Access to this data allows us to train a conditional diffusion model, TMED, that takes both the source motion and the edit text as input. We develop several baselines to evaluate our model, comparing it against models trained solely on text-motion pair datasets, and demonstrate the superior performance of our model trained on triplets. We also introduce new retrieval-based metrics for motion editing, establishing a benchmark on the evaluation set of MotionFix. Our results are promising, paving the way for further research in fine-grained motion generation. Code, models, and data are available at https://motionfix.is.tue.mpg.de/ ., Comment: SIGGRAPH Asia 2024 Camera Ready, Project page: https://motionfix.is.tue.mpg.de

Our objective is to generate Audio Descriptions (ADs) for both movies and TV series in a training-free manner. We use the power of off-the-shelf Visual-Language Models (VLMs) and Large Language Models (LLMs), and develop visual and text prompting strategies for this task. Our contributions are three-fold: (i) We demonstrate that a VLM can successfully name and refer to characters if directly prompted with character information through visual indications without requiring any fine-tuning; (ii) A two-stage process is developed to generate ADs, with the first stage asking the VLM to comprehensively describe the video, followed by a second stage utilising a LLM to summarise dense textual information into one succinct AD sentence; (iii) A new dataset for TV audio description is formulated. Our approach, named AutoAD-Zero, demonstrates outstanding performance (even competitive with some models fine-tuned on ground truth ADs) in AD generation for both movies and TV series, achieving state-of-the-art CRITIC scores., Comment: Project Page: https://www.robots.ox.ac.uk/~vgg/research/autoad-zero/

The computational power and fault-tolerance of future large-scale quantum processors derive in large part from the connectivity between the qubits. One approach to increase connectivity is to engineer qubit-qubit interactions at a distance. Alternatively, the connectivity can be increased by physically displacing the qubits. This has been explored in trapped-ion experiments and using neutral atoms trapped with optical tweezers. For semiconductor spin qubits, several studies have investigated spin coherent shuttling of individual electrons, but high-fidelity transport over extended distances remains to be demonstrated. Here we report shuttling of an electron inside an isotopically purified Si/SiGe heterostructure using electric gate potentials. First, we form static quantum dots, and study how spin coherence decays as we repeatedly move a single electron between up to five dots. Next, we create a traveling wave potential to transport an electron in a moving quantum dot. This second method shows substantially better spin coherence than the first. It allows us to displace an electron over an effective distance of 10 {\mu}m in under 200 ns with an average fidelity of 99%. These results will guide future efforts to realize large-scale semiconductor quantum processors, making use of electron shuttling both within and between qubit arrays., Comment: 15 pages, 15 figures

We provide results of our study on text-based 3D human motion retrieval and particularly focus on cross-dataset generalization. Due to practical reasons such as dataset-specific human body representations, existing works typically benchmarkby training and testing on partitions from the same dataset. Here, we employ a unified SMPL body format for all datasets, which allows us to perform training on one dataset, testing on the other, as well as training on a combination of datasets. Our results suggest that there exist dataset biases in standard text-motion benchmarks such as HumanML3D, KIT Motion-Language, and BABEL. We show that text augmentations help close the domain gap to some extent, but the gap remains. We further provide the first zero-shot action recognition results on BABEL, without using categorical action labels during training, opening up a new avenue for future research.

In this work, our goals are two fold: large-vocabulary continuous sign language recognition (CSLR), and sign language retrieval. To this end, we introduce a multi-task Transformer model, CSLR2, that is able to ingest a signing sequence and output in a joint embedding space between signed language and spoken language text. To enable CSLR evaluation in the large-vocabulary setting, we introduce new dataset annotations that have been manually collected. These provide continuous sign-level annotations for six hours of test videos, and will be made publicly available. We demonstrate that by a careful choice of loss functions, training the model for both the CSLR and retrieval tasks is mutually beneficial in terms of performance -- retrieval improves CSLR performance by providing context, while CSLR improves retrieval with more fine-grained supervision. We further show the benefits of leveraging weak and noisy supervision from large-vocabulary datasets such as BOBSL, namely sign-level pseudo-labels, and English subtitles. Our model significantly outperforms the previous state of the art on both tasks.