Search

Diplomska naloga se osredotoča na zasnovo avtomatskega testiranja programske opreme, ki se je v zadnjih letih uveljavilo kot najbolj učinkovita metoda za odkrivanje napak in zagotavljanje optimalnega delovanja. Zato je postalo avtomatsko testiranje nepogrešljivo orodje v svetu razvoja programske opreme, saj je zasenčilo staromodno ročno testiranje. V nadaljevanju naslavlja izzive, s katerimi se srečujemo pri testiranju aplikacij z govornim vmesnikom ter predstavi ustrezna orodja in ogrodja za avtomatsko testiranje tovrstnih aplikacij. Med naštetimi sta nato izbrani dve orodji, in sicer Playwright ter Selenium, ki ju konkretno predstavimo in primerjamo. Glavni cilj diplomske naloge je tako preveriti orodja za avtomatsko testiranje aplikacij z govornim vmesnikom. Poudarek je na enostavnosti, učinkovitosti ter hitrosti pisanja in izvajanja testov. Končni rezultat dela nam pove, katero orodje je najbolj primerno za testiranje aplikacij, ki zadostujejo našim potrebam. This thesis focuses on the concept of automated software testing, which has emerged in recent years as the most effective method for detecting faults and ensuring optimal performance. As a result, automated testing has become an indispensable tool in the world of software development, eclipsing old-fashioned manual testing. In the following, we address the challenges encountered when testing applications with a speech interface and present relevant tools and frameworks for automated testing of such applications. Two of these tools, Playwright and Selenium, are then selected for concrete presentation and comparison. The main objective of the thesis is thus to examine tools for automatic testing of speech-enabled applications. The focus is on simplicity, efficiency and speed of writing and running tests. The final result of the work tells us which tool is the most suitable for testing applications that meet our needs.

Researchers, data scientists, and programmers have started to recognize and take action on gender bias in data sets used to train artificial intelligence (AI). To date, almost all scholarly and practical work has focused on sexist and abusive language targeting negative representations of femininity or reactions to women, girls, femme folk, and others who identify as feminine. Little is known about how masculinities are represented in AI: positive or negative in valence; narrow or diverse in expression; toxic or prosocial in orientation; and so on. As ongoing work in feminist, queer, gender, and masculinity studies has argued, gender bias and sexism is not merely power differences and discrimination against women and gender diverse people, but also the social construction of masculinities. As such, addressing gender bias and sexism requires acknowledging and interrogating how masculinities are constructed and embedded in sociotechnical systems. Moreover, analyses need to evaluate both extreme forms of bias, such as toxic behaviour and misogynistic language, and the more subtle varieties of masculine biases that may be implicitly toxic to men and male-identifying people as well as unwelcoming to women, such as the pronouns assumed of the readership of a text. As a first step, I focus on how masculinities are represented in open natural language processing (NLP) data sets drawn from big social media data sources or crowdsourced through services such as Amazon Mechanical Turk. I use automated content analysis of the MASSIVE data set of dialogues for conversational AI (FitzGerald et al., 2022) with the dictionary created by Roberts and Utych (2019) to assess the relative degree of masculine-biased language and the misogyny meta-dictionary created by Farell et al. (2019) to assess the relative degree of more extreme expressions of toxic masculinities. After presenting the results, I propose critical next steps and strategies for expanding prosocial representations and mitigating negative expressions of masculinities in NLP data sets.

Initiatives in natural language processing (NLP) for training natural conversation and communication between people and virtual assistants (VAs) is taking off. Many are gathering and offering large, crowdsourced data sets for this purpose. An ongoing challenge is that most are English and geared towards the US context. In recognition of this, new efforts have started to be undertaken with the goal of ensuring that a diversity of languages and cultural contexts are represented. Most are crowdsourced translation activities. However, critical analyses have raised concerns about biases in translation, notably around gender and word choice. One issue that remains under-explored is how pronouns are translated from language to language. Languages vary greatly on pronoun use. For example, in English, a small set of pronouns are commonly used, but in Japanese, pronouns are less common and arguably more varied. Translators would need to make careful choices about how to translate pronouns in a case-by-case fashion, and also be aware of the cultural context of variable pronouns such as "they/them," which can be interpreted as a neutral plural pronoun or a singular non-binary pronoun. As a first step, we evaluated pronoun translations in four open NLP data sets that were translated from English to Japanese through crowdsourcing on two respects: (i) relative number of pronouns linked to certain genders and (ii) whether or not direct translations of pronouns were faithfully preserved. We aim to raise awareness of sociocultural biases that may be embedded in translation work and offer a technical solution to account for pronoun diversity and context-sensitive variability.

Surveying the literature on how the user experience (UX) of voice-based agents, interfaces, systems, and experiences has been measured.

The voice UX literature will be systematically surveyed to discover what qualitative approaches have been used and what findings there are so far. 07/20/21 - Added the following keywords to the query after an initial search in PsychInfo: qualitative OR grounded theory OR diary OR focus group* OR interview OR thematic analysis OR content analysis OR *ethnography OR phenomenology OR case study OR observations* OR discourse analysis OR participant-observation OR field stud* OR explorative stud* OR dialog* OR narrative* OR conversation analysis

This paper focuses on the development of the speech interface for controlling a Micro Air Vehicle (MAV). A speech interface in such control applications will have two distinct modules. One is the Automatic Speech Recognition (ASR) module and the other is the Natural Language Processing (NLP) module. The ASR is developed using the models built using CMU Sphinx toolkit. The NLP scheme is proposed and developed using Natural Language Toolkit (NLTK). Understanding of the speech is very important in such kind of control applications. The NLP outcome is used to invoke the Ground Control Station (GCS) commands. The results are validated in a Flight Gear simulator using Mission Planner GCS configured for MAV.

In 2019, media reports raised awareness about privacy and security violations in Conversational User Interfaces (CUI) like Alexa, Siri and Google. Users report that they perceive CUI as creepy and that they are concerned about their privacy. The European General Data Protection Regulation (GDPR) and several other laws across the globe, give users the right to control processing of their data, for example by requesting deletion of their data and it gives them the right to obtain information about their data. Furthermore, GDPR and other laws advise for seamless communication of user rights, which, currently, is poorly implemented in CUI. We used a bespoke data collection interface to generate speaking chatbots and made them available as tasks on the crowd sourcing platform Mechanical Turk. With those chatbots we simulated how privacy can be communicated in a dialogue between user and machine. We find that conversational privacy can affect user perceptions of privacy and security positively. Moreover, user choices suggest that users are interested in obtaining information on their privacy and security in dialogue form. We discuss implications and limitations of this research.

The growing popularity of speech interfaces goes hand in hand with the creation of synthetic voices that sound ever more human. Previous research has been inconclusive about whether anthropomorphic design features of machines are more likely to be associated with positive user responses or, conversely, with uncanny experiences. To avoid detrimental effects of synthetic voice design, it is therefore crucial to explore what level of human realism human interactors prefer and whether their evaluations may vary across different domains of application. In a randomized laboratory experiment, 165 participants listened to one of five female-sounding robot voices, each with a different degree of human realism. We assessed how much participants anthropomorphized the voice (by subjective human-likeness ratings, a name-giving task and an imagination task), how pleasant and how eerie they found it, and to what extent they would accept its use in various domains. Additionally, participants completed Big Five personality measures and a tolerance of ambiguity scale. Our results indicate a positive relationship between human-likeness and user acceptance, with the most realistic sounding voice scoring highest in pleasantness and lowest in eeriness. Participants were also more likely to assign real human names to the voice (e.g., “Julia” instead of “T380”) if it sounded more realistic. In terms of application context, participants overall indicated lower acceptance of the use of speech interfaces in social domains (care, companionship) than in others (e.g., information & navigation), though the most human-like voice was rated significantly more acceptable in social applications than the remaining four. While most personality factors did not prove influential, openness to experience was found to moderate the relationship between voice type and user acceptance such that individuals with higher openness scores rated the most human-like voice even more positively. Study results are discussed in the light of the presented theory and in relation to open research questions in the field of synthetic voice design. Version of record

This paper shows recent Silent Speech Interface (SSI) progress that translates tongue motions into audible speech. In our previous work and also in the current study, the prediction of fundamental frequency (F0) from Ultra-Sound Tongue Images (UTI) was achieved using articulatory-to-acoustic mapping methods based on deep learning. Here we investigated several traditional discontinuous speech-based F0 estimation algorithms for the target of UTI-based SSI system. Besides, the vocoder parameters (F0, Maximum Voiced Frequency and Mel-Generalized Cepstrum) are predicted using deep neural networks, with UTI as input. We found that those discontinuous F0 algorithms are predicted with a lower error during the articulatory-to-acoustic mapping experiments. They result in slightly more natural synthesized speech than the baseline continuous F0 algorithm. Moreover, experimental results confirmed that discontinuous algorithms (e.g. Yin) are closest to original speech in objective metrics and subjective listening test.

We developed an ultrasonic-based silent speech interface for Virtual Reality (VR). As more and more customized devices are proposed to enhance the immersion and experience of VR, our system can be used to improve the capability of interactions between users and the systems, while retaining the possibilities of using various customized devices and avoiding some limitations of traditional speech recognition. By employing the channel estimation techniques with ultrasonic waves, we can derive movement characteristics of users’ lips, which can be used to fine-tune existing speech recognition models and augmented by vast open-sourced speech datasets. Moreover, we use the speech interface to guide the initialization of customized models for new users, so that they can easily have the access to our system. A two-stage experiment has been conducted and the results show that our system can achieve 90.8% command-level accuracy and 1.3% word-error-rate in sentence-level accuracy.

Articulatory-to-acoustic mapping seeks to reconstruct speech from a recording of the articulatory movements, for example, an ultrasound video. Just like speech signals, these recordings represent not only the linguistic content, but are also highly specific to the actual speaker. Hence, due to the lack of multi-speaker data sets, researchers have so far concentrated on speaker-dependent modeling. Here, we present multi-speaker experiments using the recently published TaL80 corpus. To model speaker characteristics, we adjusted the x-vector framework popular in speech processing to operate with ultrasound tongue videos. Next, we performed speaker recognition experiments using 50 speakers from the corpus. Then, we created speaker embedding vectors and evaluated them on the remaining speakers. Finally, we examined how the embedding vector influences the accuracy of our ultrasound-to-speech conversion network in a multi-speaker scenario. In the experiments we attained speaker recognition error rates below 3%, and we also found that the embedding vectors generalize nicely to unseen speakers. Our first attempt to apply them in a multi-speaker silent speech framework brought about a marginal reduction in the error rate of the spectral estimation step., Comment: 5 pages, 3 figures, 3 tables