Search

We describe Open CALL-SLT, a framework that allows people who may only have modest computer skills to create interactive speech-enabled CALL courses on the web. User content is uploaded to a server and remotely compiled and deployed. Courses can be created at six levels of increasing sophistication, where the specification of the content ranges from simple prompt/response pairs at one end to multimedia-enabled gamified dialogues with multiple paths at the other. We briefly describe the overall framework and the different levels.

We summarise experiments carried out using a system initiative spoken CALL system in which permitted responses to prompts are defined using a minimal formalism based on templates and regular expressions and describe a simple structural learning algorithm that uses annotated data to update response definitions. Using 1 927 utterances of training data we obtained a relative improvement of 20 in the system’s ability to react differentially to correct and incorrect input measured on a previously unseen test set. The results are significant at p < 0:005.

By definition spoken dialogue CALL systems should be easy to use and understand. However interaction in this context is often far from unhindered. In this paper we introduce a formative feedback mechanism in our CALL system which can monitor interaction report errors and provide advice and suggestions to users. The distinctive feature of this mechanism is the ability to combine information from different sources and decide on the most pertinent feedback which can also be adapted in terms of phrasing style and language. We conducted experiments at three secondary schools in German speaking Switzerland and the obtained results suggest that our feedback mechanism helps students during interaction and contributes as a motivating factor.

We describe work carried out in the context of a project whose goal is to develop a web enabled serious game designed to help German speaking beginner students of English improve their generative and auditory competence. The game which is currently undergoing a first round of formal user testing is intended as a self study adjunct to normal classroom instruction. It was developed in collaboration with a secondary school teacher with the content taken from a textbook commonly used in German speaking Switzerland and offers a short course of 8 interactive lessons using a combined vocabulary of about 450 words. A lesson is structured as a short prompted dialogue between the student and the machine where the student is encouraged to use simple language in practical contexts like booking a hotel room buying clothes or ordering a meal in a restaurant. We present an overview of the architecture which is designed to balance three main considerations: it should support coherent grammatical responses and reasonably interesting dialogues but also allow course content to be constructed by personnel who do not necessarily have a computer science background. We use as an example a new strategy we have recently developed where the central idea is to add further dialogue paths to the scripts graded by their degree of “cooperativeness”. The system starts with a simple cooperative interaction and opens up increasingly "uncooperative" dialogue paths as the student gains familiarity with the game.

We present a framework CALL SLT Lite which can be used by people with only very basic software skills to create interactive multimodal speech enabled CALL games suitable for beginner/low intermediate child language learners. The games are deployed over the web and can be accessed through a normal browser and the framework is completely language independent. As the name suggests the framework grew out of an earlier platform CALL SLT which enables construction of similar games but uses a more sophisticated architecture. We review the history of the project describing the type of game we are aiming to build and our reasons for believing that they are useful and then present CALL SLT Lite and an initial evaluation comparing the performance of the two versions of the framework. The results suggest that the Lite framework although much simpler offers performance at least as good as that of the original system.

We present an English L2 child learner speech corpus produced by Swiss German L1 students in their third year of learning English which is currently in the process of being collected. The collection method uses a web enabled multimodal language game implemented using the CALL SLT platform in which subjects hold prompted conversations with an animated agent. Prompts consist of a short animated English language video clip together with a German language piece of text indicating the semantic content of the requested response. Grammar based speech understanding is used to decide whether responses are accepted or rejected and dialogue flow is controlled using a simple XML based scripting language; the scripts are written to allow multiple dialogue paths the choice being made randomly. The system is gamified using a score and badge framework with four levels of badges. We describe the application the data collection and annotation procedures and the initial tranche of data. The full corpus when complete should contain at least 5 000 annotated utterances.

We describe a Swedish version of CALL-SLT,a web-deployed CALL system that allows beginner/intermediate students to practise generativespoken language skills. Speech recognitionis grammar-based, with language modelsderived, using the Regulus platform, fromsubstantial domain-independent feature grammars.The paper focusses on the Swedishgrammar resources, which were developedby generalising the existing English featuregrammar into a shared grammar for Englishand Swedish. It turns out that this can be donevery economically: all but a handful of rulesand features are shared, and English grammaressentially ends up being treated as a reducedform of Swedish. We conclude by presentinga simple evaluation which compares theSwedish and French versions of CALL-SLT.

We describe a scalable architecture particularly well suited to cloud based computing which can be used for Web deployment of spoken dialogue systems. In common with similar platforms like WAMI and the Nuance Mobile Developer Platform we use a client/server approach in which speech recognition is carried out on the server side; our architecture however differs from these systems in offering considerably more elaborate server side functionality based on large scale grammar based language processing and generic dialogue management. We describe two substantial applications built using our framework which we argue would have been hard to construct in WAMI or NMDP. Finally we present a series of evaluations carried out using CALL SLT a speech translation game where we contrast performance in Web and desktop versions. Task Error Rate in the Web version is only slightly inferior that in the desktop one and the average additional latency is under half a second. The software is generally available for research purposes.

We present an evaluation of CALL SLT a web based CALL application based on the “translation game” idea that can be used for practicing fluency in a limited domain. The version tested was configured to teach basic restaurant French to students whose native language is Arabic. Students spent about an hour and a half each working with the system and explored between five and eight lessons. The focus was on investigating how they interacted with the system and what they learned most effectively.

We describe two versions of a prototype system for computer assisted language learning hosted on a mobile device. In both versions the student is given prompts by the machine which they then have to render in the L2. The two versions differ with respect to the modality of the prompt: one presents it as an L1 based text string while the other one presents it in pictorial form. The two versions were tested on a group of 32 high school students as part of the Geneva University Student Week. Our findings suggest that male students found the pictorial version significantly easier to use while female students preferred the text version.

We present an evaluation of CALL SLT a web deployed speech enabled platform for improving fluency in a limited domain based on the “translation game” idea ofWang and Seneff. The evaluation used 10 Chinese speaking students of French who spent an average of about three hours each practising on a set of five lessons covering elementary and intermediate grammar topics in a restaurant domain. We found significant improvements in student performance measured both in terms of their ability to be recognised correctly by the application and according to written vocabulary and grammar tests.

We present an evaluation of a Web-deployed spoken language CALL system, carried out using crowd-sourcing methods. The system, “Survival Japanese”, is a crash course in tourist Japanese implemented within the platform CALL-SLT. The evaluation was carried out over one week using the Amazon Mechanical Turk. Although we found a high proportion of attempted scammers, there was a core of 23 subjects who used the system in a responsible manner. The evidence that these subjects learned from their 111 sessions and 9092 spoken interactions was significant at P=0.001. Our conclusion is that crowd-sourcing is a potentially valid method for evaluating spoken CALL systems.

We describe a series of experiments in which we start with English to French and English to Japanese versions of an Open Source rule-based speech translation system for a medical domain, and bootstrap correspondign statistical systems. Comparative evaluation reveals that the rule-based systems are still significantly better than the statistical ones, despite the fact that considerable effort has been invested in tuning both the recognition and translation components; also, a hybrid system only marginally improved recall at the cost of a los in precision. The result suggests that rule-based architectures may still be preferable to statistical ones for safety-critical speech translation tasks. Describimos una serie de experimentos en los que comenzamos con las versiones de inglés a francés y de inglés a japonés de un sistema de traducción oral basado en reglas de código abierto para un dominio médico, e iniciamos los sistemas estadísticos correspondientes. La evaluación comparativa revela que los sistemas basados en reglas son significativamente mejores que los estadísticos, a pesar del hecho de que se ha dedicado un esfuerzo considerable en mejorar los componentes de reconocimiento y traducción; además, un sistema híbrido sólo mejoró de modo marginal la memoria a costa de una pérdida de precisión. Los resultados indican que las arquitecturas basadas en reglas podrían ser mejores para tareas de traducción oral críticas. Descrivim una sèrie d'experiments en què comencem amb les versions d'anglès a francès i d'anglès a japonès d'un sistema de traducció oral basat en regles en codi obert per a un domini mèdic, i iniciem els sistemes estadístics corresponents. L'avaluació comparativa revela que els sistemes basats en regles encara són significativament millors que els estadístics, malgrat el fet que s'ha dedicat un esforç important a millorar els components tant de reconeixement com de traducció; a més a més, un sistema híbrid només va millorar marginalment la memòria a costa d'una pèrdua de precisió. Els resultats indiquen que les arquitectures basades en regles podrien ser millors que les estadístiques per a tasques de traducció oral crítiques.

We argue that there is an urgent need for spoken dialogue translation systems in the medical domain. In this work we describe how an existing system of this kind, originally intended for a desktop PC, was adapted to run in a mobile environment. The different characteristics of the target device necessitate new interaction approaches and interface design. The new configuration uses two client applications running on two different devices, one for the doctor and one for the patient, together with a server; the components are connected over a wireless network. The system supports context-dependent translation in both directions. We give a general overview of the system, and discuss some of the relevant design issues pertaining to deployment of speech-enabled systems on mobile devices.

We describe recent work on MedSLT, a medium-vocabulary interlingua-based medical speech translation system, focussing on issues that arise when handling languages of which the grammar engineer has little or no knowledge. We describe how we can systematically create and maintain multiple forms of grammars, lexica and interlingual representations, with some versions being used by language informants, and some by grammar engineers. In particular, we describe the advantages of structuring the interlingua definition as a simple semantic grammar, which includes a human-readable surface form. We show how this allows us to rationalise the process of evaluating translations between languages lacking common speakers. The grammar-based interlingua definition can also be used in other ways. We describe two applications: a simple generic tool for debugging to-interlingua translation rules, and a method for improving speech understanding performance by rescoring N-best speech hypothesis lists. Examples presented focus on the concrete case of translation between Japanese and Arabic in both directions.

Particularly considering the requirement of high reliability, we argue that the most appropriate architecture for a medical speech translator that can be realised using today's technology combines unidirectional (doctor to patient) translation, medium-vocabulary controlled language coverage, interlingua-based translation, an embedded help component, and deployability on a hand-held hardware platform. We present an overview of the Open Source MedSLT prototype, which has been developed in accordance with these design principles. The system is implemented on top of the Regulus and Nuance 8.5 platforms, translates patient examination questions for all language pairs in the set {English, French, Japanese, Arabic, Catalan}, using vocabularies of about 400 to 1 100 words, and can be run in a distributed client/server environment, where the client application is hosted on a Nokia Internet Tablet device.

We present an overview of MedSLT, a medium vocabulary medical speech translation system, focussing on the representational issues that arise when translating temporal and causal concepts. Although flat key/value structures are strongly preferred as semantic representations in speech understanding systems, we argue that it is infeasible to handle the necessary range of concepts using only flat structures. By exploiting the specific nature of the task, we show that it is possible to implement a solution which only slightly extends the representational complexity of the semantic representation language, by permitting an optional single nested level representing a subordinate clause construct. We sketch our solutions to the key problems of producing minimally nested representations using phrase-spotting methods, and writing cleanly structured rule-sets that map temporal and phrasal representations into a canonical interlingual form.

We present an overview of MedSLT, an Open Source platform for developing limited-domain medical speech translation systems. We focus in particular on the speech understanding architecture, which uses grammar-based language models derived using corpus-based specialisation methods from a single linguistically motivated grammar, and summarise the results of two evaluations which investigate the appropriateness of these design choices. Other sections describe the interlingua and its relationship with the recognition architecture, and the current demo system.

The paper directly compares two versions of a medical speech translation system, one with a grammar based language model (GLM) recognizer and the other with a statistical language model (SLM) recognizer. We construct the GLM using a corpus-based method, so that both the GLM and the SLM can be derived from the same corpus; evaluation is carried out with respect to performance on the speech translation task. Despite using a very small training set for both the GLM and the SLM, the SLM delivers much better word error rates on unseen test material. Nonetheless, evaluating both systems on translation performance rather than word error rates, the GLM-based version of the system outperforms the SLM on the actual translation task.

We describe an implemented spoken-language dialogue system for a travel-planning domain, which accesses a commercially available travel-information web-server and supports a flexible mixed-initiative dialogue strategy. We argue, based on data from initial Wizard-of-Oz experiments, that mixed-initiative strategies are appropriate for many types of user, but require more sophisticated architectures for processing of language and dialogue; we then use these observations to motivate an architecture which combines parallel deep and shallow natural language analysis engines and an agenda-driven dialogue manager. We outline the top-level processing strategy used by the dialogue manager, and also a novel formalism, which we call Flat Utterance Description, that allows us to reduce the output of the deep and shallow language-processing engines to a common representation.

The paper describes a Swedish-language customization (S-CLE) of the SRI Core Language Engine, which has been developed at SICS from the original English-language version by replacing English-specific modules with corresponding Swedish-language versions. The S-CLE is intended to be used as a building block in a broad range of applications, such as data-base query system, machine translation systems, NL front-ends, speech-to-text/text-to-speech systems, and so on. Examples of the first two types of application already exist. The main part of the S-CLE is an extensive Swedish grammar that is compiled into parsing and generation modules. The grammar formalism is a type of unification grammar loosely based on Generalized Phrase Structure Grammar (GPSG). Generation is performed using the Semantic-Head-Driven algorithm. Analysis turns sentences into ``Quasi-Logical Form'' (QLF), a logical-form representation, while generation works in the opposite direction. Intermediate stages include processing of morphology, syntax and semantics. For knowledge-base applications, a separate module can convert QLFs into conventional scoped logical forms. After two-and-a-half years of work (approximately 45 person months), the first prototype system has a vocabulary of about 1900 words and covers a fairly broad range of possible grammatical constructions. Based on our experience in this project, we present in this paper detailed arguments to support the claim that customization of an English-language NLP system is a highly cost-effective way of constructing Swedish language systems with corresponding functionality. A shorter version of this paper appears in L. Ahrenberg (ed.): Papers from the Third Nordic Conference on Text Comprehension in Man and Machine , Link Sweden, 1992.